A wide-scale image input / output processing method for a road collapse hazard detector

Abstract

This invention relates to a wide-scale image input / output processing method for a road collapse hazard detector, belonging to the field of image processing technology. It utilizes a sliding window preprocessor with multiple widths and step sizes to repeatedly crop wide-scale ground-penetrating radar (GPR) images, forming multiple sets of input images with conventional ratios. These input images are then fed into a pre-trained target detector for road collapse hazard identification. This technique not only solves the problem of wide-scale input images but also, due to the extensive and repeated cropping of the wide-scale input images, aligns with the principle of test data augmentation, thereby improving the hazard identification rate. The road collapse hazard identification results in each cropped image are mapped back to the original wide-scale GPR image. Due to repeated cropping, the target boxes of road collapse hazards mapped to the original image may overlap. This method sorts the overlapping boxes in descending order of confidence level and removes redundancy based on the degree of overlap, confidence level, and individual coordinates to obtain the final detection result.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a wide-scale image input and output processing method for a road collapse hazard detector. Background Technology

[0002] Deep learning-based object detectors (road collapse hazard detectors), such as Faster R-CNN, scale the input image to a suitable size before detecting objects. However, 3D ground-penetrating radar collects data along the road direction, creating wide-scale images (aspect ratio > 100:1). Therefore, when object detectors directly process these wide-scale images, the objects become severely distorted as the image is scaled, sometimes even compressed to a single pixel, significantly reducing the accuracy of road collapse hazard identification.

[0003] Therefore, at this stage, it is necessary to design a wide-scale image input and output processing method for road collapse hazard detectors to solve the above problems. Summary of the Invention

[0004] The purpose of this invention is to provide a wide-scale image input / output processing method for a road collapse hazard detector, addressing the technical problems existing in the prior art. Deep learning-based target detectors (road collapse hazard detectors) such as Faster R-CNN scale the input image to a suitable size before target detection. However, 3D ground-penetrating radar collects data along the road direction, forming wide-scale images (aspect ratio > 100:1). Therefore, when target detectors directly process such wide-scale images, the target undergoes severe deformation as the image is scaled, sometimes even being compressed to a single pixel, significantly reducing the accuracy of road collapse hazard identification.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A method for wide-scale image input / output processing of a road collapse hazard detector includes the following steps:

[0007] S1: Use a sliding window preprocessor with multiple widths and step sizes to crop wide-scale ground-penetrating radar images multiple times to form multiple sets of input images with conventional proportions;

[0008] S2: Input the input image from step S1 into the trained target detector to identify potential road collapse hazards;

[0009] S3: Based on step S2, map the road collapse hazard identification results in each cropped image to the original wide-scale ground-penetrating radar image;

[0010] S4: Based on step S3, the overlapping boxes are arranged in descending order of confidence level. Redundancy is removed from the overlapping boxes according to the degree of overlap, confidence level and their respective coordinates to obtain the final detection result and output it.

[0011] Furthermore, in step S1, the specific details are as follows:

[0012] set up The sliding window height of each preprocessor is... The height is the same as the data acquired by the radar, meaning this value is constant; the width is... The sliding step size is These two items are variables; the sliding windows of different preprocessors slide forward simultaneously according to their respective sliding steps. Each slide cropps a portion of image data at a standard ratio. The sliding stops when it reaches the boundary, and each preprocessor generates a set of image datasets at a regular scale. ;Right now

[0013]

[0014] In the formula Indicates the first A set of standard-scale image datasets generated by a preprocessor. Indicates the first The preprocessor in the first The image is a standard-sized image generated by cropping.

[0015] Furthermore, width and sliding step size The range of values ​​for is as follows:

[0016]

[0017] Furthermore, in step S3, the mapping process is as follows:

[0018] The step size of each sliding window preprocessor is known. Cropping sequence number of a standard aspect ratio image And the coordinates of the top left and bottom right corners of the target bounding box. Therefore, the corresponding coordinates of the target box in the original aspect ratio image can be calculated, forming a mapping relationship.

[0019]

[0020] For each preprocessor-cropped image data of a standard scale The above processing is performed on each target bounding box.

[0021] Furthermore, in step S4, assume that now for Group of regular scale image datasets Perform bounding box prediction. For the number of preprocessors, obtain Group the target bounding boxes; the redundancy removal process is as follows:

[0022] Normalize the score and coordinates of each box to distribute the data across... Between them, the coordinates and confidence scores of the target bounding box are packaged into a data structure;

[0023] Add all the target boxes obtained from this prediction to the list. In the middle, sort this list by confidence level Sort in descending order;

[0024] Declare empty lists for box clusters and result boxes respectively. and List Each location in the data contains a set of boxes, forming a cluster; Each location in the image contains only one target box, which is derived from... The result box for the corresponding cluster;

[0025] Loop through the list The target box in the list Find matching bounding boxes; matching means that the two bounding boxes overlap to a certain degree. ,in It's about the degree of overlap. Indicates the threshold;

[0026] If no match is found, then the list... Add the target box in the list and The end of the list is used as the new element; continue to the list. The next target box in;

[0027] If a match is found, add this target box to the list. In and List The position corresponding to the matching box Place;

[0028] Use box clusters All accumulated lists The target bounding box in the middle is recalculated. The coordinates and confidence score of the bounding box in the image are calculated using the following formula:

[0029]

[0030] In the formula This represents the x-coordinates of the two vertices used to describe the bounding box. y coordinate .

[0031] Furthermore, the list When there are two target boxes, i.e., the two target boxes are as follows:

[0032]

[0033] in These represent the coordinates of the two vertices used to describe the target box A. This represents the confidence level of target box A; the meaning of the parameters for target box B is similarly understood.

[0034] pass and Calculated ;

[0035] in,

[0036]

[0037] Furthermore, the list After all target boxes have been processed, if the list needs to be readjusted... The confidence level is calculated by multiplying by the number of bounding boxes in a cluster and then dividing by the number of models. Reduce confidence levels using the following two methods:

[0038]

[0039] .

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] One of the advantages of this scheme is that it uses a sliding window preprocessor to crop a wide-scale image into a set of regular-scale images. To minimize the possibility that a single sliding window preprocessor might truncate areas that could be potential hazard points, thus affecting the detection results, multiple sliding window preprocessors with different widths and lengths can be used. This extensive cropping of the wide-scale input image aligns with the idea of ​​test data augmentation, improving the hazard identification rate. Two issues need to be addressed regarding the target detector's output. First, since the detection results are based on the cropped regular-scale image set, while actual engineering projects still use wide-scale image data, it is necessary to map the road collapse hazard identification results from the cropped images back to the original wide-scale ground-penetrating radar image. Second, the repeated cropping using multiple sliding window preprocessors results in overlapping bounding boxes for road collapse hazards mapped to the original image. Therefore, a specific algorithm is needed to remove redundant bounding boxes to obtain the final result. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the processing technology flow of an embodiment of this application.

[0043] Figure 2 This is a schematic diagram illustrating the operation of the sliding window in an embodiment of this application.

[0044] Figure 3 This is a schematic diagram of the mapping algorithm in an embodiment of this application.

[0045] Figure 4 This is a schematic diagram of the mapping processing result in an embodiment of this application.

[0046] Figure 5 This is a schematic diagram of the redundancy removal algorithm in an embodiment of this application.

[0047] Figure 6 This is a schematic diagram of the target box processing procedure in an embodiment of this application.

[0048] Figure 7 This is a schematic diagram illustrating the process of generating a result box C from two target boxes A and B, as described in an embodiment of this application. Detailed Implementation

[0049] The following description, in conjunction with the appendix of the present invention, Figure 1 -Appendix Figure 7 The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0050] Deep learning-based object detectors (road collapse hazard detectors), such as Faster R-CNN, scale the input image to a suitable size before detecting objects. However, 3D ground-penetrating radar collects data along the road direction, creating wide-scale images (aspect ratio > 100:1). Therefore, when object detectors directly process these wide-scale images, the objects become severely distorted as the image is scaled, sometimes even compressed to a single pixel, significantly reducing the accuracy of road collapse hazard identification.

[0051] Example:

[0052] like Figure 3 As shown, a wide-scale image input / output processing technique for target detectors targeting road collapse hazards is proposed.

[0053] 1) Input Technology: This technology uses a sliding window preprocessor with multiple widths and step sizes to repeatedly crop wide-scale ground-penetrating radar images, forming multiple sets of input images with conventional proportions. These images are then input into a pre-trained target detector for identifying road collapse hazards. This technology not only solves the problem of wide-scale input images but also, due to the extensive and repeated cropping of wide-scale input images, aligns with the idea of ​​test data augmentation, thereby improving the hazard identification rate.

[0054] 2) Output Technique: The road collapse hazard identification results in each cropped image are mapped to the original wide-scale ground-penetrating radar image. Due to repeated cropping, the target boxes of road collapse hazards mapped to the original image will overlap. This method sorts the overlapping boxes in descending order of confidence, and performs redundancy removal on the overlapping boxes based on the degree of overlap, confidence, and their respective coordinates to obtain the final detection result.

[0055] A schematic diagram of the wide-scale image input / output processing technology for road collapse hazard detectors is shown below. Figure 1 As shown.

[0056] Pre-trained deep learning-based target detectors (such as Faster R-CNN) scale the input image to a suitable size before detection, resulting in good detection performance only for images with normal aspect ratios. However, 3D ground-penetrating radar collects wide-scale images (aspect ratio > 100:1) along the road. If this image data is directly fed into the target detector, the target will be severely distorted as the image is scaled, and the target width may even be compressed to 1 pixel, greatly reducing the accuracy of identifying road collapse hazards.

[0057] This method uses a sliding window preprocessor to crop a wide-scale image into a set of images with a normal scale. To minimize the possibility that a single sliding window preprocessor might truncate areas of the image that could be potential hazards, thus affecting the detection results, multiple sliding window preprocessors with different widths and lengths can be used. This extensive and repeated cropping of the wide-scale input image aligns with the principle of test data augmentation, thereby improving the hazard identification rate.

[0058] set up The sliding window height of each preprocessor is... The height is the same as the data acquired by the radar, meaning this value is constant. The width is... The sliding step size is These two items are variables. The sliding windows of different preprocessors slide forward simultaneously according to their own sliding steps. Each slide cropps a portion of image data at a standard ratio. The sliding stops when it reaches the boundary, so that each preprocessor generates a set of images at a regular scale.

[0059]

[0060] In the formula Indicates the first A set of standard-scale image datasets generated by a preprocessor. Indicates the first The preprocessor in the first The standard-scale image generated by the secondary cropping is illustrated in the following diagram of the sliding window operation: Figure 2 As shown.

[0061] The width and stride of the sliding window significantly impact the computational cost of the subsequent object detector. If the width and stride are too small, the granularity is too fine, resulting in excessive cropped image data at standard proportions, significantly increasing the workload of the object detector and making it easier to truncate areas that may contain hidden dangers. Conversely, if the width and stride are too large, the granularity is too coarse, leading to insufficient utilization of the original data and affecting the performance of the object detector. Therefore, we aim for the width and stride of the sliding window to be within a certain range. After several trials, the width... and sliding step size The effect is better when the value range is as follows:

[0062]

[0063] There are two issues to address regarding the target detector's output. First, since the detection results are based on a cropped, standard-scale image set, while actual engineering projects use wide-scale image data, it's necessary to map the road collapse hazard identification results from the cropped images back to the original wide-scale ground-penetrating radar images. Second, due to repeated cropping using multiple sliding window preprocessors, the target bounding boxes for road collapse hazards mapped to the original images overlap. Therefore, a specific algorithm is needed to remove redundant target bounding boxes to obtain the final result.

[0064] Since we know the step size of each sliding window preprocessor Cropping sequence number of a standard aspect ratio image (Starting from 0), and the coordinates of the top left and bottom right corners of the target box. Therefore, the corresponding coordinates of the target box in the original aspect ratio image can be calculated, forming a mapping relationship.

[0065]

[0066] A schematic diagram of the mapping algorithm is shown below. Figure 3 As shown.

[0067] For each preprocessor-cropped image data of a standard scale The above processing is performed on each target bounding box, and the final result is shown in the diagram. Figure 4 As shown.

[0068] This shows that overlapping occurs when the target boxes are mapped onto the original aspect ratio image. This is because multiple sliding window preprocessors repeatedly crop the original image data, causing the same road collapse hazard point to be detected repeatedly by the target detector. A series of processing and calculations can be performed on the overlapping boxes to obtain the final detection result. A schematic diagram of the redundancy removal algorithm is shown below. Figure 5 As shown.

[0069] In the image, the red boxes represent actual road subsidence hazard areas, while the blue boxes represent predicted road subsidence hazard areas. After processing with a redundancy removal algorithm, the road subsidence hazard points are clearly marked, facilitating further processing by relevant personnel. The redundancy removal algorithm is as follows:

[0070] Assuming now Group( (Number of preprocessors) Standard scale image dataset Perform bounding box prediction to obtain Group the target bounding boxes. The algorithm steps are as follows:

[0071] Normalize the score and coordinates of each box to distribute the data across... Between them, the coordinates and confidence scores of the target bounding box are packaged into a data structure, such as... .

[0072] Add all the target boxes obtained from this prediction to the list. In the middle, sort this list by confidence level Sort in descending order. See diagram below. Figure 6 As shown in the figure. For clarity, only three target boxes are shown in the figure; in reality, there may be more.

[0073] Declare empty lists for box clusters and result boxes respectively. and List Each location in the data can contain a set of boxes (or a single target box), forming a cluster; Each location in the image contains only one target box, which is derived from... The result box for the corresponding cluster. The formula for generating the result box will be given later.

[0074] Loop through the list The target box in the list, try to... Find matching bounding boxes. A match means that the two bounding boxes overlap sufficiently. ,in It's about the degree of overlap. This represents the threshold.

[0075] If no match is found, then the list... Add the target box in the list and The end of the list is used as the new element; continue to the list. The next target box in the [theory / method].

[0076] If a match is found, add this target box to the list. In and List The position corresponding to the matching box Place.

[0077] Use box clusters All accumulated lists The target bounding box in the middle is recalculated. The coordinates and confidence score of the bounding box in the image are calculated using the following formula:

[0078]

[0079] In the formula This represents the x-coordinates of the two vertices used to describe the bounding box. y coordinate The calculation process will be demonstrated below using two target boxes as an example:

[0080] There are two target boxes.

[0081]

[0082] in These represent the coordinates of the two vertices used to describe the target box A. This represents the confidence level of target box A; the meaning of the parameters for target box B is similarly understood.

[0083] pass and Calculated ;like Figure 7 As shown; where

[0084]

[0085] As you can see, the confidence level of the target bounding boxes is used as the weight to calculate a new target bounding box from two existing ones. Therefore, the higher the confidence level of the target bounding box, the greater its weight and its contribution to the generation of the new target bounding box. The shape and position of the new target bounding box are more biased towards the target bounding box with the higher confidence level.

[0086] For cases with multiple bounding boxes, the calculation can be performed using the formula described above.

[0087] List After all target boxes have been processed, the list is readjusted. The confidence level is calculated by multiplying the number of bounding boxes in a cluster and then dividing by the number of models. If the number of bounding boxes in a cluster is small, it may mean that only a few models can predict this situation, indicating that it is uncommon. Therefore, we need to reduce the confidence level for this situation. The confidence level can be reduced in two ways:

[0088]

[0089] .

[0090] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A wide-scale image input / output processing method for a road collapse hazard detector, characterized in that, Includes the following steps: S1: Use a sliding window preprocessor with multiple widths and step sizes to crop wide-scale ground-penetrating radar images multiple times to form multiple sets of input images with conventional proportions; S2: Input the input image from step S1 into the trained target detector to identify potential road collapse hazards; S3: Based on step S2, map the road collapse hazard identification results in each cropped image to the original wide-scale ground-penetrating radar image; S4: Based on step S3, the overlapping boxes are sorted in descending order of confidence level, and redundancy is removed from the overlapping boxes according to the degree of overlap, confidence level and their respective coordinates to obtain the final detection result and output it. In step S4, assume that the current... Group of regular scale image datasets Perform bounding box prediction. For the number of preprocessors, obtain Group the target bounding boxes; the redundancy removal process is as follows: Normalize the score and coordinates of each box to distribute the data across... Between them, the coordinates and confidence scores of the target bounding box are packaged into a data structure; Add all the target boxes obtained from this prediction to the list. In the middle, sort this list by confidence level Sort in descending order; Declare empty lists for box clusters and result boxes respectively. and List Each location in the data contains a set of boxes, forming a cluster; Each location in the image contains only one target box, which is derived from... The result box for the corresponding cluster; Loop through the list The target box in the list Find matching bounding boxes; matching means that the two bounding boxes overlap to a certain degree. ,in It's about the degree of overlap. Indicates the threshold; If no match is found, then the list... Add the target box in the list and The end of the list is used as the new element; continue to the list. The next target box in; If a match is found, add this target box to the list. In and List The position corresponding to the matching box Place; Use box clusters All accumulated lists in The target bounding box in the middle is recalculated. The coordinates and confidence score of the bounding box in the image are calculated using the following formula: In the formula This represents the x-coordinates of the two vertices used to describe the bounding box. y coordinate .

2. The wide-scale image input / output processing method for a road collapse hazard detector as described in claim 1, characterized in that, In step S1, the specific details are as follows: set up The sliding window height of each preprocessor is... The height is the same as the data acquired by the radar, meaning this value is constant; the width is... The sliding step size is These two items are variables; the sliding windows of different preprocessors slide forward simultaneously according to their respective sliding steps. Each slide cropps a portion of image data at a standard ratio. The sliding stops when it reaches the boundary, and each preprocessor generates a set of image datasets at a normal scale. ;Right now In the formula Indicates the first A set of standard-scale image datasets generated by a preprocessor. Indicates the first The preprocessor in the first The image is a standard-sized image generated by the cropping process.

3. The wide-scale image input / output processing method for a road collapse hazard detector as described in claim 2, characterized in that, width and sliding step size The range of values ​​for is as follows: 。 4. The wide-scale image input / output processing method for a road collapse hazard detector as described in claim 3, characterized in that, In step S3, the mapping process is as follows: The step size of each sliding window preprocessor is known. Cropping sequence number of a standard aspect ratio image And the coordinates of the top left and bottom right corners of the target bounding box. Therefore, the corresponding coordinates of the target box in the original aspect ratio image can be calculated, forming a mapping relationship. For each preprocessor-cropped image data of a standard scale The above processing is performed on each target bounding box.

5. The wide-scale image input / output processing method for a road collapse hazard detector as described in claim 4, characterized in that, List When there are two target boxes, i.e., the two target boxes are as follows: in These represent the coordinates of the two vertices used to describe the target box A. This represents the confidence level of target box A; the meaning of the parameters for target box B is similarly understood. pass and Calculated ; in, 。 6. The wide-scale image input / output processing method for a road collapse hazard detector as described in claim 4, characterized in that, List After all target boxes have been processed, if the list needs to be readjusted... The confidence level is calculated by multiplying by the number of bounding boxes in a cluster and then dividing by the number of models. ; Reduce confidence using the following two methods: 。

Images