Real-time detection method and system for underground collapse hidden danger based on 3D task parallel network

Abstract

The application discloses a ground collapse hidden danger real-time detection method based on a 3D task parallel network, and has the characteristics that a ground penetrating radar data set is acquired, the ground penetrating radar data set is preprocessed to obtain a preprocessed ground penetrating radar data set, and each ground penetrating radar image in the preprocessed ground penetrating radar data set is sequentially input into a trained 3D task parallel network to obtain hidden danger position information of the ground penetrating radar image. The application can solve the technical problems that the workload of the interpretation method based on artificial means used in the existing ground collapse hidden danger detection method based on ground penetrating radar images is large and time-consuming, thereby failing to meet the real-time detection needs, and the interpretation method based on artificial means highly depends on the technology and professional level of operators, leading to different interpretation results and affecting the accuracy of the ground collapse hidden danger detection result.

Description

Technical Field

[0001] The present invention belongs to the field of urban road survey, and more specifically, relates to a real-time detection method and system for ground collapse hazards based on a 3D task parallel network. Background Art

[0002] Using ground-penetrating radar (GPR) to detect potential ground subsidence hazards is a crucial tool for ensuring urban road safety. Most existing methods rely solely on manual interpretation of GPR images to identify underground hazards. When dealing with large amounts of data, manual methods are inefficient, labor-intensive, and time-consuming, often failing to detect hidden dangers in a timely manner. Failure to promptly identify and address these hazards can easily lead to safety accidents such as urban road subsidence, endangering public safety. Therefore, real-time hazard detection for urban roads is essential.

[0003] Ground-penetrating radar (GPR) technology has been widely used in engineering surveys due to its fast scanning speed, non-destructive testing, and convenient and flexible operation. Through electromagnetic transmission and reception, combined with signal and image processing techniques, GPR can generate two-dimensional (B-Scan) and three-dimensional (C-Scan) images. Due to the complex underground environment, B-Scan images generally cannot fully reflect the structure of the underground space, while C-Scan images can provide a more complete picture of the underground area. The radar data obtained by GPR requires interpretation, and currently this is primarily done manually, with operators controlling the GPR device to obtain GPR images and identify potential hazards within them.

[0004] However, the existing methods of using ground penetrating radar images to detect ground collapse hazards have some significant shortcomings:

[0005] First, its interpretation method based on manual means is labor-intensive and time-consuming, and cannot meet the needs of real-time detection;

[0006] Second, its interpretation method based on manual means is highly dependent on the operator's technical and professional level. Different operators have different levels of expertise, which often leads to different interpretation results, thus affecting the accuracy of ground collapse hazard detection results.

[0007] Third, the results obtained by this method are not reliable enough and are prone to missed detections, false detections, etc.

[0008] Fourth, this method has poor real-time performance. Summary of the Invention

[0009] In response to the above defects or improvement needs of the prior art, the present invention provides a real-time detection method for ground collapse hazards based on a 3D task parallel network. Its purpose is to solve the technical problems that the interpretation method based on manual means used in the existing method of detecting ground collapse hazards using ground penetrating radar images has a large workload and a long time consumption, and thus cannot meet the needs of real-time detection; the interpretation method based on manual means is highly dependent on the operator's skills and professional level, resulting in different interpretation results, thereby affecting the accuracy of the ground collapse hazard detection results; the technical problem that the reliability is insufficient and easily leads to missed detections, false detections, etc., and the technical problem of poor real-time performance.

[0010] To achieve the above object, according to one aspect of the present invention, a method for real-time detection of ground subsidence hazards based on a 3D task parallel network is provided, comprising the following steps:

[0011] (1) Obtain ground penetrating radar dataset;

[0012] (2) Preprocessing the ground penetrating radar dataset obtained in step (1) to obtain a preprocessed ground penetrating radar dataset.

[0013] (3) Each ground penetrating radar image in the ground penetrating radar dataset preprocessed in step (2) is sequentially input into the trained 3D task parallel network to obtain the hidden danger location information of the ground penetrating radar image.

[0014] Preferably, step (1) is to obtain a ground penetrating radar data set consisting of multiple ground penetrating radar data from the road through a SPIDAR array ground penetrating radar, and the ground penetrating radar data includes five types: void data, loose body data, pipeline data, cavity data, background data, and hidden danger location information marked by professional technicians.

[0015] Preferably, the method of preprocessing the data set obtained in step (1) in step (2) includes the following process: first, deleting redundant information related to the ground penetrating radar equipment in the ground penetrating radar data set, then, using the bilateral filtering method to denoise the ground penetrating radar data set after the redundant information is deleted, and using a linear function to normalize the denoised ground penetrating radar data set to obtain a normalized ground penetrating radar data set, and finally, performing a data amplification operation on the normalized ground penetrating radar data set to obtain a data-amplified ground penetrating radar data set as the preprocessed ground penetrating radar data set.

[0016] Preferably, the 3D task parallel network includes a 3D-Resnet feature extraction network, a 3D feature fusion network, and a task collaboration network connected in sequence.

[0017] Preferably, the architecture of the 3D-Resnet feature extraction network is as follows:

[0018] The first layer is an input layer, which inputs a matrix with a size of 512*512*512*1;

[0019] The second layer is a first convolutional layer, which receives an image from the input layer, the size of the convolution kernel of the layer is 7*7*7, the step is 2, the number of convolution kernels is 64, the layer is filled in the SAME mode, followed by a RELU activation function and batch normalization (BN), and the output size of the layer is 256*256*256*64;

[0020] The third layer is a maximum pooling layer, the size of the pooling window is 3*3*3, and the step is (2, 2, 2), and the output size of the layer is 128*128*128*64.

[0021] The fourth layer is a second convolutional layer, which receives a matrix with a size of 128*128*128*64 from the output of the third layer, the size of the convolution kernel of the layer is 1*1*1, the step is 1, the number of convolution kernels is 256, the layer is filled in the SAME mode, followed by BN, and the output size of the layer is 128*128*128*256;

[0022] The fifth layer is a first bottleneck layer, which receives a matrix with a size of 128*128*128*64 from the output of the third layer, and the layer includes three sub-convolutional layers, wherein the size of the convolution kernel of the first layer is 1*1*1, the step is 1, the number of convolution kernels is 64, the layer is filled in the SAME mode, followed by BN and a RELU activation function; the size of the convolution kernel of the second layer is 3*3*3, the step is 1, the number of convolution kernels is 64, the layer is filled in the SAME mode, followed by BN and a RELU activation function; the size of the convolution kernel of the third layer is 1*1*1, the step is 1, the number of convolution kernels is 256, the layer is filled in the SAME mode, followed by BN, and the output size of the layer is 128*128*128*256;

[0023] The sixth layer is an addition layer, which adds the results of the fourth layer and the fifth layer, followed by a RELU activation function, and outputs a matrix with a size of 128*128*128*256;

[0024] The seventh layer is the second bottleneck layer, which receives the matrix of size 128*128*128*256 output from the sixth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 64, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, the number of convolution kernels is 256. It is padded in the same way, followed by BN, and the output size of this layer is a matrix of size 128*128*128*256.

[0025] The eighth layer is the addition layer, which adds the results of the sixth layer and the seventh layer, and then connects the RELU activation function to output a matrix of size 128*128*128*256;

[0026] The ninth layer is the third bottleneck layer, which receives the matrix of size 128*128*128*256 output from the eighth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 64, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, the number of convolution kernels is 256. It is padded in the same way, followed by BN, and the output size of this layer is a matrix of size 128*128*128*256.

[0027] The tenth layer is the addition layer, which adds the results of the eighth layer and the ninth layer, and then connects the RELU activation function to output a matrix of size 128*128*128*256;

[0028] The eleventh layer is the third convolutional layer, which receives the matrix of size 128*128*128*256 output from the tenth layer. The convolution kernel size of this layer is 1*1*1, the stride is 2, and the number of convolution kernels is 512. This layer is padded in the SAME way and then connected to BN. The output matrix of this layer is 64*64*64*512.

[0029] The twelfth layer is the fourth bottleneck layer, which receives the matrix of size 128*128*128*256 output from the tenth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 2, the number of convolution kernels is 128, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64*64*64*512.

[0030] The thirteenth layer is the addition layer, which adds the results of the eleventh layer and the twelfth layer, and then connects the RELU activation function to output a matrix of size 64*64*512;

[0031] The fourteenth layer is the fifth bottleneck layer, which receives the matrix of size 64*64*64*512 output from the thirteenth layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64*64*64*512.

[0032] The fifteenth layer is the addition layer, which adds the results of the thirteenth layer and the fourteenth layer, and then connects the RELU activation function to output a matrix of size 64*64*64*512;

[0033] The sixteenth layer is the sixth bottleneck layer, which receives the matrix of size 64*64*64*512 output from the fifteenth layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64*64*64*512.

[0034] The seventeenth layer is the addition layer, which adds the results of the fifteenth layer and the sixteenth layer, and then connects the RELU activation function to output a matrix of size 64*64*64*512;

[0035] The eighteenth layer is the seventh bottleneck layer, which receives the matrix of size 64*64*64*512 output from the seventeenth layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 128, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64*64*64*512;

[0036] The nineteenth layer is the addition layer, which adds the results of the seventeenth layer and the eighteenth layer, and then connects the RELU activation function to output a matrix of size 64*64*64*512;

[0037] The 20th layer is the fourth convolutional layer, which receives the 64*64*64*512 matrix output from the 19th layer. The convolution kernel size of this layer is 1*1*1, the stride is 2, and the number of convolution kernels is 1024. This layer is padded in the SAME way and then connected to BN. The output matrix of this layer is 32*32*32*1024.

[0038] The 21st layer is the eighth bottleneck layer, which receives the 64*64*64*512 matrix output from the 19th layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 2, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32*32*32*1024.

[0039] The 21st layer is the addition layer, which adds the results of the 20th layer and the 21st layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0040] The 22nd layer is the ninth bottleneck layer, which receives the 32*32*32*1024 matrix output from the 21st layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of 32*32*32*1024.

[0041] The 23rd layer is the addition layer, which adds the results of the 21st layer and the 22nd layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0042] The 24th layer is the tenth bottleneck layer, which receives the matrix of size 32*32*32*1024 output from the 23rd layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32*32*32*1024.

[0043] The 25th layer is the addition layer, which adds the results of the 23rd layer and the 24th layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0044] The 26th layer is the 11th bottleneck layer, which receives the 32*32*32*1024 matrix output from the 25th layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of 32*32*32*1024.

[0045] The 27th layer is the addition layer, which adds the results of the 25th layer and the 26th layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0046] The 28th layer is the twelfth bottleneck layer, which receives the matrix of size 32*32*32*1024 output from the 27th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32*32*32*1024.

[0047] The 29th layer is the addition layer, which adds the results of the 27th layer and the 28th layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0048] The 30th layer is the 13th bottleneck layer, which receives the matrix of size 32*32*32*1024 output from the 29th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32*32*32*1024.

[0049] The 31st layer is the addition layer, which adds the results of the 29th layer and the 30th layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0050] The 32nd layer is the fifth convolutional layer, which receives the matrix of size 32*32*32*1024 output from the 31st layer. The convolution kernel size of this layer is 1*1*1, the stride is 2, and the number of convolution kernels is 2048. This layer is padded in the SAME way and then connected to BN. The output matrix of this layer is 16*16*16*2048.

[0051] The 33rd layer is the 14th bottleneck layer, which receives the 32*32*32*1024 matrix output from the 31st layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 2, the number of convolution kernels is 512, and it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 16*16*16*2048.

[0052] The 34th layer is the addition layer, which adds the results of the 32nd layer and the 33rd layer, and then connects the RELU activation function to output a matrix of size 16*16*16*2048;

[0053] The 35th layer is the 15th bottleneck layer, which receives the matrix of size 16*16*16*2048 output from the 34th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 512, and it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output size of this layer is 16*16*16*2048.

[0054] The 36th layer is the addition layer, which adds the results of the 34th layer and the 35th layer, and then connects the RELU activation function to output a matrix of size 16*16*16*2048;

[0055] The 37th layer is the 16th bottleneck layer, which receives the matrix of size 16*16*2048 output from the 35th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 512, and it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 16*16*16*2048.

[0056] The 38th layer is the addition layer, which adds the results of the 36th layer and the 37th layer, and then connects the RELU activation function to output a matrix of size 16*16*16*2048;

[0057] The 40th layer is a fully connected layer. It receives the 1x1x2048 feature matrix output from the 39th layer. It has 6 output nodes and is followed by a dropout operation to prevent overfitting. The 6 nodes represent the locations of hidden dangers in the current GPR image.

[0058] Preferably, the feature fusion network includes four horizontally connected convolutional layers, the convolution kernel size is 1*1*1, and the number of convolution kernels in each layer is 256. It receives the feature matrices from the sixth, nineteenth, thirty-first, and thirty-eighth addition layers of the 3D-Resnet feature extraction network, respectively, and outputs matrices of sizes 128*128*128*256, 56*56*56*256, 32*32*32*256, and 16*16*16*256, respectively. Each output matrix is ​​downsampled by 2 times and concatenated with the adjacent matrix. Then, a convolution kernel of size 3*3*3, number 256, and step size 1 is used to perform feature calculation on each matrix obtained after splicing to obtain feature matrix pyramids of 128*128*128*256, 56*56*56*256, 32*32*32*256, and 16*16*16*256.

[0059] Preferably, the architecture in the task collaboration network is composed of a classifier and a localizer with the same structure connected in parallel;

[0060] For each of the classifier and the locator, it includes a first part, a second part, and a third part;

[0061] The first part is five sequentially connected convolutional layers with a convolution kernel size of 1*1*1, a number of 128, and a stride of 1;

[0062] The second part is the maximum pooling layer, which receives the matrix of the first part output size of 16*16*16*128 and outputs a matrix of size 1*1*1*128;

[0063] The third part is the fully connected layer, which receives the matrix of size 1*1*1*128 output by the second part, has 5 output nodes, and is followed by a random deactivation operation to prevent overfitting;

[0064] Preferably, the 3D task-parallel network is trained by the following steps:

[0065] A. Obtain a dataset consisting of ground-penetrating radar images acquired by ground-penetrating radar equipment manufactured by mainstream manufacturers, as well as location information of ground subsidence hazards annotated by professional technicians;

[0066] B. Preprocess the data set obtained in step A to obtain a preprocessed data set, and randomly divide the preprocessed data set into a training set, a validation set, and a test set;

[0067] C. Input the training set from the dataset preprocessed in step B into the 3D task parallel network to obtain an inference output of the potential ground collapse location. Input the inference output into the loss function in the 3D task parallel network to obtain a loss value.

[0068] D. Use the stochastic gradient descent algorithm and the loss value obtained in step C to optimize the loss function in the 3D task-parallel network to iteratively update the parameters in the 3D task-parallel network.

[0069] E. Repeat steps C and D for the remaining data sets in the training set part of the data set obtained in step B until the 3D task parallel network converges to the optimal state, thereby obtaining a preliminarily trained 3D task parallel network.

[0070] F. Use the validation set in the dataset obtained in step B to validate the preliminarily trained 3D task-parallel network to obtain a validated 3D task-parallel network;

[0071] G. Use the test set in the data set obtained in step B to test the verified 3D task parallel network to obtain a tested 3D task parallel network;

[0072] H. Use the coarse-grained channel pruning algorithm to compress the tested 3D task-parallel network to obtain a streamlined 3D task-parallel network as the trained 3D task-parallel network.

[0073] Preferably, the loss function used in step C is:

[0074]

[0075] Among them, D(x, y) represents the distance between the predicted box x of the 3D task parallel network and the center point y of the labeled box in the training set, c represents the diagonal length of the minimum rectangular box that can simultaneously surround the predicted box x and the labeled box, and IOU represents the ratio between the intersection of the predicted box x of the 3D task parallel network and the labeled box in the training set and the union of the two.

[0076] In the iterative process of step D, lr=0.001, batch size BatchSize=32, and the number of iterations is 200.

[0077] According to another aspect of the present invention, a real-time detection system for ground collapse hazards based on a 3D task parallel network is provided, comprising:

[0078] The first module is used to obtain the ground penetrating radar dataset;

[0079] The second module is used to preprocess the ground penetrating radar dataset obtained by the first module to obtain a preprocessed ground penetrating radar dataset.

[0080] The third module is used to input each ground penetrating radar image in the ground penetrating radar dataset preprocessed by the second module into the trained 3D task parallel network in sequence to obtain the hidden danger location information of the ground penetrating radar image.

[0081] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0082] (1) Since the present invention adopts steps (1) to (3), it uses artificial intelligence technology to realize fully automated detection of ground collapse hidden dangers in ground penetrating radar images, and can replace the operator to complete the collapse hidden danger identification work, thereby solving the technical problems of large workload and long time consumption in the existing method of using ground penetrating radar images for ground collapse hidden danger detection using manual means;

[0083] (2) Since the present invention adopts step (3), it uses a 3D task-parallel deep convolutional network, and the three subnetworks of the network respectively complete the feature extraction of three-dimensional ground penetrating radar images, the identification of ground collapse hidden danger types, and the positioning of hidden danger locations. Therefore, the network can solve the technical problem that the existing method of using ground penetrating radar images for ground collapse hidden danger detection uses an interpretation method based on manual means, which is highly dependent on the operator's technical and professional level, and different operators have different levels, resulting in different operators obtaining different interpretation results, thereby affecting the accuracy of the ground collapse hidden danger detection results;

[0084] (3) Since the present invention performs automatic recognition on three-dimensional C-Scan radar images, it can perform comprehensive feature extraction and analysis of underground spatial structure information, thereby fully reflecting the underground situation and effectively reducing the occurrence of false detection and missed detection of underground hidden dangers.

[0085] (4) Since step F is adopted in the training process of the 3D task parallel network of the present invention, it uses the model channel pruning algorithm to trim unimportant weights in the network, which greatly reduces the parameters of the model while accelerating network training, improves the efficiency of the model in processing data, and can enhance the real-time performance of the network in detecting underground hidden dangers. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 This is a flow chart of a method for real-time detection of ground subsidence hazards based on a 3D task parallel network according to the present invention;

[0087] Figure 2 3D task parallel network used in the real-time detection method of the present invention. DETAILED DESCRIPTION

[0088] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0089] The basic idea of ​​the present invention is to provide a real-time detection method for ground collapse hazards based on a 3D task parallel network. It specifically uses a 3D-Resnet feature extraction network for multi-scale feature extraction, a 3D feature fusion network for feature fusion, and a task collaborative network to predict the category and confidence of ground collapse hazards contained in the image.

[0090] like Figure 1 As shown, the present invention provides a real-time detection method for ground collapse hazards based on a 3D task parallel network, comprising the following steps:

[0091] (1) Obtain ground penetrating radar dataset;

[0092] Specifically, this step uses the SPIDAR array ground-penetrating radar to obtain a ground-penetrating radar dataset consisting of multiple ground-penetrating radar data from major roads in Longhua District, Shenzhen City, Guangdong Province (Longhua Street, Minzhi Avenue, Longlan Avenue, etc.). The ground-penetrating radar data includes five types: void data, loose body data, pipeline data, cavity data, background data, and hidden danger location information marked by professional technicians.

[0093] (2) Preprocessing the ground penetrating radar dataset obtained in step (1) to obtain a preprocessed ground penetrating radar dataset.

[0094] Specifically, the method for preprocessing the data set obtained in step (1) in this step includes the following process: first, deleting redundant information related to the ground penetrating radar equipment in the ground penetrating radar data set, then, using the bilateral filtering method to denoise the ground penetrating radar data set after the redundant information is deleted, and using a linear function to normalize the denoised ground penetrating radar data set to obtain a normalized ground penetrating radar data set, and finally, performing a data amplification operation on the normalized ground penetrating radar data set to obtain a data-amplified ground penetrating radar data set as the preprocessed ground penetrating radar data set.

[0095] Specifically, data augmentation methods include random flipping and scaling.

[0096] (3) inputting each ground penetrating radar image in the ground penetrating radar dataset preprocessed in step (2) into the trained 3D task parallel network in sequence to obtain the hidden danger location information of the ground penetrating radar image;

[0097] Specifically, the 3D task parallel network includes a sequentially connected 3D-Resnet feature extraction network, a 3D feature fusion network, and a task collaboration network.

[0098] Specifically, the architecture of the 3D-Resnet feature extraction network is as follows:

[0099] The first layer is the input layer: its input is a matrix of size 512*512*512*1;

[0100] The second layer is the first convolutional layer, which receives the image from the input layer. The size of the convolution kernel of this layer is 7*7*7, the stride is 2, and the number of convolution kernels is 64. This layer is padded in the same way, followed by the RELU activation function and batch normalization (BN). The output of this layer is a matrix of size 256*256*256*64.

[0101] The third layer is the maximum pooling layer with a pooling window size of 3*3*3 and a step size of (2, 2, 2). The output of this layer is a matrix of size 128*128*128*64.

[0102] The fourth layer is the second convolutional layer, which receives the matrix of size 128*128*128*64 output from the third layer. The convolution kernel size of this layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. This layer is padded in the SAME way and then connected to BN. The output of this layer is a matrix of size 128*128*128*256;

[0103] The fifth layer is the first bottleneck layer, which receives the matrix of size 128*128*128*64 output from the third layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN, and the output size of this layer is 128*128*128*256.

[0104] The sixth layer is the addition layer, which adds the results of the fourth layer and the fifth layer, and then connects the RELU activation function to output a matrix of size 128*128*128*256;

[0105] The seventh layer is the second bottleneck layer, which receives the matrix of size 128*128*128*256 output from the sixth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 64, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, the number of convolution kernels is 256. It is padded in the same way, followed by BN, and the output size of this layer is a matrix of size 128*128*128*256.

[0106] The eighth layer is the addition layer, which adds the results of the sixth layer and the seventh layer, and then connects the RELU activation function to output a matrix of size 128*128*128*256;

[0107] The ninth layer is the third bottleneck layer, which receives the matrix of size 128*128*128*256 output from the eighth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 64, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, the number of convolution kernels is 256. It is padded in the same way, followed by BN, and the output size of this layer is a matrix of size 128*128*128*256.

[0108] The tenth layer is the addition layer, which adds the results of the eighth layer and the ninth layer, and then connects the RELU activation function to output a matrix of size 128*128*128*256;

[0109] The eleventh layer is the third convolutional layer, which receives the matrix of size 128*128*128*256 output from the tenth layer. The convolution kernel size of this layer is 1*1*1, the stride is 2, and the number of convolution kernels is 512. This layer is padded in the SAME way and then connected to BN. The output matrix of this layer is 64*64*64*512.

[0110] The twelfth layer is the fourth bottleneck layer, which receives the matrix of size 128*128*128*256 output from the tenth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 2, the number of convolution kernels is 128, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64*64*64*512.

[0111] The thirteenth layer is the addition layer, which adds the results of the eleventh layer and the twelfth layer, and then connects the RELU activation function to output a matrix of size 64*64*512;

[0112] The fourteenth layer is the fifth bottleneck layer, which receives the matrix of size 64*64*64*512 output from the thirteenth layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64*64*64*512.

[0113] The fifteenth layer is the addition layer, which adds the results of the thirteenth layer and the fourteenth layer, and then connects the RELU activation function to output a matrix of size 64*64*64*512;

[0114] The sixteenth layer is the sixth bottleneck layer, which receives the matrix of size 64*64*64*512 output from the fifteenth layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64*64*64*512.

[0115] The seventeenth layer is the addition layer, which adds the results of the fifteenth layer and the sixteenth layer, and then connects the RELU activation function to output a matrix of size 64*64*64*512;

[0116] The eighteenth layer is the seventh bottleneck layer, which receives the matrix of size 64*64*64*512 output from the seventeenth layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 128, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64*64*64*512;

[0117] The nineteenth layer is the addition layer, which adds the results of the seventeenth layer and the eighteenth layer, and then connects the RELU activation function to output a matrix of size 64*64*64*512;

[0118] The 20th layer is the fourth convolutional layer, which receives the 64*64*64*512 matrix output from the 19th layer. The convolution kernel size of this layer is 1*1*1, the stride is 2, and the number of convolution kernels is 1024. This layer is padded in the SAME way and then connected to BN. The output matrix of this layer is 32*32*32*1024.

[0119] The 21st layer is the eighth bottleneck layer, which receives the 64*64*64*512 matrix output from the 19th layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 2, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32*32*32*1024.

[0120] The 21st layer is the addition layer, which adds the results of the 20th layer and the 21st layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0121] The 22nd layer is the ninth bottleneck layer, which receives the 32*32*32*1024 matrix output from the 21st layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of 32*32*32*1024.

[0122] The 23rd layer is the addition layer, which adds the results of the 21st layer and the 22nd layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0123] The 24th layer is the tenth bottleneck layer, which receives the matrix of size 32*32*32*1024 output from the 23rd layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32*32*32*1024.

[0124] The 25th layer is the addition layer, which adds the results of the 23rd layer and the 24th layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0125] The 26th layer is the 11th bottleneck layer, which receives the 32*32*32*1024 matrix output from the 25th layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of 32*32*32*1024.

[0126] The 27th layer is the addition layer, which adds the results of the 25th layer and the 26th layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0127] The 28th layer is the twelfth bottleneck layer, which receives the matrix of size 32*32*32*1024 output from the 27th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32*32*32*1024.

[0128] The 29th layer is the addition layer, which adds the results of the 27th layer and the 28th layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0129] The 30th layer is the 13th bottleneck layer, which receives the matrix of size 32*32*32*1024 output from the 29th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32*32*32*1024.

[0130] The 31st layer is the addition layer, which adds the results of the 29th layer and the 30th layer, and then connects the RELU activation function to output a matrix of size 32*32*32*1024;

[0131] The 32nd layer is the fifth convolutional layer, which receives the matrix of size 32*32*32*1024 output from the 31st layer. The convolution kernel size of this layer is 1*1*1, the stride is 2, and the number of convolution kernels is 2048. This layer is padded in the SAME way and then connected to BN. The output matrix of this layer is 16*16*16*2048.

[0132] The 33rd layer is the 14th bottleneck layer, which receives the 32*32*32*1024 matrix output from the 31st layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 2, the number of convolution kernels is 512, and it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 16*16*16*2048.

[0133] The 34th layer is the addition layer, which adds the results of the 32nd layer and the 33rd layer, and then connects the RELU activation function to output a matrix of size 16*16*16*2048;

[0134] The 35th layer is the 15th bottleneck layer, which receives the matrix of size 16*16*16*2048 output from the 34th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 512, and it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output size of this layer is 16*16*16*2048.

[0135] The 36th layer is the addition layer, which adds the results of the 34th layer and the 35th layer, and then connects the RELU activation function to output a matrix of size 16*16*16*2048;

[0136] The 37th layer is the 16th bottleneck layer, which receives the matrix of size 16*16*2048 output from the 35th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1*1*1, the step size is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3*3*3, the step size is 1, the number of convolution kernels is 512, and it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1*1*1, the step size is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 16*16*16*2048.

[0137] The 38th layer is the addition layer, which adds the results of the 36th layer and the 37th layer, and then connects the RELU activation function to output a matrix of size 16*16*16*2048;

[0138] The 40th layer is a fully connected layer. It receives the 1x1x2048 feature matrix output from the 39th layer. It has 6 output nodes and is followed by a dropout operation to prevent overfitting. The 6 nodes represent the locations of hidden dangers in the current GPR image.

[0139] Specifically, the feature fusion network architecture is as follows:

[0140] It includes 4 convolutional layers connected transversely, the convolution kernel size is 1*1*1, the number of convolution kernels in each layer is 256, which respectively receives the feature matrix from the sixth, nineteenth, thirty-first and thirty-eighth addition layer of the 3D-Resnet feature extraction network, and the output size is 128*128*128*256, 56*56*56*256, 32*32*32*256 and 16*16*16*256 matrix respectively. After 2 times down-sampling of each output matrix, the adjacent matrix is spliced, and then a convolution kernel with a size of 3*3*3, a number of 256 and a step of 1 is used to calculate the features of each spliced matrix to obtain a feature matrix pyramid with a size of 128*128*128*256, 56*56*56*256, 32*32*32*256 and 16*16*16*256.

[0141] Specifically, the task coordination network architecture is as follows:

[0142] After the multi-scale features obtained by the feature fusion network are multiplied and spliced, they are input into the network composed of the same structure of the classifier and the locator in parallel. The first part is five convolutional layers connected in sequence, the convolution kernel size is 1*1*1, the number is 128, and the step is 1. The second part is a max-pooling layer, which receives the matrix with an output size of 16*16*16*128 from the first part, and outputs a matrix with an output size of 1*1*1*128. The third part is a fully connected layer, which receives the matrix with an output size of 1*1*1*128 from the second part, and outputs 5 nodes, followed by a random deactivation operation to prevent overfitting. In the classifier, the five nodes represent the category of hidden dangers in the current three-dimensional radar image; in the locator, the five nodes represent the location of the corresponding hidden danger.

[0143] Specifically, the process of training the 3D task parallel network includes the following sub-steps:

[0144] A, obtaining a data set, which includes ground penetrating radar images obtained by ground penetrating radar equipment manufactured by mainstream manufacturers in the market, and location information of ground collapse hidden dangers labeled by professional technicians;

[0145] B, preprocessing the data set obtained in step A to obtain a preprocessed data set, and randomly dividing the preprocessed data set into a training set, a validation set and a test set;

[0146] Specifically, the pre-processed data set is randomly divided into 3 parts, of which 80% is used as the training set (Trainset), 10% is used as the validation set (Validation set), and 10% is used as the test set (Test set). In this example, the data set has a total of 4000 data, the training set includes 3200 data, the validation set includes 400 data, and the test set includes 400 data;

[0147] C. The training set in the pre-processed data set of step B is input into the 3D task parallel network to obtain the inference output of the ground collapse hidden danger position, and the inference output is input into the loss function in the 3D task parallel network to obtain the loss value;

[0148] D. The loss function in the 3D task parallel network is optimized using the random gradient descent algorithm and the loss value obtained in step C to iteratively update the parameters in the 3D task parallel network;

[0149] Specifically, the lr in the iterative training process is 0.001, the batch size BatchSize is 32, and the iteration number is 200.

[0150] Specifically, the loss function is:

[0151]

[0152] Wherein, D(x, y) represents the distance between the prediction frame x of the 3D task parallel network and the center point y of the labeled frame in the training set, c represents the diagonal length of the smallest rectangular frame that can simultaneously enclose the prediction frame x and the labeled frame, and IOU represents the ratio between the intersection of the prediction frame x of the 3D task parallel network and the labeled frame in the training set and the union of the two.

[0153] E. For the remaining data set in the training set part of the data set obtained in step B, the above steps C and D are repeatedly executed until the 3D task parallel network converges to the best, thereby obtaining a preliminarily trained 3D task parallel network.

[0154] F. The preliminarily trained 3D task parallel network is verified using the validation set in the data set obtained in step B to obtain a verified 3D task parallel network;

[0155] G. The verified 3D task parallel network is tested using the test set in the data set obtained in step B to obtain a tested 3D task parallel network;

[0156] H. The tested 3D task parallel network is compressed using a coarse-grained channel pruning algorithm to obtain a simplified 3D task parallel network as a trained 3D task parallel network.

[0157] Performance test results

[0158] The following are the results of model testing using ground-penetrating radar images from the test set into the trained 3D task-parallel network. The 3D task-parallel network's detection results were evaluated using accuracy, recall, and mean average precision (mAP), as shown in Table 1. The detection speed of the 3D task-parallel network was also evaluated using frames per second (FPS), as shown in Table 2.

[0159] Table 1

[0160] Hazard Category Accuracy Recall mAP empty 0.9423 0.8871 0.9108 Hollow 0.9612 0.8159 0.9268 pipeline 0.9636 0.9262 0.9741 loose body 0.9543 0.9547 0.9614

[0161] Table 2

[0162] Model FPS 3D task-parallel network (before compression) 18.24 3D task parallel network (compressed) 32.88

[0163] It can be seen from Table 1 above that the method provided by the present invention has a high accuracy and recall rate for detecting hidden dangers in ground penetrating radar images, and can meet the needs of current urban road surveys for the investigation of ground collapse hidden dangers.

[0164] It can be seen from Table 2 that the 3D task parallel network compression method provided by the present invention can improve the detection speed of the network model for ground penetrating radar images, meeting the requirements of real-time detection.

[0165] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A real-time detection method for ground collapse hazards based on a 3D task parallel network, characterized in that: The following steps are involved: (1) Obtaining ground penetrating radar dataset; (2) Preprocessing the GPR dataset obtained in step (1) to obtain a preprocessed GPR dataset; Step (2) Method for preprocessing the dataset obtained in step (1) The method includes the following steps: first, deleting redundant information related to the ground penetrating radar equipment in the ground penetrating radar dataset; then, performing denoising on the ground penetrating radar dataset after the redundant information is deleted using a bilateral filtering method; and performing normalization on the denoised ground penetrating radar dataset using a linear function to obtain a normalized ground penetrating radar dataset; and finally, performing a data augmentation operation on the normalized ground penetrating radar dataset to obtain an amplified ground penetrating radar dataset as a preprocessed ground penetrating radar dataset. (3) Each GPR image in the GPR dataset preprocessed in step (2) is sequentially input into the trained 3D task parallel network to obtain the hidden danger location information of the GPR image; the 3D task parallel network is trained through the following steps: A. Obtain a dataset consisting of ground-penetrating radar images acquired by ground-penetrating radar equipment manufactured by mainstream manufacturers, as well as location information of ground subsidence hazards annotated by professional technicians; B. Preprocess the data set obtained in step A to obtain a preprocessed data set, and randomly divide the preprocessed data set into a training set, a validation set, and a test set; C. Input the training set from the dataset preprocessed in step B into the 3D task parallel network to obtain an inference output of the potential ground collapse location. Input the inference output into the loss function in the 3D task parallel network to obtain a loss value. D. Optimize the loss function in the 3D task-parallel network using the stochastic gradient descent algorithm and the loss value obtained in step C to iteratively update the parameters in the 3D task-parallel network. E. Repeat steps C and D for the remaining data sets in the training set portion of the data set obtained in step B until the 3D task-parallel network converges to the optimal state, thereby obtaining a preliminarily trained 3D task-parallel network. F. Use the validation set in the dataset obtained in step B to validate the preliminarily trained 3D task-parallel network to obtain a validated 3D task-parallel network; G. Use the test set in the data set obtained in step B to test the verified 3D task parallel network to obtain a tested 3D task parallel network; H. Use the coarse-grained channel pruning algorithm to compress the tested 3D task-parallel network to obtain a streamlined 3D task-parallel network as the trained 3D task-parallel network.

2. The method for real-time detection of ground subsidence hazards based on a 3D task parallel network according to claim 1 is characterized in that: Step (1) is to obtain a ground penetrating radar data set consisting of multiple ground penetrating radar data from the road through the SPIDAR array ground penetrating radar, and the ground penetrating radar data includes five types: void data, loose body data, pipeline data, cavity data, background data, and hidden danger location information marked by professional technicians.

3. The method for real-time detection of ground subsidence hazards based on a 3D task parallel network according to claim 2 is characterized in that: The 3D task parallel network includes a sequentially connected 3D-Resnet feature extraction network, a 3D feature fusion network, and a task collaboration network.

4. The method for real-time detection of ground subsidence hazards based on a 3D task parallel network according to claim 1 is characterized in that: The architecture of the 3D-Resnet feature extraction network is as follows: The first layer is the input layer: its input is a matrix of size 512×512×512×1; The second layer is the first convolutional layer, which receives the image from the input layer. The size of the convolution kernel of this layer is 7×7×7, the stride is 2, and the number of convolution kernels is 64. This layer is padded in the same way, followed by the RELU activation function and batch normalization BN. The output of this layer is a matrix of size 256×256×256×64; The third layer is the maximum pooling layer with a pooling window size of 3×3×3 and a stride of (2, 2, 2). The output of this layer is a matrix of size 128×128×128×64. The fourth layer is the second convolutional layer, which receives the matrix of size 128×128×128×64 output from the third layer. The convolution kernel size of this layer is 1×1×1, the stride is 1, and the number of convolution kernels is 256. This layer is padded in the SAME way and then connected to BN. The output of this layer is a matrix of size 128×128×128×256. The fifth layer is the first bottleneck layer, which receives the matrix of size 128×128×128×64 output from the third layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 128×128×128×256. The sixth layer is the addition layer, which adds the results of the fourth layer and the fifth layer, and then connects the RELU activation function to output a matrix of size 128×128×128×256; The seventh layer is the second bottleneck layer, which receives the matrix of size 128×128×128×256 output from the sixth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN, and the output size of this layer is 128×128×128×256. The eighth layer is the addition layer, which adds the results of the sixth layer and the seventh layer, and then connects the RELU activation function to output a matrix of size 128×128×128×256; The ninth layer is the third bottleneck layer, which receives the matrix of size 128×128×128×256 output from the eighth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 1, and the number of convolution kernels is 64. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 128×128×128×256. The tenth layer is the addition layer, which adds the results of the eighth layer and the ninth layer, and then connects the RELU activation function to output a matrix of size 128×128×128×256; The eleventh layer is the third convolutional layer, which receives the matrix of size 128×128×128×256 output from the tenth layer. The convolution kernel size of this layer is 1×1×1, the stride is 2, and the number of convolution kernels is 512. This layer uses the SAME padding method and is followed by BN. The output of this layer is a matrix of size 64×64×64×512. The twelfth layer is the fourth bottleneck layer, which receives the matrix of size 128×128×128×256 output from the tenth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 2, the number of convolution kernels is 128, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64×64×64×512. The thirteenth layer is the addition layer, which adds the results of the eleventh layer and the twelfth layer, and then connects the RELU activation function to output a matrix of size 64×64×512; The fourteenth layer is the fifth bottleneck layer, which receives the matrix of size 64×64×64×512 output from the thirteenth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64×64×64×512. The fifteenth layer is the addition layer, which adds the results of the thirteenth layer and the fourteenth layer, and then connects the RELU activation function to output a matrix of size 64×64×64×512; The sixteenth layer is the sixth bottleneck layer, which receives the matrix of size 64×64×64×512 output from the fifteenth layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the step size is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the step size is 1, the number of convolution kernels is 128, it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the step size is 1, the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64×64×64×512. The seventeenth layer is the addition layer, which adds the results of the fifteenth layer and the sixteenth layer, and then connects the RELU activation function to output a matrix of size 64×64×64×512; The eighteenth layer is the seventh bottleneck layer, which receives the matrix of size 64×64×64×512 output from the seventeenth layer. This layer includes three sub-convolutional layers, where the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 1, and the number of convolution kernels is 128. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 64×64×64×512. The nineteenth layer is the addition layer, which adds the results of the seventeenth layer and the eighteenth layer, and then connects the RELU activation function to output a matrix of size 64×64×64×512; The 20th layer is the fourth convolutional layer, which receives the matrix of size 64×64×64×512 output from the 19th layer. The convolution kernel size of this layer is 1×1×1, the stride is 2, and the number of convolution kernels is 1024. This layer uses the SAME padding method and is followed by BN. The output of this layer is a matrix of size 32×32×32×1024; The 21st layer is the eighth bottleneck layer, which receives the 64×64×64×512 matrix output from the 19th layer. This layer includes three sub-convolutional layers, of which the size of the first convolution kernel is 1×1×1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the second convolution kernel is 3×3×3, the step size is 2, the number of convolution kernels is 256, and it is padded in the same way, followed by BN and RELU activation functions; the size of the third convolution kernel is 1×1×1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32×32×32×1024. The 21st layer is the addition layer, which adds the results of the 20th layer and the 21st layer, and then connects the RELU activation function to output a matrix of size 32×32×32×1024; The 22nd layer is the ninth bottleneck layer, which receives the matrix of size 32×32×32×1024 output from the 21st layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32×32×32×1024. The 23rd layer is the addition layer, which adds the results of the 21st layer and the 22nd layer, and then connects the RELU activation function to output a matrix of size 32×32×32×1024; The 24th layer is the tenth bottleneck layer, which receives the matrix of size 32×32×32×1024 output from the 23rd layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32×32×32×1024. The 25th layer is the addition layer, which adds the results of the 23rd layer and the 24th layer, and then connects the RELU activation function to output a matrix of size 32×32×32×1024; The 26th layer is the 11th bottleneck layer, which receives the matrix of size 32×32×32×1024 output from the 25th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32×32×32×1024. The 27th layer is the addition layer, which adds the results of the 25th layer and the 26th layer, and then connects the RELU activation function to output a matrix of size 32×32×32×1024; The 28th layer is the twelfth bottleneck layer, which receives the matrix of size 32×32×32×1024 output from the 27th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32×32×32×1024. The 29th layer is the addition layer, which adds the results of the 27th layer and the 28th layer, and then connects the RELU activation function to output a matrix of size 32×32×32×1024; The 30th layer is the 13th bottleneck layer, which receives the matrix of size 32×32×32×1024 output from the 29th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the step size is 1, and the number of convolution kernels is 256. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the step size is 1, and the number of convolution kernels is 1024. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 32×32×32×1024. The 31st layer is the addition layer, which adds the results of the 29th layer and the 30th layer, and then connects the RELU activation function to output a matrix of size 32×32×32×1024; The 32nd layer is the fifth convolutional layer, which receives the matrix of size 32×32×32×1024 output from the 31st layer. The convolution kernel size of this layer is 1×1×1, the stride is 2, and the number of convolution kernels is 2048. This layer uses the SAME padding method and is followed by BN. The output of this layer is a matrix of size 16×16×16×2048. The 33rd layer is the 14th bottleneck layer, which receives the matrix of size 32×32×32×1024 output from the 31st layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 2, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 16×16×16×2048. The 34th layer is the addition layer, which adds the results of the 32nd layer and the 33rd layer, and then connects the RELU activation function to output a matrix of size 16×16×16×2048; The 35th layer is the 15th bottleneck layer, which receives the matrix of size 16×16×16×2048 output from the 34th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 1, the number of convolution kernels is 512, and it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 16×16×16×2048. The 36th layer is the addition layer, which adds the results of the 34th layer and the 35th layer, and then connects the RELU activation function to output a matrix of size 16×16×16×2048; The 37th layer is the 16th bottleneck layer, which receives the matrix of size 16×16×2048 output from the 35th layer. This layer includes three sub-convolutional layers, of which the size of the convolution kernel of the first layer is 1×1×1, the stride is 1, and the number of convolution kernels is 512. It is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the second layer is 3×3×3, the stride is 1, the number of convolution kernels is 512, and it is padded in the same way, followed by BN and RELU activation functions; the size of the convolution kernel of the third layer is 1×1×1, the stride is 1, and the number of convolution kernels is 2048. It is padded in the same way, followed by BN, and the output of this layer is a matrix of size 16×16×16×2048. The 38th layer is the addition layer, which adds the results of the 36th layer and the 37th layer, and then connects the RELU activation function to output a matrix of size 16×16×16×2048; The 40th layer is a fully connected layer that receives the feature matrix of size 1×1×2048 output from the 39th layer. The number of its output nodes is 6, and a random deactivation operation is performed to prevent overfitting. The 6 nodes represent the locations of hidden dangers in the current ground penetrating radar image.

5. The method for real-time detection of ground subsidence hazards based on a 3D task parallel network according to claim 1 is characterized in that: The feature fusion network consists of four horizontally connected convolutional layers with a convolution kernel size of 1×1×1 and 256 convolution kernels in each layer. It receives the feature matrices from the sixth, nineteenth, thirty-first, and thirty-eighth addition layers of the 3D-Resnet feature extraction network, respectively, and outputs matrices of sizes 128×128×128×256, 56×56×56×256, 32×32×32×256, and 16×16×16×256, respectively. Each output matrix is ​​downsampled by a factor of 2 and concatenated with the adjacent matrices. Then, feature calculation is performed on each of the concatenated matrices using a convolution kernel of size 3×3×3, number 256, and stride 1, to obtain feature matrix pyramids of 128×128×128×256, 56×56×56×256, 32×32×32×256, and 16×16×16×256.

6. The method for real-time detection of ground subsidence hazards based on a 3D task parallel network according to claim 5 is characterized in that: The architecture in the task collaboration network is composed of a classifier and a localizer with the same structure connected in parallel; For each of the classifier and the locator, it includes a first part, a second part, and a third part; The first part is five sequentially connected convolutional layers with a convolution kernel size of 1×1×1, a number of 128, and a stride of 1; The second part is the maximum pooling layer, which receives the matrix of size 16×16×16×128 output by the first part and outputs a matrix of size 1×1×1×128; The third part is a fully connected layer, which receives the matrix of size 1×1×1×128 output by the second part, has 5 output nodes, and is followed by a random dropout operation to prevent overfitting.

7. The method for real-time detection of ground subsidence hazards based on a 3D task parallel network according to claim 1 is characterized in that: The loss function used in step C is: ， in, represents the distance between the predicted box x of the 3D task parallel network and the center point y of the labeled box in the training set, and c represents the diagonal length of the smallest rectangular box that can simultaneously surround the predicted box x and the labeled box. It represents the ratio between the intersection of the predicted box x of the 3D task parallel network and the labeled box in the training set and the union of the two; In step D, lr=0.001, batch size BatchSize=32, and the number of iterations is 200.

8. A real-time detection system for ground collapse hazards based on a 3D task parallel network, characterized in that: include: The first module is used to obtain the ground penetrating radar dataset; The second module is used to preprocess the ground penetrating radar data set obtained by the first module to obtain a preprocessed ground penetrating radar data set; step (2) is a method for preprocessing the data set obtained in step (1) The method includes the following steps: first, deleting redundant information related to the ground penetrating radar equipment in the ground penetrating radar dataset; then, performing denoising on the ground penetrating radar dataset after the redundant information is deleted using a bilateral filtering method; and performing normalization on the denoised ground penetrating radar dataset using a linear function to obtain a normalized ground penetrating radar dataset; and finally, performing a data augmentation operation on the normalized ground penetrating radar dataset to obtain an amplified ground penetrating radar dataset as a preprocessed ground penetrating radar dataset. The third module is used to input each ground-penetrating radar image in the ground-penetrating radar dataset preprocessed by the second module into the trained 3D task parallel network in sequence to obtain the hidden danger location information of the ground-penetrating radar image; the 3D task parallel network is trained by the following steps: A. Obtain a dataset consisting of ground-penetrating radar images acquired by ground-penetrating radar equipment manufactured by mainstream manufacturers, as well as location information of ground subsidence hazards annotated by professional technicians; B. Preprocess the data set obtained in step A to obtain a preprocessed data set, and randomly divide the preprocessed data set into a training set, a validation set, and a test set; C. Input the training set from the dataset preprocessed in step B into the 3D task parallel network to obtain an inference output of the potential ground collapse location. Input the inference output into the loss function in the 3D task parallel network to obtain a loss value. D. Optimize the loss function in the 3D task-parallel network using the stochastic gradient descent algorithm and the loss value obtained in step C to iteratively update the parameters in the 3D task-parallel network. E. Repeat steps C and D for the remaining data sets in the training set portion of the data set obtained in step B until the 3D task-parallel network converges to the optimal state, thereby obtaining a preliminarily trained 3D task-parallel network. F. Use the validation set in the dataset obtained in step B to validate the preliminarily trained 3D task-parallel network to obtain a validated 3D task-parallel network; G. Use the test set in the data set obtained in step B to test the verified 3D task parallel network to obtain a tested 3D task parallel network; H. Use the coarse-grained channel pruning algorithm to compress the tested 3D task-parallel network to obtain a streamlined 3D task-parallel network as the trained 3D task-parallel network.

Images