Railway tunnel portal safety state detection method, system, device, medium and program

Abstract

The invention relates to a railway tunnel portal safety state detection method, system, device, medium and program, and the method comprises the steps: carrying out the two-dimensional data collection of a railway tunnel portal to obtain a tunnel portal image, and carrying out the three-dimensional data collection of the railway tunnel portal to obtain a tunnel portal point cloud; performing multi-layer target feature detection and two-dimensional target labeling on the tunnel portal image to obtain a two-dimensional detection result; performing point cloud denoising and spatial voting clustering on the tunnel portal point cloud to obtain a primary point cloud class group; performing bounding box detection and three-dimensional target labeling on the tunnel portal point cloud according to the primary point cloud class group to obtain a three-dimensional detection result; and performing detection target alignment and scene modeling on the two-dimensional detection result and the three-dimensional detection result to obtain a standard detection result, and performing safety analysis on the standard detection result to obtain the tunnel portal safety state, thereby improving the efficiency of the tunnel portal safety state.

Description

Technical Field

[0001] The present disclosure relates to the technical field of target detection, and in particular to a method, system, device, medium and program for detecting the safety status of a railway tunnel entrance. Background Art

[0002] Heavy-load railways are usually used to transport large quantities of goods, especially bulk materials such as ore and coal. The weight of the trains and the frequency of operation are very high. Frequent train passages will accelerate the aging and damage of the tunnel entrances. The tunnel entrances are the weak link of heavy-load railways. Once problems occur, they may lead to serious accidents such as landslides and train derailments. Therefore, regular safety inspections of tunnel entrances are required to promptly discover and repair potential hidden dangers.

[0003] Traditional tunnel entrance safety detection technology is a detection method based on manual video surveillance. In actual application, the real-time and accuracy of the detection method based on manual video surveillance are low. The image resolution of traditional monitoring videos is limited and cannot provide sufficiently clear images for accurate safety status detection. In addition, traditional image processing algorithms have poor adaptability to complex scenes such as lighting changes and shadows, making it difficult to accurately detect complex situations at tunnel entrances, which may lead to low efficiency in safety status detection. Summary of the invention

[0004] The present invention provides a method, system, device, medium and program for detecting the safety status of a railway tunnel entrance. The method of spatial voting clustering is used to improve the target detection method of three-dimensional point cloud. At the same time, a two-dimensional target detection method is combined to realize multi-modal target detection of railway tunnel entrances. The efficiency of safety status detection is improved through technical means such as multi-modal fusion, deep learning target detection, three-dimensional point cloud processing and information fusion.

[0005] In a first aspect, the present disclosure provides a method for detecting the safety status of a railway tunnel entrance, comprising: performing two-dimensional data collection on the railway tunnel entrance to obtain a tunnel entrance image, and performing three-dimensional data collection on the railway tunnel entrance to obtain a tunnel entrance point cloud; performing multi-layer target feature detection and two-dimensional target annotation on the tunnel entrance image to obtain a two-dimensional detection result; performing point cloud denoising and spatial voting clustering on the tunnel entrance point cloud to obtain a primary point cloud group; performing bounding box detection and three-dimensional target annotation on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result; performing detection target alignment and scene modeling on the two-dimensional detection result and the three-dimensional detection result to obtain a standard detection result, and performing safety analysis on the standard detection result to obtain the safety status of the tunnel entrance.

[0006] In some embodiments, the performing multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image to obtain a two-dimensional detection result includes: performing multi-scale convolution and downsampling operations on the tunnel entrance image to obtain a multi-scale tunnel entrance feature map group; performing feature aggregation on the multi-scale tunnel entrance feature map group to obtain a fused tunnel entrance feature map group; performing two-dimensional bounding box prediction and content prediction on the fused tunnel entrance feature map group to obtain a primary two-dimensional detection frame group; performing non-maximum suppression on the primary two-dimensional detection frame group to obtain a standard two-dimensional detection frame group; and using the standard two-dimensional detection frame group to perform target labeling on the tunnel entrance image to obtain a two-dimensional detection result.

[0007] In some embodiments, the point cloud denoising and spatial voting clustering of the tunnel entrance point cloud to obtain a primary point cloud group includes: grid segmentation and low point point detection on the tunnel entrance point cloud to obtain an initial ground point set; plane fitting and ground point detection on the initial ground point set to obtain a secondary ground point set; ground point screening of the tunnel entrance point cloud according to the initial ground point set to obtain a primary tunnel entrance point cloud; outlier filtering of the primary tunnel entrance point cloud to obtain a denoised tunnel entrance point cloud; smoothing filtering of the denoised tunnel entrance point cloud to obtain a smoothed tunnel entrance point cloud; and spatial voting clustering of the smoothed tunnel entrance point cloud to obtain a primary point cloud group.

[0008] In some embodiments, the performing spatial voting clustering on the smooth tunnel entrance point cloud to obtain a primary point cloud cluster group includes: dividing the smooth tunnel entrance point cloud into voting areas to obtain a point cloud voting area set; and performing voting clustering on the point cloud voting area set according to the primary three-dimensional target frame group using the following formula to obtain a voting result set:

[0009]

[0010] Among them, G i represents the i-th point cloud voting area in the point cloud voting area set, V(G i ) represents the point cloud voting area G i The voting result of p represents the smooth tunnel entrance point cloud in the point cloud voting area G i The point cloud data point in , w(p) represents the weight of the point cloud data point p, and c represents the point cloud voting area G i The center point, d(p,c) 2 represents the distance between the point cloud data point p and the center point c, and σ represents the scale parameter for controlling Gaussian attenuation; the point cloud voting area set is primary clustered according to the voting result set to obtain a primary point cloud group.

[0011] In some embodiments, the method of performing bounding box detection and three-dimensional target annotation on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result includes: performing point cloud denoising on the tunnel entrance point cloud to obtain a smooth tunnel entrance point cloud; performing farthest point sampling on the primary point cloud group to obtain a sampling point cloud cluster center group; calculating the cluster center distance between each sampling point cloud cluster center in the sampling point cloud cluster center group, and merging adjacent classes of the voting point cloud group according to the cluster center distance to obtain a merged point cloud group; performing multi-scale grouping and spatial feature extraction on the merged point cloud group to obtain a multi-scale feature group set; performing three-dimensional bounding box prediction and content prediction on the smooth tunnel entrance point cloud according to the multi-scale feature group set to obtain a primary three-dimensional detection frame group; performing non-maximum suppression on the primary three-dimensional detection frame group to obtain a standard three-dimensional detection frame group; and using the standard three-dimensional detection frame group to perform target annotation on the smooth tunnel entrance point cloud to obtain a three-dimensional detection result.

[0012] In some embodiments, the two-dimensional detection result and the three-dimensional detection result are subjected to detection target alignment and scene modeling to obtain a standard detection result, including: performing detection frame alignment on the two-dimensional detection result and the three-dimensional detection result to obtain an aligned two-dimensional detection result and an aligned three-dimensional detection result; performing two-dimensional feature extraction on the aligned two-dimensional detection result to obtain two-dimensional target features; performing three-dimensional feature extraction on the aligned three-dimensional detection result to obtain three-dimensional target features; performing object detection on the aligned three-dimensional detection result based on the two-dimensional target features and the three-dimensional target features to obtain a set of identified target objects; and performing scene modeling based on the set of identified target objects to obtain a standard detection result.

[0013] In some embodiments, the safety analysis of the standard detection results to obtain the safety status of the tunnel entrance includes: extracting a set of identified target objects from the standard detection results; performing spatial correlation analysis on the set of identified target objects to obtain object correlation information; performing safety analysis on the standard detection results according to the object correlation information to obtain the safety status of the tunnel entrance.

[0014] In a second aspect, the present disclosure provides a railway tunnel entrance safety status detection system, including: a data acquisition module, used to perform two-dimensional data acquisition on the railway tunnel entrance to obtain a tunnel entrance image, and perform three-dimensional data acquisition on the railway tunnel entrance to obtain a tunnel entrance point cloud; a two-dimensional target detection module, used to perform multi-layer target feature detection and two-dimensional target annotation on the tunnel entrance image to obtain a two-dimensional detection result; a voting clustering module, used to perform point cloud denoising and spatial voting clustering on the tunnel entrance point cloud to obtain a primary point cloud group; a three-dimensional target detection module, used to perform bounding box detection and three-dimensional target annotation on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result; a safety analysis module, used to perform detection target alignment and scene modeling on the two-dimensional detection result and the three-dimensional detection result to obtain a standard detection result, and perform safety analysis on the standard detection result to obtain the tunnel entrance safety status.

[0015] In a third aspect, the present disclosure provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a method for detecting the safety status of a railway tunnel entrance described in the above aspect.

[0016] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for detecting a safety status of a railway tunnel entrance as described in the above aspect.

[0017] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of a method for detecting the safety status of a railway tunnel entrance described in the above aspects.

[0018] The present invention provides a method, system, device, medium and program for detecting the safety status of a railway tunnel entrance. By respectively collecting two-dimensional data and three-dimensional data of the railway tunnel entrance, target detection at the tunnel entrance can be achieved in two dimensions and three dimensions, thereby improving the accuracy of safety detection. By performing multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image, a two-dimensional detection result is obtained. Target detection can be performed on feature maps of different scales, thereby realizing multi-level information extraction of the tunnel entrance image, thereby improving the accuracy of target detection and safety status detection.

[0019] By performing point cloud denoising and spatial voting clustering on the tunnel entrance point cloud, a primary point cloud group is obtained, which can ensure an efficient and accurate point cloud clustering process, thereby improving the accuracy and robustness of object detection at the tunnel entrance. By performing bounding box detection and three-dimensional target annotation on the tunnel entrance point cloud according to the primary point cloud group, target detection can be performed in combination with the multi-scale spatial features of the three-dimensional point cloud data, thereby improving the accuracy of target detection and safety detection.

[0020] By performing detection target alignment and scene modeling on the two-dimensional detection results and the three-dimensional detection results, a standard detection result is obtained, which can realize two-dimensional and three-dimensional target detection and target object recognition, thereby improving the accuracy of target detection. By performing safety analysis on the standard detection results, the safety status of the tunnel entrance is obtained, and through multi-level machine learning analysis, it can be ensured that the system can comprehensively and accurately identify various safety statuses in the tunnel entrance, thereby improving the efficiency of safety status detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:

[0022] Figure 1 A flow chart of a method for detecting a safety status of a railway tunnel entrance according to a first embodiment of the present disclosure is shown.

[0023] Figure 2 A schematic diagram of a process for generating two-dimensional detection results in the first embodiment of the present disclosure is shown.

[0024] Figure 3 A schematic diagram of the process of performing point cloud denoising and spatial voting clustering in the first embodiment of the present disclosure is shown.

[0025] Figure 4 A flow chart of a method for detecting the safety status of a railway tunnel entrance according to a second embodiment of the present disclosure is shown.

[0026] Figure 5 A functional module diagram of a railway tunnel entrance safety status detection system according to a third embodiment of the present disclosure is shown.

[0027] In the drawings, identical components are provided with identical reference numerals, and the drawings are not drawn to scale.

[0028] In the drawings, the same reference numerals are used for the same components, and the drawings are not drawn to scale. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0031] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Embodiment 1

[0033] Figure 1 The following is a flow chart of a method for detecting the safety status of a railway tunnel entrance provided by an embodiment of the present disclosure. Figure 1 As shown, a method for detecting the safety status of a railway tunnel entrance comprises:

[0034] S1. Collect two-dimensional data of a railway tunnel entrance to obtain a tunnel entrance image, and collect three-dimensional data of the railway tunnel entrance to obtain a tunnel entrance point cloud.

[0035] In detail, the railway tunnel entrance refers to the tunnel entrance of a heavy-load railway, the tunnel entrance image is image data obtained by performing two-dimensional data collection on the railway tunnel entrance, and the tunnel entrance point cloud is point cloud data obtained by performing three-dimensional data collection on the railway tunnel entrance.

[0036] Specifically, a camera can be used for two-dimensional data collection. The camera has sufficiently strong imaging capabilities and can clearly capture intruding foreign objects within 70m. The camera can work around the clock and has a certain tolerance to temperature and humidity. When lighting conditions are poor, it has a certain fill light capability to meet night imaging needs.

[0037] In detail, laser radar can be used for three-dimensional data collection. The laser radar is used to perform real-time 3D measurement of the surrounding environment, accurately obtain the position and size information of each object in the environment, and cooperate with image information to achieve precise detection of the outline of obstacles or moving objects.

[0038] In the embodiment of the present invention, by respectively performing two-dimensional data collection and three-dimensional data collection on the railway tunnel entrance, target detection of the tunnel entrance can be achieved from two dimensions and three dimensions, thereby improving the accuracy of safety detection.

[0039] S2. Perform multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image to obtain a two-dimensional detection result.

[0040] In detail, the two-dimensional detection result refers to the tunnel entrance image marked with parameters such as the detection frame, the category of the detected object and the confidence level.

[0041] In the embodiment of the present invention, referring to Figure 2 As shown, the multi-layer target feature detection and two-dimensional target labeling are performed on the tunnel entrance image to obtain a two-dimensional detection result, including:

[0042] S21, performing multi-scale convolution and down-sampling operations on the tunnel entrance image to obtain a multi-scale tunnel entrance feature map group;

[0043] S22, performing feature aggregation on the multi-scale tunnel entrance feature map group to obtain a fused tunnel entrance feature map group;

[0044] S23, performing two-dimensional bounding box prediction and content prediction on the fused tunnel entrance feature map group to obtain a primary two-dimensional detection frame group;

[0045] S24, performing non-maximum suppression on the primary two-dimensional detection frame group to obtain a standard two-dimensional detection frame group;

[0046] S25. Use the standard two-dimensional detection frame group to perform target annotation on the tunnel entrance image to obtain a two-dimensional detection result.

[0047] In detail, the YOLOv4 model trained with a large number of tunnel entrance images with pre-annotated target categories and target bounding boxes can be used to perform multi-layer target feature detection, and multi-scale convolution and downsampling operations can be performed through the convolution layer of the YOLOv4 model and the cross-stage partial residual block (Cross Stage Partial Network, abbreviated) CSP.

[0048] Specifically, the multi-scale tunnel entrance feature map group contains feature maps of the tunnel entrance images of multiple sizes. The feature aggregation of the multi-scale tunnel entrance feature map group to obtain the fused tunnel entrance feature map group includes using a feature pyramid network (Feature Pyramid Network, FPN for short) of a YOLOv4 model to construct a top-down feature pyramid, combining low-resolution high-level features with high-resolution low-level features in the multi-scale tunnel entrance feature map group, using a path aggregation network (Path Aggregation Network, PAN for short) of a YOLOv4 model to construct a bottom-up feature aggregation network, and merging features of different levels through path reconnection to obtain a fused tunnel entrance feature map group.

[0049] In detail, the Head layer of the YOLOv4 model can be used to perform two-dimensional bounding box prediction and content prediction, wherein the two-dimensional bounding box prediction refers to predicting three bounding boxes of different scales at each position of each fused tunnel mouth feature map in the fused tunnel mouth feature map group, and the content prediction refers to predicting the center point position, width and height, object confidence and category probability of each predicted bounding box.

[0050] Specifically, the non-maximum suppression includes screening out bounding boxes whose confidence is lower than a preset threshold, and calculating the overlapping area of ​​the bounding boxes according to the intersection-over-union ratio, retaining the bounding box with the highest confidence, and suppressing other boxes with high overlap to avoid repeated detection of the same object.

[0051] In an embodiment of the present invention, by performing multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image, a two-dimensional detection result is obtained. By performing target detection on feature maps of different scales, multi-level information extraction of the tunnel entrance image can be achieved, thereby improving the accuracy of target detection and safety status detection.

[0052] S3. Performing point cloud denoising and spatial voting clustering on the tunnel entrance point cloud to obtain a primary point cloud cluster group.

[0053] In detail, each primary point cloud cluster in the primary point cloud cluster group is a cluster composed of multiple point clouds in the tunnel entrance point cloud.

[0054] In the embodiment of the present invention, refer to Figure 3As shown, the tunnel entrance point cloud is subjected to point cloud denoising and spatial voting clustering to obtain a primary point cloud group, including:

[0055] S31, performing grid segmentation and low point detection on the tunnel entrance point cloud to obtain an initial ground point set;

[0056] S32, performing plane fitting and ground point detection on the initial ground point set to obtain a secondary ground point set;

[0057] S33, filtering out ground points from the tunnel entrance point cloud according to the initial ground point set to obtain a primary tunnel entrance point cloud;

[0058] S34, performing outlier filtering on the primary tunnel entrance point cloud to obtain a denoised tunnel entrance point cloud;

[0059] S35, performing smoothing filtering on the denoised tunnel entrance point cloud to obtain a smoothed tunnel entrance point cloud;

[0060] S36, performing spatial voting clustering on the smooth tunnel entrance point cloud to obtain a primary point cloud cluster group.

[0061] In detail, the grid segmentation refers to dividing the point cloud into grids of equal size, and the low point detection refers to finding the lowest point in each grid and treating it as the initial ground point. The random consistency algorithm can be used for plane fitting, and points with smaller height differences are used as secondary ground points.

[0062] Specifically, outlier filtering may be performed using statistical filtering or conditional filtering, and smoothing filtering may be performed using average filtering or bilateral filtering.

[0063] In detail, the spatial voting clustering of the smooth tunnel entrance point cloud to obtain a primary point cloud group includes: dividing the smooth tunnel entrance point cloud into voting areas to obtain a point cloud voting area set; and voting clustering the point cloud voting area set according to the primary three-dimensional target frame group using the following formula to obtain a voting result set:

[0064]

[0065] Among them, G i represents the i-th point cloud voting area in the point cloud voting area set, V(G i ) represents the point cloud voting area G i The voting result of p represents the smooth tunnel entrance point cloud in the point cloud voting area G i The point cloud data point in , w(p) represents the weight of the point cloud data point p, and c represents the point cloud voting area G i The center point, d(p,c) 2represents the distance between the point cloud data point p and the center point c, and σ represents the scale parameter for controlling Gaussian attenuation; the point cloud voting area set is primary clustered according to the voting result set to obtain a primary point cloud group.

[0066] Specifically, the voting area division refers to dividing the smooth tunnel entrance point cloud into a series of equally spaced voting areas, each voting area contains a portion of data points in the smooth tunnel entrance point cloud, and the primary clustering refers to determining the density of each point cloud voting area in the point cloud voting area set according to the voting result set, and performing primary clustering according to the density.

[0067] In an embodiment of the present invention, by performing point cloud denoising and spatial voting clustering on the tunnel entrance point cloud, a primary point cloud group is obtained, which can ensure an efficient and accurate point cloud clustering process, thereby improving the accuracy and robustness of tunnel entrance object detection.

[0068] S4. Perform bounding box detection and three-dimensional object annotation on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result.

[0069] In detail, the three-dimensional detection result refers to the tunnel entrance point cloud marked with parameters such as the detection frame, detection object category and confidence level.

[0070] In an embodiment of the present invention, the method of performing bounding box detection and three-dimensional target labeling on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result includes: performing point cloud denoising on the tunnel entrance point cloud to obtain a smooth tunnel entrance point cloud; performing farthest point sampling on the primary point cloud group to obtain a sampling point cloud cluster center group; calculating the cluster center distance between each sampling point cloud cluster center in the sampling point cloud cluster center group, and merging adjacent classes of the voting point cloud group according to the cluster center distance to obtain a merged point cloud group; performing multi-scale grouping and spatial feature extraction on the merged point cloud group to obtain a multi-scale feature group set; performing three-dimensional bounding box prediction and content prediction on the smooth tunnel entrance point cloud according to the multi-scale feature group set to obtain a primary three-dimensional detection frame group; performing non-maximum suppression on the primary three-dimensional detection frame group to obtain a standard three-dimensional detection frame group; and using the standard three-dimensional detection frame group to label the smooth tunnel entrance point cloud to obtain a three-dimensional detection result.

[0071] In detail, the farthest point sampling refers to selecting the point farthest from the edge of the cluster of each voting point cloud class in the voting point cloud class group as the sampling point cloud cluster center to form a sampling point cloud cluster center group. The cluster center distance can be calculated using the Euclidean distance algorithm. The adjacent class merging refers to determining whether the cluster center distance is lower than a preset threshold. If so, the distribution or merging of the cluster is readjusted.

[0072] Specifically, the multi-scale grouping refers to dividing the point cloud data into different scale groups according to the Euclidean distance to obtain multiple hierarchical clustering clusters. PointNet can be used for spatial feature extraction. PointNet can automatically learn the spatial features of the point cloud and perform efficient three-dimensional feature representation. The fusion clustering refers to feature fusion of the multi-scale feature group set and clustering using the KNN clustering algorithm.

[0073] In detail, a pre-trained object detection model may be used to perform 3D bounding box prediction and content prediction. The object detection model includes a predicted 3D bounding box and a real 3D bounding box during training, and the following intersection-over-union loss algorithm is used for training:

[0074]

[0075] Among them, DIOU represents the intersection-over-union loss value, IOU represents the overlap ratio between the predicted 3D bounding box and the real 3D bounding box, α represents the midpoint distance weight of the bounding box, D represents the 3D distance between the predicted 3D bounding box and the real bounding box, r represents the diagonal length of the minimum enclosing cube, β represents the bounding box volume difference weight, V1 represents the volume of the predicted 3D bounding box, and V2 represents the volume of the real 3D bounding box.

[0076] In an embodiment of the present invention, by performing bounding box detection and three-dimensional target annotation on the tunnel entrance point cloud according to the primary point cloud class group, target detection can be performed in combination with multi-scale spatial features of the three-dimensional point cloud data, thereby improving the accuracy of target detection and safety detection.

[0077] S5. Perform detection target alignment and scene modeling on the two-dimensional detection result and the three-dimensional detection result to obtain a standard detection result, and perform safety analysis on the standard detection result to obtain a safety status of the tunnel entrance.

[0078] In detail, the two-dimensional detection result and the three-dimensional detection result are subjected to detection target alignment and scene modeling to obtain a standard detection result, including: performing detection frame alignment on the two-dimensional detection result and the three-dimensional detection result to obtain an aligned two-dimensional detection result and an aligned three-dimensional detection result; performing two-dimensional feature extraction on the aligned two-dimensional detection result to obtain two-dimensional target features; performing three-dimensional feature extraction on the aligned three-dimensional detection result to obtain three-dimensional target features; performing object detection on the aligned three-dimensional detection result based on the two-dimensional target features and the three-dimensional target features to obtain a set of identified target objects; and performing scene modeling based on the set of identified target objects to obtain a standard detection result.

[0079] Specifically, the detection frame alignment refers to associating the detection frame in the two-dimensional detection result with the detection frame in the three-dimensional detection result by matching spatial position and size information, so as to ensure that the detection results in different dimensions point to the same object.

[0080] In detail, two-dimensional feature extraction can be performed through a convolutional neural network, and three-dimensional feature extraction can be performed through a PointNet point cloud processing algorithm. The object detection refers to the feature fusion of two-dimensional target features and three-dimensional target features by weighted summation or concatenation, and object recognition is performed on the target features after feature fusion. Object detection can be performed by combining the depth information of the three-dimensional point cloud and the texture information of the two-dimensional image through a feature point matching algorithm to achieve higher-precision object recognition. The object recognition includes recognizing various objects and obstacles in the tunnel entrance.

[0081] Specifically, the safety analysis of the standard detection results to obtain the safety status of the tunnel entrance includes: extracting a set of identified target objects from the standard detection results; performing spatial correlation analysis on the set of identified target objects to obtain object correlation information; and performing safety analysis on the standard detection results based on the object correlation information to obtain the safety status of the tunnel entrance.

[0082] In detail, the spatial correlation analysis refers to the understanding of the entire scene by calculating and identifying the relative position, distance, direction and other information between objects in the target object set, judging the relationship between objects, such as the relationship between vehicles and roads, and the relationship between obstacles and safe passage.

[0083] Specifically, spatial correlation analysis can be performed through a global attention model, a convolutional neural network or a graph neural network, wherein the global attention model is used to capture the relationship between key areas and objects in the scene, thereby helping to understand the entire scene and judge the safety status of the tunnel entrance, and is suitable for situations where global information and complex spatial relationships need to be considered.

[0084] In detail, the convolutional neural network or the scene analysis model based on the graph neural network models the spatial relationship between objects through the graph structure, which is helpful for judging the state of the tunnel entrance. For some specific scenarios, a rule-based analysis method can also be used to analyze the position and relationship of objects in the tunnel entrance through predefined logical and statistical features.

[0085] Specifically, the safety analysis refers to identifying whether the object association information contains object association information that poses a threat. The safety analysis can be performed using one or more of a support vector machine, a decision tree, and a neural network algorithm. For example, it is determined whether the object association information contains association information between a railway route and an obstacle. If so, the safety status of the tunnel entrance is determined to be an unsafe status.

[0086] In an embodiment of the present invention, by performing detection target alignment and scene modeling on the two-dimensional detection results and the three-dimensional detection results, a standard detection result is obtained, which can realize two-dimensional and three-dimensional target detection and target object recognition, thereby improving the accuracy of target detection. By performing safety analysis on the standard detection results, the safety status of the tunnel entrance is obtained, and through multi-level machine learning analysis, it can be ensured that the system can comprehensively and accurately identify various safety statuses in the tunnel entrance.

[0087] The present invention provides a method for detecting the safety status of a railway tunnel entrance. By respectively collecting two-dimensional data and three-dimensional data of the railway tunnel entrance, target detection of the tunnel entrance can be achieved from two dimensions and three dimensions, thereby improving the accuracy of safety detection. By performing multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image, a two-dimensional detection result is obtained. Target detection can be performed on feature maps of different scales, thereby realizing multi-level information extraction of the tunnel entrance image, thereby improving the accuracy of target detection and safety status detection.

[0088] By performing point cloud denoising and spatial voting clustering on the tunnel entrance point cloud, a primary point cloud group is obtained, which can ensure an efficient and accurate point cloud clustering process, thereby improving the accuracy and robustness of object detection at the tunnel entrance. By performing bounding box detection and three-dimensional target annotation on the tunnel entrance point cloud according to the primary point cloud group, target detection can be performed in combination with the multi-scale spatial features of the three-dimensional point cloud data, thereby improving the accuracy of target detection and safety detection.

[0089] By performing detection target alignment and scene modeling on the two-dimensional detection results and the three-dimensional detection results, a standard detection result is obtained, which can realize two-dimensional and three-dimensional target detection and target object recognition, thereby improving the accuracy of target detection. By performing safety analysis on the standard detection results, the safety status of the tunnel entrance is obtained, and through multi-level machine learning analysis, it can be ensured that the system can comprehensively and accurately identify various safety statuses in the tunnel entrance, thereby improving the efficiency of safety status detection.

[0090] Embodiment 2

[0091] On the basis of the above embodiments, in order to more clearly understand the present disclosure, refer to Figure 4 As shown, the following further explains the situation in which a faster security status detection result is required in the first embodiment of the present disclosure through a second embodiment.

[0092] Figure 4 The following is a flow chart of a method for detecting the safety status of a railway tunnel entrance provided by an embodiment of the present disclosure. Figure 4As shown, a method for detecting the safety status of a railway tunnel entrance comprises:

[0093] S41, collecting two-dimensional data of the railway tunnel entrance to obtain a tunnel entrance image, and collecting three-dimensional data of the railway tunnel entrance to obtain a tunnel entrance point cloud;

[0094] S42, performing multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image to obtain a two-dimensional detection result;

[0095] S43, performing point cloud denoising and spatial density clustering on the tunnel entrance point cloud to obtain a primary point cloud cluster group;

[0096] Specifically, the Spatial Density Based Clustering is a density-based clustering method, which is mainly used to divide data into different clusters according to density distribution characteristics. The core idea is that in a data set, a region with higher density is regarded as a cluster, and a region with lower density is regarded as the boundary between clusters. This method is very suitable for data sets with non-uniform distribution and complex shapes, especially data containing noise.

[0097] S44, performing bounding box detection and three-dimensional object labeling on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result;

[0098] S45, performing detection target alignment and scene modeling on the two-dimensional detection result and the three-dimensional detection result to obtain a standard detection result, and performing safety analysis on the standard detection result to obtain a safety status of the tunnel entrance.

[0099] Embodiment 3

[0100] Based on the above embodiments, Figure 5 This is a functional module diagram of a railway tunnel entrance safety status detection system provided by an embodiment of the present disclosure. Figure 5 As shown, a railway tunnel entrance safety status detection system comprises:

[0101] The railway tunnel entrance safety state detection system 500 described in this embodiment can be installed in an electronic device. According to the functions implemented, the railway tunnel entrance safety state detection system 500 may include a data acquisition module 501, a two-dimensional target detection module 502, a voting clustering module 503, a three-dimensional target detection module 504 and a safety analysis module 505. The module described in the present disclosure may also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0102] In this embodiment, the functions of each module / unit are as follows:

[0103] The data acquisition module 501 is used to perform two-dimensional data acquisition on the railway tunnel entrance to obtain a tunnel entrance image, and perform three-dimensional data acquisition on the railway tunnel entrance to obtain a tunnel entrance point cloud;

[0104] The two-dimensional target detection module 502 is used to perform multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image to obtain a two-dimensional detection result;

[0105] The voting clustering module 503 is used to perform point cloud denoising and spatial voting clustering on the tunnel entrance point cloud to obtain a primary point cloud cluster group;

[0106] The three-dimensional object detection module 504 is used to perform bounding box detection and three-dimensional object labeling on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result;

[0107] The safety analysis module 505 is used to perform detection target alignment and scene modeling on the two-dimensional detection results and the three-dimensional detection results to obtain standard detection results, and perform safety analysis on the standard detection results to obtain the safety status of the tunnel entrance.

[0108] In detail, each module described in the railway tunnel entrance safety status detection system 500 described in the embodiment of the present disclosure adopts the same technical means as the railway tunnel entrance safety status detection method described in Example 1 when passing through, and can produce the same technical effects, which will not be repeated here.

[0109] Embodiment 4

[0110] On the basis of the above embodiments, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above embodiments.

[0111] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0112] In some implementations of this embodiment, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0113] The processor may include, but is not limited to, one or more processors or microprocessors, etc. Each processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components to execute the methods in the above embodiments.

[0114] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the computer-readable storage medium may include but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD ROM, DVD ROM, Blu-ray disc, etc.).

[0115] The computer-readable storage medium may also store at least one computer executable program, such as a computer-readable instruction. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The computer-readable storage medium may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0116] In addition, the computer device may also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.

[0117] The processor may communicate with external devices via an I / O bus via a wired or wireless network.

[0118] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0119] In the embodiments provided in the present disclosure, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0120] It should be noted that in the present disclosure, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0121] Although the embodiments disclosed in the present disclosure are as above, the above contents are only embodiments adopted for facilitating the understanding of the present disclosure and are not intended to limit the present disclosure. Any technician in the technical field to which the present disclosure belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present disclosure, but the scope of patent protection of the present disclosure shall still be subject to the scope defined in the attached claims.

Claims

1. A method for detecting the safety status of a railway tunnel entrance, characterized in that: include: Performing two-dimensional data collection on the railway tunnel entrance to obtain a tunnel entrance image, and performing three-dimensional data collection on the railway tunnel entrance to obtain a tunnel entrance point cloud; Performing multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image to obtain a two-dimensional detection result; Performing point cloud denoising and spatial voting clustering on the tunnel entrance point cloud to obtain a primary point cloud group; Performing bounding box detection and three-dimensional target annotation on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result; The two-dimensional detection result and the three-dimensional detection result are subjected to detection target alignment and scene modeling to obtain a standard detection result, and a safety analysis is performed on the standard detection result to obtain a safety status of the tunnel entrance.

2. The railway tunnel entrance safety status detection method according to claim 1, characterized in that: The performing multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image to obtain a two-dimensional detection result includes: Performing multi-scale convolution and down-sampling operations on the tunnel entrance image to obtain a multi-scale tunnel entrance feature map group; Performing feature aggregation on the multi-scale tunnel entrance feature map group to obtain a fused tunnel entrance feature map group; Performing two-dimensional bounding box prediction and content prediction on the fused tunnel entrance feature map group to obtain a primary two-dimensional detection frame group; Performing non-maximum suppression on the primary two-dimensional detection frame group to obtain a standard two-dimensional detection frame group; The standard two-dimensional detection frame group is used to perform target annotation on the tunnel entrance image to obtain a two-dimensional detection result.

3. The railway tunnel entrance safety status detection method according to claim 1, characterized in that: The step of performing point cloud denoising and spatial voting clustering on the tunnel entrance point cloud to obtain a primary point cloud group includes: Performing grid segmentation and low point detection on the tunnel entrance point cloud to obtain an initial ground point set; Performing plane fitting and ground point detection on the initial ground point set to obtain a secondary ground point set; Screening the tunnel entrance point cloud for ground points according to the initial ground point set to obtain a primary tunnel entrance point cloud; Performing outlier filtering on the primary tunnel entrance point cloud to obtain a denoised tunnel entrance point cloud; Performing smoothing filtering on the denoised tunnel entrance point cloud to obtain a smoothed tunnel entrance point cloud; The smooth tunnel entrance point cloud is subjected to spatial voting clustering to obtain a primary point cloud cluster group.

4. The railway tunnel entrance safety status detection method according to claim 3, characterized in that: The performing spatial voting clustering on the smooth tunnel entrance point cloud to obtain a primary point cloud cluster group includes: Dividing the smooth tunnel entrance point cloud into voting areas to obtain a point cloud voting area set; The voting result set is obtained by performing voting clustering on the point cloud voting area set according to the primary three-dimensional target frame group using the following formula: Among them, G i represents the i-th point cloud voting area in the point cloud voting area set, V(G i ) represents the point cloud voting area G i The voting result of p represents the smooth tunnel entrance point cloud in the point cloud voting area G i The point cloud data point in , w(p) represents the weight of the point cloud data point p, and c represents the point cloud voting area G i The center point, d(p,c) 2 represents the distance between the point cloud data point p and the center point c, and σ represents the scale parameter that controls the Gaussian attenuation; The point cloud voting area set is subjected to primary clustering according to the voting result set to obtain a primary point cloud cluster group.

5. The railway tunnel entrance safety status detection method according to claim 1, characterized in that: The step of performing bounding box detection and three-dimensional target labeling on the tunnel entrance point cloud according to the primary point cloud group to obtain a three-dimensional detection result includes: Performing point cloud denoising on the tunnel entrance point cloud to obtain a smooth tunnel entrance point cloud; Performing farthest point sampling on the primary point cloud cluster group to obtain a sampled point cloud cluster center group; Calculating the cluster center distance between each sampling point cloud cluster center in the sampling point cloud cluster center group, and merging adjacent classes of the voting point cloud cluster group according to the cluster center distance to obtain a merged point cloud cluster group; Performing multi-scale grouping and spatial feature extraction on the merged point cloud cluster group to obtain a multi-scale feature group set; Performing 3D bounding box prediction and content prediction on the smooth tunnel entrance point cloud according to the multi-scale feature set to obtain a primary 3D detection frame set; Performing non-maximum suppression on the primary three-dimensional detection frame group to obtain a standard three-dimensional detection frame group; The standard three-dimensional detection frame group is used to perform target annotation on the smooth tunnel entrance point cloud to obtain a three-dimensional detection result.

6. The railway tunnel entrance safety status detection method according to claim 1, characterized in that: The performing detection target alignment and scene modeling on the two-dimensional detection result and the three-dimensional detection result to obtain a standard detection result includes: Performing detection frame alignment on the two-dimensional detection result and the three-dimensional detection result to obtain an aligned two-dimensional detection result and an aligned three-dimensional detection result; Performing two-dimensional feature extraction on the aligned two-dimensional detection result to obtain two-dimensional target features; Performing three-dimensional feature extraction on the aligned three-dimensional detection result to obtain three-dimensional target features; Performing object detection on the aligned three-dimensional detection results according to the two-dimensional target features and the three-dimensional target features to obtain a set of identified target objects; Scene modeling is performed based on the identified target object set to obtain standard detection results.

7. A railway tunnel entrance safety status detection system, characterized in that: include: A data acquisition module, used to acquire two-dimensional data of the railway tunnel entrance to obtain a tunnel entrance image, and to acquire three-dimensional data of the railway tunnel entrance to obtain a tunnel entrance point cloud; A two-dimensional target detection module is used to perform multi-layer target feature detection and two-dimensional target labeling on the tunnel entrance image to obtain a two-dimensional detection result; A voting clustering module, used for performing point cloud denoising and spatial voting clustering on the tunnel entrance point cloud to obtain a primary point cloud cluster group; A three-dimensional target detection module, used to perform bounding box detection and three-dimensional target annotation on the tunnel entrance point cloud according to the primary point cloud group, to obtain a three-dimensional detection result; The safety analysis module is used to perform detection target alignment and scene modeling on the two-dimensional detection results and the three-dimensional detection results to obtain standard detection results, and to perform safety analysis on the standard detection results to obtain the safety status of the tunnel entrance.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the railway tunnel entrance safety status detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting the safety status of a railway tunnel entrance described in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for detecting the safety status of a railway tunnel entrance described in any one of claims 1 to 6 are implemented.

Images