A tunnel holographic perception method and system of a lightweight multi-scale attention mechanism

Abstract

The present application relates to the technical field of image processing, in particular to a tunnel holographic perception method and system of a lightweight multi-scale attention mechanism, the method comprising: identifying suspected crack areas in each segmentation range of the tunnel based on holographic perception data of the tunnel; determining the consistency degree of the trend between the current suspected crack branch and other suspected crack branches in the segmentation range; determining the crack matching degree of each pixel point in the suspected crack branch, the target suspected crack branch being a suspected crack branch in an adjacent segmentation range that is tangent to the current segmentation range and has adjacent pixel points with the suspected crack branch in the current segmentation range; and determining the crack area range of the tunnel based on the crack matching degree of each pixel point. In this way, the present application improves the reliability and accuracy of the identification result of the tunnel cracks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a tunnel holographic perception method and system based on a lightweight multi-scale attention mechanism. BACKGROUND

[0002] As a component of transportation infrastructure, the structural safety of a tunnel is directly related to the efficiency of traffic. In the long-term service process, the lining structure of the tunnel is prone to cracks and other diseases due to the influence of multiple factors such as geological subsidence, vehicle load, temperature change and environmental erosion. If these cracks are not found and repaired in time, they will gradually expand to cause a decrease in the load-bearing capacity of the structure, and even cause serious problems such as lining peeling and water leakage, which threaten the safe operation of the tunnel. Therefore, the accurate and efficient detection of cracks in the tunnel is the core link to ensure the health and long-term stable operation of the tunnel structure. However, the detection environment inside the tunnel is complex, which brings many challenges to crack identification. On the one hand, the lighting conditions in the tunnel are uneven, and there are local areas with alternating light and dark, which easily leads to unclear crack features in the image. On the other hand, the lining surface is often attached with water stains, stains, construction residual marks and other interference objects, the static forms of which are highly similar to the fine cracks, and it is difficult to distinguish them directly by vision. At the same time, some fine cracks are only millimeters wide, which are represented as weak gray changes in the image and are easily covered by background information, further increasing the detection difficulty. Under this background, the traditional tunnel crack detection methods gradually reveal obvious limitations and cannot meet the detection requirements of high precision and high reliability.

[0003] In some scenarios, machine vision technology based on a single static form feature is often used to detect cracks in tunnels. Traditional methods based on machine vision identify cracks only based on the static form of the crack area (such as length, width, gray gradient, etc.), lack multi-dimensional analysis of features and key attention to key areas. For example, when such methods face water stains, oil stains or surface scratches that are very similar to the static form of cracks, they cannot effectively distinguish the essential differences between them. Although water stains and other interference areas are similar to cracks in terms of gray distribution and contour form, they do not have the continuity, expansibility and depth characteristics unique to cracks. Traditional methods lack the ability to finely screen and distinguish features, and are prone to misjudgment of such interference areas as cracks, resulting in low reliability and accuracy of the identification results of tunnel cracks. SUMMARY

[0004] In order to solve the technical problem of low reliability and accuracy of the identification results of tunnel cracks, the purpose of the present application is to provide a tunnel holographic perception method and system based on a lightweight multi-scale attention mechanism.

[0005] To solve the above technical problems, the technical solution adopted is as follows:

[0006] In a first aspect, an embodiment of the present application provides a tunnel holographic perception method based on a lightweight multi-scale attention mechanism, comprising: identifying suspected crack regions in each segmentation range of a tunnel based on holographic perception data of the tunnel; determining a trend consistency degree between a current suspected crack branch and other suspected crack branches in the segmentation range according to adjacent pixel points of the suspected crack branch; determining a crack matching degree of each pixel point in the suspected crack branch according to a target suspected crack branch, the trend consistency degree, a distance between adjacent light-dark boundaries of the suspected crack branch in the segmentation range, pixel points of a crack boundary in the segmentation range, and all pixel points in the segmentation range, the target suspected crack branch being a suspected crack branch in an adjacent segmentation range that is tangent to the current segmentation range and has adjacent pixel points with the suspected crack branch in the current segmentation range; and determining a crack region range of the tunnel based on the crack matching degrees of the pixel points.

[0007] Optionally, determining the trend consistency degree between the current suspected crack branch and the other suspected crack branches in the segmentation range according to the adjacent pixel points of the suspected crack branch in the suspected crack region comprises: skeletonizing the suspected crack region in each segmentation range to obtain a skeleton image of the suspected crack region; determining the suspected crack branch from the skeleton image according to pixel points in the skeleton image; determining an extension direction of the suspected crack branch based on a pointing direction and a unit length of the adjacent pixel points in the suspected crack branch; and determining the trend consistency degree between each suspected crack branch and the other suspected crack branches in each segmentation range according to an included angle between the extension directions of the suspected crack branches.

[0008] Optionally, determining the trend consistency degree between each suspected crack branch and the other suspected crack branches in each segmentation range according to the included angle between the extension directions of the suspected crack branches comprises: determining the included angle between the extension directions of the current suspected crack branch and the other suspected crack branches in the same segmentation range; counting a first number of the other suspected crack branches whose included angles are greater than a first threshold value, and obtaining a minimum value in the included angles; and determining the trend consistency degree between each suspected crack branch and the other suspected crack branches in each segmentation range according to a second number of all suspected crack branches in the same segmentation range, the first number, and the minimum value in the included angles.

[0009] Optionally, the crack matching degree of each pixel in the suspected crack branch is determined based on the target suspected crack branch, the consistency of its trend, the distance between adjacent light and dark boundaries of the suspected crack branch in the segmentation range, the pixels at the crack boundary in the segmentation range, and all pixels in the segmentation range. This includes: determining the splicing fit between the suspected crack branch in the current segmentation range and the target suspected crack branch based on the angle between the extension directions of the suspected crack branch in the current segmentation range and the target suspected crack branch, the third number of adjacent pixels, and the fourth number of pixels of the suspected crack branch in the current segmentation range on the boundary of the current segmentation range; and based on the splicing fit... The degree of consistency with the trend is used to determine the static division performance of the suspected crack branches within the segmentation range; the crack conformity of the suspected crack branches within the segmentation range is determined based on the distance between adjacent light and dark boundaries of the suspected crack branches within the segmentation range and the static division performance of the suspected crack branches; the crack conformity, the third number of pixels at the crack boundary within the segmentation range, and the fourth number of all pixels within the segmentation range are used to determine the crack belonging degree of each pixel in the suspected crack branches within the segmentation range; and the crack matching degree of each pixel in the suspected crack branches is determined based on the crack belonging degree of each pixel in the suspected crack branches in all segmentation ranges containing it.

[0010] Optionally, determining the static division performance of the suspected crack branches within the segmentation range based on the splicing fit degree and the trend consistency degree includes: counting the fifth number of adjacent segmentation ranges tangent to the suspected crack branch within the current segmentation range whose splicing fit degree with the target suspected crack branch is greater than the second threshold, and the sixth number of all adjacent segmentation ranges tangent to the current segmentation range within the current segmentation range; and determining the static division performance of the suspected crack branches within the segmentation range based on the fifth number, the sixth number, and the trend consistency degree.

[0011] Optionally, the crack conformity of the suspected crack branches in the segmentation range is determined based on the distance between adjacent light and dark boundaries of the suspected crack branches and the static division performance of the suspected crack branches. This includes: determining the smoothness of light and dark changes of the suspected crack branches in the segmentation range under different angles based on the distance between adjacent light and dark boundaries of the suspected crack branches in the segmentation range; obtaining the maximum value in the static division performance of each suspected crack branch in the segmentation range; and determining the crack conformity of the suspected crack branches in the segmentation range based on the static division performance of each suspected crack branch in the segmentation range, the maximum value in the static division performance of each suspected crack branch, and the smoothness of light and dark changes.

[0012] Optionally, based on the crack conformity, the third number of pixels at the crack boundary in the segmentation range, and the fourth number of all pixels in the segmentation range, the crack belonging degree of each pixel in the suspected crack branch in the segmentation range is determined by: determining the crack belonging degree of each pixel in the suspected crack branch in the segmentation range according to the ratio between the third number and the fourth number and the crack conformity.

[0013] Optionally, determining the crack region range of the tunnel based on the crack matching degree of each pixel includes: normalizing the crack matching degree to obtain a normalized matching degree; classifying pixels with a normalized matching degree greater than a third threshold into the crack region; merging adjacent pixels of pixels in the crack region to obtain the crack region range of the tunnel.

[0014] Optionally, identifying suspected crack regions within each segmentation range of the tunnel based on holographic perception data includes: inputting the holographic perception data of the tunnel into a trained network model for crack region identification, thereby obtaining suspected crack regions of the tunnel. The trained network model is obtained by training on historical point cloud data and historical video stream data acquired from high-definition cameras and LiDAR deployed in the tunnel; performing cylindrical unfolding on the point cloud data from the holographic perception data of the tunnel surface, mapping the points on the three-dimensional surface of the tunnel onto a two-dimensional plane to obtain a rectangular image; sliding windows of different sizes across the rectangular image with a certain step size to extract image blocks; matching the image blocks with the point cloud data of the tunnel surface before unfolding, and using the corresponding point cloud range as the segmentation range; and obtaining all suspected crack regions within the segmentation range.

[0015] In a second aspect, embodiments of the present invention provide a lightweight multi-scale attention mechanism tunneling holographic perception system, comprising: a processor and a memory; wherein the memory is used to store a computer program that can run on the processor; the processor is used to execute the program stored in the memory to implement the steps of the lightweight multi-scale attention mechanism tunneling holographic perception method mentioned in the first aspect.

[0016] This invention offers the following advantages: By employing a dual verification of the consistency of the trend between the current suspected crack branch and other suspected crack branches within the segmented range, along with the crack matching degree, it achieves refined screening of suspected cracks. By analyzing adjacent pixels of suspected crack branches, the consistency of the trend between different suspected crack branches can be determined, quickly eliminating interference areas that do not conform to the crack trend pattern (such as messy textures of water stains or dirt). Furthermore, by combining multi-dimensional parameters such as the target suspected crack branch (related cracks in adjacent segmented ranges), the distance between light and dark boundaries, and the percentage of pixels at the crack boundary, the crack matching degree can be calculated. This further distinguishes between interference areas that are similar in shape but fundamentally different from real cracks, accurately eliminating such misjudgments and improving the accuracy of crack identification to the pixel level, significantly reducing the false negative and false positive rates. When determining the crack matching degree, the parameter of the distance between the light and dark boundaries of adjacent suspected crack branches within the segmented range can effectively offset the interference caused by uneven illumination and avoid feature distortion caused by changes in illumination. Furthermore, the correlation analysis between the suspected crack branch and other suspected crack branches within the current segmentation range establishes a crack spatial continuity verification mechanism across segmentation ranges. Even if there is a large area of ​​stain covering the lining surface, as long as there are identifiable crack features in adjacent tangent segmentation ranges, the crack segments obscured by stains can be connected by calculating the consistency of the crack trend and the crack matching degree. This overcomes the scenario limitation of misjudging crack fractures due to local stain obscuration and enables continuous crack identification in complex environments. This improves the reliability and accuracy of tunnel crack identification results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a lightweight multi-scale attention mechanism tunneling holographic perception method provided in one embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a three-dimensional image and an unfolded image of the tunnel interior provided in one embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of a light-dark boundary line provided in one embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of the structure of a lightweight multi-scale attention mechanism tunneling holographic perception system provided in one embodiment of the present invention. Detailed Implementation

[0022] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a lightweight multi-scale attention mechanism tunneling holographic perception method and system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] This invention addresses the scenario where a multi-scale attention mechanism significantly improves the accuracy of identifying minute cracks in tunnels by capturing crack features at different scales in parallel. It enhances the response of key area features, suppresses background interference, strengthens the model's robustness under complex lighting and contamination conditions, effectively avoids missing minute cracks, and greatly improves detection reliability. This invention analyzes the consistency of suspected crack regions across different segmentation ranges based on the trend of these regions and the connectivity of suspected crack regions within tangent segmentation ranges. It eliminates interference from crack identification in tunnel walls by analyzing the changes in light and shadow of suspected crack regions under different camera perspectives. Finally, it uses the crack behavior at a single location across segmentation ranges at different scales as weights to determine the actual crack consistency of suspected crack regions existing across multiple segmentation ranges.

[0025] The following description, in conjunction with the accompanying drawings, details the specific scheme of the lightweight multi-scale attention mechanism tunneling holographic perception method provided by this invention.

[0026] Example 1:

[0027] Please see Figure 1 The flowchart illustrates a lightweight multi-scale attention mechanism tunneling holographic perception method according to an embodiment of the present invention, including:

[0028] Step S101: Identify suspected crack areas within each segmented range of the tunnel based on the holographic perception data of the tunnel.

[0029] Specifically, the holographic sensing data in this embodiment of the invention includes point cloud data and video data within the tunnel. The specific process for acquiring holographic sensing data is as follows: Lidar and high-definition cameras are installed at regular intervals at the top or high on the sidewalls of the tunnel. The Lidar and cameras are securely mounted using seismic-resistant brackets. Harsh environments such as vibration, humidity, and dust within the tunnel must be considered, and equipment meeting high protection levels such as IP67 must be selected. Then, a stable power supply network (using Power over Ethernet (PoE) or centralized power supply via industrial switches) and a high-speed communication network (Gigabit Ethernet or fiber optic) are deployed to ensure that massive amounts of data can be transmitted to edge computing nodes or a central server in real time without delay. All Lidar and cameras are connected to the same time source (such as a GPS clock based on satellite navigation system signals), and each frame of data is timestamped with millisecond-level precision to ensure time synchronization of the acquired data.

[0030] Furthermore, the lidar emits laser beams at a frequency of 10-20 frames per second, receives the returned signals, and generates point cloud data. Each frame of point cloud data contains the three-dimensional coordinates (X, Y, Z) and reflection intensity information of millions of points. High-definition cameras acquire video stream data (RGB images) at a frequency of 25-30 frames per second (or higher). Through a deployed high-speed network, the raw point cloud data and video stream data are transmitted in real time to a central cloud platform for processing.

[0031] Furthermore, in this embodiment of the invention, after preprocessing the point cloud data and video stream data, the pixel information (such as color and texture) of the camera image is mapped to each point of the LiDAR point cloud using the multi-sensor fusion (MSF) method, and each 3D point is assigned an RGB color value to generate a true color point cloud.

[0032] Furthermore, the crack regions are manually and precisely labeled in the true-color point cloud, with each point marked as either "crack" or "background". The training dataset is then manually expanded by rotating, translating, scaling, and adjusting color and brightness of the point cloud to improve the model's robustness. These operations prepare and enhance the data.

[0033] Furthermore, as an optional embodiment of the present invention, identifying suspected crack regions within each segmentation range of a tunnel based on holographic perception data of the tunnel includes: inputting the holographic perception data of the tunnel into a trained network model for crack region identification to obtain suspected crack regions of the tunnel; the trained network model is obtained by training historical point cloud data and historical video stream data of the tunnel obtained by high-definition cameras and lidar deployed in the tunnel; performing cylindrical unfolding on the point cloud data in the holographic perception data of the tunnel surface to map the points on the three-dimensional surface of the tunnel onto a two-dimensional plane to obtain a rectangular image; sliding windows of different sizes across the rectangular image with a certain step size to extract image blocks; matching the image blocks with the point cloud data of the tunnel surface before unfolding, and using the corresponding point cloud range as the segmentation range; and obtaining all suspected crack regions within the segmentation range.

[0034] Specifically, in this embodiment of the invention, a selected network model is trained using labeled data, such as a voxel-based method. The network parameters of the network model are continuously optimized through a loss function (such as cross-entropy loss) to obtain a network model for identifying cracks in tunnels. This network model is then used to identify and label crack areas in the current tunnel, which are then designated as suspected crack areas in the following steps.

[0035] Furthermore, since the tunnel surface is cylindrical, the resulting point cloud of the tunnel surface is cylindrically unfolded. Points on the three-dimensional, curved surface of the tunnel are mapped onto a two-dimensional plane, thus establishing a coordinate system that facilitates analysis while fully preserving the three-dimensional geometric properties (including depth / height information) of each point. For example, ... Figure 2 As shown, Figure 2 This is a schematic diagram of a three-dimensional image and an unfolded image of the tunnel interior provided in one embodiment of the present invention. On the rectangular image of the unfolded cylindrical surface of the tunnel, windows of different sizes (e.g., 256x256, 512x512, 1024x1024 pixels) are slid with a certain step size (50% overlap) to capture image patches. These image patches are then mapped to the point cloud of the tunnel surface before unfolding, and the corresponding point cloud range is used as a single segmentation range for crack region analysis. Then, the network model described in the above embodiment of the present invention is used to obtain the suspected crack regions within each segmentation range.

[0036] Step S102: Based on the adjacent pixels of the suspected crack branch in the suspected crack region, determine the degree of consistency between the trend of the current suspected crack branch and other suspected crack branches within the segmentation range.

[0037] Furthermore, material deterioration, groundwater leakage, and excessive pressure from the surrounding rock above the tunnel can all lead to cracks in the tunnel. The above-described embodiment of the invention uses a network model of the overall point cloud to identify suspected crack areas, which is easily interfered with by scratches, water stains, etc., on the tunnel surface. Therefore, by segmenting different identification ranges, suspected crack areas are further judged within each segment. Since adjacent crack areas are caused by the same force and have similar trends, while other interferences such as scratches and protrusions do not have this characteristic, this embodiment of the invention uses this characteristic to further judge suspected crack areas within the segmented range. Specifically, the consistency of the trends of suspected crack areas in different segmented ranges and the connection between suspected crack areas in tangent segmented ranges are used to analyze the degree of conformity of different suspected crack areas. The changes in light and shadow of suspected crack areas under different camera perspectives are used to eliminate interference from crack identification in the tunnel wall. The crack performance at a single location in segmented ranges of different scales is used as a weight to judge the actual crack conformity of suspected crack areas existing in multiple segmented ranges.

[0038] Furthermore, as an optional embodiment of the present invention, determining the degree of consistency of the trend between the current suspected crack branch and other suspected crack branches within a segmented range based on the adjacent pixels of the suspected crack branch in the suspected crack region includes: skeletonizing the suspected crack region within each segmented range to obtain a skeleton image of the suspected crack region; determining the suspected crack branch from the skeleton image based on the pixels in the skeleton image; determining the extension direction of the suspected crack branch based on the pointing direction and unit length of the adjacent pixels in the suspected crack branch; and determining the degree of consistency of the trend between each suspected crack branch and other suspected crack branches within each segmented range based on the angle between the extension directions of each suspected crack branch and other suspected crack branches.

[0039] Specifically, this embodiment of the invention performs binarization on the segmented area to ensure that the crack is pure white. The suspected crack region is skeletonized using the `skimage.morphology.skeletonize` function in Python, resulting in a skeleton image of the suspected crack region. The purpose of skeletonization is to reduce the crack width to 1 pixel while preserving its original topological structure (the connection relationships of all branches remain unchanged). Then, this embodiment of the invention defines a 3x3 convolution kernel to traverse each pixel of the skeleton image (corresponding to the unfolded 2D image). The number (N) of white pixels in the 8-neighborhood of the current pixel is calculated. When N=1, the number of white pixels is the skeleton endpoint. When N=3, it is a connecting point; when N=2, it is a path point, which is a regular point in the line. The Depth-First Search (DFS) algorithm is used to start from each endpoint and trace forward along the path points until all points reachable from the current endpoint have been traversed. When points are reachable from each other, they are divided into a suspected crack branch.

[0040] Furthermore, taking segmentation range j as an example, for a single suspected crack branch z in segmentation range j, the pointing direction from adjacent pixel i-1 to pixel i in its corresponding skeleton image is calculated. . To indicate direction Using direction as the unit length and length as the unit length, the corresponding modulus is obtained. The modulus obtained by combining all adjacent pixels i-1 and pixel i in a single suspected crack branch z. The modulus is summed, and the direction corresponding to the resulting modulus is taken as the extension direction of the suspected crack branch z. .

[0041] Furthermore, as an optional embodiment of the present invention, determining the degree of consistency of the trend between each suspected crack branch and other suspected crack branches within each segmentation range, based on the angle between the extension directions of each suspected crack branch and other suspected crack branches, includes: determining the angle between the extension directions of the current suspected crack branch and other suspected crack branches within the same segmentation range; counting the first number of other suspected crack branches whose angles are greater than a first threshold, and obtaining the minimum value among the angles; and determining the degree of consistency of the trend between each suspected crack branch and other suspected crack branches within each segmentation range based on the second number, the first number, and the minimum value among all suspected crack branches within the same segmentation range.

[0042] Specifically, in this embodiment of the invention, the number of suspected crack branches in the segmented range j is determined. When the number of suspected crack branches in the segmented range j is greater than 2, the angle between the extension direction of a single suspected crack branch z and other suspected crack branches k in the same segmented range j is calculated (less than 180 degrees). .statistics The first number of other suspected crack branches exceeding the first threshold And compare and obtain each included angle minimum value The first threshold can be set according to the actual situation; in this embodiment of the invention, it is set to 90 degrees.

[0043] Furthermore, in this embodiment of the invention, a second number of all suspected crack branches within the same segmentation range j is counted. Then, taking a suspected crack branch z within the same segmentation range j as an example, this embodiment of the invention uses the following formula to calculate the degree of consistency in trend between the suspected crack branch z within the same segmentation range j and other suspected crack branches within the same segmentation range j:

[0044]

[0045] In the above formula, This indicates the degree of consistency in trend between a suspected crack branch z within the same segmentation range j and other suspected crack branches within the same segmentation range j. This indicates the first number of other suspected crack branches with an angle greater than the first threshold. This represents the second number of all suspected crack branches within the same segmentation range j. This represents the minimum angle between the extension directions of the current suspected crack branch z and other suspected crack branches within the same segmentation range j. Let represent an exponential function with base e. Where, when the suspected crack branch z in the segmentation range j... The first number of other suspected crack branches The second number of all suspected crack branches in the segmentation range j Percentage Larger, and minimum value The larger the value, the greater the difference in the extension direction between the suspected crack branch z and other suspected crack branches in the segmented range j, the more inconsistent their trends are, and the less consistent they are with the characteristics of actual cracks.

[0046] Step S103: Based on the target suspected crack branches, the degree of consistency of trend, the distance between adjacent light and dark boundaries of the suspected crack branches in the segmentation range, the pixels at the crack boundary in the segmentation range, and all pixels in the segmentation range, determine the crack matching degree of each pixel in the suspected crack branches.

[0047] Among them, the suspected crack branch is the suspected crack branch in the adjacent segmentation range that is tangent to the current segmentation range and has adjacent pixels to the suspected crack branch in the current segmentation range.

[0048] Specifically, adjacent pixels refer to pixels within the 8-neighborhood of a pixel on a suspected crack branch. This embodiment of the invention judges the static conformity of a suspected crack based on the connectivity of the suspected crack regions within the tangent segmentation range and the consistency of the trends of the suspected crack branches. Interference is eliminated by assessing the uniformity of the change in the position of the light-dark boundary line of the same suspected crack region obtained from cameras at different locations. Specifically, the more consistent the stitching between a suspected crack region in a single segmentation range and the suspected crack regions in its tangent adjacent segmentation ranges, and the more consistent their extension trends, the more accurately the division of this suspected crack region conforms to the actual situation.

[0049] Furthermore, as an optional embodiment of the present invention, the crack matching degree of each pixel in the suspected crack branch is determined based on the target suspected crack branch, the degree of consistency of trend, the distance between adjacent light and dark boundaries of the suspected crack branch in the segmentation range, the pixels at the crack boundary in the segmentation range, and all pixels in the segmentation range. This includes: determining the splicing fit between the suspected crack branch in the current segmentation range and the target suspected crack branch based on the angle between the extension directions of the suspected crack branch in the current segmentation range and the target suspected crack branch, the third number of adjacent pixels, and the fourth number of pixels of the suspected crack branch in the current segmentation range on the boundary of the current segmentation range. Based on the degree of splicing fit and the degree of trend consistency, the static division performance of the suspected crack branches within the segmentation range is determined; based on the distance between adjacent light and dark boundaries of the suspected crack branches within the segmentation range and the static division performance of the suspected crack branches, the crack conformity of the suspected crack branches within the segmentation range is determined; based on the crack conformity, the third number of pixels at the crack boundary within the segmentation range, and the fourth number of all pixels within the segmentation range, the crack belonging degree of each pixel in the suspected crack branches within the segmentation range is determined; based on the crack belonging degree of each pixel in the suspected crack branches in all segmentation ranges containing it, the crack matching degree of each pixel in the suspected crack branches is determined.

[0050] Specifically, in this embodiment of the invention, for a suspected crack branch z extending to the boundary of a segmentation range j, a suspected crack branch k is obtained from multiple segmentation ranges p that are tangent to segmentation range j and have adjacent pixels to pixels on the suspected crack branch z. Then, the ratio of the third number of adjacent pixels between the suspected crack branch z and the boundary of the segmentation range p tangent to segmentation range j, and the fourth number of pixels on the boundary of the suspected crack branch z at segmentation range j is calculated. .

[0051] Furthermore, the angle between the extension directions of the suspected crack branch z and the suspected crack branch k is calculated (less than 180 degrees) through the above embodiments of the present invention. When the ratio Larger, and The smaller the size, the better the suspected crack branch z in segmentation range j matches the suspected crack branch k in segmentation range p, and the more accurate the segmentation of the suspected crack branch z in segmentation range j is.

[0052] Furthermore, in this embodiment of the invention, the degree of fit between the suspected crack branch z in the segmented range j and the suspected crack branch k in the tangent segmented range p is obtained using the following formula:

[0053]

[0054] In the above formula, This indicates the degree of fit between the suspected crack branch z in segmentation range j and the suspected crack branch k in the tangent segmentation range p. This represents the ratio of the third quantity to the fourth quantity. This represents the angle between the extension directions of the suspected crack branch z in segmentation range j and the suspected crack branch k in segmentation range p that is tangent to it. This represents an exponential function with base e.

[0055] Furthermore, embodiments of the present invention utilize max-min normalization to... Normalization is performed to obtain the splicing fit degree. Its range is [0, 1].

[0056] Furthermore, as an optional embodiment of the present invention, determining the static division performance of the suspected crack branches within the segmentation range based on the splicing fit degree and the trend consistency degree includes: counting the fifth number of adjacent segmentation ranges tangent to the target suspected crack branch within the current segmentation range whose splicing fit degree is greater than a second threshold, and the sixth number of all adjacent segmentation ranges tangent to the current segmentation range within the current segmentation range; determining the static division performance of the suspected crack branches within the segmentation range based on the fifth number, the sixth number, and the trend consistency degree.

[0057] Specifically, in this embodiment of the invention, the second threshold can be selected according to the actual situation, and in this embodiment, the value is 0.7. This embodiment of the invention calculates the degree of fit between the suspected crack branch z within the segmentation range j and the suspected crack branch k in the tangent segmentation range p. The fifth quantity of the tangent segmentation range The sixth number of all segmentation ranges tangent to segmentation range j ratio Therefore, the static division of suspected crack branches z within the segmentation range j can be calculated using the following formula, based on the fifth and sixth quantities and the degree of trend consistency:

[0058]

[0059] In the above formula, This represents the static partitioning of the suspected crack branch z within the partitioning range j. This indicates the degree of fit between the suspected crack branch z in segmentation range j and the suspected crack branch k in the tangent segmentation range p. The fifth quantity of the tangent segmentation range. This represents the sixth number of all tangent segmentation ranges in segmentation range j. This indicates the degree of consistency in trend between a suspected crack branch z within the same segmentation range j and other suspected crack branches within the same segmentation range j. Specifically, when a suspected crack branch z is not tangent to the segmentation range boundary, the degree of consistency in trend is considered. Static partitioning representation of suspected crack branch z When the proportion of the tangent segmentation range with a higher fit is... The degree of consistency between the trend of suspected fracture branch z and other suspected fracture branches is relatively large. The larger the value, the better the static division of the suspected crack branch z within the segmentation range j, and the more it matches the actual crack area.

[0060] Furthermore, some interference appears remarkably similar to cracks in the overall point cloud generated by multiple radar and camera data. Unlike interference (e.g., water stains, scratches), the shadows in the crack area change smoothly and continuously with the movement of the camera position, without jumps or abrupt changes. Therefore, the accuracy of crack area segmentation is further judged based on the uniformity of the positional changes of the light-dark boundary lines in the suspected crack area obtained from cameras at different positions. As an optional embodiment of the present invention, determining the crack conformity of the suspected crack branches in the segmentation range based on the distance between adjacent light-dark boundary lines of the suspected crack branches in the segmentation range and the static segmentation performance of the suspected crack branches includes: determining the smoothness of the light-dark changes of the suspected crack branches in the segmentation range under different angles based on the distance between adjacent light-dark boundary lines of the suspected crack branches in the segmentation range; obtaining the maximum value in the static segmentation performance of each suspected crack branch in the segmentation range; and determining the crack conformity of the suspected crack branches in the segmentation range based on the static segmentation performance of each suspected crack branch in the segmentation range, the maximum value in the static segmentation performance of each suspected crack branch, and the smoothness of the light-dark changes.

[0061] Specifically, in this embodiment of the invention, cameras are numbered according to their extension order from tunnel entrance to exit. For a suspected crack branch z within a segmented range j, an image of the suspected crack branch z within the segmented range j is acquired in camera m, converted to grayscale, and the light-dark boundary line in the image of the suspected crack branch z is obtained using Canny edge detection. The position of the obtained light-dark boundary line is then mapped to the point cloud. For example, as shown... Figure 3 As shown, Figure 3 This is a schematic diagram of a light-dark boundary line provided in one embodiment of the present invention. Figure 3 In the comparison, the positional changes of adjacent light and shadow boundaries of the same suspected crack branch within the same segmented area obtained from adjacent cameras are shown, such as... Figure 3Given the terminator line b and the terminator line b-1 adjacent to the left of terminator line b, the distance between pixel x in terminator line b-1 and the nearest pixel x' in terminator line b is obtained using the Euclidean distance formula. Further, obtain the average distance between all pixels in the light-dark boundary line b-1 and the nearest pixel in the light-dark boundary line b. This is the distance between adjacent light and shadow boundaries that represent suspected crack branches within the segmented area. Further, in this embodiment of the invention, the mean of the average distances between adjacent light and shadow boundaries obtained from all adjacent cameras is calculated. Then, the smoothness of the brightness variation of the suspected crack branch z within the segmentation range j under different angles is calculated using the following formula: .in, This indicates the number of cameras capable of capturing the suspected crack branch z within the segmented range j.

[0062] Furthermore, in this embodiment of the invention, the maximum value in the static division performance of each suspected crack branch within the segmentation range j is obtained by comparison. Therefore, the crack conformity of the suspected crack branch z within the segmented range j can be calculated using the following formula:

[0063]

[0064] In the above formula, This represents the crack conformity of the suspected crack branch z within the segmentation range j. This represents the maximum value in the static partitioning of each suspected crack branch within the segmentation range j. This represents the static partitioning of the suspected crack branch z within the partitioning range j. The smoothness of the brightness variation of the suspected crack branch z in the segmented range j under different angles of photography. This indicates the number of cameras capable of capturing the suspected crack branch z within the segmented range j. This represents the average distance between all pixels in the light-dark boundary line b-1, which is adjacent to the left side of the light-dark boundary line b, and the nearest pixel in the light-dark boundary line b. This represents the average distance between adjacent light and dark boundaries obtained from all adjacent cameras. Wherein, the smoothness of light and dark transitions... The larger the area, the more the suspected crack area conforms to the requirements of a crack under changes in light and dark, and the more interference can be eliminated. The larger the value, the greater the static consistency of the suspected crack area compared to other suspected crack branches. Combined with the appearance under light and dark changes, the larger both values ​​are, the greater the crack consistency of the suspected crack branch.

[0065] Furthermore, in this embodiment of the invention, the crack conformity description weight at the same location under different segmentation ranges is obtained based on the proportion of boundary points in suspected crack regions within different segmentation ranges. When a suspected crack region contains a large number of boundaries, it means that the suspected crack branch is the dominant feature of that region. The interference of background pixels (such as a complete lining surface) is minimized, and the algorithm can more "focused" on analyzing the target itself. Therefore, the larger the proportion of crack boundary points in all pixels within the segmentation range, the more accurate the crack division within this segmentation range, and the greater its reference value. Therefore, as an optional embodiment of the present invention, determining the crack belonging degree of each pixel in the suspected crack branch within the segmentation range based on crack conformity, the third number of pixels at crack boundaries in the segmentation range, and the fourth number of all pixels in the segmentation range includes: determining the crack belonging degree of each pixel in the suspected crack branch within the segmentation range based on the ratio between the third and fourth numbers and the crack conformity.

[0066] Specifically, in this embodiment of the invention, the third number of pixels at the crack boundary in the segmentation range j is obtained. And the fourth number of all pixels in the segmentation range j Calculate the third number of pixels at the crack boundary within the segmentation range j. The fourth number of all pixels in the segmentation range j ratio Therefore, the crack degree of each pixel c in the suspected crack branch z of the segmentation range j is calculated using the following formula:

[0067]

[0068] In the above formula, The crack degree of each pixel c in the suspected crack branch z of the segmentation range j is indicated. This represents the crack conformity of the suspected crack branch z within the segmentation range j. This represents the third number of pixels at the crack boundary within the segmentation range j. This represents the fourth number of pixels in the segmentation range j. Where, when the ratio... The larger the value, the more accurate the crack region division obtained by segmentation range j, and the more valuable it is for evaluating the crack consistency of cracks at the same location across multiple segmentation ranges. In this case, the crack consistency of the suspected crack branch z within segmentation range j should be greater.

[0069] Furthermore, for a single point c in the point cloud of the tunnel wall, the degree of crack belonging in different segmentation ranges containing it is calculated. The mean value is used as the crack matching degree at point c. .

[0070] Step S104: Determine the range of the crack area in the tunnel based on the crack matching degree of each pixel.

[0071] Specifically, determining the crack region range of the tunnel based on the crack matching degree of each pixel includes: normalizing the crack matching degree to obtain a normalized matching degree; classifying pixels with a normalized matching degree greater than a third threshold into the crack region; merging adjacent pixels of pixels in the crack region to obtain the crack region range of the tunnel.

[0072] In this embodiment of the invention, the third threshold can be selected based on actual conditions; in this embodiment, it is set to 0.8. This embodiment of the invention utilizes maximum-minimum normalization to determine the crack matching degree. After normalization, we get Its range is [0,1]. When When the value is greater than 0.8, it is determined to belong to a crack region. Pixels identified as belonging to crack regions are merged with their adjacent pixels to obtain multiple crack region ranges in the tunnel. Adjacent pixels can be pixels within the 8-neighborhood of a given pixel.

[0073] Furthermore, this embodiment of the invention uses the above method to obtain the location of the crack area on the tunnel surface, and transmits the crack matching degree of the crack area and its corresponding coordinates to the database for storage. SQL queries are used to obtain the coordinates and crack matching degree of points within different crack areas, and the results are visualized in a table. A mask of the crack location is also obtained, and the crack area is marked in the point cloud on the tunnel surface. For example, as shown in Table 1, Table 1 shows the coordinates, numbers, and crack matching degrees of points within different crack areas.

[0074] Table 1. Coordinates, numbers, and crack matching degree of points within different crack regions.

[0075]

[0076] The table above shows the relationship between point number, point coordinates, and crack matching degree.

[0077] The technical solution provided by this invention achieves refined screening of suspected cracks by dual verification of the consistency of the trend between the current suspected crack branch and other suspected crack branches within the segmentation range and the crack matching degree. By analyzing the adjacent pixels of the suspected crack branch, the consistency of the trend between different suspected crack branches can be judged, and interference areas that do not conform to the crack trend pattern (such as messy textures of water stains and dirt) can be quickly eliminated. Furthermore, by combining multi-dimensional parameters such as the target suspected crack branch (related cracks in adjacent segmentation ranges), the distance between the light and dark boundary lines, and the pixel ratio of the crack boundary, the crack matching degree can be calculated, which can further distinguish interference areas with similar shapes but different essences from real cracks, accurately eliminate such misjudgments, improve the accuracy of crack identification to the pixel level, and significantly reduce the false negative rate and false positive rate. When determining the crack matching degree, the parameter of the distance between the light and dark boundary lines of adjacent suspected crack branches in the segmentation range can be introduced, which can effectively offset the interference caused by uneven illumination and avoid feature distortion caused by changes in illumination. Furthermore, the correlation analysis between the suspected crack branch and other suspected crack branches within the current segmentation range establishes a crack spatial continuity verification mechanism across segmentation ranges. Even if there is a large area of ​​stain covering the lining surface, as long as there are identifiable crack features in adjacent tangent segmentation ranges, the crack segments obscured by stains can be connected by calculating the consistency of the crack trend and the crack matching degree. This overcomes the scenario limitation of misjudging crack fractures due to local stain obscuration and enables continuous crack identification in complex environments. This improves the reliability and accuracy of tunnel crack identification results.

[0078] Furthermore, this embodiment of the invention analyzes the consistency of suspected crack regions across different segmented ranges based on the degree of trend uniformity and the connectivity of suspected crack regions within tangent segmented ranges. Interference in crack identification within the tunnel wall is eliminated by analyzing the changes in light and shadow over suspected crack regions from different camera perspectives. By using the crack behavior at a single location across segmented ranges of different scales as a weight to determine the actual crack consistency across suspected crack regions existing in multiple segmented ranges, the detection of minute cracks on the tunnel surface becomes more accurate, effectively preventing major tunnel accidents.

[0079] Example 2:

[0080] Corresponding to the lightweight multi-scale attention mechanism tunneling holographic sensing method provided in the above embodiments, based on the same technical concept, this invention also provides a lightweight multi-scale attention mechanism tunneling holographic sensing system. This lightweight multi-scale attention mechanism tunneling holographic sensing system is used to execute the above-described lightweight multi-scale attention mechanism tunneling holographic sensing method. Figure 4 This is a schematic diagram of the structure of a lightweight multi-scale attention mechanism tunneling holographic sensing system according to an embodiment of the present invention, as shown below. Figure 4As shown. Lightweight multi-scale attention mechanism tunneling holographic perception systems can vary considerably depending on configuration or performance. They may include one or more processors 401 and memory 402. Memory 402 stores computer programs that can run on processor 401. Processor 401 executes the programs stored in memory 402 to achieve the above... Figure 1 The various steps in the method embodiment. Memory 402 can be temporary or persistent storage. The application stored in memory 402 may include one or more modules (not shown in the figures), each module may include a series of computer-executable instructions for a tunneling holographic perception system with a lightweight multi-scale attention mechanism.

[0081] Furthermore, the processor 401 can be configured to communicate with the memory 402 and execute a series of computer-executable instructions in the memory 402 on the lightweight multi-scale attention mechanism tunneling holographic perception system. The lightweight multi-scale attention mechanism tunneling holographic perception system may also include one or more power supplies 403, one or more wired or wireless network interfaces 404, one or more input / output interfaces 405, and one or more keyboards 406.

[0082] Specifically, in this embodiment, the lightweight multi-scale attention mechanism tunneling holographic perception system includes a processor, a communication interface, a memory, and a communication bus; wherein, the processor, communication interface, and memory communicate with each other via the bus; the memory is used to store computer programs; and the processor is used to execute the programs stored in the memory to achieve the above... Figure 1 The various steps in the method embodiments are the same as those in the above method embodiments, and have the same beneficial effects. To avoid repetition, the embodiments of the present invention will not be described again here.

[0083] It should be noted that the lightweight multi-scale attention mechanism tunneling holographic perception system provided in this embodiment of the invention and the lightweight multi-scale attention mechanism tunneling holographic perception method provided in this embodiment of the invention are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned lightweight multi-scale attention mechanism tunneling holographic perception method, and has the same or similar beneficial effects. The repeated parts will not be described again.

[0084] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0085] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0086] This invention also proposes a computer-readable storage medium storing one or more programs, which, when executed by a lightweight multi-scale attention mechanism tunneling holographic perception system comprising multiple applications, cause the lightweight multi-scale attention mechanism tunneling holographic perception system to perform... Figure 1 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods in the preceding method embodiments, and will not be repeated here.

[0087] The computer-readable storage media include read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, etc.

Claims

1. A lightweight multi-scale attention mechanism-based tunneling holographic perception method, characterized in that, The lightweight multi-scale attention mechanism tunneling holographic perception method includes: Based on the holographic perception data of the tunnel, suspected crack areas within each segmented range of the tunnel are identified; Based on the adjacent pixels of the suspected crack branch in the suspected crack region, determine the degree of consistency between the current suspected crack branch and other suspected crack branches within the segmentation range. Based on the target suspected crack branch, the trend consistency, the distance between adjacent light and dark boundaries of the suspected crack branch in the segmentation range, the pixels of the crack boundary in the segmentation range, and all pixels in the segmentation range, the crack matching degree of each pixel in the suspected crack branch is determined. The target suspected crack branch is the suspected crack branch in the adjacent segmentation range that is tangent to the current segmentation range and has adjacent pixels with the suspected crack branch in the current segmentation range. The crack region range of the tunnel is determined based on the crack matching degree of each pixel. Determining the crack matching degree of each pixel in the suspected crack branch includes: Based on the angle between the extension direction of the suspected crack branch in the current segmentation range and the target suspected crack branch, the third number of adjacent pixels, and the fourth number of pixels of the suspected crack branch in the current segmentation range on the boundary of the current segmentation range, the splicing fit between the suspected crack branch in the current segmentation range and the target suspected crack branch is determined. Based on the degree of splicing fit and the degree of trend consistency, the static division of the suspected crack branches within the segmentation range is determined; Based on the distance between adjacent light and dark boundary lines of the suspected crack branches in the segmented range and the static division performance of the suspected crack branches, the crack conformity of the suspected crack branches in the segmented range is determined. Based on the crack conformity, the third number of pixels at the crack boundary in the segmentation range, and the fourth number of all pixels in the segmentation range, the crack belonging degree of each pixel in the suspected crack branch in the segmentation range is determined; The crack matching degree of each pixel in the suspected crack branch is determined based on the crack belonging degree of each pixel in all segmentation ranges including it.

2. The tunneling holographic perception method with a lightweight multi-scale attention mechanism according to claim 1, characterized in that, The step of determining the degree of consistency in trend between the current suspected crack branch and other suspected crack branches within the segmentation range based on the adjacent pixels of the suspected crack branch in the suspected crack region includes: Skeletonization is performed on the suspected crack regions within each segmentation range to obtain skeleton images of the suspected crack regions; Suspected crack branches are identified from the skeleton image based on the pixels in the skeleton image; The extension direction of the suspected crack branch is determined based on the pointing direction and unit length of adjacent pixels in the suspected crack branch; Based on the angle between the extension directions of each suspected crack branch and other suspected crack branches, the degree of consistency between the trends of each suspected crack branch and other suspected crack branches within each segmented range is determined.

3. The tunneling holographic perception method with a lightweight multi-scale attention mechanism according to claim 2, characterized in that, The step of determining the degree of consistency in the trend between each suspected crack branch and other suspected crack branches within each segmented range, based on the angle between their extension directions, includes: Determine the angle between the extension directions of the current suspected crack branch and other suspected crack branches within the same segmentation range; Count the first number of other suspected crack branches whose included angle is greater than the first threshold, and obtain the minimum value among the included angles; Based on the second number, the first number, and the minimum value of the included angle among all suspected crack branches within the same segmentation range, the degree of consistency of the trend between each suspected crack branch and other suspected crack branches within each segmentation range is determined.

4. The tunneling holographic perception method with a lightweight multi-scale attention mechanism according to claim 1, characterized in that, The determination of the static division of the suspected crack branches within the segmentation range based on the degree of splicing fit and the degree of trend consistency includes: The fifth number of adjacent segmentation ranges within the current segmentation range whose splicing fit with the target suspected crack branch is greater than the second threshold is counted, and the sixth number of all adjacent segmentation ranges within the current segmentation range that are tangent to the current segmentation range is counted. Based on the fifth quantity, the sixth quantity, and the degree of consistency of the trend, the static division performance of the suspected crack branches within the segmentation range is determined.

5. The tunneling holographic perception method with a lightweight multi-scale attention mechanism according to claim 1, characterized in that, The step of determining the crack conformity of the suspected crack branch in the segmented range based on the distance between adjacent light and dark boundary lines of the suspected crack branch in the segmented range and the static division performance of the suspected crack branch includes: Based on the distance between adjacent light and dark boundaries of the suspected crack branches within the segmented range, the smoothness of light and dark changes of the suspected crack branches within the segmented range under different angles of photography is determined. Obtain the maximum value in the static division representation of each suspected crack branch within the segmentation range; Based on the static division performance of each suspected crack branch within the segmentation range, the maximum value in the static division performance of each suspected crack branch, and the smoothness of the brightness change, the crack conformity of the suspected crack branch within the segmentation range is determined.

6. The tunneling holographic perception method with a lightweight multi-scale attention mechanism according to claim 1, characterized in that, The determination of the crack belonging degree of each pixel in the suspected crack branch within the segmentation range based on the crack conformity, the third number of pixels at the crack boundary in the segmentation range, and the fourth number of all pixels in the segmentation range includes: Based on the ratio between the third quantity and the fourth quantity and the crack conformity, the crack belonging degree of each pixel in the suspected crack branch within the segmentation range is determined.

7. The tunneling holographic perception method with a lightweight multi-scale attention mechanism according to claim 1, characterized in that, Determining the crack region range of the tunnel based on the crack matching degree of each pixel includes: The crack matching degree is normalized to obtain the normalized matching degree; Pixels with a normalized matching degree greater than a third threshold are assigned to the crack region, and adjacent pixels of the pixels in the crack region are merged to obtain the crack region range of the tunnel.

8. The tunneling holographic perception method with a lightweight multi-scale attention mechanism according to claim 1, characterized in that, The identification of suspected crack regions within each segmented area of ​​the tunnel based on tunnel-based holographic sensing data includes: The holographic perception data of the tunnel is input into the trained network model to identify crack areas and obtain the suspected crack areas of the tunnel. The trained network model is obtained by training the historical point cloud data and historical video stream data of the tunnel through high-definition cameras and lidar deployed in the tunnel. The point cloud data in the holographic perception data of the tunnel surface is cylindrically unfolded, and the points on the three-dimensional surface of the tunnel are mapped to a two-dimensional plane to obtain a rectangular image. By sliding windows of different sizes across the rectangular image at certain step sizes, image blocks are captured. The image block is mapped to the point cloud data of the tunnel surface before it is unfolded, and the corresponding point cloud range is used as the segmentation range. Obtain all suspected crack areas within the segmented range.

9. A lightweight, multi-scale attention mechanism-based tunneling holographic sensing system, characterized in that, include: Processor and memory; wherein the memory is used to store computer programs that can run on the processor; A processor for executing a program stored in memory to implement the steps of the tunneling holographic perception method with a lightweight multi-scale attention mechanism as described in any one of claims 1-8.

Images