A deep learning-based slope crack detection method and system

Abstract

The application discloses a kind of based on deep learning's side slope crack detection method and system, it is related to side slope detection technical field, including, acquisition side slope multi-source inspection image data and through pre-processing, obtain side slope image data to be identified;According to side slope image data to be identified, slope surface structure constraint model is constructed using geometric texture feature layer and thermal anomaly feature layer, and based on slope surface structure constraint model, structure constraint analysis is carried out to side slope crack, and slope surface feature constraint atlas is formed;Band target pattern recognition algorithm is used to match and identify crack candidate area from slope surface feature constraint atlas, and crack candidate band is formed;Crack candidate band is guided to crack continuity, obtains crack continuous guide band, and performs deep learning identification to crack continuous guide band, generates initial crack identification area.The application constrains deep learning identification process by crack continuity guide band, and enhances the continuous extraction capacity of slender crack.

Description

Technical Field

[0001] This invention relates to the field of slope detection technology, and in particular to a method and system for detecting slope cracks based on deep learning. Background Technology

[0002] With the increasingly complex long-term service environments of open-pit mines, highway high slopes, hydraulic slopes, and geotechnical engineering projects, surface cracks on slopes have become an important visual indicator of slope deformation evolution, shallow instability development, and local structural deterioration. In recent years, the combination of UAV low-altitude inspection, visible light / thermal infrared joint imaging, and deep learning methods has gradually enabled slope crack detection to have a foundation for large-scale, non-contact, automated, and highly timely applications. Existing research also shows that using UAVs to acquire high-resolution images and combining them with deep learning models can achieve automatic extraction of rock mass cracks, rock joints, and similar slender structures. At the same time, multi-source remote sensing and thermal infrared information play an auxiliary role in perceiving abnormal areas in complex environments.

[0003] Existing technologies still have two shortcomings in slope crack scenarios. On the one hand, most existing methods still rely on directly inputting the entire image into the detection or segmentation network, lacking specific constraint mechanisms for common features in slope scenarios such as bedding textures, erosion patterns, shadow bands, weathering textures, and thermal anomalies. This results in insufficient focus in locating crack candidate regions, easily leading to misidentification or missed identification. On the other hand, existing deep learning methods are prone to problems such as insufficient crack continuity, local fractures, regional adhesion, and unstable subsequent category determination, making it difficult to generate continuous crack identification information suitable for engineering interpretation. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a deep learning-based slope crack detection method to solve the problems of inaccurate crack candidate region localization and insufficient crack continuity identification capability.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a deep learning-based method for detecting slope cracks, comprising: acquiring multi-source inspection image data of a slope and preprocessing it to obtain image data of the slope to be identified; constructing a slope structure constraint model based on the image data of the slope to be identified using a geometric texture feature layer and a thermal anomaly feature layer, and performing structural constraint analysis on the slope cracks based on the slope structure constraint model to form a slope feature constraint map; using a strip target pattern recognition algorithm to match and identify crack candidate regions from the slope feature constraint map to form crack candidate zones; guiding the crack candidate zones to obtain crack continuity guide zones, and performing deep learning recognition on the crack continuity guide zones to generate an initial crack identification region; and performing semantic consistency verification and category determination based on the initial crack identification region and in combination with the slope feature constraint map and crack candidate zones to form slope crack identification information.

[0007] As a preferred embodiment of the deep learning-based slope crack detection method of the present invention, the multi-source inspection image data of the slope includes visible light images, thermal infrared images, depth images and pose registration image information. The preprocessing includes time synchronization, spatial registration, and noise suppression; The slope image data to be identified includes visible light recognition images, thermal infrared recognition images, depth recognition images, and pose-related recognition image information.

[0008] As a preferred embodiment of the deep learning-based slope crack detection method of the present invention, the specific steps for constructing a slope structure constraint model based on the slope image data to be identified using a geometric texture feature layer and a thermal anomaly feature layer are as follows: A geometric texture feature layer is used to extract local gradients from visible light recognition images and depth recognition images, and to perform texture direction and surface undulation change analysis to obtain geometric texture feature vectors; A thermal anomaly feature layer is used to perform neighborhood temperature difference comparison and thermal gradient change extraction on thermal infrared recognition images to obtain thermal anomaly feature vectors. The geometric texture feature layer and the thermal anomaly feature layer are correlated in parallel, and a slope structure constraint model is constructed by feature splicing.

[0009] As a preferred embodiment of the deep learning-based slope crack detection method of the present invention, the specific steps of performing structural constraint analysis on slope cracks based on a slope structure constraint model to form a slope feature constraint map are as follows: Based on the slope structure constraint model, spatial coupling mapping is performed on geometric texture feature vectors and thermal anomaly feature vectors to generate crack constraint feature sets; Spatial connectivity organization and directional consistency reorganization of the crack constraint feature set are performed to form a slope feature constraint map.

[0010] As a preferred embodiment of the deep learning-based slope crack detection method of the present invention, the specific steps of using a strip target pattern recognition algorithm to match and identify crack candidate regions from the slope feature constraint map to form crack candidate zones are as follows: Strip retrieval and strip merging are performed on the slope feature constraint map, and the number of consecutive registrations and the number of interrupted registrations are counted to obtain strip target identification information; Based on the strip target identification information, the strip target matching value is calculated, and the strip target matching value is subjected to discontinuity tolerance compensation verification and interruption offset suppression in combination with the number of consecutive registrations and the number of interrupted registrations to identify crack candidate regions; The candidate crack regions are merged into strips, and the number of consecutive registrations of adjacent candidate crack regions is counted to organize the breakpoint continuation, forming a candidate crack zone.

[0011] As a preferred embodiment of the deep learning-based slope crack detection method of the present invention, the specific steps for guiding the crack candidate zone to obtain a crack continuity guide zone are as follows: The candidate fracture zones are constrained and organized in a banded manner to obtain banded connectivity information. Based on the banded connectivity information, the candidate fracture zones are organized into continuous banded structures to obtain continuous fracture distribution information. Based on the continuous distribution information of cracks, the crack candidate zones are guided by the crack extension trend to obtain the continuous crack guidance zone.

[0012] As a preferred embodiment of the deep learning-based slope crack detection method of the present invention, the specific steps of performing deep learning recognition on the continuous crack guide zone to generate an initial crack recognition region are as follows: Deep learning is used to extract features and classify categories from image data of slopes to be identified based on continuous crack guide strips, thereby obtaining crack identification feature information. Based on the crack identification feature information, the crack region response is determined in the continuous crack guide zone to generate the initial crack identification region.

[0013] As a preferred embodiment of the deep learning-based slope crack detection method of the present invention, the specific steps of performing semantic consistency verification and category determination based on the initial crack identification area and combined with the slope feature constraint map and crack candidate zones are as follows: Based on the initial crack identification area and combined with the slope feature constraint map and crack candidate zone, semantic correspondence verification is performed to obtain semantic verification information; The semantic consistency score is calculated based on the semantic verification information, and the crack category is determined for the initial crack identification area based on the semantic consistency score to obtain crack category information.

[0014] As a preferred embodiment of the deep learning-based slope crack detection method of the present invention, the slope crack identification information is formed by regional screening and false crack removal of the initial crack identification area based on crack category information.

[0015] Secondly, this invention provides a deep learning-based slope crack detection system, comprising: an image acquisition module for acquiring multi-source inspection image data of a slope and preprocessing it to obtain slope image data to be identified; a constraint map module for constructing a slope structure constraint model based on the slope image data to be identified using a geometric texture feature layer and a thermal anomaly feature layer, and performing structural constraint analysis on slope cracks based on the slope structure constraint model to form a slope feature constraint map; a candidate zone identification module for matching and identifying crack candidate regions from the slope feature constraint map using a strip target pattern recognition algorithm to form crack candidate zones; a continuous guidance identification module for guiding the crack candidate zones to obtain continuous crack guidance zones, and performing deep learning recognition on the continuous crack guidance zones to generate an initial crack identification region; and an identification information generation module for performing semantic consistency verification and category determination based on the initial crack identification region and in combination with the slope feature constraint map and crack candidate zones to form slope crack identification information.

[0016] The beneficial effects of this invention are as follows: by constructing a slope structure constraint model that integrates geometric texture feature layers and thermal anomaly feature layers, and generating a slope feature constraint map, the prior constraint screening of slope crack-related areas is realized, reducing the interference of complex backgrounds on deep learning recognition; and by guiding the deep learning recognition process with crack continuity, the continuous extraction capability of slender cracks is enhanced, thereby improving the accuracy, stability and engineering applicability of slope crack recognition. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a deep learning-based slope crack detection method.

[0019] Figure 2 This is a schematic diagram of a deep learning-based slope crack detection system.

[0020] Figure 3 The flowchart for identifying crack candidate bands.

[0021] Figure 4 This is a flowchart for semantic validation.

[0022] Figure 5 A schematic diagram illustrating the trajectory changes of slope cracks.

[0023] Figure 6 A schematic diagram comparing the indicators of slope crack identification effectiveness. Detailed Implementation

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0027] Reference Figures 1-6 This is one embodiment of the present invention, which provides a slope crack detection method based on deep learning, including the following steps: S1: Collect multi-source inspection image data of the slope and obtain the image data of the slope to be identified through preprocessing.

[0028] S1.1: Multi-source inspection image data of slopes includes visible light images, thermal infrared images, depth images, and pose registration image information.

[0029] Specifically, the inspection equipment is controlled to move along the slope area according to the slope inspection route. When the inspection equipment reaches each collection position, the visible light imaging equipment is activated to continuously take pictures of the corresponding slope area. The images corresponding to each collection position are numbered and registered according to the collection sequence to obtain visible light images.

[0030] While completing the imaging of the slope area corresponding to the visible light imaging device, the thermal infrared imaging device is activated to perform thermal imaging acquisition of the slope area corresponding to the same acquisition location, and the images are numbered and registered according to the acquisition sequence corresponding to the visible light images to obtain thermal infrared images.

[0031] After acquiring visible light and thermal infrared images corresponding to the same acquisition location, laser ranging is performed on the slope area corresponding to the same acquisition location. Based on the distance and elevation difference information obtained from the laser ranging, a depth image is generated. The images are then organized and registered according to the same acquisition location to obtain the depth image.

[0032] During the acquisition of visible light images, thermal infrared images, and depth images, the shooting position, shooting posture, and positional correspondence between visible light images, thermal infrared images, and depth images are recorded at each acquisition location. The images are then associated and registered according to their respective acquisition locations to obtain pose registration image information.

[0033] S1.2: Preprocessing includes time synchronization, spatial registration, and noise suppression.

[0034] Specifically, the visible light image, thermal infrared image, depth image, and pose registration image information are read in the acquisition order. The acquisition time corresponding to the visible light image, thermal infrared image, and depth image is extracted, and the visible light image, thermal infrared image, and depth image with the same acquisition time at the same acquisition location are associated and registered.

[0035] Based on the correspondence between the shooting position and shooting posture recorded in the pose registration image information, the slope areas corresponding to the visible light image, thermal infrared image and depth image at the same acquisition position are determined. Then, using the slope area in the visible light image as the corresponding reference, the same slope area in the thermal infrared image and the same slope area in the depth image are mapped and aligned.

[0036] Isolated bright spots and spurious areas in visible light images are marked as noise areas and removed from subsequent identification and processing objects. Isolated thermal noise points in thermal infrared images are marked as noise areas and removed from subsequent identification and processing objects. Single-point mutations in depth images are marked as noise areas and removed from subsequent identification and processing objects.

[0037] The slope image data to be identified includes visible light recognition images, thermal infrared recognition images, depth recognition images, and pose-related recognition image information.

[0038] S2: Based on the image data of the slope to be identified, a slope structure constraint model is constructed using a geometric texture feature layer and a thermal anomaly feature layer. Based on the slope structure constraint model, a structural constraint analysis is performed on the slope cracks to form a slope feature constraint map.

[0039] S2.1: The geometric texture feature layer is used to extract local gradients from the visible light recognition image and the depth recognition image, and the texture direction and surface undulation change analysis are performed to obtain the geometric texture feature vector.

[0040] Specifically, the visible light recognition image and depth recognition image are read, and the slope area is divided into regions in the horizontal and vertical directions to obtain local regions arranged in order of slope location. The geometric texture feature layer is used to read the visible light recognition image and depth recognition image corresponding to the same slope area, and the pixel content in the visible light recognition image is read one by one. The grayscale changes of adjacent pixels in the horizontal and vertical directions are compared to determine the location of local gradient changes.

[0041] Based on the extension and distribution of local gradient changes in the local region, the texture direction of the corresponding local region is obtained; the depth distribution corresponding to the same local region in the depth recognition image is read, the depth changes within the local region and between adjacent local regions are compared, and the information on surface undulation changes of the corresponding local region is obtained by organizing the depth change distribution.

[0042] The local gradient changes, texture direction, and surface undulation changes corresponding to the same local area are combined and registered in a unified order to obtain the geometric texture feature vector.

[0043] It should be noted that the geometric texture feature vector is a feature representation formed by combining and registering the local gradient changes, texture direction, and surface undulation changes of the same slope area in the visible light recognition image and the depth recognition image.

[0044] S2.2: The thermal anomaly feature layer is used to compare the neighborhood temperature difference and extract the thermal gradient change of the thermal infrared recognition image to obtain the thermal anomaly feature vector.

[0045] Specifically, the thermal anomaly feature layer is used to read the thermal infrared recognition image. Starting from the initial boundary of the slope area, image areas of the same width are sequentially cropped along the horizontal direction of the visible light recognition image. Within each horizontal image area, image areas of the same height are sequentially cropped along the vertical direction. The temperature distribution in each local area is read one by one.

[0046] Select the current local area, then read the temperature distribution of the local areas adjacent to the current local area, and compare the temperature distribution of the current local area with the corresponding temperature distribution of the adjacent local areas one by one to obtain the temperature difference change of the neighborhood corresponding to the current local area.

[0047] Within the current local area, the temperature change direction between corresponding positions is compared one by one according to the arrangement order of adjacent positions to obtain the thermal gradient change situation corresponding to the current local area; the neighborhood temperature difference change situation and thermal gradient change situation corresponding to the same local area are combined and registered in a unified order to obtain the thermal anomaly feature vector.

[0048] It should be noted that the thermal anomaly feature vector is a feature representation formed by combining and registering the changes in neighborhood temperature difference and thermal gradient in the same slope area in the thermal infrared identification image.

[0049] S2.3: The geometric texture feature layer and the thermal anomaly feature layer are associated in parallel, and a slope structure constraint model is constructed by feature splicing.

[0050] Specifically, each slope area is numbered and a corresponding relationship is established, so that each slope area corresponds to a set of geometric texture feature vectors and a set of thermal anomaly feature vectors.

[0051] The geometric texture feature vectors and thermal anomaly feature vectors corresponding to the same slope area are arranged in a uniform order, so that the contents of the geometric texture feature vector and the contents of the thermal anomaly feature vector are arranged in a corresponding order under the same slope area.

[0052] The geometric texture feature vectors and thermal anomaly feature vectors, after being arranged in parallel, are continuously spliced ​​together in sequential order to form crack constraint feature records for the corresponding slope areas. The crack constraint feature records for each slope area are then centrally summarized according to the order of the slope areas to obtain the slope structure constraint model.

[0053] S2.4: Based on the slope structure constraint model, spatial coupling mapping is performed on the geometric texture feature vector and thermal anomaly feature vector to generate a crack constraint feature set.

[0054] Specifically, geometric texture feature vectors and thermal anomaly feature vectors for corresponding slope regions are extracted from multiple sets of slope image data to be identified, and then paired and organized according to the same slope region. Each pair of paired geometric texture feature vectors and thermal anomaly feature vectors is then labeled. The labeled geometric texture feature vectors and thermal anomaly feature vectors are input into the slope structure constraint model in the order of slope regions, and the slope structure constraint model outputs the constraint representation of the corresponding slope region. The constraint representation is compared with the labeled content of the corresponding slope region item by item, and the corresponding difference between the constraint representation and the labeled content is used as the basis for adjustment to update the corresponding mapping relationship in the slope structure constraint model, thereby obtaining the trained slope structure constraint model.

[0055] Read the crack constraint feature records corresponding to each slope region in the trained slope structure constraint model, and read the geometric texture feature vector and thermal anomaly feature vector of each crack constraint feature record one by one according to the arrangement order of the slope regions.

[0056] The geometric texture feature vector and thermal anomaly feature vector in the same crack constraint feature record are arranged in a uniform positional order to correspond to each other, so that the information on texture direction change and surface undulation change in the geometric texture feature vector representing the same slope area is in a one-to-one correspondence with the information on neighborhood temperature difference change and thermal gradient change in the thermal anomaly feature vector representing the same slope area.

[0057] After completing the one-to-one correspondence, the geometric texture feature vector and thermal anomaly feature vector are merged and registered according to the same slope area to form the crack constraint feature record of the corresponding slope area; the crack constraint feature records corresponding to all slope areas are centrally organized according to the arrangement order of the slope areas to generate the crack constraint feature set.

[0058] S2.5: Spatial connectivity organization and directional consistency reorganization of the crack constraint feature set to form a slope feature constraint map.

[0059] Specifically, the crack constraint feature records registered in the crack constraint feature set according to the slope region are read. Starting from the first crack constraint feature record, the next crack constraint feature record is read in sequence, and the adjacency relationship between the slope region corresponding to the previous crack constraint feature record and the slope region corresponding to the next crack constraint feature record is compared in the arrangement of slope regions.

[0060] When there are adjacent relationships between the preceding and following cracks in the slope area arrangement, the preceding crack constraint feature record and the following crack constraint feature record are registered as the same connected group, and the next crack constraint feature record is read until it is no longer possible to register them as the same connected group.

[0061] Read the crack constraint feature records in the same connected group one by one, and read the texture direction change information and thermal gradient change information respectively. Compare whether the texture direction change information and thermal gradient change information in adjacent crack constraint feature records are extended in the same direction. When both the texture direction change information and thermal gradient change information are extended in the same direction, register the corresponding crack constraint feature record as the same direction consistent group, until all crack constraint feature records in the same connected group are registered.

[0062] According to the arrangement order of slope areas, the directional consistency groups that have completed the spatial connectivity organization and directional consistency reorganization are re-registered to form a slope feature constraint map.

[0063] It should be noted that the slope feature constraint map is a structured distribution map formed by centrally organizing the constraint content related to cracks in the slope area. The slope feature constraint map can pre-screen the slope areas related to cracks in visible light recognition images, thermal infrared recognition images and depth recognition images and establish corresponding relationships. This reduces the interference of bedding textures, shadow bands and local anomalies in the subsequent crack candidate area recognition and deep learning recognition process, thereby improving the accuracy and continuity of slope crack recognition.

[0064] S3: A strip target pattern recognition algorithm is used to match and identify crack candidate regions from the slope feature constraint map to form crack candidate zones.

[0065] S3.1: Perform strip retrieval and strip merging on the slope feature constraint map, and count the number of consecutive registrations and the number of interrupted registrations to obtain strip target identification information.

[0066] Specifically, the content of slope regions registered in the slope feature constraint map according to the slope region arrangement order is read. Starting from the first slope region content, the content of the next slope region is read in sequence. The continuity of the previous slope region content and the next slope region content in terms of directional consistency is compared. When the directional consistency is maintained, a continuous registration is recorded, and the previous slope region content and the next slope region content are organized into the same strip retrieval group. When the directional consistency is interrupted, an interruption registration is recorded.

[0067] Read adjacent strip search groups sequentially according to the slope area arrangement order. Compare the connection between the ending slope area content of the previous strip search group and the starting slope area content of the next strip search group in the slope area arrangement, and compare the continuity of the directional consistency compilation content between the previous and next strip search groups. When the connection is maintained and the directional consistency compilation content continues to be continuous, record a continuity entry and organize the adjacent strip search groups into the same strip merge group. When the connection is interrupted or the directional consistency compilation content is interrupted, record an interruption entry.

[0068] The content of the slope area corresponding to each strip-shaped merging group, the range of the slope area arrangement, the number of consecutive registrations and the number of interrupted registrations are centrally registered to obtain strip-shaped target identification information.

[0069] S3.2: Calculate the matching value of the strip target based on the strip target identification information, and perform discontinuous tolerance compensation verification and interruption offset suppression on the matching value of the strip target by combining the number of consecutive registrations and the number of interrupted registrations, and identify the crack candidate region.

[0070] Specifically, the strip target identification information is read, and each strip group is registered in the order of slope area arrangement. First, the slope area content, the starting position of the slope area arrangement, and the ending position of the slope area arrangement in each strip group are read one by one.

[0071] Read the slope area content registered in the same strip-shaped merging group according to the slope area arrangement order one by one, and compare the continuity of the content of adjacent slope areas in terms of directional consistency. When the content of adjacent slope areas remains continuous in terms of directional consistency, record a continuous registration. When the content of adjacent slope areas is interrupted in terms of directional consistency, do not record a continuous registration. After all the slope area content in the same strip-shaped merging group has been compared, count the number of continuous registrations and obtain the directional consistency continuity value.

[0072] The number of slope regions included in the same strip-shaped merging group is counted. Based on the arrangement difference between the starting and ending positions of the slope regions (slope region arrangement span) and the directional consistency continuous value, the strip target matching value corresponding to each strip-shaped merging group is calculated one by one. The expression is: ; in, For the matching value of the strip target, The quantity of content in the slope area. For directional consistency, continuous values ​​are compiled. The span of the slope area is arranged. This represents the maximum span of the slope area. This represents the minimum span of the slope area.

[0073] The matching values ​​of each strip target are compared centrally according to the arrangement order of the slope areas, and the number of consecutive registrations and the number of interrupted registrations corresponding to each strip merging group are read simultaneously. The number of consecutive registrations and the number of interrupted registrations in the same strip merging group are compared. When the number of consecutive registrations is greater than the number of interrupted registrations, the interruption situation in the corresponding strip merging group is determined to be in an acceptable state, and the matching value of the corresponding strip target is retained. When the number of interrupted registrations is greater than the number of consecutive registrations, the interruption situation in the corresponding strip merging group is determined to be in a discontinuous offset state, and the corresponding strip merging group is removed from the set of comparison objects of strip target matching values.

[0074] The strip-shaped merging groups that have completed the discontinuous tolerance compensation verification and whose corresponding slope area content remains continuously arranged are identified as candidate crack areas.

[0075] S3.3: Perform strip merging on the crack candidate regions and count the number of consecutive registrations of adjacent crack candidate regions to organize the breakpoint continuation and form crack candidate zones.

[0076] Specifically, the candidate crack regions registered in the order of slope regions are read. Starting from the first candidate crack region, the next candidate crack region is read in sequence. The connection between the previous and next candidate crack regions in the slope region arrangement is compared. When the connection between the previous and next candidate crack regions in the slope region arrangement remains continuous, a continuous registration is recorded. The previous and next candidate crack regions are then grouped into the same strip and merged. The next candidate crack region is read again until the connection between the previous and next candidate crack regions in the slope region arrangement is interrupted.

[0077] After completing the strip merging, each strip merging group is read one by one according to the slope area arrangement order. The connection between the ending slope area of ​​the previous strip merging group and the starting slope area of ​​the next strip merging group in the slope area arrangement is compared. When the ending slope area and the starting slope area are continuously connected in the slope area arrangement, a continuity registration is recorded, and the previous strip merging group and the next strip merging group are organized into the same connected group. When there is only a discontinuity slope area between the ending slope area and the starting slope area, and the previous strip merging group and the next strip merging group are connected one after the other, the previous strip merging group and the next strip merging group are organized into the same discontinuity continuation group.

[0078] The candidate crack regions corresponding to each connected group and each discontinuity continuation group are re-registered according to the slope region arrangement order to form a crack candidate zone.

[0079] It should be noted that the strip target pattern recognition algorithm can retrieve, merge, and match the scattered crack-related areas in the slope feature constraint map according to the strip extension relationship, thereby extracting continuous areas that are closer to the actual distribution of cracks to form crack candidate zones. Through the strip target pattern recognition algorithm, the interference of slope surface bedding texture, shadow bands, and local abnormal areas on subsequent identification can be reduced, thereby improving the accuracy and continuity of crack candidate area extraction.

[0080] S4: Perform crack continuity guidance on crack candidate zones to obtain crack continuity guidance zones, and perform deep learning recognition on crack continuity guidance zones to generate initial crack recognition regions.

[0081] S4.1: Perform banded connectivity constraint processing on the candidate fracture zones to obtain banded connectivity information, and organize the candidate fracture zones into continuous banded structures based on the banded connectivity information to obtain continuous fracture distribution information.

[0082] Specifically, the candidate crack zones registered according to the slope area arrangement are extracted. First, the candidate crack regions contained in each candidate crack zone are read one by one, and the connection between adjacent candidate crack regions in the slope area arrangement is compared. When adjacent candidate crack regions are continuous in the slope area arrangement, the adjacent candidate crack regions are registered as the same strip-shaped connected group to obtain strip-shaped connectivity information.

[0083] Read the candidate crack regions contained in each banded connected group one by one, and continue to compare the arrangement and continuity of adjacent candidate crack regions in the same banded connected group in the candidate crack zone. Organize adjacent candidate crack regions that are continuous in the slope area arrangement and maintain continuity in the candidate crack zone into the same continuous banded region.

[0084] The candidate crack regions and slope regions corresponding to each continuous strip area are re-registered in the order of slope regions to obtain information on the continuous distribution of cracks.

[0085] S4.2: Based on the continuous distribution information of cracks, guide the crack extension trend of the crack candidate zone to obtain the continuous crack guidance zone.

[0086] Specifically, the candidate crack regions and slope regions contained in each continuous strip area are read one by one. Starting from the initial candidate crack region of each continuous strip area, the next candidate crack region is read in sequence. The extension of the previous and next candidate crack regions in the continuous strip area is compared. When the next candidate crack region continues to be arranged along the extension direction of the previous candidate crack region in the slope region, the previous candidate crack region and the next candidate crack region are registered as the same crack extension segment.

[0087] The candidate crack regions and slope regions corresponding to all crack extension segments are re-registered in the order of slope region arrangement to obtain the continuous crack guide zone.

[0088] S4.3: Based on the continuous crack guide zone, perform deep learning feature extraction and category discrimination on the slope image data to be identified, and obtain crack identification feature information.

[0089] Specifically, the extension segments of each crack in the continuous guide zone of the crack and the corresponding slope area arrangement range of each crack extension segment are extracted. Then, according to the slope area arrangement range corresponding to each crack extension segment, the visible light recognition image, thermal infrared recognition image, depth recognition image and pose association recognition image information of the corresponding slope area are extracted from the slope image data to be identified.

[0090] First, extract edge texture changes from the visible light recognition image, thermal anomaly changes from the thermal infrared recognition image, and surface undulation changes from the depth recognition image according to the order of slope regions to obtain feature extraction content. Based on the distribution relationship between the feature extraction content and the slope regions corresponding to the continuous crack guidance zone, classify whether the slope region belongs to the crack region. When the slope region belongs to the crack region, the corresponding slope region is identified as the crack response region; when the slope region does not belong to the crack region, the corresponding slope region is identified as the non-crack response region to obtain category discrimination content.

[0091] The extracted features and category information corresponding to each crack extension segment are centrally registered according to the order of the slope area to obtain crack identification feature information.

[0092] It should be noted that deep learning features refer to the representational information extracted after the slope image data to be identified is input into the deep learning recognition process. Deep learning features can comprehensively reflect the edge texture changes in visible light recognition images, thermal anomaly changes in thermal infrared recognition images, and surface undulation changes in depth recognition images, thereby providing a basis for subsequent determination of whether the corresponding slope area belongs to the crack area.

[0093] S4.4: Based on the crack identification feature information, determine the crack region response of the continuous crack guide zone and generate the initial crack identification region.

[0094] Specifically, the process involves sequentially reading each crack extension segment and the corresponding slope area arrangement range within the continuous crack guide zone, then reading the crack identification feature information corresponding to each crack extension segment, comparing the category judgment content corresponding to each slope area within the same crack extension segment, determining that the corresponding slope area maintains crack response when the corresponding slope area is determined to be a crack response area, and determining that the corresponding slope area does not maintain crack response when the corresponding slope area is determined to be a non-crack response area, and registering the adjacent slope areas as the same crack identification segment when the crack response is interrupted. When the crack response is interrupted, the registration of the current crack identification segment ends, and the crack area response judgment is restarted from the next slope area that maintains crack response.

[0095] The slope area corresponding to each crack identification segment is re-registered according to the slope area arrangement order to generate the initial crack identification area.

[0096] S5: Based on the initial crack identification area and combined with the slope feature constraint map and crack candidate zone, perform semantic consistency verification and category determination to form slope crack identification information.

[0097] S5.1: Based on the initial crack identification area and combined with the slope feature constraint map and crack candidate zone, perform semantic correspondence verification to obtain semantic verification information.

[0098] Specifically, the slope area arrangement range corresponding to each initial crack identification area is read one by one, and then the content corresponding to the same slope area arrangement range in the slope feature constraint map and crack candidate zone is read. The starting position, ending position and continuous coverage range of the initial crack identification area, the content corresponding to the slope feature constraint map and the content corresponding to the crack candidate zone in the slope area arrangement range are checked segment by segment.

[0099] When the start position, end position, and continuous coverage area all correspond, the corresponding initial crack identification area is registered as a semantically corresponding area; when any two of the start position, end position, and continuous coverage area correspond, the corresponding initial crack identification area is registered as a semantically partially corresponding area; when the start position, end position, and continuous coverage area do not correspond, the corresponding initial crack identification area is registered as a semantically non-corresponding area.

[0100] The semantically corresponding regions, semantically partially corresponding regions, and semantically non-corresponding regions are registered in a centralized manner according to the slope region order to obtain semantic verification information.

[0101] S5.2: Calculate the semantic consistency score based on the semantic verification information, and determine the crack category of the initial crack identification area based on the semantic consistency score to obtain crack category information.

[0102] Specifically, semantic verification information is registered according to the slope region arrangement order, and the initial crack identification regions registered according to the slope region arrangement order are read simultaneously; the number of slope regions in the same initial crack identification region is counted for semantically corresponding regions, semantically partially corresponding regions, and semantically non-corresponding regions, respectively; the number of slope regions, the number of slope regions corresponding to semantically partially corresponding regions, and the number of slope regions corresponding to semantically non-corresponding regions are obtained; and the semantic consistency score is calculated, with the expression as follows: ; in, Score semantic consistency. This represents the number of slope regions corresponding to the semantically defined region. This represents the number of slope regions corresponding to the semantic part. This represents the number of slope regions corresponding to semantically mismatched regions. The maximum number of slope regions that maintain a continuous arrangement between semantically corresponding regions and semantically partial corresponding regions.

[0103] Read the semantic consistency score corresponding to each initial crack identification region one by one, and perform crack category determination on the initial crack identification region according to the semantic consistency score corresponding to the same initial crack identification region. Register the initial crack identification region whose semantic consistency score meets the crack identification condition as crack category region, and register the initial crack identification region whose semantic consistency score does not meet the crack category condition as pseudo crack category region.

[0104] Crack category areas and pseudo-crack category areas are centrally registered according to the slope area order to obtain crack category information.

[0105] It should be noted that the crack identification conditions include distribution conditions, response conditions, and correspondence conditions. The distribution condition is that the corresponding slope area is located within the range of slope areas corresponding to the continuous crack guide zone. The response condition is that the corresponding slope area is identified as a crack response area after deep learning identification. The correspondence condition is that the corresponding slope area corresponds to the content of the constraint area in the slope feature constraint map and the content of the candidate zone in the crack candidate zone in terms of the range of slope areas. When the distribution condition, response condition, and correspondence condition are all met, the corresponding slope area is determined to satisfy the crack identification conditions.

[0106] S5.3: Based on the crack category information, the initial crack identification area is screened and false cracks are removed to form slope crack identification information.

[0107] Specifically, the crack category information registered in the order of slope regions is read, and the initial crack identification area registered in the order of slope regions is read simultaneously; first, the slope region arrangement range corresponding to each initial crack identification area is read one by one, and then the crack category area and pseudo crack category area corresponding to the same slope region arrangement range are read.

[0108] The initial crack identification areas that are registered as crack categories and whose slope area arrangement range remains continuous are retained, while the areas registered as false crack categories are removed. The retained initial crack identification areas are read one by one according to the slope area arrangement order. The connection between adjacent initial crack identification areas in the slope area arrangement is compared. Adjacent initial crack identification areas that maintain continuity in the slope area arrangement are grouped into the same area for screening.

[0109] The slope area arrangement range corresponding to each screening group is re-registered according to the slope area arrangement order to form slope crack identification information.

[0110] like Figure 5The invention described herein, under the condition of sequential unfolding of a slope area, performs prior screening, continuous guided identification, and subsequent verification of trajectory change relationships in crack-related areas. First, time synchronization, spatial registration, and noise suppression are performed on multi-source inspection image data of the slope to obtain image data of the slope to be identified. Then, local gradient changes, texture direction, and surface undulation changes are extracted from visible light and depth recognition images to form geometric texture feature vectors. Neighborhood temperature difference changes and thermal gradient changes are extracted from thermal infrared recognition images to form thermal anomaly feature vectors. The two types of feature vectors are then spliced ​​together according to the same slope area to construct a slope structure constraint model and generate a slope feature constraint map. Subsequently, strip retrieval and merging are performed on the slope feature constraint map. The continuity of adjacent slope regions in terms of directional consistency and connectivity in the slope region arrangement are compared. Then, based on the content quantity of the slope region, the continuity value of directional consistency, and the span of the slope region arrangement, strip target matching values ​​are calculated. Strip merge groups that meet the conditions are identified as crack candidate regions, completing the prior screening and forming crack candidate strips. Next, strip connectivity constraint processing and crack extension trend guidance are performed on the crack candidate strips to form continuous crack guidance strips. Based on these continuous crack guidance strips, deep learning recognition is performed to generate initial crack recognition regions. The results in the figure show that after the above processing, the slope crack recognition information trajectory is closer to the reference crack distribution trajectory, indicating that the present invention can reduce interference from complex backgrounds and enhance the ability to continuously extract slender cracks.

[0111] like Figure 6 The invention demonstrates the improvement in slope crack identification performance at the overall performance level. Direct deep learning recognition directly performs deep learning feature extraction and category discrimination on the slope image data to be identified. Constraint map enhancement recognition first constructs a slope structure constraint model before recognition, generates a slope feature constraint map, and performs strip retrieval and strip merging on the slope feature constraint map. Based on the directional consistency of the continuous value, the number of contents in the slope area, and the span of the slope area arrangement, the strip target matching value is calculated, and the strip merged group that meets the crack candidate conditions is determined as the crack candidate area. The complete method of this invention continues to form a continuous crack guide zone on the basis of prior screening, generates an initial crack identification area, and performs semantic consistency verification and category determination in combination with the slope feature constraint map and crack candidate zone. Specifically, it checks the start position, end position, and continuous coverage of the same slope area arrangement range, divides it into semantically corresponding areas, semantically partially corresponding areas, and semantically non-corresponding areas, counts the number of corresponding slope areas, calculates the semantic consistency score, and registers the initial crack identification area as a crack category area or a pseudo-crack category area, completing the pseudo-crack elimination and area screening. The results in the figure show that the complete method of the present invention has higher accuracy, F1 value and consistency rate of repeated identification in the same scene, and lower false detection rate, indicating that the present invention can simultaneously improve accuracy, stability and engineering applicability.

[0112] This embodiment also provides a deep learning-based slope crack detection system, comprising: an image acquisition module, which acquires multi-source inspection image data of the slope and preprocesses it to obtain slope image data to be identified; a constraint map module, which constructs a slope structure constraint model based on the slope image data to be identified using a geometric texture feature layer and a thermal anomaly feature layer, and performs structural constraint analysis on the slope cracks based on the slope structure constraint model to form a slope feature constraint map; a candidate zone identification module, which uses a strip target pattern recognition algorithm to match and identify crack candidate regions from the slope feature constraint map to form crack candidate zones; a continuous guidance identification module, which guides the crack candidate zones to obtain continuous crack guidance zones, and performs deep learning recognition on the continuous crack guidance zones to generate an initial crack identification region; and an identification information generation module, which performs semantic consistency verification and category determination based on the initial crack identification region and in combination with the slope feature constraint map and crack candidate zones to form slope crack identification information.

[0113] This embodiment also provides a computer device applicable to the deep learning-based slope crack detection method, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the deep learning-based slope crack detection method proposed in the above embodiment.

[0114] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0115] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the deep learning-based slope crack detection method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0116] In summary, this invention achieves prior constraint screening of slope crack-related areas by constructing a slope structure constraint model that integrates geometric texture feature layers and thermal anomaly feature layers, and generating a slope feature constraint map, thereby reducing the interference of complex backgrounds on deep learning recognition. Furthermore, by guiding the deep learning recognition process with crack continuity constraints, the continuous extraction capability of slender cracks is enhanced, thus improving the accuracy, stability, and engineering applicability of slope crack recognition.

[0117] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep learning-based method for detecting slope cracks, characterized in that, include: Collect multi-source inspection image data of the slope and obtain the image data of the slope to be identified through preprocessing; Based on the slope image data to be identified, a slope structure constraint model is constructed using a geometric texture feature layer and a thermal anomaly feature layer. Based on the slope structure constraint model, structural constraint analysis is performed on the slope cracks to form a slope feature constraint map. A strip target pattern recognition algorithm is used to match and identify crack candidate regions from the slope feature constraint map, forming crack candidate strips; Crack continuity guidance is performed on the crack candidate zone to obtain the crack continuity guidance zone, and deep learning recognition is performed on the crack continuity guidance zone to generate the initial crack recognition region. Based on the initial crack identification area and combined with the slope feature constraint map and crack candidate zone, semantic consistency verification and category determination are performed to form slope crack identification information.

2. The slope crack detection method based on deep learning as described in claim 1, characterized in that, The multi-source inspection image data of the slope includes visible light images, thermal infrared images, depth images, and pose registration image information; The preprocessing includes time synchronization, spatial registration, and noise suppression; The slope image data to be identified includes visible light recognition images, thermal infrared recognition images, depth recognition images, and pose-related recognition image information.

3. The slope crack detection method based on deep learning as described in claim 1, characterized in that, The specific steps for constructing a slope structure constraint model based on the slope image data to be identified using a geometric texture feature layer and a thermal anomaly feature layer are as follows: A geometric texture feature layer is used to extract local gradients from visible light recognition images and depth recognition images, and to perform texture direction and surface undulation change analysis to obtain geometric texture feature vectors; A thermal anomaly feature layer is used to perform neighborhood temperature difference comparison and thermal gradient change extraction on thermal infrared recognition images to obtain thermal anomaly feature vectors. The geometric texture feature layer and the thermal anomaly feature layer are correlated in parallel, and a slope structure constraint model is constructed by feature splicing.

4. The slope crack detection method based on deep learning as described in claim 3, characterized in that, The structural constraint analysis of slope cracks based on the slope structure constraint model, and the generation of slope feature constraint map, are performed in the following specific steps: Based on the slope structure constraint model, spatial coupling mapping is performed on geometric texture feature vectors and thermal anomaly feature vectors to generate crack constraint feature sets; Spatial connectivity organization and directional consistency reorganization of the crack constraint feature set are performed to form a slope feature constraint map.

5. The slope crack detection method based on deep learning as described in claim 1, characterized in that, The method of using a strip target pattern recognition algorithm to match and identify crack candidate regions from the slope feature constraint map to form crack candidate zones is as follows: Strip retrieval and strip merging are performed on the slope feature constraint map, and the number of consecutive registrations and the number of interrupted registrations are counted to obtain strip target identification information; Based on the strip target identification information, the strip target matching value is calculated, and the strip target matching value is subjected to discontinuity tolerance compensation verification and interruption offset suppression in combination with the number of consecutive registrations and the number of interrupted registrations to identify crack candidate regions; The candidate crack regions are merged into strips, and the number of consecutive registrations of adjacent candidate crack regions is counted to organize the breakpoint continuation, forming a candidate crack zone.

6. The slope crack detection method based on deep learning as described in claim 5, characterized in that, The specific steps for guiding the fracture candidate zone to obtain a fracture continuity guide zone are as follows: The candidate fracture zones are constrained and organized in a banded manner to obtain banded connectivity information. Based on the banded connectivity information, the candidate fracture zones are organized into continuous banded structures to obtain continuous fracture distribution information. Based on the continuous distribution information of cracks, the crack candidate zones are guided by the crack extension trend to obtain the continuous crack guidance zone.

7. The slope crack detection method based on deep learning as described in claim 1 or 6, characterized in that, The specific steps for performing deep learning recognition on the continuous guide strip of the crack to generate an initial crack recognition region are as follows: Deep learning is used to extract features and classify categories from image data of slopes to be identified based on continuous crack guide strips, thereby obtaining crack identification feature information. Based on the crack identification feature information, the crack region response is determined in the continuous crack guide zone to generate the initial crack identification region.

8. The slope crack detection method based on deep learning as described in claim 7, characterized in that, The specific steps for performing semantic consistency verification and category determination based on the initial crack identification area and combined with the slope feature constraint map and crack candidate zones are as follows: Based on the initial crack identification area and combined with the slope feature constraint map and crack candidate zone, semantic correspondence verification is performed to obtain semantic verification information; The semantic consistency score is calculated based on the semantic verification information, and the crack category is determined for the initial crack identification area based on the semantic consistency score to obtain crack category information.

9. The slope crack detection method based on deep learning as described in claim 1, characterized in that, The slope crack identification information is formed by regional screening and false crack removal of the initial crack identification area based on crack category information.

10. A deep learning-based slope crack detection system, based on the deep learning-based slope crack detection method according to any one of claims 1 to 9, characterized in that, include: The image acquisition module collects multi-source inspection image data of the slope and obtains the image data of the slope to be identified through preprocessing; The constraint map module constructs a slope structure constraint model based on the slope image data to be identified using a geometric texture feature layer and a thermal anomaly feature layer. Based on the slope structure constraint model, it performs structural constraint analysis on slope cracks to form a slope feature constraint map. The candidate zone identification module uses a strip target pattern recognition algorithm to match and identify crack candidate regions from the slope feature constraint map, forming crack candidate zones; The continuous guidance and recognition module guides the candidate cracks to obtain continuous crack guidance zones, and performs deep learning recognition on the continuous crack guidance zones to generate an initial crack recognition region. The identification information generation module performs semantic consistency verification and category determination based on the initial crack identification area and in combination with the slope feature constraint map and crack candidate zone to form slope crack identification information.

Images