Intelligent crack recognition method and system based on semi-supervised learning and improved YOLOv8

Abstract

The application discloses a kind of based on semi-supervised learning and improved YOLOv8's fissure intelligent identification method and system, belong to fissure identification technical field.First, by semi-supervised labeling strategy constructs high-quality fissure image dataset, and adopts three-stage process to realize the standardization conversion and visual verification of labeled data;Second, based on improved YOLOv8n-seg light weight instance segmentation network, fusion multi-scale feature pyramid and triple data enhancement strategy, through three branch detection head realizes the collaborative training and optimization of classification, positioning and segmentation;Finally, via confidence filtering, non-maximum suppression, mask optimization and coordinate system restoration system post-processing, output structured recognition results containing boundary box and pixel-level mask.The application has significant advantages in improving small target detection capability, boundary positioning accuracy and model generalization performance, and is suitable for geological engineering safety monitoring and disaster warning scene.

Description

Technical Field

[0001] This invention relates to the field of crack recognition technology, specifically to a crack intelligent recognition method and system based on semi-supervised learning and improved YOLOv8. Background Technology

[0002] As a pillar energy source in my country's energy structure, coal's strategic importance continues to strengthen with the deepening of industrialization. Under the dual constraints of "dual carbon" targets and energy security, the coal industry is undergoing a strategic transformation from scale expansion to quality improvement, and deep resource development has become a key path to ensure energy supply.

[0003] Coal roadway excavation is a special underground engineering project that constructs stable passages in coal-bearing strata with deep burial depths, poor lithology, and complex engineering environments. As mining depth increases, the geomechanical environment becomes increasingly complex: when the mining depth exceeds 800 meters, the ground stress exhibits a non-linear growth characteristic. The superposition of tectonic stress and mining-induced stress leads to a significant deterioration of the surrounding rock stress environment, triggering a chain reaction of intensified mine pressure manifestation, large roadway deformation, and frequent rockbursts, posing a severe challenge to mine safety. As the core engineering component of coal mine production, roadways undertake crucial functions such as underground ventilation, material and coal transportation. The fracture network developed within the surrounding rock, if poorly detected and not controlled in a timely manner, often leads to safety accidents such as roof falls, water inrushes, and gas outbursts, seriously threatening life and property. The mechanism of surrounding rock fracture development is a core scientific question concerning the causes of deep disasters. Its randomness, density, and directional distribution characteristics make accurate identification and quantitative characterization the technical basis for rock mass stability assessment and disaster prediction and early warning, which is of decisive significance for ensuring the safety of deep mining.

[0004] In the development of rock fracture identification technology, research methods have evolved from traditional image processing to intelligent analysis. Early research mainly relied on traditional image processing techniques such as threshold segmentation and edge detection, extracting morphological parameters such as fracture length and pit area to achieve defect identification and geometric measurement. While these methods have promoted the development of fracture quantitative analysis to some extent, their feature extraction capabilities are limited, and they heavily rely on prior knowledge to set processing parameters, leading to three prominent limitations in practical applications: First, low processing efficiency, making it difficult to meet the real-time analysis needs of massive fracture images in engineering sites; second, insufficient generalization ability, resulting in poor identification stability when faced with the diversity of rock fracture morphology, changes in illumination, and background interference; and third, difficulty in balancing accuracy and speed, failing to meet the technical requirements of modern mines for "rapid perception—accurate identification—dynamic monitoring" of fractures. Although borehole imaging technology has achieved visualized data acquisition of the internal structure of rock strata through anchor-supported boreholes, providing direct observational basis for key aspects such as rock mass stability assessment and water inrush prediction, its data accuracy and acquisition efficiency directly determine the reliability of the three-dimensional structural surface network reconstruction. When there is a deviation in borehole positioning or insufficient imaging resolution, it will lead to distortion of the three-dimensional spatial distribution features, which in turn will affect the accuracy of the restoration of the real structural surface network in the tunnel excavation direction. Summary of the Invention

[0005] The technical problem this invention aims to solve is to address the shortcomings of existing technologies by proposing an improved YOLOv8 structural fracture intelligent identification method that integrates semi-supervised learning. This method utilizes advanced artificial intelligence technology to achieve accurate fracture identification and quantitative analysis, providing core technical support for geological safety monitoring and promoting the intelligent development of deep disaster prevention and rock mass stability assessment. To solve the above problem, this invention adopts the following technical solution:

[0006] First, this invention proposes a crack intelligent identification method based on semi-supervised learning and improved YOLOv8, comprising the following steps:

[0007] S1. Acquire fracture images; use LabelMe tool to perform semi-supervised fine-grained annotation on the fracture images, and achieve data standardization through a three-stage transformation process, including parsing JSON annotation files, performing normalized coordinate mapping, and generating YOLO format label files; establish a contour visualization verification mechanism to ensure annotation quality;

[0008] S2. By constructing an improved YOLOv8n-seg network architecture, integrating multiple data augmentation strategies and multi-scale feature fusion mechanisms, a three-branch detection head is designed to achieve multi-task collaborative learning of classification, localization and segmentation, and a composite loss function is used for end-to-end optimization.

[0009] S3. Input the preprocessed image into the trained model and construct a four-step post-processing process including confidence filtering, non-maximum suppression, mask optimization and coordinate system restoration. Finally, generate a structured output for gap recognition that includes bounding boxes and pixel-level masks.

[0010] Preferably, in step S1, the LabelMe tool is used to perform semi-supervised fine-grained annotation of the crack image, specifically including:

[0011] Construct closed polygonal regions by clicking point by point along the edges of the crack shapes in the image;

[0012] Set a uniform label name for each label and establish a mapping relationship between the label name and the category ID.

[0013] Preferably, in step S1, data standardization is achieved through a three-stage transformation process, specifically including:

[0014] Parse the JSON annotation file to obtain image size information and coordinates of the vertices of the annotated polygons, and perform independent instance annotation on all visible crack regions in the image;

[0015] The pixel coordinate system is mapped to the normalized coordinate system through a linear transformation. The specific transformation formula is as follows:

[0016] ;

[0017] ;

[0018] in, These are the original pixel coordinates. and These are the width and height of the image, respectively. , These are the normalized pixel coordinates, and the normalized coordinate values ​​are strictly limited to the range (0, 1).

[0019] The output data is organized according to the standard format of the YOLO segmentation task, generating a text file containing category IDs and normalized coordinate sequences, with each fracture instance represented as a single line of data.

[0020] Preferably, step S2 specifically includes:

[0021] A network architecture based on the improved YOLOv8n-seg was constructed, and a lightweight instance segmentation model was selected as the foundation;

[0022] The configuration includes multiple data augmentation strategies such as spatial geometric transformation, mirror flipping, and advanced Mosaic enhancements;

[0023] A multi-scale feature pyramid is generated using an improved CSPDarknet53 backbone network, and feature fusion is achieved via PANet and SPPF neck network.

[0024] The design includes a three-branch detection head comprising a classification branch, a bounding box regression branch, and an instance mask branch, and optimizes it using Sigmoid activation, CIoU loss, and a composite loss of Dice and binary cross-entropy.

[0025] A multi-task loss function is constructed using a weighted combination of classification loss, bounding box loss, and segmentation loss, and the network is trained and optimized end-to-end.

[0026] Preferably, the improved YOLOv8n-seg network architecture uses an improved CSPDarknet53 as the backbone network; the improved CSPDarknet53 optimizes the gradient information flow through cross-stage local connection design and generates feature pyramids with three different scales, P3, P4 and P5, to capture the detailed texture and edge information of the crack, the medium-scale crack structure and the large-scale crack region, respectively.

[0027] Preferably, step S3 specifically includes:

[0028] All candidate prediction boxes output by the model are initially screened based on a set confidence threshold of 0.1 to filter out low-confidence detection results;

[0029] The candidate boxes after screening are sorted in descending order of confidence, the overlap is calculated using the intersection-union ratio, and redundant boxes are suppressed based on an IoU threshold of 0.5 to ensure that only one optimal detection box is retained for each crack instance.

[0030] For each preserved instance, the segmentation mask probability map is sequentially activated by the Sigmoid function, optimized for contours, and closed by morphological operations to smooth the boundaries, fill the internal holes, and remove noise.

[0031] The post-processed bounding box coordinates and segmentation mask are inversely mapped back to the coordinate system of the original input image based on the scaling and padding parameters during preprocessing, generating a final structured output that is spatially aligned with the original image.

[0032] Meanwhile, this invention proposes a crack intelligent recognition system based on semi-supervised learning and an improved YOLOv8, characterized by comprising:

[0033] The data preprocessing and format conversion module is configured to perform the following processes: acquiring fracture images; using the LabelMe tool to perform semi-supervised fine-grained annotation on the fracture images; achieving data standardization through a three-stage conversion process, including parsing JSON annotation files, performing normalized coordinate mapping, and generating YOLO format label files; and establishing a contour visualization verification mechanism to ensure annotation quality.

[0034] The model training and optimization module is configured to perform the following process: by constructing an improved YOLOv8n-seg network architecture, integrating multiple data augmentation strategies and multi-scale feature fusion mechanisms, designing a three-branch detection head to achieve multi-task collaborative learning of classification, localization and segmentation, and using a composite loss function for end-to-end optimization;

[0035] The model inference and prediction module is configured to perform the following process: inputting the preprocessed image into the trained model, constructing a four-step post-processing process including confidence filtering, non-maximum suppression, mask optimization and coordinate system restoration, and finally generating a structured output for gap recognition that includes bounding boxes and pixel-level masks.

[0036] Meanwhile, the present invention proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in the present invention.

[0037] Furthermore, the present invention proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed, it implements the steps of the method described in the present invention.

[0038] Finally, the present invention provides a computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described in the present invention.

[0039] The present invention adopts the above technical solution and has the following technical effects compared with the prior art:

[0040] (1) In terms of data preprocessing, this invention constructs a systematic multi-stage processing framework, which integrates four major modules: deterministic data partitioning, semi-supervised fine annotation, coordinate transformation standardization and visualization verification. It adopts normalized coordinate transformation to adapt to the input requirements of deep learning models and combines contour visualization mechanism to achieve dual verification of annotation quality. This full-process standardized processing strategy significantly improves the consistency of dataset quality and the stability of model training. In particular, it shows excellent data adaptability and engineering practicality when facing complex crack morphology and multi-scale features.

[0041] (2) In terms of model architecture and training strategy, this invention has constructed a complete technical system integrating lightweight design, multi-scale feature fusion and adaptive loss optimization. By selecting the YOLOv8n-seg lightweight instance segmentation network and fusing the improved CSPDarknet53 backbone network and PANet+SPPF neck network, and combining multi-level spatial transformation, mirror rotation and Mosaic advanced data augmentation strategies, a multi-task loss function optimization mechanism with three branches of classification-detection-segmentation is constructed. This enables accurate capture and segmentation of multi-scale fracture features under limited computing resources, effectively solving a series of technical problems of missed detection of small targets, inaccurate boundary positioning and insufficient model generalization ability in complex geological environments by traditional methods.

[0042] (3) In terms of post-processing optimization, this invention integrates a systematic multi-step processing framework, which integrates confidence filtering, non-maximum suppression and mask post-processing modules into the deep learning inference system. Through low confidence threshold setting and contour optimization strategy, it dynamically adapts to different image resolutions and crack morphology distributions, solving the problems of high false negative rate, inaccurate boundary positioning and insufficient cross-scale generalization ability of traditional methods in complex scenarios.

[0043] (4) This invention constructs a semi-supervised learning framework that integrates the advantages of human prior knowledge and deep learning. It combines refined manual annotation with automated feature learning. The semi-supervised strategy retains the accuracy of manual annotation and gives full play to the generalization ability of deep learning. While ensuring the accuracy of crack boundary positioning, it effectively improves the model's adaptability to different geological conditions and different imaging environments. It solves the industry problem of insufficient generalization performance of traditional methods in scenarios with scarce samples and provides an efficient and reliable crack detection solution for engineering practice. Attached Figure Description

[0044] Figure 1 This invention relates to a flowchart of a crack intelligent identification method based on semi-supervised learning and improved YOLOv8.

[0045] Figure 2 These are the 8 manually labeled training set crack maps involved in the embodiment.

[0046] Figure 3 These are two manually labeled crack maps of the test set involved in the embodiment.

[0047] Figure 4 These are the 10 predicted fracture maps involved in the example. Detailed Implementation

[0048] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Example 1: This example proposes a crack intelligent identification method based on semi-supervised learning and an improved YOLOv8. This method focuses on the intelligent identification and quantitative characterization of cracks in geological scenes, falling within the scope of integrated applications of computer vision, machine learning, and road safety engineering. Considering the balance between computational efficiency and accuracy, and taking into account the specific nature of crack detection tasks and the practical constraints of limited hardware resources, a complete crack detection solution is constructed. Through data preprocessing and format conversion, model training and optimization, and model inference and prediction, intelligent crack identification is ultimately achieved.

[0050] refer to Figure 1 The method includes the following steps:

[0051] Step 1: Data Preprocessing and Format Conversion

[0052] 1. Data collection and partitioning strategies:

[0053] The system collects crack images to construct a structural surface dataset. A rigorous data partitioning strategy is employed, randomly dividing the dataset into a training set and a validation set at an 8:2 ratio. A fixed random seed (RANDOM_SEED=42) is set to ensure consistency in each partition, facilitating experimental reproduction and result comparison.

[0054] 2. Detailed annotation:

[0055] The LabelMe annotation tool was used for semi-supervised manual annotation. Closed polygonal regions were constructed by clicking along the edge of the crack morphology point by point. A unified label name was set for each annotation, and a mapping relationship between the label name and the category ID was established to ensure the consistency of the annotation information. Figure 2 The annotation diagram of the training set is shown. Figure 3 The annotation diagram of the test set is shown.

[0056] 3. Three-stage annotation conversion:

[0057] The first stage involves data parsing and extraction. The JSON annotation file is parsed to obtain image size information and the coordinates of the labeled polygon vertices. All visible crack regions in the image are labeled as independent instances. A single image may contain multiple crack instances, each requiring separate processing.

[0058] The second stage involves coordinate system transformation. A linear transformation maps the pixel coordinate system to a normalized coordinate system. The specific transformation formula is as follows:

[0059] ,

[0060] ,

[0061] in, These are the original pixel coordinates. and These are the width and height of the image, respectively. , These are the normalized pixel coordinates. The normalized coordinate values ​​are strictly limited to the range of (0, 1) to adapt to the relative coordinate system requirements of the YOLO model.

[0062] The third stage involves data format standardization. The output data is organized according to the standard format of the YOLO segmentation task, generating a text file containing category IDs and normalized coordinate sequences, with each fracture instance represented as a single line of data.

[0063] 4. Contour visualization verification mechanism:

[0064] OpenCV's drawing functions are used to draw contours based on raw integer coordinates (unnormalized), fully preserving the original contour information for subsequent visualization verification. A green contour line (2 pixels thick) is drawn on the original image to generate a debug image for manual verification of the conversion's correctness. When processing each data partition, image file copying, label format conversion, and visualization image generation operations are performed simultaneously.

[0065] Step 2: Model Training and Optimization

[0066] 1. Model architecture selection:

[0067] The lightweight instance segmentation model YOLOv8n-seg was selected as the basic model architecture. The pre-trained weights of this model provide general feature extraction capabilities learned on large datasets, laying a solid foundation for crack detection tasks.

[0068] 2. Configure the training parameter system:

[0069] Configure a complete training parameter system, including basic parameters and data augmentation strategies.

[0070] The basic training parameters include setting the number of training epochs, image size, and batch size. Specifically, the number of training epochs is set to 500 to ensure that the model converges fully; the image size is 640×640 pixels to balance detail preservation and computational efficiency; and the batch size is 4 to achieve the optimal trade-off between gradient stability and memory efficiency.

[0071] During training, various data augmentation techniques are employed, which are achieved through affine transformations. The data augmentation strategies include spatial transformations, mirror rotations, and advanced augmentations. Spatial transformations include rotation, translation, and scaling; mirror rotations include horizontal and vertical flips.

[0072] (1) Spatial transformation:

[0073] The rotation angle of random rotation is Randomly selected from within the range, the transformation matrix is:

[0074] ,

[0075] in The rotation angle is in radians.

[0076] Randomly translate in the horizontal and vertical directions, the translation amount is... Within the range of the image size, the transformation matrix is:

[0077] ,

[0078] in ; ; It is the balance ratio coefficient, usually 0.1 (representing 10%). It is the width of the image (in pixels); It is the height of the image (pixel value); These are random numbers that are uniformly distributed within an interval, which allows for randomization of the translation direction and distance.

[0079] The scaling factor of random scaling is Random selection is made within the range. The transformation matrix is:

[0080] ,

[0081] in This is the scaling factor.

[0082] The spatial transformation is specifically set to rotation (±15°), translation (±10%), and scaling (±10%).

[0083] (2) Mirror rotation:

[0084] Flip horizontally flips the image left and right, transforming it into:

[0085]

[0086] Vertical flipping transforms the image vertically, resulting in:

[0087]

[0088] The mirror flip is specifically set to horizontal flip (50%) and vertical flip (50%).

[0089] (3) Advanced Enhancement:

[0090] Advanced enhancement using Mosaic data augmentation (100% enabled) was performed by stitching four images together into one image using a 2×2 grid, while simultaneously adjusting the annotation boxes. Specifically, four images were randomly selected and placed in the top left, top right, bottom left, and bottom right positions, respectively. The size of each image was then adjusted and placed in its corresponding position, while the coordinates of the annotation boxes were also adjusted.

[0091] 3. Model architecture processing:

[0092] Forward propagation passes the input image through the backbone network, neck network, and detection head, and outputs and integrates the final prediction results.

[0093] Input layer processing:

[0094] The model receives fixed-size input images, specifically 640×640 pixel RGB three-channel images. Pixel values ​​undergo two-stage standardization: first, normalization converts the 0-255 pixel value range to a floating-point range of 0-1; then, standardization is performed using statistics from the ImageNet dataset (mean [0.485, 0.456, 0.406], standard deviation [0.229, 0.224, 0.225]) for each channel. This standardization ensures that the input data distribution is consistent with the data distribution during model training, helping to maintain the model's recognition performance. Finally, the system converts the processed image data into PyTorch tensor format and adjusts its dimensions: from a height × width × number of channels layout to a number of channels × height × width layout, while adding a batch dimension to form a 4-dimensional tensor that meets the model's input requirements.

[0095] Backbone network feature extraction:

[0096] The YOLOv8 backbone uses an improved Cross-Stage Network (CSPNet) as its feature extraction framework, addressing the gradient redundancy problem of traditional CNNs through optimized gradient design. It employs an improved CSPDarknet53 as the backbone network, optimizing gradient information flow through cross-stage local connectivity design to generate a multi-scale feature pyramid.

[0097] (1) The spatial size of the P3 feature layer is 80×80, and the number of channels is 256. This layer maintains a high spatial resolution, corresponding to an 8x downsampling factor for the input image. It is good at capturing the details, textures and edge information of cracks, and has excellent detection capabilities for small cracks.

[0098] (2) The spatial size of the P4 feature layer is 40×40, and the number of channels is 512. As an intermediate layer feature, the downsampling factor is 16 times, which achieves a balance between spatial details and semantic information and can identify medium-scale crack structures.

[0099] (3) The spatial size of the P5 feature layer is 20×20, and the number of channels is 1024. This layer has the largest receptive field, a downsampling ratio of 32 times, and contains rich semantic information and contextual features, making it suitable for the overall recognition of large-scale crack areas.

[0100] Neck network feature fusion optimization:

[0101] A combined architecture of PANet (Path Aggregation Network) and SPPF (Spatial Pyramid Pooling Fast) modules is adopted to achieve effective fusion and enhancement of multi-level features.

[0102] (1) The top-down path integrates high-level semantic features (P5) with lower-level features (P4, P3) through upsampling, enabling semantic information to propagate to the shallower layers and enhancing the semantic expressive power of detailed features. This is defined as:

[0103] ,

[0104] in, This represents the enhanced feature map obtained by fusing deep semantics and shallow details; It is responsible for upsampling deep features to match their spatial dimensions with those of shallow features, thereby transferring high-level semantic information (such as object categories and global context) contained in the deep layers to the shallow layers, improving the ability of shallow features to understand high-level semantics while maintaining detail resolution. This indicates element-wise addition or feature concatenation operations; Indicates the first The feature map of the layer, usually Increasing the corresponding network depth increases; Indicates compared to the first A feature map at a deeper level; Indicates the first The result of adjusting the number of channels of layer features by 1×1 convolution is intended to unify the channel dimensions of features at different levels, so as to facilitate subsequent fusion.

[0105] (2) The bottom-up approach fuses the low-level detailed features (P3) with the upper-level features (P4, P5) through downsampling, enabling spatial detail information to be transmitted to deeper layers and improving the localization accuracy of semantic features. This is defined as:

[0106] ,

[0107] in, This indicates the result obtained after bottom-up path fusion. Layer-enhanced feature maps; Shallow features are downsampled to transfer rich spatial details (such as edges, textures and small object features) to deeper layers, effectively compensating for the loss of detail caused by multiple downsampling. Indicates the first The feature map of layer 1 has a higher resolution than that of layer 2. layer; Indicates the first The intermediate feature maps of a layer are usually the results of the top-down path output or the adjusted backbone network features.

[0108] (3) The SPPF module extracts contextual information within different receptive fields through parallel multi-scale pooling operations (kernel sizes of 5×5, 9×9, and 13×13), significantly expanding the model's receptive field while maintaining low computational complexity. The spatial pyramid pooling formula is:

[0109] ,

[0110] in, Indicates the input feature map; This indicates the size of the pooling kernel, with values ​​of 5, 9, and 13, corresponding to the local, mid-range, and global context extraction levels, respectively. Indicates the size used is The max pooling operation maintains a constant output space size through appropriate padding. This indicates that a splicing operation is performed along the channel dimension, fusing the feature maps output by multi-scale pooling in the channel direction.

[0111] After processing by the neck network, an optimized multi-scale feature representation is generated:

[0112] (1) The F3 fusion feature is 80×80×128, which strengthens the combination of detailed information and semantic background, and is particularly suitable for the accurate segmentation of small cracks.

[0113] (2) The F4 fusion feature is 40×40×256, which achieves the best balance between semantic information and spatial details, and serves as the main detection level.

[0114] (3) The F5 fusion feature is 20×20×512, rich in high-level semantic information and global context, and is responsible for the identification of large-scale crack regions.

[0115] Multi-task output of the detection head:

[0116] The detection head is based on the anchor point mechanism and performs multi-task learning in parallel at three feature levels to achieve end-to-end instance segmentation.

[0117] (1) The classification prediction branch is responsible for determining whether each candidate region contains a crack target and outputting a category confidence score. Multi-label classification is implemented using the Sigmoid activation function, which can simultaneously handle the identification of multiple crack instances. The Sigmoid activation function can map the original linear output to a probability-based confidence score, as shown in the formula:

[0118] ,

[0119] in, This represents the output after Sigmoid activation, indicating the prediction confidence that the candidate region belongs to the corresponding category; The original linear output (logits) of the classification branch typically corresponds to the score of a candidate region (such as an anchor box or target proposal) on a certain category. Let represent the Sigmoid function, which maps any real number to the interval (0, 1), thus converting it into an interpretable confidence probability.

[0120] This branch optimizes based on binary cross-entropy loss to ensure accurate identification of cracks. It uses BCELoss (binary cross-entropy loss) as the classification loss. For multi-class classification, it uses the sigmoid activation function instead of softmax, allowing each class to be classified independently. The formula is:

[0121] ,

[0122] in, It is the sample size; It is a real label, 0 or 1; It is the predicted probability, output by the Sigmoid function.

[0123] (2) Bounding box regression branch: accurately locate the specific spatial position of the crack in the image, and output the coordinate parameters of the bounding box, including the coordinates of the center point. ,width and height .use The (Complete Intersection over Union) loss serves as the optimization objective for bounding box regression. Losses in tradition Based on this, the overlapping area, center point distance, and aspect ratio consistency are also considered, thus providing a more comprehensive and stable guide for accurate bounding box regression. The loss function is defined as follows:

[0124] ,

[0125] in, The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. ; This represents the Euclidean distance between the center points of two bounding boxes; The diagonal length of the smallest bounding rectangle that covers both the predicted and ground truth boxes; It is a weighting function that balances different loss terms. It is a metric that measures the consistency of aspect ratio between the predicted bounding box and the ground truth bounding box.

[0126] (3) Instance mask branch: Generate pixel-level binary segmentation masks to accurately depict the contour shape of the cracks. Based on the combination mechanism of prototype mask and instance coefficients, a set of shared prototype masks is first learned, then specific combination coefficients are predicted for each detected instance, and finally instance-specific segmentation results are generated through matrix multiplication. The prototype mask mechanism is defined as follows:

[0127] ,

[0128] in, Indicates the first A shared prototype mask; For the first The instance corresponds to the first The design incorporates the combination coefficients of a prototype mask. While maintaining a lightweight model, it achieves flexible modeling and high-precision segmentation of cracks with different morphologies.

[0129] For segmentation tasks, a combined optimization strategy of Dice loss and binary cross-entropy loss is adopted to effectively handle the foreground-background class imbalance problem. The composite segmentation loss is typically composed of binary cross-entropy and Dice loss, and is defined as follows:

[0130] ,

[0131] Dice loss focuses on overlapping regions, which is beneficial for handling class imbalance problems. It is defined as:

[0132] ,

[0133] in, Indicates the first The actual label value of each pixel (takes 0 or 1); It is the probability that the model predicts the pixel as positive. It is a smoothing term that prevents division by zero.

[0134] The definition of the split binary cross-entropy is:

[0135] .

[0136] 4. Loss Function System

[0137] YOLOv8's loss function typically consists of three parts: classification loss, bounding box loss, and segmentation loss (for segmentation tasks). The total loss is a weighted sum of these loss terms:

[0138] ,

[0139] in, These are weighting coefficients used to balance the importance of different loss terms.

[0140] Step 3: Model Inference and Prediction

[0141] The preprocessed image is input into the trained YOLOv8n-seg model, and the original prediction output is obtained through forward propagation. Then, the prediction results output by the model are subjected to systematic post-processing operations, including confidence filtering, NMS (Non-Maximum Suppression), and mask post-processing, etc., to finally generate accurate and reliable crack detection and segmentation results.

[0142] (1) Confidence filtering:

[0143] During forward inference, the model generates a large number of candidate bounding boxes, many of which are low-confidence false positives or background noise. To improve detection efficiency and accuracy, a preliminary screening is first performed based on a set confidence threshold (set to 0.1). The mathematical expression for confidence filtering is defined as:

[0144] ,

[0145] This threshold setting is relatively low to ensure that difficult samples such as those with small cracks and blurred edges are not missed, while effectively filtering out obviously erroneous detection results. This step significantly reduces the computational burden of subsequent processing, allowing for optimized allocation of system resources.

[0146] (2) NMS (Non-Maximum Suppression):

[0147] For the remaining detection boxes after confidence filtering, there may be multiple highly overlapping predictions pointing to the same crack instance. The NMS algorithm uses an iterative filtering mechanism to ensure that only the most representative detection box is retained for each physical crack. The specific steps are as follows: First, the detection boxes are sorted in descending order of confidence; then, the detection box with the highest confidence is iteratively selected as the benchmark, and its IoU (Intersection Over Union) with the remaining detection boxes is calculated and converted into a probability value. If the IoU exceeds a set threshold of 0.5, the detection box is suppressed. This process is repeated until all detection boxes have been processed. The formula for calculating IoU overlap is:

[0148] ,

[0149] in, Indicates the predicted bounding box; This represents the baseline bounding box. The IoU value ranges from [0,1]. A larger value indicates a higher degree of spatial overlap between the two detection boxes, and a greater likelihood that they point to the same target.

[0150] The NMS filtering criteria are:

[0151] ,

[0152] in, This indicates traversing all bounding boxes except the current reference box. In addition, the indices of all other detection boxes to be compared; This is the preset overlap threshold.

[0153] (3) Post-mask processing:

[0154] For instance segmentation tasks, the model outputs a pixel-level segmentation mask for each detected crack instance. The original mask prediction may contain jagged edges, internal holes, or noise interference, requiring fine post-processing to improve visual quality and measurement accuracy. The specific steps are as follows: First, apply the mask probability map output by the model... The function is then used to obtain the boundary contour of the binary mask using OpenCV's contour lookup algorithm. Polygon approximation or morphological operations are applied to smooth the contour curve and eliminate irregular jagged edges. Subsequently, morphological closing operations are used to fill the small holes inside the mask, while area filtering is used to remove isolated noise pixel regions.

[0155] (4) Coordinate system restoration:

[0156] Since the input image undergoes size normalization and center padding preprocessing during the model inference stage, all prediction results (bounding box coordinates and segmentation mask) are based on the preprocessed image coordinate system. To obtain accurate detection results corresponding to the original input image, an inverse coordinate transformation must be performed. Specifically, the following steps are taken: First, based on the recorded scaling ratio and padding offset, the detection box coordinates are mapped back from the preprocessed image space to the original image space. Then, through inverse scaling and cropping operations, the segmentation mask is adjusted from the standard size (640×640) to the original image size. Finally, it is ensured that the restored detection results correspond precisely to the original image in spatial location, avoiding geometric distortion introduced by coordinate transformation.

[0157] After the above systematic post-processing workflow, the original predictions generated by the model are transformed into structured and visualized final outputs.

[0158] Figure 4 The images shown are 10 training results of the YOLOv8n-seg model. The YOLOv8n-seg model exhibits excellent fracture identification performance in complex geological environments.

[0159] Example 2: This example proposes a crack intelligent identification system based on semi-supervised learning and improved YOLOv8, characterized by including:

[0160] The data preprocessing and format conversion module is configured to perform the following processes: acquiring fracture images; using the LabelMe tool to perform semi-supervised fine-grained annotation on the fracture images; achieving data standardization through a three-stage conversion process, including parsing JSON annotation files, performing normalized coordinate mapping, and generating YOLO format label files; and establishing a contour visualization verification mechanism to ensure annotation quality.

[0161] The model training and optimization module is configured to perform the following process: by constructing an improved YOLOv8n-seg network architecture, integrating multiple data augmentation strategies and multi-scale feature fusion mechanisms, designing a three-branch detection head to achieve multi-task collaborative learning of classification, localization and segmentation, and using a composite loss function for end-to-end optimization;

[0162] The model inference and prediction module is configured to perform the following process: inputting the preprocessed image into the trained model, constructing a four-step post-processing process including confidence filtering, non-maximum suppression, mask optimization and coordinate system restoration, and finally generating a structured output for gap recognition that includes bounding boxes and pixel-level masks.

[0163] Example 3: This example proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method as described in this invention.

[0164] Example 4: This example proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed, it implements the steps of the method described in this invention.

[0165] Example 5: The present invention proposes a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the method described in the present invention.

[0166] It should be noted that the processing flow of embodiments 2-5 corresponds to the specific steps of the method provided in embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the method provided in embodiment 1 of the present invention.

[0167] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0168] The specific implementation schemes described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific implementation schemes of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present invention should fall within the scope of protection of the present invention.

Claims

1. A crack intelligent recognition method based on semi-supervised learning and improved YOLOv8, characterized in that, Includes the following steps: Data preprocessing and format conversion: Crack images were acquired, and the LabelMe tool was used to perform semi-supervised fine-grained annotation on the crack images. Data standardization was achieved through a three-stage conversion process, including parsing JSON annotation files, performing normalized coordinate mapping, and generating YOLO format label files; a contour visualization verification mechanism was established to ensure annotation quality. Model Training and Optimization: First, a lightweight instance segmentation model is selected as the foundation, and multiple data augmentation strategies including spatial geometric transformation, mirror flipping, and advanced Mosaic enhancement are configured. Second, a multi-scale feature pyramid is generated based on the CSPDarknet53 backbone network, and feature fusion is achieved through PANet and SPPF neck networks to construct a network architecture based on YOLOv8n-seg. Then, a three-branch detection head is designed to achieve multi-task collaborative learning of classification, localization, and segmentation, and a composite loss function is used for end-to-end optimization. Model inference and prediction: The preprocessed image is input into the trained model, and a four-step post-processing process is constructed, including confidence filtering, non-maximum suppression, mask optimization and coordinate system restoration, which finally generates a structured output for gap recognition containing bounding boxes and pixel-level masks.

2. The method according to claim 1, characterized in that, The process of using the LabelMe tool to perform semi-supervised, refined annotation of the crack image specifically includes: Construct closed polygonal regions by clicking point by point along the edges of the crack shapes in the image; Set a uniform label name for each label and establish a mapping relationship between the label name and the category ID.

3. The method according to claim 1, characterized in that, The data standardization process achieved through a three-stage transformation includes: Parse the JSON annotation file to obtain image size information and coordinates of the vertices of the annotated polygons, and perform independent instance annotations on all visible crack regions in the image; The pixel coordinate system is mapped to the normalized coordinate system through a linear transformation. The specific transformation formula is as follows: ； ； in, These are the original pixel coordinates. and These are the width and height of the image, respectively. , These are the normalized pixel coordinates, and the normalized coordinate values ​​are strictly limited to the range (0, 1). The output data is organized according to the standard format of the YOLO segmentation task, generating a text file containing category IDs and normalized coordinate sequences, with each fracture instance represented as a single line of data.

4. The method according to claim 1, characterized in that, A three-branch detection head is designed to achieve multi-task collaborative learning of classification, localization, and segmentation. End-to-end optimization using a composite loss function is employed, specifically including: The design includes a three-branch detection head comprising a classification branch, a bounding box regression branch, and an instance mask branch, and optimizes it using Sigmoid activation, CIoU loss, and a composite loss of Dice and binary cross-entropy respectively. A multi-task loss function is constructed using a weighted combination of classification loss, bounding box loss, and segmentation loss, and the network is trained and optimized end-to-end.

5. The method according to claim 1, characterized in that, The multi-scale feature pyramid generated based on the CSPDarknet53 backbone network is The gradient information flow is optimized by designing cross-stage local connectivity, and feature pyramids with three different scales, P3, P4 and P5, are generated to capture the detailed texture and edge information of cracks, the medium-scale crack structure and the large-scale crack region, respectively.

6. The method according to claim 1, characterized in that, The construction process includes a four-step post-processing workflow: confidence filtering, non-maximum suppression, mask optimization, and coordinate system restoration. The final generated structured output for gap recognition, containing bounding boxes and pixel-level masks, specifically includes: All candidate prediction boxes output by the model are initially screened based on a set confidence threshold of 0.1 to filter out low-confidence detection results; The candidate boxes after screening are sorted in descending order of confidence, the overlap is calculated using the intersection-union ratio, and redundant boxes are suppressed based on an IoU threshold of 0.5 to ensure that only one optimal detection box is retained for each crack instance. For each preserved instance, the segmentation mask probability map is sequentially activated by the Sigmoid function, optimized for contours, and closed by morphological operations to smooth the boundaries, fill the internal holes, and remove noise. The post-processed bounding box coordinates and segmentation mask are inversely mapped back to the coordinate system of the original input image based on the scaling and padding parameters during preprocessing, generating a final structured output that is spatially aligned with the original image.

7. A crack intelligent recognition system based on semi-supervised learning and improved YOLOv8, characterized in that, include: The data preprocessing and format conversion module is configured to perform the following processes: acquiring fracture images, using the LabelMe tool to perform semi-supervised fine-grained annotation on the fracture images, achieving data standardization through a three-stage conversion process, including parsing JSON annotation files, performing normalized coordinate mapping, and generating YOLO format label files; and establishing a contour visualization verification mechanism to ensure annotation quality. The model training and optimization module is configured to perform the following process: First, a lightweight instance segmentation model is selected as the base, and multiple data augmentation strategies, including spatial geometric transformation, mirror flipping, and advanced Mosaic enhancement, are configured. Second, a multi-scale feature pyramid is generated based on the CSPDarknet53 backbone network, and feature fusion is achieved through PANet and SPPF neck networks to construct a network architecture based on YOLOv8n-seg. Then, a three-branch detection head is designed to achieve multi-task collaborative learning of classification, localization, and segmentation, and end-to-end optimization is performed using a composite loss function. The model inference and prediction module is configured to perform the following process: inputting the preprocessed image into the trained model, constructing a four-step post-processing process including confidence filtering, non-maximum suppression, mask optimization and coordinate system restoration, and finally generating a structured output for gap recognition that includes bounding boxes and pixel-level masks.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed, it implements the steps of the method as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 6.

Images