Indoor ice surface defect detection method based on deep learning

Abstract

The invention discloses an ice surface defect detection and quantification system and method based on deep learning. The system comprises a data acquisition module, a data enhancement module, a neural network training module, a binocular stereoscopic vision-based detection module and an information fusion and analysis module. The method comprises the following steps: firstly, acquiring image data of ice surface defects by using various camera devices through a data acquisition module, and covering different illumination conditions and defect types; and then, the data enhancement module performs rotation, overturning, contrast adjustment and noise addition on the acquired data so as to improve data diversity and generalization ability of the neural network. And then, a multi-scale feature extraction module (MSTG-Net) and a detail enhancement convolution module (MSDSC) are introduced into a neural network training module, and the segmentation feature extraction of the ice surface defect is realized by training the improved YOLOv8 neural network. The detection module based on binocular stereo vision acquires an ice surface image through a binocular camera, and obtains defect mark information and a depth map in combination with the trained neural network model. And finally, further determining the three-dimensional position, the surface area size and the spatial morphological characteristics of the defect by combining the information fusion and analysis module with a binocular stereoscopic vision distance measurement result. According to the method, high-precision identification and positioning of ice surface defects are realized, reliable data support is provided for subsequent ice surface repair planning, and the method has the advantages of high precision, strong robustness, wide applicability and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and specifically relates to a binocular ice surface defect detection system based on multi-scale feature enhancement. By combining a multi-scale feature extraction network (MSTG-Net), a detail enhancement convolution module (MSDSC) and a lightweight shared detail enhancement detection head (DSEE-Head), the accuracy and efficiency of ice surface defect detection are significantly improved. Background Art

[0002] With the widespread use of indoor ice rinks, timely detection and repair of ice surface defects have become an important task to ensure the safety of athletes. However, traditional methods based on manual detection have problems such as low efficiency and high missed detection rate. Although traditional image processing methods can partially solve this problem, they still have certain limitations for complex optical reflections on the ice surface and the diversity of defect types. With the rapid development of deep learning technology, the target detection method based on convolutional neural networks has shown high accuracy and real-time performance. However, in the actual application of ice surface defect detection, due to the multi-scale characteristics of defects and factors such as ice surface reflection and ice debris interference, the existing deep learning models still need to be improved in detection accuracy.

[0003] In order to solve the above problems, the present invention proposes an ice surface defect detection system based on an improved YOLOv8 network architecture. By combining a multi-scale feature enhancement network (MSTG-Net), a detail enhancement convolution module (MSDSC) and a lightweight shared detail enhancement detection head (DSEE-Head), the overall performance of defect detection is significantly improved. Summary of the invention

[0004] The present invention aims to provide an efficient target detection system based on deep learning, which can realize accurate detection and area calculation of ice surface defects in indoor ice rinks by introducing a multi-scale feature extraction module (MSTG-Net), a detail enhancement convolution module (MSDSC), and a lightweight shared detail enhancement detection head (DSEE-Head) in combination with binocular vision technology.

[0005] To achieve the above object, the present invention provides the following technical solution: an efficient target detection system based on deep learning, comprising:

[0006] Data acquisition module: used to collect image data, including obtaining images of ice surface defects from multiple data sources such as cameras and surveillance videos.

[0007] Data enhancement module: Enhance the collected images by performing processing such as rotation, flipping, contrast adjustment, and noise addition to simulate the state of the ice surface under different lighting and humidity conditions and increase the diversity of the data set.

[0008] Neural network training module: Build a deep learning network based on the improved YOLOv8, introduce a multi-scale feature extraction module (MSTG-Net), a detail enhancement convolution module (MSDSC), a lightweight shared detail enhancement detection head (DSEE-Head), etc.

[0009] Among them, MSTG-Net is used to extract the global and detail features of ice surface defects through multi-scale convolution features; MSDSC realizes the extraction of detail features through a variety of convolution kernels; DSEE-Head is used to improve the model's ability to capture details in defect detection.

[0010] Detection module based on binocular stereo vision: The ice surface image is obtained through a binocular camera, input into a trained model, and combined with parallax calculation to obtain the depth information of the defect, thereby improving the accuracy of defect positioning.

[0011] Information fusion and analysis module: Fusion of depth information with neural network segmentation results to accurately calculate the location, depth and area of ​​defects, providing data support for ice rink maintenance.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] Efficient multi-scale feature extraction and detail enhancement detection:

[0014] The introduction of the multi-scale feature extraction network (MSTG-Net) and the detail enhancement convolution module (MSDSC) effectively enhances the model's ability to capture details of ice surface defects and improves detection accuracy. Specifically, multi-scale convolution kernels (such as 1×1, 3×3, and 5×5) extract features of multiple scales through different receptive fields, ensuring the model's ability to detect large-scale defects and small cracks.

[0015] For example, through the MSDSC module, the feature refinement after convolution of the feature map can be expressed by the following formula:

[0016]

[0017] Among them, F input t is the input feature map, Conv i is the convolution operation of different scales, w i is the fusion weight coefficient. This feature fusion process improves the model's sensitivity to defects of different sizes, thereby significantly enhancing the detail detection effect.

[0018] Binocular stereo vision combined with deep learning:

[0019] The binocular camera system can accurately locate ice surface defects, and combined with the neural network segmentation results, the area and volume of the defects can be calculated. The specific area calculation formula is:

[0020]

[0021] Where A is the total area of ​​the defect, p i Represents the mask value of the i-th pixel, A pixel The actual area size corresponding to each pixel.

[0022] The calculation of depth information is based on the disparity map of the binocular camera, and the depth of the defect can be determined by the following formula:

[0023]

[0024] Where D is the depth, f is the focal length of the camera, B is the baseline distance of the binocular camera, and d is the parallax. Using this formula, the system can accurately reconstruct the three-dimensional shape of the defect, ensuring that the measurement error of the defect position, area and depth is within ±0.5mm.

[0025] High robustness:

[0026] The system of the present invention can maintain high detection accuracy under different lighting conditions and complex environments through the combination of data enhancement and multi-scale feature extraction, and adapt to various ice surface defect detection tasks. In order to solve the problems of ice surface reflection and blurred defect edges, the model realizes the effective fusion of low-level features (such as edge information) and high-level features (such as semantic information) through the feature pyramid network (FPN), thereby improving the detection robustness and accuracy.

[0027] Under different lighting conditions, the model's detection accuracy fluctuation remained within ±1.2 percentage points, demonstrating the system's adaptability in complex environments.

[0028] Real-time:

[0029] The introduction of the lightweight shared detail enhancement detection head (DSEE-Head) enables the model to significantly reduce computational overhead while maintaining high accuracy. DSEE-Head reduces the number of parameters by sharing convolution kernels and enhances key features through detail convolution. Compared with traditional methods, the model inference time is reduced by about 20%, and the time to process a 1080p image is shortened to 30ms, which can be applied to real-time monitoring and detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0031] Figure 1This is a system flow chart of the present invention, illustrating the complete process of data collection, data enhancement, neural network training, detection based on binocular stereo vision, information fusion and analysis.

[0032] Figure 2 It is a hardware structure application of the present invention, which is installed on a mobile device to realize a rapid detection function.

[0033] Figure 3 Figure 1 is the module structure diagram of MSTG-Net (Multi-Scale Tensor Grouping Network). MSTG-Net aims to improve the detection accuracy of ice surface defects by neural networks, especially the defect feature extraction capability at different scales. Ice surface defects have various forms, including different depths, sizes and texture features, so the feature extraction module is required to have the ability to capture and integrate multi-scale information. MSTG-Net divides the input feature map into multiple tensor groups of different sizes, and then applies multi-scale convolution operations to each group of tensors to ensure that information at each scale can be effectively extracted and represented. The input image feature map of the MSTG-Net module is divided into several tensor groups, each of which corresponds to different scale features. Convolution kernels of 1×1, 3×3, 5×5 and 7×7 are applied to each tensor group, respectively, and features of different scales are captured through the diversity of convolution kernels. The convolution results of all tensor groups are spliced ​​and fused to form a joint expression of multi-scale features. This multi-scale feature extraction method can effectively improve the model's detection performance for ice surface defects, especially in identifying tiny cracks and large-area depressions.

[0034] Figure 4 The figure is the structure diagram of the MSDSC (Multi-Scale Detail Shared Convolution) module. The core of the MSDSC module is the detail enhancement convolution, which focuses on enhancing the edges and details of ice surface defects through shared multi-scale convolution operations. Defects on the ice surface usually have blurred boundaries or subtle features, and traditional convolution layers are difficult to capture these details. MSDSC combines detail features of different scales by sharing multi-scale convolutions to enhance the sensitivity to defect details. The MSDSC module uses different dilations and shared convolution kernels to achieve detail enhancement. Convolution operations with different dilation rates can capture more contextual information, thereby enhancing the details of the defect edges. The sharing mechanism of convolution kernels effectively reduces the number of parameters, improves the computational efficiency and lightweight of the model, and ensures the ability to capture details. Finally, by integrating and activating feature maps at different dilation rates, the MSDSC module can find a balance between computational efficiency and detection accuracy, which is particularly suitable for detecting ice surface defects with complex edges.

[0035] Figure 5 This is the structural diagram of the DSEE-Head (Detail Shared Enhanced Detection Head) module. DSEE-Head is a detection head module based on a lightweight shared structure, which is specifically used to enhance the ability to capture the boundaries and details of ice surface defects. Since ice surface defects often have subtle differences in detection, DSEE-Head introduces a shared detail enhancement convolution (DEConv-GN) layer to strengthen the segmentation of edges and small defects in the final detection stage of the model. DSEE-Head uses a lightweight shared detail convolution structure (DEConv-GN) to reduce redundant parameters and improve detection efficiency. The detection head performs multiple convolutions and feature enhancements on the input feature map through a shared detail convolution layer, so that the final output detection result is more responsive to the defect edge. In addition, DSEE-Head uses a dynamic feature scale (Scale) module to adapt to different defect sizes and shapes, ensuring that the model has good performance in ice surface defect detection of various sizes. Through detail enhancement and lightweight design, DSEE-Head not only significantly improves the detection accuracy of ice surface defects, but also ensures that the model can run efficiently with lower computing resources, making it suitable for real-time application scenarios. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Example: Figure 1 As shown, the present invention provides an efficient ice surface defect detection system based on deep learning, comprising the following modules:

[0038] Data acquisition module: The data acquisition module is used to collect ice surface image data from cameras, surveillance videos and other equipment at the ice rink. The collected images cover a variety of scenes such as different lighting, humidity and ice surface conditions. The ice surface is collected through a binocular camera to ensure the diversity of image data and the acquisition of three-dimensional information.

[0039] Extension of implementation method: In an optional embodiment, the data acquisition module can collect ice surface images from different cameras (including visible light cameras and infrared cameras), so as to obtain high-quality images under different lighting conditions, ensuring the accuracy of subsequent defect detection. By using a binocular camera system to obtain images, not only can the planar image information of the ice surface be collected, but also the depth information of the ice surface can be obtained through stereo matching, which is very critical for three-dimensional measurement and volume calculation in subsequent defect analysis.

[0040] The core of binocular camera stereo matching is to use the disparity between the left and right cameras to calculate the depth of the scene. Assuming that the baseline length of the two cameras is B, the focal length is f, and the coordinate difference (disparity) of the left and right camera imaging planes is d, the depth Z of a point can be calculated by the following formula:

[0041]

[0042] in:

[0043] Z: The distance from the object point to the camera (depth).

[0044] f: Focal length of the camera, usually obtained from the camera's intrinsic parameter matrix.

[0045] B: The baseline length between the binocular cameras, that is, the physical distance between the two cameras.

[0046] d: Disparity, which is the lateral pixel offset of corresponding points in the left and right camera images.

[0047] Through the above formula, the system can calculate the depth information of each pixel on the ice surface, thereby generating a depth map. Combining the depth map with the two-dimensional segmentation information, the ice surface defects can be measured in three dimensions, including the depth, area, volume and other characteristics of the defects.

[0048] For some more complex ice surface defects, obtaining the volume information of the defects is an important basis for ice surface repair. Assuming that the depth map of the defects has been obtained through binocular matching, the depth in each defect area can be integrated to calculate its volume:

[0049] V = ∫∫ Ω Z(x,y)dxdy

[0050] in:

[0051] V: volume of the defect.

[0052] Ω: The area range of the defect.

[0053] Z(x,y): Depth value of the defect area in the depth map.

[0054] Data enhancement module: The data enhancement module performs various enhancement processes on the collected images, including rotation, flipping, adding noise, adjusting contrast, and other operations to expand the ice surface defect dataset, simulate different environments and lighting conditions, and improve the robustness of the model.

[0055] Extension of implementation method: In an optional embodiment, the data enhancement module can also apply pixel-level enhancement techniques, such as contrast enhancement, brightness adjustment, adding pseudo-random noise, etc., to simulate various possible ice surface scenes and improve the adaptability of the deep learning model to complex scenes. By adding image noise and performing data perturbation, the model can learn how to accurately detect defects in noisy data, thereby enhancing the performance of the system in a real environment.

[0056] Neural network training module: The neural network training module is responsible for training the deep learning model based on the improved YOLOv8 architecture. The networks used include the multi-scale feature extraction module (MSTG-Net), the detail enhancement convolution module (MSDSC), and the lightweight shared detail enhancement detection head (DSEE-Head). MSTG-Net is responsible for extracting features of ice surface defects from different scales, MSDSC is used to enhance the capture of detail features through multiple convolution kernels, and DSEE-Head improves the accuracy of target detection through detail convolution enhancement.

[0057] Neural network training module: The neural network training module is used to train deep learning models based on the improved YOLOv8 architecture. This module integrates the multi-scale feature extraction module (MSTG-Net), the detail enhancement convolution module (MSDSC), and the lightweight shared detail enhancement detection head (DSEE-Head) to improve the detection accuracy of ice surface defects.

[0058] Multi-scale feature extraction module (MSTG-Net): The MSTG-Net module is designed to extract features of ice surface defects at different scales, and capture defect information of different sizes through a combination of multiple convolution kernels (such as 1×1, 3×3, and 5×5). For ice surface defects, there are usually cracks as small as a few millimeters, as well as cracks and depressions as large as tens of centimeters. MSTG-Net covers these different size intervals by setting the receptive fields of different convolution kernels. 1×1 convolution is used to capture local fine information, 3×3 convolution is used for medium-scale feature extraction, and 5×5 convolution is responsible for capturing larger-scale defect features, thereby ensuring that the network has multi-scale perception capabilities when detecting ice surface defects.

[0059] Detail Enhanced Convolution Module (MSDSC): The MSDSC module enhances the capture of detail features through a variety of convolution kernels, and is particularly suitable for the detection of irregular defects and blurred edge areas on the ice surface. The MSDSC module uses Detail-Enhanced Convolution (DECConv), which focuses on improving the model's attention to high-frequency detail information. For small cracks (usually in the range of 1mm to 5mm in size) and defects with blurred edges, MSDSC ensures that high-frequency information is enhanced by superimposing multiple refinement convolution operations, allowing the model to accurately detect small defects with blurred edges and irregularities.

[0060] Lightweight Shared Detail Enhanced Detection Head (DSEE-Head): The DSEE-Head module improves the accuracy of target detection through detail convolution enhancement, especially for defect areas on the ice surface with complex edges and multi-scale features. DSEE-Head adopts a lightweight shared convolution architecture and uses the same convolution kernel to process feature maps of different scales to reduce redundant calculations and improve detection efficiency. At the same time, the introduction of detail enhancement convolution makes the model more sensitive to the edge information of different defects, especially when detecting small defects (scale less than 10mm), which can significantly improve the accuracy.

[0061] By training the improved YOLOv8 architecture through the neural network training module, the system can adaptively learn multi-scale features and perform well in processing large-area cracks (such as more than 10cm) and small-scale depressions (such as 1mm to 10mm). The combination of each module ensures that the network can not only focus on the overall structure of large defects, but also effectively detect tiny details, thereby improving the overall accuracy and robustness of ice surface defect detection.

[0062] Multi-scale feature extraction module (MSTG-Net) implementation process:

[0063] Input feature grouping: First, the input feature map is divided into multiple different sub-feature groups, each group representing a different scale.

[0064] Multi-convolution kernel convolution operation: Convolution kernels of different sizes are applied to each sub-feature group, including 1×1, 3×3, and 5×5 convolution kernels, to capture information at different scales. The 1×1 convolution kernel is used to fuse information between channels, and the 3×3 convolution kernel and the 5×5 convolution kernel are used to capture more extensive spatial information and context.

[0065] Feature fusion: By using the cross-scale feature fusion mechanism, the convolution results of each scale are concatenated or weighted summed to generate a multi-scale feature map.

[0066] Output results: Finally, the fused feature map is used for subsequent defect identification and segmentation.

[0067] Detail Enhancement Convolution Module (MSDSC)

[0068] Implementation process:

[0069] Detail-sensitive convolutional layer: MSDSC introduces a set of convolutional layers with different dilation rates, using convolution kernels with dilation rates of 1, 3, and 5. This can expand the receptive field while maintaining feature resolution and obtain more spatial detail information.

[0070] Feature concatenation and integration: After the convolution operation is completed, the obtained feature maps are merged through concatenation, and then the channel information is integrated through 1×1 convolution to generate an enhanced feature map.

[0071] Output feature map: The enhanced feature map is input into the downstream detection network for defect segmentation.

[0072] Lightweight Shared Detail Enhanced Detection Head (DSEE-Head)

[0073] Implementation process:

[0074] Convolution module sharing: DSEE-Head uses a lightweight shared convolution module (Shared Convolution Module) to perform multiple detail enhancement convolution operations on the input features. This sharing mechanism processes the same feature layer layer by layer through multiple shared convolution kernels.

[0075] Detail-Enhanced Convolution Operation: A set of detail-enhanced convolution layers (DEConv) are used, which are optimized to pay special attention to the edge parts and local details of the features. For each detection layer, by applying multiple detail convolution operations, the model can more accurately capture the shape and edge features of the defect.

[0076] Classification and regression output: The feature map after shared detail enhancement is then input into the classification and regression network for the final defect classification and location output.

[0077] The three modules MSTG-Net, MSDSC, and DSEE-Head optimize the YOLOv8 network from three aspects: multi-scale feature extraction, detail enhancement, and lightweight detection head:

[0078] MSTG-Net helps capture features of different scales, allowing the model to maintain high accuracy when faced with complex and diverse defects.

[0079] MSDSC enhances the detail resolution of feature maps, making edge detection more accurate.

[0080] DSEE-Head reduces the computational overhead of the detection head and improves the inference speed through the lightweight design of shared detail convolution, thus ensuring the needs of real-time detection.

[0081] The synergy of these three modules enables the improved YOLOv8 network to have higher detection accuracy, boundary segmentation accuracy, and fast real-time performance in indoor ice surface defect detection.

[0082] Extension of implementation methods: In a specific embodiment, the neural network training module uses a labeled ice surface defect dataset to train the model and optimizes the model parameters to minimize the loss function. In order to improve the performance of the model, optimization algorithms such as stochastic gradient descent (SGD) and Adam are used, and techniques such as learning rate decay and early stopping are combined during the training process to prevent overfitting and improve the generalization ability of the model. MSTG-Net uses multi-scale feature extraction to enable the model to effectively capture global and local features when facing ice surface defects of different sizes, thereby improving the comprehensiveness of detection. The MSDSC module captures the detailed features of defects by using convolution kernels of various sizes. DSEE-Head reduces the computational burden of the network while improving the sensitivity of detection to details.

[0083] Detection module based on binocular stereo vision: This module obtains ice surface images through a binocular camera system, uses parallax calculation to obtain depth information of ice surface defects, and inputs it into a trained neural network model to extract and mark the characteristic areas of ice surface defects, thereby achieving three-dimensional positioning of ice surface defects.

[0084] Extension of implementation method: In an optional embodiment, the detection module based on binocular vision realizes the spatial positioning of ice surface defects through stereo matching technology. Binocular vision obtains the depth information of the defect area by calculating the parallax between the left and right camera images, so as to further calculate the volume and area of ​​the defect. Through the combination of depth information, the system can not only obtain the two-dimensional outline of the defect, but also accurately infer the depth of the defect and realize the stereo analysis of three-dimensional defects, which is of great reference value for ice surface repair work.

[0085] Information fusion and analysis module: The information fusion and analysis module integrates the depth information obtained by the binocular camera with the segmentation results of the neural network to achieve a comprehensive analysis of ice surface defects, including area calculation, position confirmation, depth measurement, etc. The final results are used to provide scientific data support for the maintenance and repair of the ice rink.

[0086] Area calculation: For the area calculation of ice surface defects, first obtain the binary mask image of the defect through the segmentation result of the neural network. In the mask, each pixel represents a part of the ice surface defect. The area of ​​the defect is calculated by the following formula:

[0087]

[0088] Where A is the area of ​​the defect, N is the number of defective pixels in the mask, and M i represents the value of the i-th pixel (0 or 1), Δx and Δy represent the width and height of the pixel in the actual physical space (in millimeters or centimeters). Through the calibration of the binocular camera, the pixel space can be mapped to the actual space to ensure the accuracy of the area calculation.

[0089] Position confirmation: The position of the defect is confirmed by matching the left and right images obtained by the binocular camera, and combining the depth map information to determine the three-dimensional coordinates (X, Y, Z) of the defect. The core of position confirmation is to use the disparity calculation principle of the binocular camera, obtain the depth information of the object through the disparity map, and combine the internal and external parameters of the camera to determine the spatial position of the defect. The calculation formula is:

[0090]

[0091] Among them, Z is the depth of the defect relative to the camera, f is the focal length of the camera, B is the baseline distance of the binocular camera, and d is the disparity (the pixel difference between corresponding points in the left and right images). After determining the depth Z, the image coordinates are converted to world coordinates through the camera calibration matrix to obtain the three-dimensional position (X, Y, Z) of the defect.

[0092] Depth measurement: Through the depth map generated by binocular vision, the system can accurately measure the depth of the defect. Depth measurement is used to evaluate the depth of the defect, that is, the distance from the surface of the ice to the bottom of the defect. The depth measurement formula is:

[0093] D=Z max -Z min

[0094] Where D is the depth of the defect, Z max and Z min The maximum and minimum depth values ​​in the defect area are represented respectively. In this way, the depth of the defect can be quantified to help ice rink maintenance personnel determine the ice thickness and amount of filling material required for repair work.

[0095] Through the above information fusion and analysis modules, the system can fully obtain the area, location and depth information of ice surface defects, thereby providing detailed quantitative data support for ice rink maintenance and repair work. These data can be used to formulate accurate repair plans and improve the efficiency and effectiveness of ice surface maintenance.

[0096] Extension of implementation method: In an optional embodiment, the information fusion and analysis module matches and fuses the defect area segmented by the neural network with the depth information obtained by binocular vision, and adopts weighted average or depth optimization method to further improve the positioning accuracy of defects and the reliability of three-dimensional measurement. Through precise positioning and depth analysis, the system can output the location, size, shape and depth information of defects in real time and generate a complete defect analysis report. In addition, the detection results can also be displayed through a graphical user interface (GUI), which is convenient for managers to comprehensively monitor and maintain the status of the ice rink.

[0097] Summary of implementation effects:

[0098] Improved accuracy: The improved YOLOv8 model has improved the mean average precision (mAP) of ice surface defect detection by 2 percentage points, especially in the identification of complex defects such as small cracks and dents. In experiments on edge detection and detail recognition, the detection accuracy has increased by 1.8 percentage points, significantly enhancing the system's ability to identify defect edges and details.

[0099] Efficiency improvement and manual comparison: Compared with traditional manual detection methods, the improved YOLOv8 model has significant efficiency advantages. Manual detection of ice surface defects usually requires experienced staff to check one by one on the ice rink. The whole process is time-consuming and easily affected by subjective factors such as light and angle. In contrast, the improved YOLOv8 model is based on an automated detection process, which can cover the entire ice rink in a short time and provide detection results in real time. In addition, due to the lightweight model structure and parameter optimization, the introduction of DSEE-Head has further increased the model inference speed by 15%, ensuring real-time detection in an environment with complex ice surface defects.

Claims

1. The ice surface defect detection method based on improved YOLOv8 deep learning is characterized by: include, A data acquisition module, used to collect image data of the indoor ice rink; The data enhancement module preprocesses and enhances the collected image data, including operations such as rotation, flipping, contrast adjustment, and noise addition, to improve the diversity of data and the generalization ability of the neural network, thereby improving the robustness of the detection model under different environmental conditions; A neural network training module is used to build a network model based on the improved YOLOv8 architecture, including a multi-scale feature extraction module (MSTG-Net), a detail enhancement convolution module (MSDSC), and a lightweight shared detail enhancement detection head (DSEE-Head) to perform feature segmentation and extraction of ice surface defects; ①MSTG-Net module: MSTG-Net (Multi-Scale Tensor Grouping Network) aims to improve the detection accuracy of ice surface defects by neural networks, especially the defect feature extraction capability at different scales. Ice surface defects have various forms, so the feature extraction module is required to have the ability to capture and integrate multi-scale information. MSTG-Net divides the input feature map into multiple tensor groups of different sizes, and then applies multi-scale convolution operations to each group of tensors to ensure that information at each scale can be effectively extracted and represented. The input image feature map of the MSTG-Net module is divided into several tensor groups, each of which corresponds to different scale features. Convolution kernels of 1×1, 3×3, 5×5 and 7×7 are applied to each tensor group, respectively, and features of different scales are captured through the diversity of convolution kernels. The convolution results of all tensor groups are spliced ​​and fused to form a joint expression of multi-scale features. This multi-scale feature extraction method can effectively improve the detection performance of the model for ice surface defect segmentation and extraction, especially in identifying tiny cracks and large-area depressions. ②MSDSC module: The core of the MSDSC (Multi-Scale Detail Shared Convolution) module is detail enhancement convolution, which focuses on enhancing the edges and details of ice surface defects through shared multi-scale convolution operations. Defects on the ice surface usually have blurred boundaries or subtle features, and traditional convolution layers have difficulty capturing these details. MSDSC combines detail features of different scales by sharing multi-scale convolutions to enhance sensitivity to defect details. The MSDSC module uses different dilations and shared convolution kernels to achieve detail enhancement. Convolution operations with different dilation rates can capture more contextual information, thereby enhancing the details of the defect edges. The sharing mechanism of convolution kernels effectively reduces the number of parameters, improves the computational efficiency and lightweight of the model, and ensures the ability to capture details. Finally, by integrating and activating feature maps at different dilation rates, it is particularly suitable for detecting ice surface defects with complex edges. ③DSEE-Head detection head: DSEE-Head (Detail Shared Enhanced Detection Head) is a detection head module based on a lightweight shared structure, which is specially used to enhance the ability to capture the boundaries and details of ice surface defects. Since ice surface defects often have subtle differences in detection, DSEE-Head introduces a shared detail enhanced convolution (DEConv_GN) layer to enhance the segmentation of edges and small defects in the final detection stage of the model. DSEE-Head uses a lightweight shared detail convolution structure (DEConv_GN) to reduce redundant parameters and improve detection efficiency. The detection head performs multiple convolutions and feature enhancements on the input feature map through a shared detail convolution layer, making the final output detection result more responsive to the defect edge. In addition, DSEE-Head uses a dynamic feature scale module to adapt to different defect sizes and shapes, ensuring that the model performs well in ice surface defect detection of various sizes. Through detail enhancement and lightweight design, DSEE-Head not only significantly improves the detection accuracy of ice surface defects, but also ensures that the model can run efficiently with lower computing resources, making it suitable for real-time application scenarios. The detection module based on binocular stereo vision is used to obtain ice surface images through binocular cameras and combine them with the trained neural network model to detect defect features and extract depth information. The depth information of defects is obtained through the parallax of the binocular camera, and the location and size of ice surface defects can be accurately identified by combining with the improved YOLOv8 model; The information fusion and analysis module is used to fuse the depth information of binocular vision with the segmentation features to calculate the three-dimensional position and area information of ice surface defects. The segmentation results of the neural network and the depth information of binocular vision are combined to achieve accurate positioning and area calculation of ice surface defects.

2. The ice surface defect detection system according to claim 1, characterized in that: The data acquisition module can obtain ice surface images from multiple data sources such as cameras and surveillance videos to ensure the comprehensiveness and diversity of ice surface defect detection data.

3. The ice surface defect detection system according to claim 1, characterized in that: The data enhancement module expands the data by simulating different environmental conditions such as lighting, humidity and angle to enhance the detection robustness and generalization ability of the model.

4. The ice surface defect detection system according to claim 1, characterized in that: The information fusion and analysis module accurately locates and calculates the area of ​​ice surface defects by combining the neural network segmentation results with the depth information of binocular vision, so as to provide accurate data support in the subsequent ice surface repair process.

5. The ice surface defect detection system according to claim 1, characterized in that: The detection result output module can display the location, depth and area information of the defect in a graphical manner, or output the detection results to other systems for further processing to achieve real-time monitoring and maintenance management.

Images