A road pavement crack segmentation method and device based on improved YOLOv8

Abstract

The application discloses a road pavement crack segmentation method and device based on an improved YOLOv8, and relates to the field of image processing.The method comprises the following steps: constructing a road pavement crack segmentation model based on the improved YOLOv8, wherein a spectral fusion convolution layer is used to replace a two-dimensional convolution layer in a backbone network of a YOLOv8 network, a full-scale pyramid pooling module is used to replace an SPPF module in the backbone network of the YOLOv8 network, and a cross-scale fusion network is used in a neck network; acquiring a road pavement image to be detected and inputting the image to the trained road pavement crack segmentation model, performing feature extraction on the image through the backbone network to obtain multi-scale image features, inputting the multi-scale image features into the neck network, performing feature fusion on the features through the cross-scale fusion network to obtain cross-scale fusion features, inputting the cross-scale fusion features into a head network, and obtaining segmentation results of each crack in the features. The application solves the problems of complex background, large crack scale difference and detail loss in the road pavement crack segmentation task.

Description

Technical Field

[0001] This invention relates to the field of image processing, and specifically to a method and apparatus for road surface crack segmentation based on an improved YOLOv8. Background Technology

[0002] Road surface quality monitoring is crucial for ensuring road safety and maintaining transportation infrastructure. Road surfaces are continuously subjected to various adverse factors during daily use, including heavy traffic flow, vehicle load, temperature fluctuations, humidity, and weathering. These factors can lead to surface defects such as longitudinal cracks, transverse cracks, and potholes. Such damage not only accelerates road surface deterioration but also affects driving comfort, increases vehicle operating costs, and poses significant safety hazards. The continuous pressure of heavy traffic and frequent extreme weather events accelerate the formation and evolution of road surface cracks. The cost difference between early repair and later major overhauls is significant, making the need for road surface crack segmentation technology increasingly urgent in the engineering field. Therefore, developing timely, accurate, and efficient detection and segmentation methods is essential for reliable road surface condition assessment, which is crucial for infrastructure protection, public safety, and extending road surface lifespan.

[0003] Currently, object detection solves the problem of locating and classifying multiple objects in an image. However, methods for detecting the distribution path, shape, and density of cracks in targets cannot provide high-precision detection information about these cracks. To measure cracks more accurately, they must be detected at the pixel level. Segmentation aims to identify target objects at the pixel level. Pixel segmentation is divided into instance segmentation (i.e., pixel-level segmentation of each individual target object, mainly used in complex scenes with multiple target objects of the same type) and semantic segmentation (i.e., pixel-level segmentation of all target objects of the same type, mainly used to identify a small number of target objects). However, crack detection methods based on semantic segmentation have certain limitations, such as only providing accurate information about the location and extent of cracks, but not distinguishing between different cracks. Due to insufficient segmentation performance and small amount of available data, semantic segmentation methods cannot completely detect some particularly subtle cracks. Instance segmentation can solve this problem well by specifying a unique and individual process to distinguish cracks and obtain boundary information. However, instance segmentation has insufficient ability to extract differential features of multi-scale defects (especially small-scale and large-scale defects); its ability to represent defects that require deep semantic features (such as longitudinal and transverse cracks) is limited, resulting in low accuracy of instance segmentation. Furthermore, road surface crack segmentation is more likely to be difficult due to complex backgrounds, large differences in crack scales, and loss of details. Summary of the Invention

[0004] The purpose of this application is to propose a road surface crack segmentation method and device based on an improved YOLOv8 to address the aforementioned technical problems.

[0005] In a first aspect, the present invention provides a road surface crack segmentation method based on an improved YOLOv8, comprising the following steps:

[0006] A road surface crack segmentation model based on an improved YOLOv8 was constructed and trained to obtain a trained road surface crack segmentation model. The road surface crack segmentation model consists of a backbone network, a neck network, and a head network connected in sequence. In the backbone network, a spectral fusion convolutional layer is used to replace the two-dimensional convolutional layer in the YOLOv8 backbone network, a full-scale pyramid pooling module is used to replace the SPPF module in the YOLOv8 backbone network, and a cross-scale fusion network is used in the neck network.

[0007] The road surface image to be detected is acquired and input into the trained road surface crack segmentation model. First, the backbone network is used for feature extraction to obtain multi-scale image features. The multi-scale image features are input into the neck network and then fused through the cross-scale fusion network to obtain cross-scale fused features. The cross-scale fused features are input into the head network to obtain the segmentation result of each crack in the road surface image to be detected.

[0008] Preferably, the spectrum fusion convolutional layer includes a channel partitioning module, a first extraction path, a second extraction path, a third extraction path, a fourth extraction path, a first fusion path, a second fusion path, a first splicing layer, a first convolutional layer, a first batch of normalization layers, and a first SiLU activation function layer; the first fusion path includes a second SiLU activation function layer, and the second fusion path includes a third SiLU activation function layer;

[0009] The input features of the spectral fusion convolutional layer are first divided into high-frequency features and low-frequency features according to different numbers of channels by the channel segmentation module;

[0010] The high-frequency features are input into the first extraction path and the second extraction path respectively to obtain the first path features and the second path features;

[0011] The low-frequency features are input into the third extraction path and the fourth extraction path respectively to obtain the third path features and the fourth path features;

[0012] The first path features and the third path features are input into the first fusion path, where they are first weighted and then passed through the second SiLU activation function layer to obtain the first fusion features.

[0013] The second path features and the fourth path features are input into the second fusion path, first weighted, then passed through the third SiLU activation function layer, and then bilinear interpolation is performed to obtain the second fusion features.

[0014] The first and second fusion features are processed through the first splicing layer to obtain the spliced ​​features. The spliced ​​features are then processed through the first convolutional layer and the first batch of normalization layers to obtain the intermediate features.

[0015] The input features of the spectral fusion convolutional layer are residually concatenated with the intermediate features and then input into the first SiLU activation function layer to obtain the output features of the spectral fusion convolutional layer.

[0016] Preferably, the first extraction path includes a second convolutional layer and a second batch normalization layer connected in sequence; the second extraction path includes a first average pooling layer, a third convolutional layer and a third batch normalization layer connected in sequence; the third extraction path includes a fourth convolutional layer, a fourth batch normalization layer and an upsampling module connected in sequence; and the fourth extraction path includes a fifth convolutional layer and a fifth batch normalization layer connected in sequence.

[0017] Preferably, the full-scale pyramid pooling module includes a sixth convolutional layer, a seventh convolutional layer, a first processing branch, a second processing branch, a third processing branch, a fourth processing branch, a fifth processing branch, a sixth processing branch, a seventh processing branch, a second concatenation layer, a grouped convolutional layer, an adaptive weight calculation module, an eighth convolutional layer, and a ninth convolutional layer. The sixth, eighth, and ninth convolutional layers each use a 1×1 kernel, while the seventh convolutional layer uses a 3×3 kernel. The first processing branch preserves the original features, and the second, third, fourth, and fifth processing branches are respectively the first max pooling layer, the second max pooling layer, and the third max pooling layer. The system consists of a third max pooling layer, a second average pooling layer, and a third max pooling layer. The first, second, and third max pooling layers employ max pooling operations with kernel sizes of 3×3, 5×5, and 7×7, respectively. The second average pooling layer employs an average pooling operation with a kernel size of 5×5. The sixth and seventh processing branches are a depthwise separable convolutional layer and a dilated convolutional layer, respectively. The adaptive weight calculation module includes an adaptive average pooling layer, a tenth convolutional layer, a sixth batch normalization layer, a ReLU activation function layer, an eleventh convolutional layer, a seventh batch normalization layer, and a Sigmoid activation function layer, connected in sequence. The tenth and eleventh convolutional layers employ convolution operations with a kernel size of 1×1.

[0018] The input features of the full-scale pyramid pooling module are sequentially passed through the sixth and seventh convolutional layers to obtain enhanced features, as shown in the following equation:

[0019] ;

[0020] in, This represents the input features of the full-scale pyramid pooling module. and These represent convolution operations with kernel sizes of 1×1 and 3×3, respectively. Indicates enhanced features;

[0021] The enhanced features are processed in parallel through the first, second, third, fourth, fifth, sixth, and seventh processing branches to obtain the first, second, third, fourth, fifth, sixth, and seventh processed features, as shown in the following formula:

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] The first processing feature is the enhancement feature. , , , These represent max pooling operations with pooling kernels of 3×3, 5×5, and 7×7, respectively. This indicates an average pooling operation with a 5×5 pooling kernel. Represents depthwise convolution. This represents pointwise convolution. Indicates dilated convolution. , , , , and These represent the second, third, fourth, fifth, sixth, and seventh processing features, respectively.

[0029] The first, second, third, fourth, fifth, sixth, and seventh processing features are sequentially processed through the second splicing layer and the grouped convolutional layer to obtain the third fused feature.

[0030] The third fusion feature is input into the adaptive weight calculation module to obtain the adaptive weight. The third fusion feature and the adaptive weight are multiplied channel by channel to obtain the fourth fusion feature. The fourth fusion feature and the input feature of the full-scale pyramid pooling module are then residually connected after passing through the eighth and ninth convolutional layers to obtain the output feature of the full-scale pyramid pooling module, as shown in the following formula:

[0031] ;

[0032] in, Indicates the third fusion feature, This indicates an adaptive average pooling layer. This indicates a batch normalization operation. This indicates multiplication by channel. express Activation function express Activation function This represents the output characteristics of the full-scale pyramid pooling module.

[0033] Preferably, the head network includes a first detection head, a second detection head, and a third detection head arranged in parallel, and the multi-scale image features include the output features of the full-scale pyramid pooling module, the output features of the third C2f module in the backbone network, and the output features of the second C2f module in the backbone network.

[0034] Preferably, the cross-scale fusion network includes 3 upsampling modules, 11 spectral fusion convolutional layers, 4 C2f modules, and 11 splicing layers. The output features of the full-scale pyramid pooling module in the backbone network are input to the first upsampling module in the cross-scale fusion network to obtain the first feature; the output features of the full-scale pyramid pooling module in the backbone network are input to the first spectral fusion convolutional layer in the cross-scale fusion network to obtain the second feature; the second feature is then passed through the second upsampling module and the second spectral fusion convolutional layer to obtain the third feature; the output features of the third C2f module in the backbone network are input to the third spectral fusion convolutional layer to obtain the fourth feature; the third and fourth features are then input to the cross-scale fusion layer. The first concatenation layer in the network yields the fifth feature; the fifth feature and the first feature are input into the second concatenation layer in the cross-scale fusion network to obtain the sixth feature; the sixth feature passes through the first C2f module in the cross-scale fusion network to obtain the seventh feature; the seventh feature passes through the third upsampling module in the cross-scale fusion network to obtain the eighth feature; the output feature of the second C2f module in the backbone network is input into the fourth spectral fusion convolutional layer in the cross-scale fusion network to obtain the ninth feature; the ninth feature and the third feature pass through the third concatenation layer in the cross-scale fusion network to obtain the tenth feature; the tenth feature and the eighth feature are input into the fourth concatenation layer in the cross-scale fusion network to obtain the eleventh feature; the eleventh feature is then... After passing through the second C2f module and the fifth spectral fusion convolutional layer in the cross-scale fusion network, the twelfth feature is obtained. The twelfth and ninth features are input into the fifth stitching layer of the cross-scale fusion network to obtain the thirteenth feature. The thirteenth feature is input into the first detection head to obtain the segmentation result of the first size. The twelfth feature passes through the sixth spectral fusion convolutional layer in the cross-scale fusion network to obtain the fourteenth feature. The seventh and twelfth features are input into the sixth stitching layer of the cross-scale fusion network to obtain the fifteenth feature. The fifteenth feature passes through the seventh spectral fusion convolutional layer in the cross-scale fusion network to obtain the sixteenth feature. The sixteenth and fourteenth features are input into the seventh stitching layer of the cross-scale fusion network to obtain... The seventeenth feature is passed through the third C2f module and the eighth spectral fusion convolutional layer in the cross-scale fusion network to obtain the eighteenth feature. The eighteenth feature and the fourth feature are input into the eighth stitching layer in the cross-scale fusion network to obtain the nineteenth feature. The nineteenth feature is input into the second detection head to obtain the segmentation result of the second size. The eighteenth feature and the output feature of the full-scale pyramid pooling module in the backbone network are input into the ninth stitching layer in the cross-scale fusion network to obtain the twentieth feature. The twentieth feature is passed through the ninth spectral fusion convolutional layer in the cross-scale fusion network to obtain the twenty-first feature. The eighteenth feature is passed through the tenth spectral fusion convolutional layer in the cross-scale fusion network to obtain the twenty-second feature.The twenty-second and twenty-first features are input into the tenth stitching layer of the cross-scale fusion network to obtain the twenty-third feature. The twenty-third feature then passes through the fourth C2f module and the eleventh spectral fusion convolutional layer of the cross-scale fusion network to obtain the twenty-fourth feature. The twenty-fourth feature and the second feature are input into the eleventh stitching layer of the cross-scale fusion network to obtain the twenty-fifth feature. The twenty-fifth feature is then input into the third detection head to obtain the segmentation result for the third size.

[0035] Secondly, the present invention provides a road surface crack segmentation device based on an improved YOLOv8, comprising:

[0036] The model building module is configured to build and train a road surface crack segmentation model based on an improved YOLOv8, resulting in a trained road surface crack segmentation model. The road surface crack segmentation model includes a backbone network, a neck network, and a head network connected in sequence. In the backbone network, the road surface crack segmentation model replaces the two-dimensional convolutional layers in the YOLOv8 network with spectral fusion convolutional layers, replaces the SPPF module in the YOLOv8 network with a full-scale pyramid pooling module, and employs a cross-scale fusion network in the neck network.

[0037] The prediction module is configured to acquire the road surface image to be detected and input it into the trained road surface crack segmentation model. First, the backbone network performs feature extraction to obtain multi-scale image features. The multi-scale image features are input into the neck network and then fused through the cross-scale fusion network to obtain cross-scale fused features. The cross-scale fused features are input into the head network to obtain the segmentation result of each crack in the road surface image to be detected.

[0038] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0039] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0040] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The road surface crack segmentation method based on improved YOLOv8 proposed in this invention replaces the two-dimensional convolutional layer in the backbone network of the existing YOLOv8 network with a spectrum fusion convolutional layer, thereby enhancing the feature extraction capability of the road surface crack segmentation model by utilizing high-frequency details and low-frequency overall information.

[0043] (2) The road surface crack segmentation method based on the improved YOLOv8 proposed in this invention replaces the SPPF module in the backbone network of the existing YOLOv8 network with a full-scale pyramid pooling module, thereby expanding the receptive field of the model, capturing a wider range of contextual information, and thus enhancing the ability of the road surface crack segmentation model to capture features of different sizes.

[0044] (3) The road surface crack segmentation method based on the improved YOLOv8 proposed in this invention adopts a cross-scale fusion network in the neck network of the road surface crack segmentation model. Through dense cross-layer connections and multi-feature fusion, it enhances semantic understanding while maintaining details, thereby improving the segmentation accuracy of the road surface crack segmentation model for images. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic flowchart illustrating an embodiment of the road surface crack segmentation method based on the improved YOLOv8 in this application.

[0047] Figure 2 This is a schematic diagram of the structure of a road surface crack segmentation model based on the improved YOLOv8 road surface crack segmentation method according to an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the spectral fusion convolutional layer in an embodiment of the road surface crack segmentation method based on the improved YOLOv8, according to an embodiment of this application.

[0049] Figure 4 This is a schematic diagram of the full-scale pyramid pooling module of the road surface crack segmentation method based on the improved YOLOv8, as an embodiment of this application.

[0050] Figure 5 This is a schematic diagram of a road surface crack segmentation device based on the improved YOLOv8, as an embodiment of this application.

[0051] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0053] Figure 1 This application illustrates an embodiment of a road surface crack segmentation method based on an improved YOLOv8, comprising the following steps:

[0054] S1. Construct and train a road surface crack segmentation model based on the improved YOLOv8 to obtain the trained road surface crack segmentation model. The road surface crack segmentation model includes a backbone network, a neck network, and a head network connected in sequence. In the backbone network, the road surface crack segmentation model replaces the two-dimensional convolutional layer in the backbone network of the YOLOv8 network with a spectral fusion convolutional layer, replaces the SPPF module in the backbone network of the YOLOv8 network with a full-scale pyramid pooling module, and adopts a cross-scale fusion network in the neck network.

[0055] For details, please refer to Figure 2The road surface crack segmentation model mentioned in the embodiments of this application is an improvement on the existing YOLOv8 network. The specific improvements are mainly in the backbone and neck networks, while the head network adopts the existing YOLOv8 network head. In the backbone network, the road surface crack segmentation model in the embodiments of this application replaces the standard two-dimensional convolutional layer in the backbone network of the existing YOLOv8 network with a SpectraFusion Convolutional Layer (SFC), and replaces the SPPF module in the backbone network of the existing YOLOv8 network with an OmniScale Pyramid Pooling (OSPP) module. After the replacement, the backbone network of the road surface crack segmentation model includes a SpectraFusion Convolutional Layer, four feature extraction units, and an OmniScale Pyramid Pooling module connected in sequence. Each feature extraction unit includes a SpectraFusion Convolutional Layer and a C2f module connected in sequence. In the neck network portion, the road surface crack segmentation model in the embodiments of this application replaces the optimized PA-FPN (Path Aggregation Network-Feature Pyramid Network) used in the neck network of the existing YOLOv8 network with a CrossScale-FusionNet, and also replaces the standard two-dimensional convolutional layer in the neck network with a spectral fusion convolutional layer.

[0056] The following section details the structure of the spectrum fusion convolutional layer, the full-scale pyramid pooling module, and the cross-scale fusion network.

[0057] In a specific embodiment, the spectrum fusion convolutional layer includes a channel partitioning module, a first extraction path, a second extraction path, a third extraction path, a fourth extraction path, a first fusion path, a second fusion path, a first splicing layer, a first convolutional layer, a first batch of normalization layers, and a first SiLU activation function layer; the first fusion path includes a second SiLU activation function layer, and the second fusion path includes a third SiLU activation function layer;

[0058] The input features of the spectral fusion convolutional layer are first divided into high-frequency features and low-frequency features according to different numbers of channels by the channel segmentation module;

[0059] The high-frequency features are input into the first extraction path and the second extraction path respectively to obtain the first path features and the second path features;

[0060] The low-frequency features are input into the third extraction path and the fourth extraction path respectively to obtain the third path features and the fourth path features;

[0061] The first path features and the third path features are input into the first fusion path, where they are first weighted and then passed through the second SiLU activation function layer to obtain the first fusion features.

[0062] The second path features and the fourth path features are input into the second fusion path, first weighted, then passed through the third SiLU activation function layer, and then bilinear interpolation is performed to obtain the second fusion features.

[0063] The first and second fusion features are processed through the first splicing layer to obtain the spliced ​​features. The spliced ​​features are then processed through the first convolutional layer and the first batch of normalization layers to obtain the intermediate features.

[0064] The input features of the spectral fusion convolutional layer are residually concatenated with the intermediate features and then input into the first SiLU activation function layer to obtain the output features of the spectral fusion convolutional layer.

[0065] For details, please refer to Figure 3 The spectral fusion convolutional layer consists of a channel partitioning module, a first extraction path, a second extraction path, a third extraction path, a fourth extraction path, a first fusion path, a second fusion path, a first concatenation layer, a first batch of normalization layers, and a first SiLU activation function layer. The first convolutional layer uses a 3×3 kernel, the first SiLU activation function layer uses the SiLU activation function, and the channel partitioning module is used to process the input features of the spectral fusion convolutional layer. Based on the number of channels, features are categorized into high-frequency and low-frequency characteristics, as shown in the following formula:

[0066] ;

[0067] ;

[0068] in, The number of channels for the input features of the spectral fusion convolutional layer. Indicates rounding down. The proportion of low-frequency channels, This refers to the number of high-frequency channels. This refers to the number of low-frequency channels. In one example, The default value is 0.25.

[0069] Therefore, the input features of the spectral fusion convolutional layer are determined according to the number of high-frequency channels and the number of low-frequency channels. Classified into high-frequency features and low frequency characteristics Among them, high-frequency features The number of channels is Low-frequency characteristics The number of channels is The high-frequency features will be analyzed later. and low frequency characteristics Input them into the four extraction paths respectively.

[0070] In a specific embodiment, the first extraction path includes a second convolutional layer and a second batch normalization layer connected in sequence; the second extraction path includes a first average pooling layer, a third convolutional layer, and a third batch normalization layer connected in sequence; the third extraction path includes a fourth convolutional layer, a fourth batch normalization layer, and an upsampling module connected in sequence; and the fourth extraction path includes a fifth convolutional layer and a fifth batch normalization layer connected in sequence.

[0071] Specifically, in the embodiments of this application, four extraction paths are used to extract features from four paths respectively. The four extraction paths are the first extraction path, the second extraction path, the third extraction path, and the fourth extraction path. The first extraction path consists of a second convolutional layer and a second batch normalization layer; the second extraction path consists of a first average pooling layer, a third convolutional layer, and a third batch normalization layer; the third extraction path consists of a fourth convolutional layer, a fourth batch normalization layer, and an upsampling module; and the fourth extraction path consists of a fifth convolutional layer and a fifth batch normalization layer. The second, third, fourth, and fifth convolutional layers all use a 1×1 kernel for convolution; the first average pooling layer uses a 2×2 kernel for average pooling; the second, third, fourth, and fifth batch normalization layers all use batch normalization; and the upsampling module uses an upsampling operation.

[0072] The specific formulas for the four extraction paths are as follows:

[0073] First extraction path: ;

[0074] Second extraction path: ;

[0075] Third extraction path: ;

[0076] Fourth extraction path: ;

[0077] in, This represents a convolution operation with a kernel size of 1×1. This indicates the average pooling operation. This indicates a batch normalization operation. Indicates an upsampling operation; , , and These represent the first path feature, the second path feature, the third path feature, and the fourth path feature, respectively. Among them, the first and second path features are high-frequency components, while the third and fourth path features are low-frequency components.

[0078] The four path features output from the four extraction paths are then input into the first and second fusion paths. The first fusion path includes weighted processing and a second SiLU activation function layer, while the second fusion path includes weighted processing, a third SiLU activation function layer, and bilinear interpolation. Specifically, both the second and third SiLU activation function layers use the SiLU activation function. Intra-group fusion is performed on high-frequency and low-frequency components separately, using weighted processing. The fused features are all processed by the SiLU activation function, with the low-frequency components undergoing scaling via bilinear interpolation after activation to obtain the first and second fused features. Finally, the first and second fused features obtained after the above processing are concatenated to obtain the concatenated features. These concatenated features are then processed sequentially through the first convolutional layer and the first batch of normalization layers, followed by a residual connection operation with the input features of the spectral fusion convolutional layer. Finally, the output features of the spectral fusion convolutional layer are output through the SiLU activation function, as shown in the following formula:

[0079] High-frequency component fusion: ;

[0080] Low-frequency component fusion: ;

[0081] ;

[0082] in, and These represent the first fusion feature and the second fusion feature, respectively. and These represent high-frequency weights and low-frequency weights, respectively, both of which are learnable weights. This indicates a convolution operation with a kernel size of 3×3. This represents the bilinear interpolation operation. This indicates a splicing operation. This represents the SiLU activation function. This represents the output features of the spectral fusion convolutional layer. This spectral fusion convolutional layer utilizes high-frequency details and low-frequency overall information to enhance the model's feature extraction capabilities.

[0083] In a specific embodiment, the full-scale pyramid pooling module includes a sixth convolutional layer, a seventh convolutional layer, a first processing branch, a second processing branch, a third processing branch, a fourth processing branch, a fifth processing branch, a sixth processing branch, a seventh processing branch, a second concatenation layer, a grouped convolutional layer, an adaptive weight calculation module, an eighth convolutional layer, and a ninth convolutional layer. The sixth, eighth, and ninth convolutional layers each employ a 1×1 kernel, while the seventh convolutional layer employs a 3×3 kernel. The first processing branch preserves the original features, and the second, third, fourth, and fifth processing branches are respectively the first max pooling layer, the second max pooling layer, the third max pooling layer, the fourth max pooling layer, the fifth max pooling layer, and the sixth max pooling layer. The system consists of a first max pooling layer, a third max pooling layer, and a second average pooling layer. The first, second, and third max pooling layers use max pooling operations with kernels of 3×3, 5×5, and 7×7, respectively. The second average pooling layer uses an average pooling operation with a kernel of 5×5. The sixth and seventh processing branches are depthwise separable convolutional layers and dilated convolutional layers, respectively. The adaptive weight calculation module includes an adaptive average pooling layer, a tenth convolutional layer, a sixth batch normalization layer, a ReLU activation function layer, an eleventh convolutional layer, a seventh batch normalization layer, and a Sigmoid activation function layer connected in sequence. The tenth and eleventh convolutional layers use convolution operations with a kernel size of 1×1.

[0084] The input features of the full-scale pyramid pooling module are sequentially passed through the sixth and seventh convolutional layers to obtain enhanced features, as shown in the following equation:

[0085] ;

[0086] in, This represents the input features of the full-scale pyramid pooling module. and These represent convolution operations with kernel sizes of 1×1 and 3×3, respectively. Indicates enhanced features;

[0087] The enhanced features are processed in parallel through the first, second, third, fourth, fifth, sixth, and seventh processing branches to obtain the first, second, third, fourth, fifth, sixth, and seventh processed features, as shown in the following formula:

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] The first processing feature is the enhancement feature. , , , These represent max pooling operations with pooling kernels of 3×3, 5×5, and 7×7, respectively. This indicates an average pooling operation with a 5×5 pooling kernel. Represents depthwise convolution. This represents pointwise convolution. Indicates dilated convolution. , , , , and These represent the second, third, fourth, fifth, sixth, and seventh processing features, respectively.

[0095] The first, second, third, fourth, fifth, sixth, and seventh processing features are sequentially processed through the second splicing layer and the grouped convolutional layer to obtain the third fused feature.

[0096] The third fusion feature is input into the adaptive weight calculation module to obtain the adaptive weight. The third fusion feature and the adaptive weight are multiplied channel by channel to obtain the fourth fusion feature. The fourth fusion feature and the input feature of the full-scale pyramid pooling module are then residually connected after passing through the eighth and ninth convolutional layers to obtain the output feature of the full-scale pyramid pooling module, as shown in the following formula:

[0097] ;

[0098] in, Indicates the third fusion feature, This indicates an adaptive average pooling layer. This indicates a batch normalization operation. This indicates multiplication by channel. express Activation function express Activation function This represents the output characteristics of the full-scale pyramid pooling module.

[0099] For details, please refer to Figure 4The full-scale pyramid pooling module consists of a sixth convolutional layer, a seventh convolutional layer, a first processing branch, a second processing branch, a third processing branch, a fourth processing branch, a fifth processing branch, a sixth processing branch, a seventh processing branch, a second concatenation layer, a grouped convolutional layer, an adaptive weight calculation module, an eighth convolutional layer, and a ninth convolutional layer. The first processing branch preserves the original features. The second, third, and fourth processing branches employ the first, second, and third max-pooling layers, respectively, with kernels of 3×3, 5×5, and 7×7. The fifth processing branch uses a second average pooling layer with a 5×5 kernel. The sixth and seventh processing branches employ depthwise separable convolutional (DW) layers and dilated convolutional layers, respectively. The depthwise separable convolutional layer is composed of sequentially connected depthwise convolutional layers and pointwise convolutional layers. The sixth and seventh convolutional layers use convolution operations with kernel sizes of 1×1 and 3×3, respectively; the eighth and ninth convolutional layers both use convolution operations with kernel size of 1×1.

[0100] First, the input features of the full-scale pyramid pooling module are... The data is sequentially fed into the sixth and seventh convolutional layers for feature enhancement and extraction to obtain enhanced features. Subsequently, features will be enhanced. The input is fed into the seven processing branches configured in parallel, and the specific calculation formula is as follows:

[0101] First processing branch: Preserve original features ;

[0102] Second processing branch: ;

[0103] Third processing branch: ;

[0104] Fourth processing branch: ;

[0105] Fifth processing branch: ;

[0106] Sixth processing branch: ;

[0107] Seventh processing branch: ;

[0108] After the seven processing branches have completed their respective feature processing, the processed features output by all processing branches are first concatenated, and then the concatenated processed features are fed into the grouped convolutional layer for further processing, finally generating the third fused feature. As shown in the following formula:

[0109] ;

[0110] in, This indicates a grouped convolutional layer.

[0111] The third fusion feature The input is fed into the adaptive weight calculation module to calculate its corresponding adaptive weights. This module consists of an adaptive average pooling layer, a tenth convolutional layer, a sixth batch normalization layer, a ReLU activation function layer, an eleventh convolutional layer, a seventh batch normalization layer, and a Sigmoid activation function layer. Both the tenth and eleventh convolutional layers use 1×1 kernels. The third fusion feature is then... The fourth fused feature is obtained by multiplying it channel-by-channel with its corresponding adaptive weights. After being processed by the eighth convolutional layer, the fourth fused feature is joined with the input feature of the full-scale pyramid pooling module after being processed by the ninth convolutional layer, and finally output as the output feature of the full-scale pyramid pooling module.

[0112] In a specific embodiment, the head network includes a first detection head, a second detection head, and a third detection head arranged in parallel, and the multi-scale image features include the output features of the full-scale pyramid pooling module, the output features of the third C2f module in the backbone network, and the output features of the second C2f module in the backbone network.

[0113] Specifically, after feature extraction by the backbone network, the output features of the full-scale pyramid pooling module, the output features of the third C2f module in the backbone network, and the output features of the second C2f module in the backbone network constitute multi-scale image features, which are then further input into the neck network and feature fusion is performed using a cross-scale fusion network to obtain cross-scale fused features.

[0114] In a specific embodiment, the cross-scale fusion network includes 3 upsampling modules, 11 spectral fusion convolutional layers, 4 C2f modules, and 11 splicing layers. The output features of the full-scale pyramid pooling module in the backbone network are input to the first upsampling module in the cross-scale fusion network to obtain the first feature; the output features of the full-scale pyramid pooling module in the backbone network are input to the first spectral fusion convolutional layer in the cross-scale fusion network to obtain the second feature; the second feature is then passed through the second upsampling module and the second spectral fusion convolutional layer to obtain the third feature; the output features of the third C2f module in the backbone network are input to the third spectral fusion convolutional layer to obtain the fourth feature; the third and fourth features are then input to the cross-scale fusion network. The first concatenation layer in the scale fusion network yields the fifth feature; the fifth feature and the first feature are input into the second concatenation layer in the scale fusion network to obtain the sixth feature; the sixth feature passes through the first C2f module in the scale fusion network to obtain the seventh feature; the seventh feature passes through the third upsampling module in the scale fusion network to obtain the eighth feature; the output feature of the second C2f module in the backbone network is input into the fourth spectral fusion convolutional layer in the scale fusion network to obtain the ninth feature; the ninth feature and the third feature pass through the third concatenation layer in the scale fusion network to obtain the tenth feature; the tenth feature and the eighth feature are input into the fourth concatenation layer in the scale fusion network to obtain the eleventh feature; the eleventh feature... The first feature is obtained by sequentially passing through the second C2f module and the fifth spectral fusion convolutional layer in the cross-scale fusion network. The twelfth feature, along with the ninth feature, is input into the fifth stitching layer of the cross-scale fusion network to obtain the thirteenth feature. The thirteenth feature is then input into the first detection head to obtain the segmentation result of the first size. The twelfth feature passes through the sixth spectral fusion convolutional layer in the cross-scale fusion network to obtain the fourteenth feature. The seventh feature, along with the twelfth feature, is input into the sixth stitching layer of the cross-scale fusion network to obtain the fifteenth feature. The fifteenth feature passes through the seventh spectral fusion convolutional layer in the cross-scale fusion network to obtain the sixteenth feature. The sixteenth feature, along with the fourteenth feature, is input into the seventh stitching layer of the cross-scale fusion network to obtain... The process proceeds to the seventeenth feature; this seventeenth feature is then passed through the third C2f module and the eighth spectral fusion convolutional layer in the cross-scale fusion network to obtain the eighteenth feature; the eighteenth feature and the fourth feature are input into the eighth stitching layer in the cross-scale fusion network to obtain the nineteenth feature; the nineteenth feature is input into the second detection head to obtain the segmentation result of the second size; the eighteenth feature and the output feature of the full-scale pyramid pooling module in the backbone network are input into the ninth stitching layer in the cross-scale fusion network to obtain the twentieth feature; the twentieth feature is passed through the ninth spectral fusion convolutional layer in the cross-scale fusion network to obtain the twenty-first feature; the eighteenth feature is passed through the tenth spectral fusion convolutional layer in the cross-scale fusion network to obtain the twenty-second feature;The twenty-second and twenty-first features are input into the tenth stitching layer of the cross-scale fusion network to obtain the twenty-third feature. The twenty-third feature then passes through the fourth C2f module and the eleventh spectral fusion convolutional layer of the cross-scale fusion network to obtain the twenty-fourth feature. The twenty-fourth feature and the second feature are input into the eleventh stitching layer of the cross-scale fusion network to obtain the twenty-fifth feature. The twenty-fifth feature is then input into the third detection head to obtain the segmentation result for the third size.

[0115] Specifically, to enhance semantic understanding while preserving detail, embodiments of this application design a cross-scale fusion network, replacing the optimized PA-FPN used in the neck network of the existing YOLOv8 network. The cross-scale fusion network mentioned in the embodiments of this application consists of 3 upsampling modules, 11 spectral fusion convolutional layers, 4 C2f modules, and 11 splicing layers. Specific connection relationships are detailed in [reference needed]. Figure 2 , Figure 2In this diagram, C represents the concatenation layer. First, the output features of the full-scale pyramid pooling module in the backbone network are passed through the first upsampling module of the cross-scale fusion network to obtain the first feature. Then, the output features of the full-scale pyramid pooling module in the backbone network are passed through the first spectral fusion convolutional layer (SFC) of the cross-scale fusion network to obtain the second feature. The second feature map is then passed through the second upsampling module and the second SFC of the cross-scale fusion network to obtain the third feature. The third feature is concatenated with the fourth feature obtained by passing the output features of the third C2f module in the backbone network through the third SFC of the cross-scale fusion network to obtain the fifth feature. Finally, the fifth feature is concatenated with the first feature to obtain... The sixth feature is input into the first C2f module of the cross-scale fusion network to obtain the seventh feature. The seventh feature is then passed through the third upsampling module of the cross-scale fusion network to obtain the eighth feature. The output feature of the second C2f module in the backbone network is passed through the fourth SFC of the cross-scale fusion network to obtain the ninth feature. The ninth feature is concatenated with the third feature to obtain the tenth feature. The tenth feature is then concatenated with the eighth feature to obtain the eleventh feature. The eleventh feature is passed through the second C2f module and the fifth SFC of the cross-scale fusion network to obtain the twelfth feature. The twelfth feature is concatenated with the ninth feature to obtain the thirteenth feature, which is then input into the network. The first feature is input into the first detection head P3 to obtain the segmentation result of the first size; the twelfth feature passes through the sixth SFC of the cross-scale fusion network to obtain the fourteenth feature; the twelfth feature is concatenated with the seventh feature to obtain the fifteenth feature; the fifteenth feature passes through the seventh SFC of the cross-scale fusion network to obtain the sixteenth feature; the sixteenth feature is concatenated with the fourteenth feature to obtain the seventeenth feature; the seventeenth feature passes through the third C2f and the eighth SFC of the cross-scale fusion network to obtain the eighteenth feature; the eighteenth feature is concatenated with the fourth feature to obtain the nineteenth feature; the nineteenth feature is input into the second detection head P4 to obtain the segmentation result of the second size; the... The 18th feature is concatenated with the output features of the full-scale pyramid pooling module in the backbone network to obtain the 20th feature. The 20th feature passes through the 9th SFC of the cross-scale fusion network to obtain the 21st feature. The 18th feature passes through the 10th SFC of the cross-scale fusion network to obtain the 22nd feature. The 21st feature is concatenated with the 21st feature to obtain the 23rd feature. The 23rd feature passes through the 4th C2f and 11th SFC of the cross-scale fusion network to obtain the 24th feature. The 24th feature is concatenated with the 2nd feature to obtain the 25th feature. The 25th feature is input into the third detector P5 to obtain the segmentation result of the third size. The 13th, 19th, and 25th features constitute the cross-scale fusion features.

[0116] Compared to the neck network structure of the existing YOLOv8 network, the embodiments of this application additionally incorporate the parts of the backbone network with rich details from the multi-scale images. While effectively preventing the loss of feature details, it further enhances the correlation and complementarity of features at different levels, and improves the richness and robustness of the overall features.

[0117] The road surface crack segmentation model mentioned in the embodiments of this application is trained using the open-source Crack dataset to obtain a trained road surface crack segmentation model. This dataset contains cracks and other defects, and the samples in it correspond to the actual conditions of defect images in real road surface scenarios, exhibiting problems such as complex backgrounds and large differences in crack scale. Using this dataset to train the model can better improve the model's effectiveness in segmenting road surface defects in complex environments.

[0118] The effectiveness of the road surface crack segmentation model mentioned in the embodiments of this application will be illustrated below through specific experiments.

[0119] The importance of each module was further verified through ablation experiments. The results are shown in Table 1. Table 1 shows the ablation experiment results of the road surface crack segmentation model, where Miou is the average pixel crossover ratio.

[0120] Table 1

[0121]

[0122] Experimental results show that without the cross-scale fusion network, the mean intersection-over-union (MIoU) of the model decreases by 1.8%, confirming the crucial role of the cross-scale fusion network in preserving detailed features while enhancing semantic understanding. When neither the spectral fusion convolutional layer nor the cross-scale fusion network is used, the MIoU reduction further expands to 3%, fully demonstrating a significant synergistic effect between the spectral fusion convolutional layer and the cross-scale fusion network. With the spectral fusion convolutional layer, the full-scale pyramid pooling module, and the cross-scale fusion network fully integrated, the model's MIoU reaches 60.2%, a 3.3% improvement compared to the baseline model. This performance breakthrough can effectively improve the accuracy of road defect segmentation tasks in real-world applications.

[0123] S2. Acquire the road surface image to be detected and input it into the trained road surface crack segmentation model. First, the backbone network performs feature extraction to obtain multi-scale image features. The multi-scale image features are input into the neck network and then fused through the cross-scale fusion network to obtain cross-scale fused features. The cross-scale fused features are input into the head network to obtain the segmentation result of each crack in the road surface image to be detected.

[0124] Specifically, the trained road surface crack segmentation model in the embodiments of this application is deployed. After deployment, the road surface image to be detected is input into the trained road surface crack segmentation model, and the segmentation result of each crack can be obtained.

[0125] Further reference Figure 5 As an implementation of the methods shown in the above figures, this application provides an embodiment of a road surface crack segmentation device based on an improved YOLOv8, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0126] This application provides a road surface crack segmentation device based on an improved YOLOv8, comprising:

[0127] Model building module 1 is configured to build and train a road surface crack segmentation model based on an improved YOLOv8, resulting in a trained road surface crack segmentation model. The road surface crack segmentation model includes a backbone network, a neck network, and a head network connected in sequence. In the backbone network, the road surface crack segmentation model replaces the two-dimensional convolutional layer in the YOLOv8 network with a spectral fusion convolutional layer, replaces the SPPF module in the YOLOv8 network with a full-scale pyramid pooling module, and uses a cross-scale fusion network in the neck network.

[0128] Prediction module 2 is configured to acquire the road surface image to be detected and input it into the trained road surface crack segmentation model. First, the backbone network performs feature extraction to obtain multi-scale image features. The multi-scale image features are input into the neck network and then fused through the cross-scale fusion network to obtain cross-scale fused features. The cross-scale fused features are input into the head network to obtain the segmentation result of each crack in the road surface image to be detected.

[0129] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 6 As shown, the electronic device of this embodiment includes a processor 601 and a memory 602; wherein the memory 602 is used to store computer execution instructions; and the processor 601 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0130] Alternatively, the memory 602 can be either standalone or integrated with the processor 601.

[0131] When the memory 602 is set up independently, the electronic device also includes a bus 603 for connecting the memory 602 and the processor 601.

[0132] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 601, implement the above method.

[0133] This invention also provides a computer program product, including a computer program that, when executed by a processor 601, implements the above-described method.

[0134] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0135] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0136] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0137] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 601 to execute some steps of the methods of the various embodiments of this application.

[0138] It should be understood that the processor 601 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 601 can be any conventional processor 601. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 601 executing the steps, or as a combination of hardware and software modules within the processor 601 executing the steps.

[0139] The memory 602 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0140] Bus 603 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 603 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 603 in the accompanying drawings of this application is not limited to only one bus 603 or one type of bus 603.

[0141] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0142] An exemplary storage medium is coupled to a processor 601, enabling the processor 601 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor 601. The processor 601 and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor 601 and the storage medium can exist as discrete components in an electronic device or a host device.

[0143] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A road surface crack segmentation method based on an improved YOLOv8, characterized in that, Includes the following steps: A road surface crack segmentation model based on an improved YOLOv8 was constructed and trained to obtain a trained road surface crack segmentation model. The road surface crack segmentation model includes a backbone network, a neck network, and a head network connected in sequence. In the backbone network, a spectral fusion convolutional layer replaces the two-dimensional convolutional layer in the YOLOv8 network backbone network. The spectral fusion convolutional layer includes a channel partitioning module, a first extraction path, a second extraction path, a third extraction path, a fourth extraction path, a first fusion path, a second fusion path, a first splicing layer, a first convolutional layer, a first batch of normalization layers, and a first SiLU activation function layer. The first fusion path includes a second SiLU activation function layer, and the second fusion path includes a third SiLU activation function layer. The input features of the spectral fusion convolutional layer are first divided into high-frequency features and low-frequency features according to different numbers of channels by the channel segmentation module; The high-frequency features are input into the first extraction path and the second extraction path respectively to obtain the first path features and the second path features; The low-frequency features are input into the third extraction path and the fourth extraction path respectively to obtain the third path features and the fourth path features; The first path feature and the third path feature are input into the first fusion path, first weighted, and then passed through the second SiLU activation function layer to obtain the first fusion feature; The second path feature and the fourth path feature are input into the second fusion path, first weighted, then passed through the third SiLU activation function layer, and then bilinear interpolation is performed to obtain the second fusion feature; The first fusion feature and the second fusion feature are passed through the first splicing layer to obtain the spliced ​​feature. The spliced ​​feature is then passed through the first convolutional layer and the first batch of normalization layers to obtain the intermediate feature. The input features of the spectral fusion convolutional layer are residually concatenated with the intermediate features and then input into the first SiLU activation function layer to obtain the output features of the spectral fusion convolutional layer. A full-scale pyramid pooling module replaces the SPPF module in the backbone network of the YOLOv8 network, and a cross-scale fusion network is used in the neck network. The full-scale pyramid pooling module includes a sixth convolutional layer, a seventh convolutional layer, a first processing branch, a second processing branch, a third processing branch, a fourth processing branch, a fifth processing branch, a sixth processing branch, a seventh processing branch, a second concatenation layer, a grouped convolutional layer, an adaptive weight calculation module, an eighth convolutional layer, and a ninth convolutional layer. The cross-scale fusion network includes three upsampling modules, eleven spectral fusion convolutional layers, four C2f modules, and eleven concatenation layers. The output features of the full-scale pyramid pooling module in the backbone network are input into the first upsampling module in the cross-scale fusion network to obtain the first feature; the output features of the full-scale pyramid pooling module in the backbone network are input into the first spectral fusion convolutional layer in the cross-scale fusion network to obtain the second feature. The road surface image to be detected is acquired and input into the trained road surface crack segmentation model. First, the backbone network is used for feature extraction to obtain multi-scale image features. The multi-scale image features are input into the neck network and then fused through the cross-scale fusion network to obtain cross-scale fused features. The cross-scale fused features are input into the head network to obtain the segmentation result of each crack in the road surface image to be detected.

2. The road surface crack segmentation method based on the improved YOLOv8 according to claim 1, characterized in that, The first extraction path includes a second convolutional layer and a second batch normalization layer connected in sequence; the second extraction path includes a first average pooling layer, a third convolutional layer, and a third batch normalization layer connected in sequence; the third extraction path includes a fourth convolutional layer, a fourth batch normalization layer, and an upsampling module connected in sequence; and the fourth extraction path includes a fifth convolutional layer and a fifth batch normalization layer connected in sequence.

3. The road surface crack segmentation method based on the improved YOLOv8 according to claim 1, characterized in that, The sixth, eighth, and ninth convolutional layers each employ a 1×1 kernel, while the seventh convolutional layer employs a 3×3 kernel. The first processing branch preserves the original features. The second, third, fourth, and fifth processing branches are respectively a first max pooling layer, a second max pooling layer, a third max pooling layer, and a second average pooling layer. The first, second, and third max pooling layers employ pooling kernels of 3×3 and 5×5, respectively. The second average pooling layer uses a 7×7 max pooling operation and a 5×5 average pooling operation. The sixth and seventh processing branches are a depthwise separable convolutional layer and a dilated convolutional layer, respectively. The adaptive weight calculation module includes an adaptive average pooling layer, a tenth convolutional layer, a sixth batch normalization layer, a ReLU activation function layer, an eleventh convolutional layer, a seventh batch normalization layer, and a Sigmoid activation function layer connected in sequence. The tenth and eleventh convolutional layers each use a 1×1 convolutional operation. The input features of the full-scale pyramid pooling module are sequentially passed through the sixth and seventh convolutional layers to obtain enhanced features, as shown in the following formula: ； in, This represents the input features of the full-scale pyramid pooling module. and These represent convolution operations with kernel sizes of 1×1 and 3×3, respectively. Indicates enhanced features; The enhanced features are processed in parallel through the first processing branch, the second processing branch, the third processing branch, the fourth processing branch, the fifth processing branch, the sixth processing branch, and the seventh processing branch, respectively, to obtain the first processed feature, the second processed feature, the third processed feature, the fourth processed feature, the fifth processed feature, the sixth processed feature, and the seventh processed feature, as shown in the following formula: ； ； ； ； ； ； Wherein, the first processing feature is the enhancement feature. , , , These represent max pooling operations with pooling kernels of 3×3, 5×5, and 7×7, respectively. This indicates an average pooling operation with a 5×5 pooling kernel. Represents depthwise convolution. This represents pointwise convolution. Indicates dilated convolution. , , , , and These represent the second, third, fourth, fifth, sixth, and seventh processing features, respectively. The first, second, third, fourth, fifth, sixth, and seventh processing features are sequentially passed through the second splicing layer and the grouped convolutional layer to obtain the third fused feature; The third fusion feature is input to the adaptive weight calculation module to obtain adaptive weights. The third fusion feature and the adaptive weights are multiplied channel by channel to obtain the fourth fusion feature. The fourth fusion feature and the input feature of the full-scale pyramid pooling module are then residually connected after passing through the eighth and ninth convolutional layers to obtain the output feature of the full-scale pyramid pooling module, as shown in the following formula: ； in, Indicates the third fusion feature, This indicates an adaptive average pooling layer. This indicates a batch normalization operation. This indicates multiplication by channel. express Activation function express Activation function This represents the output characteristics of the full-scale pyramid pooling module.

4. The road surface crack segmentation method based on the improved YOLOv8 according to claim 1, characterized in that, The head network includes a first detection head, a second detection head, and a third detection head arranged in parallel. The multi-scale image features include the output features of the full-scale pyramid pooling module, the output features of the third C2f module in the backbone network, and the output features of the second C2f module in the backbone network.

5. The road surface crack segmentation method based on the improved YOLOv8 according to claim 4, characterized in that, The second feature is sequentially passed through the second upsampling module and the second spectral fusion convolutional layer to obtain the third feature; the output feature of the third C2f module in the backbone network is input into the third spectral fusion convolutional layer to obtain the fourth feature; the third and fourth features are input into the first splicing layer in the cross-scale fusion network to obtain the fifth feature; the fifth feature and the first feature are input into the second splicing layer in the cross-scale fusion network to obtain the sixth feature; the sixth feature is passed through the first C2f module in the cross-scale fusion network to obtain the seventh feature; the seventh feature is passed through the third upsampling module in the cross-scale fusion network to obtain the eighth feature; the output feature of the second C2f module in the backbone network is input into the fourth spectral fusion convolutional layer in the cross-scale fusion network to obtain the ninth feature; the ninth feature and the third feature are passed through the third splicing layer in the cross-scale fusion network to obtain the tenth feature; The tenth and eighth features are input into the fourth stitching layer of the cross-scale fusion network to obtain the eleventh feature; the eleventh feature is then passed through the second C2f module and the fifth spectral fusion convolutional layer of the cross-scale fusion network to obtain the twelfth feature; the twelfth feature and the ninth feature are input into the fifth stitching layer of the cross-scale fusion network to obtain the thirteenth feature, which is then input into the first detection head to obtain a segmentation result of the first size; the twelfth feature is passed through the sixth spectral fusion convolutional layer of the cross-scale fusion network to obtain the fourteenth feature; the seventh feature and the twelfth feature are input into the sixth stitching layer of the cross-scale fusion network to obtain the fifteenth feature; the fifteenth feature is passed through the seventh spectral fusion convolutional layer of the cross-scale fusion network to obtain the sixteenth feature; the sixteenth feature and the fourteenth feature are input into the seventh stitching layer of the cross-scale fusion network to obtain the seventeenth feature; The seventeenth feature is sequentially passed through the third C2f module and the eighth spectral fusion convolutional layer in the cross-scale fusion network to obtain the eighteenth feature; the eighteenth feature and the fourth feature are input into the eighth stitching layer in the cross-scale fusion network to obtain the nineteenth feature, which is then input into the second detection head to obtain the segmentation result of the second size; the eighteenth feature and the output feature of the full-scale pyramid pooling module in the backbone network are input into the ninth stitching layer in the cross-scale fusion network to obtain the twentieth feature; the twentieth feature is passed through the ninth spectral fusion convolutional layer in the cross-scale fusion network to obtain the second... The eleventh feature; the eighteenth feature is passed through the tenth spectral fusion convolutional layer in the cross-scale fusion network to obtain the twenty-second feature; the twenty-second feature and the twenty-first feature are input into the tenth stitching layer in the cross-scale fusion network to obtain the twenty-third feature; the twenty-third feature is passed through the fourth C2f module and the eleventh spectral fusion convolutional layer in the cross-scale fusion network to obtain the twenty-fourth feature; the twenty-fourth feature and the second feature are input into the eleventh stitching layer in the cross-scale fusion network to obtain the twenty-fifth feature; the twenty-fifth feature is input into the third detection head to obtain the segmentation result of the third size.

6. A road surface crack segmentation device based on an improved YOLOv8, characterized in that, include: The model building module is configured to build and train a road surface crack segmentation model based on an improved YOLOv8, resulting in a trained road surface crack segmentation model. The road surface crack segmentation model comprises a backbone network, a neck network, and a head network connected in sequence. In the backbone network, a spectral fusion convolutional layer replaces the two-dimensional convolutional layer in the YOLOv8 network's backbone network. This spectral fusion convolutional layer includes a channel partitioning module, a first extraction path, a second extraction path, a third extraction path, a fourth extraction path, a first fusion path, a second fusion path, a first splicing layer, a first convolutional layer, a first batch of normalization layers, and a first SiLU activation function layer. The first fusion path includes a second SiLU activation function layer, and the second fusion path includes a third SiLU activation function layer. The input features of the spectral fusion convolutional layer are first divided into high-frequency features and low-frequency features according to different numbers of channels by the channel segmentation module; The high-frequency features are input into the first extraction path and the second extraction path respectively to obtain the first path features and the second path features; The low-frequency features are input into the third extraction path and the fourth extraction path respectively to obtain the third path features and the fourth path features; The first path feature and the third path feature are input into the first fusion path, first weighted, and then passed through the second SiLU activation function layer to obtain the first fusion feature; The second path feature and the fourth path feature are input into the second fusion path, first weighted, then passed through the third SiLU activation function layer, and then bilinear interpolation is performed to obtain the second fusion feature; The first fusion feature and the second fusion feature are passed through the first splicing layer to obtain the spliced ​​feature. The spliced ​​feature is then passed through the first convolutional layer and the first batch of normalization layers to obtain the intermediate feature. The input features of the spectral fusion convolutional layer are residually concatenated with the intermediate features and then input into the first SiLU activation function layer to obtain the output features of the spectral fusion convolutional layer. A full-scale pyramid pooling module replaces the SPPF module in the backbone network of the YOLOv8 network, and a cross-scale fusion network is used in the neck network. The full-scale pyramid pooling module includes a sixth convolutional layer, a seventh convolutional layer, a first processing branch, a second processing branch, a third processing branch, a fourth processing branch, a fifth processing branch, a sixth processing branch, a seventh processing branch, a second concatenation layer, a grouped convolutional layer, an adaptive weight calculation module, an eighth convolutional layer, and a ninth convolutional layer. The cross-scale fusion network includes three upsampling modules, eleven spectral fusion convolutional layers, four C2f modules, and eleven concatenation layers. The output features of the full-scale pyramid pooling module in the backbone network are input into the first upsampling module in the cross-scale fusion network to obtain the first feature; the output features of the full-scale pyramid pooling module in the backbone network are input into the first spectral fusion convolutional layer in the cross-scale fusion network to obtain the second feature. The prediction module is configured to acquire a road surface image to be detected and input it into the trained road surface crack segmentation model. First, the model performs feature extraction through the backbone network to obtain multi-scale image features. The multi-scale image features are then input into the neck network and fused through the cross-scale fusion network to obtain cross-scale fused features. The cross-scale fused features are then input into the head network to obtain the segmentation result of each crack in the road surface image to be detected.

7. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images