Method and device for detecting road surface spilling based on deep learning, electronic equipment and program product

Abstract

The application discloses a kind of based on deep learning's road surface throwing detection method, device, electronic equipment and program product.The method is realized by the detection model of training completion, model uses LMSADet detection head, multi-scale feature extraction and spatial attention mechanism are introduced in task branch, multi-scale modeling is decoupled from backbone network and neck network and integrated in detection head, to directly optimize the local details and scale difference of throwing target to fit detection task.For suppressing background interference and improving the recognition effect of fuzzy boundary, the neck network adds MSHA module to efficiently capture the semantic relationship between the thrown object and the background and enhance the regional understanding ability.The backbone network introduces C3ESP module, extracts deep features by stacking depth separable convolution, and combines residual optimization fusion to avoid information loss;At the same time, PEMA attention mechanism is introduced, the importance of different receptive field features is dynamically adjusted, the model focuses on key features, more comprehensive data information is captured, so that the detection performance is significantly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a road surface throwing detection method based on deep learning, a road surface throwing detection device, an electronic device and a computer program product. BACKGROUND

[0002] With the continuous advancement of urbanization, municipal infrastructure construction is gradually improved. Road surface throwing, as an indispensable important part of urban roads, is directly related to the safety of citizens' travel and life and property safety. Road surface throwing not only bears important functions such as drainage and traffic, but also plays an important role in protecting underground pipelines, maintaining road flatness and ensuring urban operation. However, with the increase of the service life of road surface throwing, the change of environmental conditions and the influence of external damage, road surface throwing displacement, damage and even loss frequently occur, which brings serious hidden dangers to pedestrians, vehicles and public safety.

[0003] Although deep learning can currently provide new technical support for automatic detection of road surface throwing damage, the images collected in different scenes are easily affected by the corresponding environment, reducing the accuracy of road surface throwing detection, because road surface throwing is distributed in various scenes such as urban roads, squares, parks and around buildings. SUMMARY

[0004] The application provides a road surface throwing detection method based on deep learning, a road surface throwing detection device, an electronic device and a computer program product, which can enhance the perception ability of the model to small targets in a complex background, thereby improving the accuracy of road surface throwing detection.

[0005] In a first aspect, the application provides a road surface throwing detection method based on deep learning, comprising:

[0006] extracting features of an image based on a backbone network of a pre-trained road surface throwing detection model, to obtain image features; the image to be detected includes a road surface;

[0007] fusing the image features based on a neck network of the road surface throwing detection model, to obtain target fusion features;

[0008] detecting the target fusion features based on a detection network of the road surface throwing detection model, to obtain a detection result of road surface throwing in the image to be detected;

[0009] The detection network comprises an LMSADet detection head, which enhances the target fusion features based on multi-scale fusion and spatial attention, and detects the enhanced target fusion features to obtain the detection result.

[0010] In a case that the image to be detected includes a road surface spilling, the detection result includes a predicted bounding box of the road surface spilling and a predicted spilling category.

[0011] Optionally, the LMSADet detection head includes a bounding box regression detection branch for outputting the predicted bounding box, and a category prediction branch for outputting the predicted spilling category, the branch network structures of the bounding box regression detection branch and the category prediction branch are the same; the branch network structure includes a multi-scale convolution structure, a first concatenation layer, a spatial attention structure, a channel fusion structure, a second concatenation layer, a first DWCBS layer, a second DWCBS layer, and a first convolution layer; for the target fusion feature input to the bounding box regression detection branch or the category prediction branch:

[0012] The target fusion feature is subjected to at least two series convolution operations in different scales through the multi-scale convolution structure, to obtain series convolution features corresponding to each scale; the series convolution operation includes a depth separable convolution operation and a convolution operation executed in sequence;

[0013] The features corresponding to each scale are concatenated in the channel dimension through the first concatenation layer, to obtain first concatenation features;

[0014] The first concatenation features are subjected to a channel compression operation, a channel expansion operation, and a normalization operation in sequence through the spatial attention structure, to obtain spatial attention weights corresponding to each scale;

[0015] In each scale, the corresponding spatial attention weights and the corresponding mixed convolution features are weighted and fused through the channel fusion structure, to obtain spatial enhancement features corresponding to each scale;

[0016] The spatial enhancement features of each scale are concatenated through the second concatenation layer, to obtain attention-aware features;

[0017] The attention-aware features are subjected to two DWCBS operations and a convolution operation in sequence through the first DWCBS layer, the second DWCBS layer, and the first convolution layer, to obtain the predicted bounding box or the predicted spilling category;

[0018] The first convolution layer corresponding to the bounding box regression detection branch is set to have an output channel number of The first convolution layer corresponding to the category prediction branch is set to have an output channel number of n c .

[0019] Optionally, the neck network is provided with an MSHA module, the MSHA module comprising a MSCA submodule and a MSSA submodule, the MSCA submodule being connected in series with the MSSA submodule, the MSCA submodule being configured to perform channel-attention-based multi-scale feature extraction and fusion in a channel attention dimension; and the MSSA submodule being configured to perform spatial-attention-based multi-scale feature extraction and fusion in a spatial dimension.

[0020] Optionally, the MSCA submodule comprises a first mixed pooling layer, a third concatenation layer, a dimension adjustment layer, a multi-scale convolution layer, a fourth concatenation layer, a second convolution layer, a first activation layer, and a first fusion layer; and for a first input feature input into the MSHA module:

[0021] The first input feature is subjected to an average pooling operation and a maximum pooling operation by the first mixed pooling layer to obtain a first pooling result and a second pooling result;

[0022] The first pooling result and the second pooling result are concatenated by the concatenation layer to obtain a first mixed pooling result;

[0023] The first mixed pooling result after the dimension adjustment is subjected to different scale convolution operations by the multi-scale convolution layer to obtain first convolution features corresponding to the respective scales;

[0024] The first convolution features of the respective scales are concatenated by the third concatenation layer to obtain third concatenation features;

[0025] The third concatenation features subjected to the convolution by the second convolution layer are subjected to an activation operation by the first activation layer to obtain channel weights;

[0026] The first input feature and the channel weights are fused by the fusion layer to obtain multi-scale channel enhanced features.

[0027] Optionally, the MSSA submodule comprises a second mixed pooling layer, a fifth concatenation layer, a multi-scale convolution weighting structure, a sixth concatenation layer, a second activation layer, and a second fusion layer; and for multi-scale channel enhanced features input into the MSSA submodule:

[0028] The multi-scale channel enhanced features are subjected to an average pooling operation and a maximum pooling operation by the second mixed pooling layer to obtain a third pooling result and a fourth pooling result;

[0029] The third pooling result and the fourth pooling result are concatenated by the fifth concatenation layer to obtain a second mixed pooling result;

[0030] At each scale, the second mixed pooling result is subjected to a convolution operation and a weighting operation of the corresponding scale by the multi-scale convolution weighting structure in sequence to obtain a corresponding weighted feature; the weight corresponding to the weighting operation is obtained through learning;

[0031] The weighted features of each scale are spliced through the sixth splicing layer to obtain weighted fusion features;

[0032] The weighted fusion features are subjected to an activation operation through the second activation layer to obtain spatial weights;

[0033] The multi-scale channel enhanced features and the spatial weights are fused through the fusion layer to obtain multi-scale channel-spatial enhanced features.

[0034] Optionally, the backbone network is provided with a C3ESP module, and the C3ESP module includes a third convolutional layer, a residual branch, a multi-scale perception branch, a seventh splicing layer, and a fourth convolutional layer; for the second input feature input into the C3ESP module:

[0035] The second input feature is subjected to a channel compression operation through the third convolutional layer to obtain a compressed feature;

[0036] The compressed feature is subjected to a multi-scale deep feature extraction operation through the multi-scale perception branch to obtain a deep feature;

[0037] The deep feature and the compressed feature are spliced through the seventh splicing layer to obtain a seventh splicing feature;

[0038] The seventh splicing feature is subjected to a channel expansion operation through the fourth convolutional layer to obtain a second output feature corresponding to the second input feature.

[0039] Optionally, the multi-scale perception branch includes n ESPBlocks connected in series, and each ESPBlock includes a progressive depth separable convolution structure, an eighth splicing layer, a fifth convolutional layer, a PEMA layer, and a ninth splicing layer; for a third input feature input into the ESPBlock:

[0040] The third input feature is subjected to a progressive multi-level depth separable convolution operation through the progressive depth separable convolution structure to obtain deep convolution features of each level; the input corresponding to the first level is the third input feature, and the input of the non-first level is the deep convolution feature corresponding to the previous level;

[0041] The deep convolution features of each level are spliced through the eighth splicing layer to obtain fused deep convolution features;

[0042] The fused deep convolution features are subjected to a channel fusion operation through the fifth convolutional layer;

[0043] The fused deep convolution features after the channel fusion are subjected to a hybrid attention mechanism through the PEMA layer to obtain a hybrid enhanced feature;

[0044] The third input and the hybrid enhanced feature are spliced through the ninth splicing layer to obtain a third output feature corresponding to the third input feature.

[0045] In a second aspect, the present application provides a road surface scattering detection device, comprising:

[0046] an extraction module configured to perform feature extraction on a to-be-detected image based on a backbone network of a pre-trained road surface scattering detection model to obtain image features, wherein the to-be-detected image comprises a road surface;

[0047] a fusion module configured to perform fusion on the image features based on a neck network of the road surface scattering detection model to obtain fused features;

[0048] a detection module configured to perform detection on the fused features based on a detection network of the road surface scattering detection model to obtain a detection result of road surface scattering in the to-be-detected image;

[0049] The detection network comprises an LMSADet detection head, which is configured to enhance target fused features based on multi-scale fusion and spatial attention, and to detect the enhanced target fused features to obtain the detection result.

[0050] In the case where the to-be-detected image comprises road surface scattering, the detection result comprises a predicted bounding box of the road surface scattering and a predicted scattering category.

[0051] In a third aspect, the present application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0052] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, wherein the computer program is executable by a processor to implement the steps of the method of the first aspect.

[0053] In a fifth aspect, the present application provides a computer program product, which comprises a computer program, wherein the computer program is executable by one or more processors to implement the steps of the method of the first aspect.

[0054] Compared with the prior art, the present application has the beneficial effects that the present application implements the detection process based on a pre-trained road surface scattering detection model. First, the backbone network performs deep feature extraction on the input image to capture multi-level representations of the road surface region and potential scattering targets. Then, the neck network effectively fuses features of different levels to form target fused features containing rich context information. Finally, the detection network analyzes the fused features to output the detection result, which can give the corresponding predicted bounding box and predicted category in the presence of road surface scattering objects.

[0055] The model introduces a lightweight multi-scale perception (LMSADet) detection head. Unlike traditional methods that place multi-scale modeling in the backbone network or neck network, integrating the multi-scale mechanism into the detection head enables the model to directly focus on the detection task itself. This design can more accurately optimize the local details and scale differences of the road throw target, avoid unnecessary global calculations in the early and middle feature layers, and effectively reduce the computational complexity while maintaining detection accuracy. Further combined with a spatial attention mechanism, the LMSADet detection head can highlight key regional features, suppress background interference, and improve the ability to identify blurred boundaries.

[0056] In summary, the model integrated with the LMSADet detection head not only improves the detection accuracy and robustness of road throw objects, but also significantly optimizes the computational efficiency, achieving efficient and lightweight precise detection, thereby meeting the real-time and accuracy requirements in practical applications.

[0057] It can be understood that the beneficial effects of the above-mentioned second aspect to the fifth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0059] Figure 1 is a network structure schematic diagram of a road throw detection model provided by the embodiments of the present application;

[0060] Figure 2 is a flowchart of a road throw detection method based on deep learning provided by the embodiments of the present application;

[0061] Figure 3 is a network structure schematic diagram of an LMSADet detection head provided by the embodiments of the present application;

[0062] Figure 4 is a network structure schematic diagram of an MSHA module provided by the embodiments of the present application;

[0063] Figure 5 is a network structure schematic diagram of a C3ESP module provided by the embodiments of the present application;

[0064] Figure 6FIG. 1 is a structural schematic diagram of a road surface scattering detection device provided by an embodiment of the present application.

[0065] Figure 7 FIG. 2 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0066] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, persons having ordinary skill in the art will appreciate that embodiments of the application can be practiced without the specific details, and that the present application is not limited to the details by virtue of the inclusive language used herein. In other instances, the well-known systems, circuits, and methods have not been described in detail so as not to unnecessarily obscure the description of the present application.

[0067] Since road surface scattering is distributed in diversified scenes such as urban roads, squares, parks, and periphery of buildings, images collected in different scenes are easily affected by the corresponding environment, which reduces the accuracy of road surface scattering detection.

[0068] To solve this problem, the present application finds that in the related art, multi-scale modeling methods are mostly concentrated in backbone networks or neck networks (such as FPN, PANet, etc.), which can improve the detection performance through multi-level feature fusion, but such methods often introduce a large amount of additional computational overhead, including complex up-sampling, down-sampling or cross-layer connection operations, which is difficult to meet the real-time and lightweight requirements in the vehicle deployment scene.

[0069] To further meet the real-time and lightweight requirements in the vehicle deployment scene, the present application provides a road surface scattering detection model, which includes a backbone network, a neck network, and a detection head. Specifically, the backbone network is used for feature extraction of an input image to obtain image features; the neck network is used for feature fusion of the image features to obtain target fusion features; and the detection head is used for detection of the target fusion features to obtain a detection result of road surface scattering in a to-be-processed image. In the presence of road surface scattering objects, corresponding prediction bounding boxes and prediction categories can be given.

[0070] The road surface scattering detection model adopts an LMSADet detection head. The LMSADet detection head integrates a multi-scale mechanism into the detection head. Compared with a traditional method that models multi-scale in a backbone network or a neck network, the LMSADet detection head can make the model more directly face the detection task itself. The LMSADet detection head can more accurately optimize the local details and scale differences of the road surface scattering target, avoid unnecessary global calculation in the early and middle feature layers, and thus effectively reduce the computational complexity while maintaining the detection accuracy. In addition, based on the spatial attention mechanism, the LMSADet detection head can highlight key regional features, suppress background interference, and improve the recognition ability of blurred boundaries.

[0071] As can be seen, the road surface scattering detection model can not only improve the detection accuracy and robustness of road surface scattering objects, but also significantly optimize the model calculation efficiency, realize efficient and lightweight accurate detection, and balance the real-time and accuracy requirements in practical applications.

[0072] In some embodiments, the LMSADet detection head includes a bounding box regression detection branch for outputting a predicted bounding box, and a class prediction branch for outputting a predicted scattering class. The branch network structures of the bounding box regression detection branch and the class prediction branch are the same, which can realize sharing of different task-specific feature extraction methods, guarantee the consistency and sufficiency of feature utilization, and at the same time avoid redundant calculation caused by repeated construction of complex structures, thereby improving the detection efficiency.

[0073] In order to enable different branches to achieve different detection tasks, in the final output stage, the two branches are distinguished in the output channel number of the first convolutional layer: the output channel number of the first convolutional layer of the bounding box regression detection branch is set to , which is used to finely regress the position of the predicted bounding box; and the output channel number of the first convolutional layer of the class prediction branch is set to nc (total number of classes), which is used to output the prediction probability of each class. In this way, different task targets obtain consistent representation ability through the same feature processing path, and then complete their own prediction tasks with the help of the difference in the output channel number of the first convolutional layer.

[0074] The design of the branch network structure focuses on the cooperation of multi-scale and attention mechanism. By combining multi-scale convolution, splicing and spatial attention mechanism, the model can capture target details at different scales and highlight key areas while suppressing irrelevant backgrounds. In addition, in order to balance the lightweight design of the model and the detection accuracy, the DWCBS (Deep Separable Convolution + BatchNorm + Activation) structure and residual fusion are used to effectively reduce the calculation amount while avoiding information loss and ensuring high-precision detection. Specifically, the branch network structure includes a multi-scale convolution structure, a first splicing layer, a spatial attention structure, a channel fusion structure, a second splicing layer, a first DWCBS layer, a second DWCBS layer, and a first convolution layer.

[0075] In this embodiment, the LMSADet detection head realizes efficient decoupling of boundary box regression and class classification through the method of "shared enhancement module + differentiated output convolution", which can not only improve the precision and robustness of the model in road debris detection, but also ensure the lightweight and real-time detection of the model.

[0076] In the scene of road debris identification, complex road environment and unclear ground conditions can cause significant interference, affecting the detection accuracy and reliability of the model. Various visual elements commonly seen on urban roads or highways, such as traffic signs, lane lines, vehicle shadows, etc., may be similar to the appearance of debris in the image, increasing the risk of false positives. In addition, changes in lighting conditions (such as day, night, and lighting differences under different weather conditions) can cause image quality to decline, and low-light conditions may produce noise or blurring, making it difficult to accurately identify small-sized debris. Changes in ground texture and color can also affect the visibility of debris, and dark or light-colored debris can be easily overlooked in the corresponding background. Partial occlusion and incomplete view (such as occlusion by other objects or limited camera angle) further increase the difficulty of identification, while ground stains and water reflection can cause false features, interfering with model judgment. Dynamic environmental changes (such as dust and fog generated during vehicle travel) can also make the image unclear, reducing the identification ability.

[0077] In some embodiments, to address the above problems, the neck network is provided with an MSHA module, which includes an MSCA submodule and an MSSA submodule. The MSCA submodule and the MSSA submodule are connected in series, the MSCA submodule is used to perform channel attention-based multi-scale feature extraction and fusion in the channel attention dimension, and the MSSA submodule is used to perform spatial attention-based multi-scale feature extraction and fusion in the spatial dimension. Through multi-scale feature extraction, global and local feature fusion, context information capture, and channel and spatial attention combination, the performance and reliability of the model are significantly improved.

[0078] In some embodiments, the MSCA submodule sequentially comprises a first mixed pooling layer, a third concatenation layer, a dimension adjustment layer, a multi-scale convolution layer, a fourth concatenation layer, a second convolution layer, a first activation layer, and a first fusion layer. This structure integrates multi-scale feature extraction into the channel attention and spatial attention mechanism, enabling the model to flexibly obtain key information under different receptive fields, improving the ability to balance details and global features, and enhancing robustness and discriminability in complex background environments.

[0079] In some embodiments, the MSSA submodule comprises a second mixed pooling layer, a fifth concatenation layer, a multi-scale convolution weighting structure, a sixth concatenation layer, a second activation layer, and a second fusion layer. This structure efficiently extracts global context information through adaptive pooling, while combining multi-scale convolution to obtain local detail features, achieving dynamic fusion of global and local features, thereby significantly enhancing feature expression ability and improving adaptability to complex scenes.

[0080] In some embodiments, in the MSHA module, the MSCA and MSSA submodules each undertake different feature enhancement tasks and form complementary advantages.

[0081] Among them, the MSCA submodule introduces multi-scale feature extraction into the channel attention and spatial attention mechanism, combining mixed pooling, concatenation, convolution, and fusion operations, enabling the model to simultaneously obtain local details and global context information under different receptive fields. This design not only strengthens key region features but also effectively suppresses background interference, improving the model's robustness in complex road surface environments.

[0082] The MSSA submodule, on the other hand, focuses on the dynamic fusion of global and local features. By extracting global semantics through adaptive pooling, and combining a multi-scale convolution weighting structure to extract local features, and through concatenation, activation, and fusion operations to achieve efficient complementarity of the two types of features. This module enables the model to capture large-scale context relationships while maintaining sensitivity to subtle litter features, thereby significantly enhancing overall feature expression ability.

[0083] The MSCA module focuses on attention enhancement under multi-scale receptive fields, highlighting key features and suppressing interference; the MSSA module focuses on dynamic balance of global and local features, ensuring information integrity and expression ability. The synergistic effect of the two modules enables the detection model to maintain lightweight while balancing precision, robustness, and efficiency, significantly improving the practical effect of road litter detection.

[0084] The size of the road surface scattering object ranges widely, from small stones to large piles of discarded construction materials. The images show great differences. The small scattering objects occupy a small number of pixels, and the model is difficult to detect and identify. Large piles of objects cover most of the image area, and it is difficult to capture details. The model needs to capture the fine features of small-scale scattering objects and understand the overall distribution and morphological structure of large-scale scattering objects, which requires higher feature extraction networks. Single-scale feature representation cannot balance detail and overall information, and effective feature extraction needs to switch between different scales to accurately identify and classify various scattering objects.

[0085] In some embodiments, to solve the above problems, a C3ESP module is introduced in the backbone network. The C3ESP module includes a third convolutional layer, a residual branch, a multi-scale perception branch, a seventh splicing layer, and a fourth convolutional layer. Through the multi-scale perception branch, the feature information is gradually extracted and deepened under the premise of ensuring computational efficiency. At the same time, the residual branch is used to realize efficient feature fusion, avoiding information loss in the multi-layer transmission process, thereby improving the expression ability and robustness of the model to key features.

[0086] In some embodiments, the multi-scale perception branch includes n ESPBlocks connected in series, and the ESPBlock includes a progressive depth separable convolution structure, an eighth splicing layer, a fifth convolutional layer, a PEMA layer, and a ninth splicing layer. This structure gradually extracts and refines the features by stacking multiple 3x3 depth separable convolutions while ensuring computational efficiency. At the same time, the PEMA attention mechanism is innovatively introduced, which can dynamically adjust the importance of different receptive field features, enabling the model to focus on key area features and more comprehensively and accurately capture data information, thereby effectively improving the overall feature expression ability.

[0087] In some embodiments, the road surface scattering detection model can be improved based on the YOLO series model due to the following advantages of the YOLO series model:

[0088] Both are end-to-end single-stage detection frameworks that can achieve efficient object detection. Compared with two-stage detectors such as Faster R-CNN, YOLO directly regresses bounding boxes and classes through a single forward pass, improving detection speed and making it suitable for real-time applications. Its structure is constantly optimized, such as the introduction of Anchor-Free mechanism, feature pyramid (FPN, PAN), etc., enhancing the detection ability of small and multi-scale targets. In addition, the YOLO series of models are continuously optimized in terms of lightweight, computational efficiency, robustness, etc., making them have good adaptability in embedded devices and cloud inference scenarios. YOLOv8 is a major upgrade of the YOLO series, supporting object detection, image classification, and instance segmentation tasks. Its architecture consists of a backbone network, a neck network, and a detection head. The backbone network uses C2f modules to improve feature extraction efficiency, the neck network uses PANet structure to enhance multi-scale feature fusion, and the detection head introduces Anchor-Free design and DFL loss to improve detection accuracy and flexibility. In addition, YOLOv8 provides n / s / m / l / x five model variants to adapt to different scenarios. In the road spill detection task, YOLOv8s is selected for optimization to balance accuracy and real-time performance and improve detection effect.

[0089] Based on this, when constructing a road spill detection model, YOLOv8s can be preferably selected. Exemplarily, if YOLOv8s is used as the basis network for improvement, through the LMSADet detection head, MSHA module, and C3ESP module, the network structure of the road spill detection model can be referred to Figure 1 .

[0090] The road surface debris detection model adopts modular design, and each sub-module cooperates with each other, giving consideration to lightweight, high precision and robustness. Among them, the LMSADet detection head integrates multi-scale modeling and spatial attention mechanism at the detection end, through the boundary box regression branch and the class prediction branch sharing the feature enhancement structure, only the task is distinguished in the output layer channel number, so that the model can directly optimize the local details and scale differences of the debris target while ensuring the simple structure, effectively improving the detection accuracy and inference efficiency. The MSHA module realizes efficient fusion of cross-scale features by introducing multi-scale convolution and spatial attention mechanism, enhances the model's understanding ability of the semantic relationship between road debris and background, and can better suppress background interference and improve the detection performance under fuzzy boundaries. The C3ESP module cooperates with the multi-scale perception branch and the residual branch in the backbone network, extracts and refines feature information with the help of stacked depth separable convolution structure, and combines residual connection to ensure the integrity of feature fusion and avoid information loss. The three work together to make the model maintain a lightweight structure in complex road surface environment, while having strong feature expression ability and detection robustness, so as to realize fast and accurate identification of road surface debris.

[0091] In some embodiments, in order to ensure that the trained road surface debris detection model can meet the expectations and accurately understand and detect road surface debris of different scales in different scenes, a special data set can be created for training the road surface debris detection model. Specifically, a data set named Road Surface Debris (RSD) is created, which is collected by a vehicle-mounted camera under various environmental conditions, including 10000 images with a resolution of 1280x720. Each image is manually annotated.

[0092] For example, in order to provide a high-quality data set, in addition to ensuring the number of images, images covering different scenes such as urban roads, squares, parks, and areas around buildings can also be collected.

[0093] For example, the images can contain common coverings such as vehicles, people, vegetation, warning signs (warning triangles), etc.

[0094] For example, each scene can include multiple time periods and weather conditions such as daytime, nighttime, sunny, rainy, and foggy, to increase the complexity and comprehensiveness of the scene, and each image can be configured with a corresponding txt label file detailing the location and category of the road surface debris.

[0095] Such a data set can support research and application in the fields of intelligent transportation systems, autonomous vehicles, and road safety monitoring, to improve road safety and traffic efficiency.

[0096] To ensure the effectiveness of model training and the reliability of evaluation results, the data set in the present application is divided into a training set (7000 images), a validation set (1000 images) and a test set (2000 images) in a ratio of 7:1:2.

[0097] Preferably, during the process of dividing the data set, special attention is paid to maintaining the balance of sample distribution between the data sets, ensuring that both have similar statistical characteristics in terms of road surface scattering types, sizes, background environments, etc. This can avoid evaluation bias due to uneven sample distribution and make the validation results more representative and reliable.

[0098] In some embodiments, in order to comprehensively and accurately measure the performance of each version of the road surface scattering detection model, after training at least one version of the road surface scattering detection model based on the training set to convergence, the converged model can be evaluated by the validation set to avoid model overfitting, verify the generalization of the model, and ensure that the model can run stably after deployment.

[0099] Specifically, the performance of the road surface scattering detection model on the validation set can be evaluated according to preset conditions.

[0100] Exemplarily, these preset conditions can include performance indicators such as the intersection over union (IoU), detection accuracy and recall rate of the road surface scattering detection device for road surface scattering.

[0101] IoU is an indicator for evaluating the degree of overlap between the predicted bounding box and the true bounding box, and is the ratio of the intersection to the union of the predicted bounding box and the true bounding box. It plays a key role in determining whether it is correct detection. The calculation formula of IoU can be written as:

[0102]

[0103] Precision (p) is also called precision, which refers to the proportion of correct predictions to positive, as shown in the formula:

[0104]

[0105] Recall (R) is also called recall, which refers to the proportion of correct predictions to positive, as shown in the formula:

[0106]

[0107] Average Precision (AP) is calculated from precision and recall. According to the recall value, a line graph of precision is drawn, and the area under the line is calculated, as shown in the formula:

[0108]

[0109] The mean average precision (mAP) refers to an average value of average precisions AP of C different road surface scattering categories, as shown in the formula:

[0110]

[0111] That is, after verifying each version of the road surface scattering detection model through the verification set, the road surface scattering detection model of each version can be comprehensively evaluated based on the above several indicators, so as to determine the road surface scattering detection model with the best performance from each version as the trained road surface scattering detection model.

[0112] In some embodiments, the model running environment includes an Intel Xeon Platinum 8255C processor, 314 GB of memory, an NVIDIA Tesla V100 32 GB graphics card, and a CentOS 8.5.2 (64-bit) operating system. The deep neural network is built based on the PyTorch framework, the input image size is [640, 640], and a multi-scale training strategy is used. The batch size is set to 64, the training is performed for 300 epochs, the AdamW optimizer is used, the initial learning rate is 0.01, and the cosine decay strategy is combined for optimization.

[0113] In some embodiments, the road surface scattering detection model training combines classification loss and regression loss for optimization. The binary classification cross-entropy loss (BCE Loss) is used for classification loss to determine the anchor box category; the regression loss is composed of SIoU Loss and DFL Loss to measure the error between the predicted box and the real box. The TAL dynamic matching method is used for positive and negative sample matching strategy, the optimization target is allocated, and the detection accuracy is improved.

[0114] Based on the network structure of the road surface scattering detection model of the foregoing embodiments, the present application provides a deep learning-based road surface scattering detection method.

[0115] The deep learning-based road surface scattering detection method provided by the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not make any limitation on the specific type of electronic device.

[0116] In order to illustrate the technical solutions provided in the present application, the electronic device will be taken as the execution subject to explain each embodiment.

[0117] Figure 2 A schematic flowchart of a deep learning-based road surface spill detection method provided by the present application is shown, which includes:

[0118] Step 210: The electronic device performs feature extraction on the to-be-detected image based on the backbone network of the pre-trained road surface spill detection model.

[0119] Step 220: The electronic device fuses the image features based on the neck network of the road surface spill detection model to obtain fused features.

[0120] Step 230: The electronic device detects the fused features based on the detection network of the road surface spill detection model to obtain the detection result of the road surface spill in the to-be-detected image.

[0121] In the present embodiment, the model first extracts features of the input image through the backbone network to obtain image features containing road surface information; then fuses the multi-level features through the neck network to form target fused features; finally, the detection network analyzes the target fused features to output a prediction result, including the bounding box of the spill and its category. In this process, the model uses the LMSADet detection head, which combines multi-scale fusion and spatial attention mechanism to effectively enhance the expression of target features. Unlike placing the multi-scale mechanism in the backbone or neck network, integrating it into the detection head is closer to the final detection task and can directly optimize the local details and scale differences of the spill. On the one hand, it avoids redundant global calculation at the early and middle feature layers, reducing the model complexity; on the other hand, it improves the inference efficiency while ensuring high accuracy, thereby realizing fast and accurate detection of road surface spills.

[0122] In some embodiments, the LMSADet detection head includes a bounding box regression detection branch for outputting a predicted bounding box, and a category prediction branch for outputting a predicted spill category, and the branch network structures of the bounding box regression detection branch and the category prediction branch are the same; the branch network structure includes a multi-scale convolution structure, a first concatenation layer, a spatial attention structure, a channel fusion structure, a second concatenation layer, a first DWCBS layer, a second DWCBS layer, and a first convolution layer; wherein the output channel number of the first convolution layer corresponding to the bounding box regression detection branch is set to , and the output channel number of the first convolution layer corresponding to the category prediction branch is set to n c .

[0123] For the target fused features input into the bounding box regression detection branch or the category prediction branch:

[0124] Step A1, the electronic device performs at least two cascaded convolution operations on the target fusion feature through a multi-scale convolution structure to obtain cascaded convolution features corresponding to each scale.

[0125] Step A2, the electronic device splices the features corresponding to each scale in the channel dimension through a first splicing layer to obtain first spliced features.

[0126] Step A3, the electronic device sequentially performs channel compression, channel expansion and normalization operations on the first spliced features through a spatial attention structure to obtain spatial attention weights corresponding to each scale.

[0127] Step A4, the electronic device, for each scale, performs weighted fusion of the corresponding spatial attention weights and the corresponding mixed convolution features through a channel fusion structure to obtain corresponding spatial enhancement features.

[0128] Step A5, the electronic device splices the spatial enhancement features of each scale through a second splicing layer to obtain attention perception features.

[0129] Step A6, the electronic device sequentially performs two DWCBS and convolution operations on the attention perception features through a first DWCBS layer, a second DWCBS layer and a first convolution layer to obtain a predicted bounding box or a predicted scattering category.

[0130] The electronic device utilizes a multi-scale convolution structure to perform cascaded convolution operations at least at two scales to capture feature representations under different receptive fields and obtain multi-scale cascaded convolution features. The cascaded convolution operation corresponding to each scale includes sequentially performed depthwise separable convolution operation and convolution operation. Then the features obtained at each scale are spliced in the channel dimension to form first spliced features containing multi-scale information. Then the spatial attention structure is used to perform channel compression, expansion and normalization on the first spliced features to generate spatial attention weights corresponding to each scale to highlight key information. Further, the channel fusion structure is used to weight and fuse the spatial attention weights of each scale and the corresponding convolution features to obtain more discriminative spatial enhancement features. Finally, the attention perception features are processed by two depthwise separable convolutions (DWCBS) and convolution layers to output a predicted bounding box or a scattering category.

[0131] In the embodiments of the present application, the LMSADet detection head obtains different receptive field features through multi-scale convolution, highlights key areas with the help of spatial attention, and realizes fine enhancement by combining channel fusion. Finally, accurate and efficient bounding box regression and class prediction are realized under the lightweight structure, which significantly improves the detection performance and robustness.

[0132] In some embodiments, Figure 3 The LMSADet detection head network structure is shown. From Figure 3 It can be seen that the branch network structure of the bounding box regression detection branch and the class prediction branch is the same, and the target fusion features input to the bounding box regression detection branch or the class prediction branch B is the batch size, C is the number of channels, H and W are the height and width, respectively. The specific operations are as follows:

[0133] First, perform serial convolution operations (depth separable convolution operation + ordinary convolution) under different scales (3x3, 5x5 and 7x7) on X to extract serial convolution features with scale adaptability under different receptive fields:

[0134]

[0135] Then, the serial convolution features extracted under different scales are spliced in the channel dimension to form a comprehensive feature identifier with better discriminability—the first spliced feature.

[0136] Finally, the spatial attention weight corresponding to each scale is calculated according to , and it is multiplied element by element with the corresponding serial convolution feature to enhance the key area and suppress the non-key area under different scales, to obtain the spatial enhancement feature under each scale. The spatial enhancement features under each scale are spliced to generate the attention-aware feature used for task prediction.

[0137] Through the two DWCBS layers (depth separable convolution + normalization + SiLU) and the first convolution layer, the attention-aware feature is sequentially executed twice DWCBS and convolution operation, and the final detection result is obtained.

[0138] Specifically, referring to the following formula, for the bounding box regression detection branch, based on the limitation of the output channel number of the first convolution layer, the predicted bounding box of the target O reg , specifically the bounding box coordinates. For the class prediction branch, the confidence of the corresponding class is output to obtain the predicted class of the target O ​cls , specifically, the confidence score corresponding to the throwing category.

[0139]

[0140]

[0141] In this embodiment, the detection head is designed to adopt a unified branch network structure, and at the output end, the tasks of boundary box regression and category prediction are separated by differentiating the number of output channels of the first convolutional layer. Specifically, the boundary box regression branch can directly generate the predicted boundary box coordinates of the target under the limitation of the number of output channels, while the category prediction branch outputs the confidence score of the corresponding category, thereby obtaining the predicted throwing category of the target. This design not only ensures simple structure and efficient calculation, but also enables simultaneous prediction of position and category, improving detection accuracy and inference efficiency, and has high practical value.

[0142] In some embodiments, for different detection scales, corresponding LMSADet detection heads can be configured respectively. As shown in Figure 1 When the detection task is divided into three scales, three decoupled LMSADet detection heads are set up, and target information is extracted from different scale feature layers correspondingly. Finally, the output results of the three detection heads are spliced into the same image, and non-maximum suppression processing is performed to remove redundant detection boxes, and invalid predictions are filtered by combining the confidence threshold, to obtain the final output image. In the output image, not only the predicted boundary boxes of various types of road surface throwings are labeled, but also the corresponding category information is attached (if necessary, the confidence score can also be labeled at the same time, so as to accurately detect and visually present multi-scale and multi-category targets.

[0143] In some embodiments, the MSHA module introduced in the neck network is composed of MSCA and MSSA sub-modules in series, the former realizes the extraction and fusion of multi-scale features in the channel dimension, which can highlight key channel features and suppress redundant information; the latter extracts and fuses multi-scale features in the spatial dimension, which strengthens the model's perception of the spatial distribution of the target region. The synergistic effect of the two enables the model to have stronger feature expression and discrimination ability in both channel and spatial dimensions, thereby significantly improving the detection accuracy and robustness of road surface throwings in complex scenes.

[0144] In some embodiments, the MSCA sub-module includes a first mixed pooling layer, a third splicing layer, a dimension adjustment layer, a multi-scale convolutional layer, a fourth splicing layer, a second convolutional layer, a first activation layer, and a first fusion layer; for the first input feature input into the MSHA module:

[0145] Step B1, the electronic device performs an average pooling operation and a maximum pooling operation on the first input feature respectively through the first mixed pooling layer to obtain a first pooling result and a second pooling result.

[0146] Step B2, the electronic device splices the first pooling result and the second pooling result through a splicing layer to obtain a first mixed pooling result.

[0147] Step B3, the electronic device performs a convolution operation of different scales on the first mixed pooling result after the dimension adjustment through a multi-scale convolution layer to obtain first convolution features corresponding to each scale.

[0148] The electronic device performs a convolution operation of different scales on the first mixed pooling result in the multi-scale convolution layer to extract convolution features under a multi-receptive field.

[0149] Step B4, the electronic device splices the first convolution features of each scale through a third splicing layer to obtain third splicing features.

[0150] Step B5, the electronic device performs an activation operation on the third splicing features convolved by the second convolution layer through a first activation layer to obtain channel weights.

[0151] Step B6, the electronic device fuses the first input feature and the channel weights through a fusion layer to obtain multi-scale channel enhanced features.

[0152] The electronic device first performs an average pooling operation and a maximum pooling operation on the input feature through the first mixed pooling layer. For example, on a 4x4 feature map, the average pooling calculates the mean value of each region pixel, and the maximum pooling takes the maximum value of the region, thereby obtaining two complementary pooling results. Then, the two results are spliced in the channel dimension to form a first mixed pooling result containing global statistical information and local significant features. Subsequently, the device performs a convolution operation of different scales on the first mixed pooling result through a multi-scale convolution layer, for example, using 3x3, 5x5 and 7x7 convolution kernels, respectively capturing local details, neighborhood relationships and larger range context features, and obtaining multi-scale convolution features. Then, these features are spliced again to obtain multi-scale fused splicing features. In order to further emphasize key information, the splicing features will be convolved first, and then mapped to the [0, 1] interval through an activation function (such as Sigmoid) to generate weight coefficients of each channel. For example, if a certain channel contributes more to target recognition, the weight value will be close to 1. Finally, the device multiplies the weight and the original input feature channel by channel, thereby realizing the enhancement of important channels, obtaining multi-scale channel enhanced features, which not only retains the original information, but also highlights the key features.

[0153] In the embodiment, the MSCA sub-module can consider both global information and local details in the feature extraction process. The hybrid pooling operation effectively preserves different statistical features, the multi-scale convolution further expands the receptive field range, and the splicing and fusion process ensures complete expression of multi-scale information; at the same time, the introduction of channel weight enables the model to dynamically highlight key channel features. The multi-scale channel enhanced features obtained ultimately improve the feature expression ability, and significantly enhance the model's discriminant performance and detection accuracy for targets in complex scenes.

[0154] In some embodiments, the MSSA sub-module includes a second hybrid pooling layer, a fifth splicing layer, a multi-scale convolution weighting structure, a sixth splicing layer, a second activation layer, and a second fusion layer; for the multi-scale channel enhanced features input into the MSSA sub-module:

[0155] Step C1, the electronic device performs average pooling operation and maximum pooling operation on the multi-scale channel enhanced features respectively through the second hybrid pooling layer, to obtain third pooling results and fourth pooling results.

[0156] Step C2, the electronic device splices the third pooling results and the fourth pooling results through the fifth splicing layer to obtain second hybrid pooling results.

[0157] Step C3, the electronic device performs convolution operation and weighting operation of corresponding scales on the second hybrid pooling results in turn through the multi-scale convolution weighting structure at each scale, to obtain corresponding weighted features. The weight corresponding to the weighting operation is obtained through learning;

[0158] Step C4, the electronic device splices the weighted features of each scale through the sixth splicing layer to obtain weighted fusion features.

[0159] Step C5, the electronic device performs activation operation on the weighted fusion features through the second activation layer to obtain spatial weight.

[0160] Step C6, the electronic device fuses the multi-scale channel enhanced features and the spatial weight through the fusion layer to obtain multi-scale channel-spatial enhanced features.

[0161] In the MSSA submodule, the electronic device first performs average pooling and maximum pooling operations on the input multi-scale channel enhanced features through a second hybrid pooling layer. For example, on an 8x8 feature map, average pooling obtains the overall mean of the region, reflecting global background information; maximum pooling obtains the maximum value of the region, highlighting the significant target features. The third pooling result and the fourth pooling result obtained in this way are concatenated in a channel in a fifth concatenation layer to form a second hybrid pooling result with more information. Then, at each scale, the device uses a multi-scale convolution weighting structure to sequentially perform convolution and weighting processing on the result, such as extracting features of different receptive fields through 3x3, 5x5, and 7x7 convolution kernels, and then weighting these features by combining the weights learned by training, to output weighted features corresponding to each scale. Subsequently, the weighted features of all scales are concatenated into a weighted fusion feature by a sixth concatenation layer.

[0162] To further highlight the key regions, the weighted fusion feature enters the second activation layer to generate weight coefficients for each spatial position through nonlinear mapping, for example, the regions close to the target edge can obtain higher weights. Finally, the electronic device fuses these spatial weights with the multi-scale channel enhanced features of the original input (i.e., the output of the MSCA submodule) position by position through the fusion layer to obtain the final multi-scale channel-spatial enhanced features, thereby achieving comprehensive strengthening of the features.

[0163] In this embodiment, the MSSA submodule not only combines the global and local complementary information brought by average pooling and maximum pooling, but also realizes adaptive modeling of features under different receptive fields through a multi-scale convolution weighting structure, and further dynamically highlights key regions using a spatial attention mechanism. The final multi-scale channel-spatial enhanced features can not only fully express semantic information at different scales, but also strengthen the spatial distribution characteristics of target regions, thereby significantly improving the discrimination ability and detection accuracy of the model in complex backgrounds.

[0164] In some embodiments, Figure 4 A network structure diagram of the MSHA module is shown, which is based on Figure 4 the network structure, for the first input feature , first performs adaptive global average pooling and global maximum pooling to obtain a first pooling result and a second pooling result , and adds them to obtain a first hybrid pooling result :

[0165]

[0166]

[0167]

[0168] In the channel dimension, multi-scale 1D convolution operations are performed on , and the first convolution features corresponding to each scale are spliced to obtain third spliced features :

[0169]

[0170]

[0171] Then, the channel attention weight based on attention is generated using convolution and Sigmoid activation function , and is applied to , to obtain multi-scale channel enhanced features .

[0172]

[0173]

[0174] Based on , the global average pooling and the global maximum pooling are calculated to obtain third and fourth pooling results, and they are spliced in the channel dimension:

[0175]

[0176]

[0177] In the spatial dimension, multi-scale 2D convolution operations are performed on , and the weighted sum of multi-scale convolution results is obtained using learnable weights to obtain weighted fusion features :

[0178]

[0179]

[0180] Finally, the spatial attention weight is generated using Sigmoid activation function, and is applied to , to obtain output multi-scale channel-spatial enhanced features :

[0181]

[0182] In the embodiment, the MSHA module extracts and fuses multi-scale features in the channel dimension through mixed pooling, multi-scale convolution and channel weighting, which can effectively highlight key channels and suppress redundant information; the latter realizes dynamic fusion of global and local features in the spatial dimension through pooling, convolution weighting and spatial attention mechanism, which can highlight the saliency of the target region and improve the perception ability of the spatial distribution. The two work together to enable the model to enhance feature expression in both channel and spatial dimensions, not only improving the adaptability to complex backgrounds, but also significantly enhancing the recognition ability of different scales and fuzzy boundary targets, thereby comprehensively improving the detection accuracy and robustness.

[0183] In some embodiments, the backbone network is provided with a C3ESP module, and the C3ESP module includes a third convolutional layer, a residual branch, a multi-scale perception branch, a seventh splicing layer, and a fourth convolutional layer. For the second input feature input into the C3ESP module:

[0184] Step D1, the electronic device performs a channel compression operation on the second input feature through the third convolutional layer to obtain a compressed feature.

[0185] Step D2, the electronic device performs a multi-scale deep feature extraction operation on the compressed feature through the multi-scale perception branch to obtain a deep feature.

[0186] Step D3, the electronic device splices the deep feature and the compressed feature through the seventh splicing layer to obtain a seventh splicing feature.

[0187] Step D2, the electronic device performs a channel expansion operation on the seventh splicing feature through the fourth convolutional layer to obtain a second output feature corresponding to the second input feature.

[0188] In the backbone network, the electronic device first performs a channel compression operation on the input second input feature through the third convolutional layer, for example, compressing a feature map originally with 256 channels to 64 channels, thereby reducing the calculation amount and highlighting the main information. Subsequently, the compressed feature is sent to the multi-scale perception branch, in which a plurality of deep separable convolutions are sequentially performed for multi-scale feature extraction, for example, a plurality of convolutions are connected in series to gradually expand the receptive field, thereby simultaneously capturing local edge details and larger range of context information, to obtain a deep feature. Then, the electronic device splices the deep feature and the initial compressed feature in the seventh splicing layer to realize complementary fusion of shallow and deep information. Finally, the splicing result performs a channel expansion operation through the fourth convolutional layer, for example, expanding 64 channels back to 256 channels, to ensure that the output feature matches the dimension of the subsequent layers of the network, thereby obtaining the corresponding second output feature.

[0189] In the embodiment, based on the C3ESP module, the electronic device reduces redundant information and computational overhead through channel compression, the multi-scale perception branch extracts rich feature representations under different receptive fields, the seventh concatenation layer realizes effective fusion of deep and shallow features, and the channel expansion ensures complete transmission of information after compression and fusion. The overall design improves the feature expression capability while avoiding information loss and redundant calculation, enabling the model to capture more detailed and diverse features under the premise of ensuring lightweight, thereby enhancing the adaptability and robustness to complex scenes.

[0190] In some embodiments, the multi-scale perception branch includes n ESPBlocks connected in series, the ESPBlock includes a progressive depth separable convolution structure, an eighth concatenation layer, a fifth convolution layer, a PEMA layer, and a ninth concatenation layer; for the third input feature input into the ESPBlock:

[0191] Step E1, the electronic device performs a progressive multi-level depth separable convolution operation on the third input feature through the progressive depth separable convolution structure to obtain depth convolution features of each level. The input corresponding to the first level is the third input feature, and the input of the non-first level is the depth convolution feature corresponding to the previous level;

[0192] Step E2, the electronic device splices the depth convolution features of each level through the eighth concatenation layer to obtain fused depth convolution features.

[0193] Step E3, the electronic device performs a channel fusion operation on the fused depth convolution features through the fifth convolution layer.

[0194] Step E4, the electronic device performs a hybrid attention mechanism on the fused depth convolution features after channel fusion through the PEMA layer to obtain hybrid enhanced features.

[0195] Step E5, the electronic device splices the third input and the hybrid enhanced features through the ninth concatenation layer to obtain the third output feature corresponding to the third input feature.

[0196] In the multi-scale perception branch, the electronic device first performs a multi-level convolution operation on the input third input feature through a progressive depth separable convolution structure. For example, the first convolution layer directly processes the input feature map (such as a feature map of 64x64x12864\ 64\ 64\ 12864×64×128), to obtain the first layer of depth convolution feature; then each non-first level uses the output of the previous layer as input to continue to perform depth separable convolution, and extracts higher-order and more abstract features layer by layer. Then, the electronic device splices the depth convolution features of each level in the channel dimension through the eighth splicing layer to form a fused depth convolution feature, thereby integrating multi-scale information of different levels. Subsequently, the fused feature enters the fifth convolution layer to perform channel fusion operation, for example, compressing the number of channels after splicing from 512 back to 256, to maintain the compactness of the feature dimension and fuse the information of each layer. After that, the PEMA layer applies a hybrid attention mechanism to the feature after channel fusion, highlights key information by dynamically adjusting the channel and spatial weights, and obtains a hybrid enhanced feature. Finally, the electronic device splices the third input feature and the hybrid enhanced feature in the ninth splicing layer to obtain the third output feature corresponding to the third input feature, realizing the integration of shallow layer and enhanced features.

[0197] In this embodiment, through the above processing, the ESPBlock can extract multi-scale features at different levels, and integrate information through splicing and channel fusion, and the PEMA layer further strengthens the key channels and spatial regions, thereby significantly enhancing the expression ability and discrimination ability of the model for complex targets, while maintaining the balance between lightweight calculation and multi-scale perception.

[0198] In some embodiments, Figure 5 The network structure schematic diagram of the C3ESP module and the ESPBlock is shown. For an ESPBlock, assuming that the third input feature is , the ESPBlock will enhance the extraction of features and the flow of information through the following steps:

[0199] Initialize a list Y containing the input X :

[0200]

[0201] For each convolution layer i (from 1 to n), the output of the previous layer is taken as the input of the current layer, and a 3x3 depth separable convolution operation is performed. Here, the input of the first layer is X , and the input of each subsequent layer is the output of the previous layer:

[0202]

[0203] wherein, The convolution result (i.e., deep convolution feature) of each convolution layer will be added to the list Y .

[0204] The deep convolution features of all levels are connected with the initial input , and then a 1x1 convolution layer is used for channel fusion:

[0205]

[0206] Finally, the importance of features with different receptive field sizes is adjusted through the PEMA attention mechanism.

[0207]

[0208] wherein, the PEMA attention mechanism can be represented as:

[0209]

[0210]

[0211]

[0212] By setting the C3ESP module and the series ESPBlock on the basis of the ESP module, the present application can realize efficient extraction and fusion of shallow and deep, multi-scale features by using a double-branch feature shunting mechanism combined with progressive deep separable convolution and PEMA attention mechanism. This design not only enhances the model's representation ability for fine-grained features and targets in complex backgrounds, but also effectively reduces redundant calculations and improves inference efficiency, thereby realizing lightweight calculation while maintaining high precision and significantly improving overall detection performance.

[0213] In some embodiments, the detection method of each of the above embodiments is implemented based on a trained road surface scattering detection model. The road surface scattering detection model cooperatively applies the LMSADet detection head, the MSHA module, and the C3ESP module, and can realize the advantages of modular design. The LMSADet detection head uses multi-scale feature fusion and spatial attention mechanism, can directly perform boundary box regression and class prediction for targets of different scales, optimizes local detail processing, and at the same time avoids unnecessary global calculation of early and middle feature layers, thereby maintaining high precision and improving inference efficiency. The MSHA module uses the MSCA and MSSA sub-modules in series to perform multi-scale feature extraction and fusion in the channel and spatial dimensions respectively, combines channel and spatial attention mechanisms, dynamically highlights key information, and enhances the model's perception ability for complex backgrounds and targets of different scales. The C3ESP module and its internal ESPBlock structure realize efficient extraction and fusion of shallow and deep, multi-scale features through double-branch feature splitting, progressive depth separable convolution, and PEMA attention mechanism, reduce redundant calculation, and enhance fine-grained feature representation ability. The three work together to realize accurate detection of high-precision, multi-scale, and multi-class targets under the premise of lightweight, and significantly improve the robustness and overall performance in complex scenarios.

[0214] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0215] The road surface scattering detection method based on deep learning corresponding to the above embodiments, Figure 6 The structure block diagram of the road surface scattering detection device 6 provided by the embodiments of the present application is shown, and only the parts related to the embodiments of the present application are shown for ease of description.

[0216] Referring to Figure 6 The road surface scattering detection device 6 comprises:

[0217] The extraction module 61 is configured to perform feature extraction on a to-be-detected image based on a backbone network of a pre-trained road surface scattering detection model, to obtain image features; the to-be-detected image comprises a road surface;

[0218] The fusion module 62 is configured to fuse the image features based on a neck network of the road surface scattering detection model, to obtain target fusion features;

[0219] The detection module 63 is configured to detect the target fusion features based on a detection network of the road surface scattering detection model, to obtain a detection result of road surface scattering in the to-be-detected image;

[0220] The detection network comprises an LMSADet detection head, the LMSADet detection head enhances the target fusion feature based on multi-scale fusion and spatial attention, and detects the enhanced target fusion feature to obtain a detection result.

[0221] In the case that the to-be-detected image comprises road surface scattering, the detection result comprises a predicted bounding box of the road surface scattering and a predicted scattering category.

[0222] Optionally, the LMSADet detection head comprises a bounding box regression detection branch for outputting the predicted bounding box, and a category prediction branch for outputting the predicted scattering category, the branch network structures of the bounding box regression detection branch and the category prediction branch are the same; the branch network structure comprises a multi-scale convolution structure, a first concatenation layer, a spatial attention structure, a channel fusion structure, a second concatenation layer, a first DWCBS layer, a second DWCBS layer and a first convolution layer; the detection module 63 comprises a detection unit, configured to:

[0223] For the target fusion feature input into the bounding box regression detection branch or the category prediction branch:

[0224] The multi-scale convolution structure is configured to perform at least two series convolution operations on the target fusion feature in different scales to obtain series convolution features corresponding to the different scales; the series convolution operation comprises a depthwise separable convolution operation and a convolution operation performed in sequence.

[0225] The first concatenation layer is configured to concatenate the features corresponding to the different scales in the channel dimension to obtain first concatenation features.

[0226] The spatial attention structure is configured to perform a channel compression operation, a channel expansion operation and a normalization operation in sequence on the first concatenation features to obtain spatial attention weights corresponding to each scale.

[0227] The channel fusion structure is configured to perform weighted fusion on the spatial attention weights corresponding to each scale and the mixed convolution features corresponding to each scale to obtain spatial enhancement features corresponding to each scale.

[0228] The second concatenation layer is configured to concatenate the spatial enhancement features of the different scales to obtain attention-aware features.

[0229] The first DWCBS layer, the second DWCBS layer and the first convolution layer are configured to perform DWCBS and convolution operations in sequence on the attention-aware features twice to obtain the predicted bounding box or the predicted scattering category.

[0230] The first convolution layer corresponding to the bounding box regression detection branch is configured to set an output channel number as The first convolution layer corresponding to the category prediction branch is configured to set an output channel number as n c .

[0231] Optionally, the neck network is provided with an MSHA module, the MSHA module comprising a MSCA submodule and a MSSA submodule, the MSCA submodule being connected in series with the MSSA submodule, the MSCA submodule being configured to perform channel-attention-based multi-scale feature extraction and fusion in a channel attention dimension; and the MSSA submodule being configured to perform spatial-attention-based multi-scale feature extraction and fusion in a spatial dimension.

[0232] Optionally, the MSCA submodule comprises a first mixed pooling layer, a third concatenation layer, a dimension adjustment layer, a multi-scale convolution layer, a fourth concatenation layer, a second convolution layer, a first activation layer, and a first fusion layer; and the fusion module 62 comprises a fusion unit, the fusion unit being configured to:

[0233] for a first input feature input into the MSHA module:

[0234] performing an average pooling operation and a maximum pooling operation on the first input feature respectively through the first mixed pooling layer to obtain a first pooling result and a second pooling result;

[0235] concatenating the first pooling result and the second pooling result through the concatenation layer to obtain a first mixed pooling result;

[0236] performing different scale convolution operations on the first mixed pooling result after the dimension adjustment through the multi-scale convolution layer to obtain first convolution features corresponding to each scale;

[0237] concatenating the first convolution features of each scale through the third concatenation layer to obtain third concatenation features;

[0238] performing an activation operation on the third concatenation features convolved by the second convolution layer through the first activation layer to obtain channel weights;

[0239] fusing the first input feature and the channel weights through the fusion layer to obtain multi-scale channel enhanced features.

[0240] Optionally, the MSSA submodule comprises a second mixed pooling layer, a fifth concatenation layer, a multi-scale convolution weighting structure, a sixth concatenation layer, a second activation layer, and a second fusion layer; and the fusion unit is specifically configured to:

[0241] for the multi-scale channel enhanced features input into the MSSA submodule:

[0242] performing an average pooling operation and a maximum pooling operation on the multi-scale channel enhanced features respectively through the second mixed pooling layer to obtain a third pooling result and a fourth pooling result;

[0243] concatenating the third pooling result and the fourth pooling result through the fifth concatenation layer to obtain a second mixed pooling result;

[0244] The second mixed pooling result is sequentially subjected to a convolution operation and a weighting operation of a corresponding scale by a multi-scale convolution weighting structure for each scale to obtain a corresponding weighted feature; and the weight corresponding to the weighting operation is obtained through learning.

[0245] The weighted features of each scale are spliced through a sixth splicing layer to obtain a weighted fusion feature.

[0246] The weighted fusion feature is subjected to an activation operation through a second activation layer to obtain a spatial weight.

[0247] The multi-scale channel-enhanced feature and the spatial weight are fused through a fusion layer to obtain a multi-scale channel-spatial enhanced feature.

[0248] Optionally, the backbone network is provided with a C3ESP module, and the C3ESP module includes a third convolution layer, a residual branch, a multi-scale perception branch, a seventh splicing layer, and a fourth convolution layer; the extraction module 61 includes an extraction unit, and the extraction unit is configured to:

[0249] For the second input feature input into the C3ESP module:

[0250] The second input feature is subjected to a channel compression operation through the third convolution layer to obtain a compressed feature.

[0251] The compressed feature is subjected to a multi-scale deep feature extraction operation through the multi-scale perception branch to obtain a deep feature.

[0252] The deep feature and the compressed feature are spliced through the seventh splicing layer to obtain a seventh splicing feature.

[0253] The seventh splicing feature is subjected to a channel expansion operation through the fourth convolution layer to obtain a second output feature corresponding to the second input feature.

[0254] Optionally, the multi-scale perception branch includes n ESPBlocks connected in series, and each ESPBlock includes a progressive deep separable convolution structure, an eighth splicing layer, a fifth convolution layer, a PEMA layer, and a ninth splicing layer; and the extraction unit is specifically configured to:

[0255] For the third input feature input into the ESPBlock:

[0256] The third input feature is subjected to a progressive multi-level deep separable convolution operation through the progressive deep separable convolution structure to obtain deep convolution features of each level; the input corresponding to the first level is the third input feature, and the input of a non-first level is the deep convolution feature corresponding to the previous level;

[0257] The deep convolution features of each level are spliced through the eighth splicing layer to obtain a fused deep convolution feature.

[0258] Channel fusion is performed on the fused deep convolutional features through the fifth convolutional layer;

[0259] By applying a hybrid attention mechanism to the fused deep convolutional features after channel fusion through a PEMA layer, hybrid enhanced features are obtained.

[0260] The third input is concatenated with the hybrid enhancement feature through the ninth concatenation layer to obtain the third output feature corresponding to the third input feature.

[0261] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0262] Figure 7 This is a schematic diagram of the physical layer structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 7 of this embodiment includes: at least one processor 70 ( Figure 7 The diagram shows only one processor, memory 71, and a computer program 72 stored in memory 71 that can run on at least one processor 70. When processor 70 executes computer program 72, it implements the steps in any of the above embodiments of the deep learning-based road spill detection method. Figure 2 Steps 210-230 are shown.

[0263] The processor 70 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0264] In some embodiments, memory 71 may be an internal storage unit of electronic device 7, such as a hard disk or memory of electronic device 7. In other embodiments, memory 71 may also be an external storage device of electronic device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 7.

[0265] Further, the memory 71 can include both an internal storage unit of the electronic device 7 and an external storage device. The memory 71 is used to store operating systems, application programs, BootLoader, data, and other programs, such as program codes of computer programs, and the like. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0266] It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the above-mentioned device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for convenient distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0267] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps in each method embodiment.

[0268] The embodiment of the present application provides a computer program product, which, when running on an electronic device, enables the electronic device to execute the steps in each method embodiment.

[0269] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct related hardware to complete. The above-mentioned computer program can be stored in a computer readable storage medium, and the computer program can implement the steps of each method embodiment when executed by a processor. The above-mentioned computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The above-mentioned computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0270] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0271] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0272] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the above-described apparatus / network device embodiments are only schematic, for example, the division of the above-mentioned modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutually can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0273] The units described as separate parts above can or can not be physically separate, and the parts shown as units can or can not be physical units, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0274] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A deep learning-based method for detecting road spillage, characterized in that, include: The backbone network based on the pre-trained road spill detection model extracts features from the image to be detected, thus obtaining image features; The image to be detected includes the road surface; The neck network of the road spill detection model is used to fuse the image features to obtain the target fused features; The detection network based on the road surface spillage detection model detects the target fusion features to obtain the detection result of road surface spillage in the image to be detected; The detection network includes an LMSADet detection head, which enhances the target fusion features based on multi-scale fusion and spatial attention, and detects the enhanced target fusion features to obtain the detection result. If the image to be detected includes road spillage, the detection result includes the predicted bounding box of the road spillage and the predicted spillage category; The LMSADet detection head includes a bounding box regression detection branch for outputting the predicted bounding box and a category prediction branch for outputting the predicted spray category. The branch network structures of the bounding box regression detection branch and the category prediction branch are identical. The branch network structure includes a multi-scale convolutional structure, a first concatenation layer, a spatial attention structure, a channel fusion structure, a second concatenation layer, a first DWCBS layer, a second DWCBS layer, and a first convolutional layer. Both the first DWCBS layer and the second DWCBS layer include depthwise separable convolution, normalization, and the SiLU function. The target fusion feature is applied to the input bounding box regression detection branch or the category prediction branch. The target fusion feature is subjected to at least two scales of concatenated convolution operations through the multi-scale convolution structure to obtain concatenated convolution features corresponding to each scale; the concatenated convolution operation includes depthwise separable convolution operation and ordinary convolution operation performed sequentially. The first stitching layer stitches together the features corresponding to each scale along the channel dimension to obtain the first stitched feature. The spatial attention structure is used to sequentially perform channel compression, channel expansion and normalization operations on the first spliced ​​feature to obtain the spatial attention weights corresponding to each scale. For each scale, the corresponding spatial attention weights and the corresponding concatenated convolutional features are weighted and fused through the channel fusion structure to obtain the corresponding spatial enhancement features; The spatial enhancement features at various scales are stitched together by the second stitching layer to obtain attention perception features; The attention-aware features are subjected to two DWCBS operations and one convolution operation in sequence through the first DWCBS layer, the second DWCBS layer and the first convolutional layer to obtain the predicted bounding box or the predicted scattering category. Wherein, the number of output channels of the first convolutional layer corresponding to the bounding box regression detection branch is set to be The number of output channels set for the first convolutional layer corresponding to the category prediction branch is n c The This indicates the number of output channels for the predicted bounding box; 4 represents the four regression components of the bounding box. reg_max This represents the maximum value of the discrete regression interval corresponding to each regression component; the... n c This indicates the predicted number of categories for the scattering type.

2. The method for detecting road spillage as described in claim 1, characterized in that, The neck network is equipped with an MSHA module, which includes an MSCA submodule and an MSSA submodule. The MSCA submodule is connected in series with the MSSA submodule. The MSCA submodule is used to perform multi-scale feature extraction and fusion based on channel attention in the channel attention dimension. The MSSA submodule is used to perform multi-scale feature extraction and fusion based on spatial attention in the spatial dimension.

3. The method for detecting road spillage as described in claim 2, characterized in that, The MSCA submodule includes a first hybrid pooling layer, a third concatenation layer, a dimension adjustment layer, a multi-scale convolutional layer, a fourth concatenation layer, a second convolutional layer, a first activation layer, and a first fusion layer; for the first input feature input to the MSHA module: The first input feature is subjected to average pooling and max pooling operations by the first hybrid pooling layer to obtain the first pooling result and the second pooling result. The first pooling result and the second pooling result are combined through the third splicing layer to obtain the first mixed pooling result; The first hybrid pooling result after dimensionality adjustment is subjected to convolution operations of different scales through the multi-scale convolutional layer to obtain the first convolutional features corresponding to each scale. The third convolutional feature is obtained by concatenating the first convolutional features at each scale through the fourth concatenation layer. The first activation layer performs an activation operation on the third concatenated feature that has been convolved by the second convolutional layer to obtain the channel weights. The first input feature is fused with the channel weights through the fusion layer to obtain multi-scale channel enhancement features.

4. The method for detecting road spillage as described in claim 3, characterized in that, The MSSA submodule includes a second hybrid pooling layer, a fifth concatenation layer, a multi-scale convolutional weighted structure, a sixth concatenation layer, a second activation layer, and a second fusion layer; for the multi-scale channel enhancement features input to the MSSA submodule: The second hybrid pooling layer performs average pooling and max pooling operations on the multi-scale channel enhancement features to obtain the third pooling result and the fourth pooling result. The third pooling result and the fourth pooling result are combined through the fifth splicing layer to obtain the second mixed pooling result; For each scale, the second hybrid pooling result is sequentially subjected to convolution and weighting operations of the corresponding scale through the multi-scale convolution weighting structure to obtain the corresponding weighted features; the weights corresponding to the weighting operations are obtained through learning. The weighted features at each scale are stitched together through the sixth stitching layer to obtain weighted fused features; The spatial weights are obtained by performing an activation operation on the weighted fusion features through the second activation layer. The multi-scale channel enhancement features are fused with the spatial weights through the fusion layer to obtain multi-scale channel-spatial enhancement features.

5. The method for detecting road surface spillage as described in any one of claims 1-4, characterized in that, The backbone network is equipped with a C3ESP module, which includes a third convolutional layer, a residual branch, a multi-scale perceptual branch, a seventh splicing layer, and a fourth convolutional layer; the second input feature input to the C3ESP module is: The second input feature is compressed by performing channel compression on the third convolutional layer to obtain compressed features; The compressed features are subjected to multi-scale deep feature extraction operations through the multi-scale sensing branch to obtain deep features; The deep features and the compressed features are spliced ​​together through the seventh splicing layer to obtain the seventh splicing feature; The fourth convolutional layer performs a channel expansion operation on the seventh concatenated feature to obtain the second output feature corresponding to the second input feature.

6. The method for detecting road spillage as described in claim 5, characterized in that, The multi-scale perceptual branch comprises n cascaded ESPBlocks, each ESPBlock including a progressively deep separable convolutional structure, an eighth concatenation layer, a fifth convolutional layer, a PEMA layer, and a ninth concatenation layer; the third input feature input to the ESPBlock is: The progressively deep separable convolutional structure is used to perform progressively multi-level deep separable convolutional operations on the third input feature to obtain deep convolutional features at each level. The input corresponding to the first level is the third input feature, and the input to non-first levels is the depthwise convolutional feature corresponding to the previous level; The eighth stitching layer stitches together the deep convolutional features of each layer to obtain fused deep convolutional features. The channel fusion operation is performed on the fused deep convolutional features through the fifth convolutional layer; The PEMA layer performs a hybrid attention mechanism on the fused depthwise convolutional features after channel fusion to obtain hybrid enhanced features; The third input feature is concatenated with the hybrid enhancement feature through the ninth concatenation layer to obtain the third output feature corresponding to the third input feature.

7. A road surface spillage detection device, characterized in that, include: The extraction module is used to extract features from the image to be detected based on the backbone network of the pre-trained road spill detection model, and obtain image features. The image to be detected includes the road surface; The fusion module is used to fuse the image features based on the neck network of the road spill detection model to obtain the target fused features; The detection module is used to detect the target fusion features based on the detection network of the road surface spillage detection model, and obtain the detection result of road surface spillage in the image to be detected; The detection network includes an LMSADet detection head, which enhances the target fusion features based on multi-scale fusion and spatial attention, and detects the enhanced target fusion features to obtain the detection result. If the image to be detected includes road spillage, the detection result includes the predicted bounding box of the road spillage and the predicted spillage category; The LMSADet detection head includes a bounding box regression detection branch for outputting the predicted bounding boxes and a category prediction branch for outputting the predicted spray categories. The branch network structures of the bounding box regression detection branch and the category prediction branch are identical. The branch network structure includes a multi-scale convolutional structure, a first stitching layer, a spatial attention structure, a channel fusion structure, a second stitching layer, a first DWCBS layer, a second DWCBS layer, and a first convolutional layer. Both the first DWCBS layer and the second DWCBS layer include depthwise separable convolution, normalization, and the SiLU function. The detection module includes a detection unit, which is used for: Target fusion features for the input bounding box regression detection branch or class prediction branch: By performing concatenated convolution operations at at least two scales on the target fusion features through a multi-scale convolution structure, the concatenated convolution features corresponding to each scale are obtained; the concatenated convolution operations include depthwise separable convolution operations and ordinary convolution operations performed sequentially. The first stitching layer stitches the features corresponding to each scale along the channel dimension to obtain the first stitched feature. The spatial attention structure is used to sequentially perform channel compression, channel expansion and normalization operations on the first spliced ​​feature to obtain the spatial attention weights corresponding to each scale. For each scale, the corresponding spatial attention weights are weighted and fused with the corresponding concatenated convolutional features through a channel fusion structure to obtain the corresponding spatial augmentation features; The spatial enhancement features at various scales are stitched together by the second stitching layer to obtain attention perception features; By performing two DWCBS operations and one convolution operation on the attention-aware features through the first DWCBS layer, the second DWCBS layer and the first convolutional layer, the predicted bounding box or the predicted scattering category is obtained. The first convolutional layer corresponding to the bounding box regression detection branch has the following output channel number: The number of output channels for the first convolutional layer corresponding to the category prediction branch is set to... n c The This indicates the number of output channels for the predicted bounding box; 4 represents the four regression components of the bounding box. reg_max This represents the maximum value of the discrete regression interval corresponding to each regression component; the... n c This indicates the total number of predicted scattering categories.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the deep learning-based road spill detection method as described in any one of claims 1 to 6.

9. A computer program product, said computer program product storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the deep learning-based road spill detection method as described in any one of claims 1 to 6.

Images