Real-time road disease recognition embedded method and system based on lightweight CNN and attention mechanism

Abstract

The application discloses a kind of based on lightness CNN and attention mechanism's road disease real-time identification embedded method and system.The method is by constructing integrated reconfigurable convolution and channel attention light weight feature extraction module, using grouping convolution and multi-scale feature fusion network of cross-stage connection and configuring three levels of prediction head focusing on different size diseases, significantly improve the feature perception and positioning ability to subtle disease.Model training uses the compound loss function of fusion classification, regression and attention perception, and is optimized in combination with course learning, difficult example mining and other adaptive strategies.Finally, through network pruning, quantization and knowledge distillation technology, the model size is compressed, and finally deployed on edge computing device, using inference acceleration engine to realize high-precision, low-delay real-time detection and response of road disease.The application effectively solves the problem of low detection accuracy and slow speed of small target disease detection on embedded terminal in the prior art.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and embedded artificial intelligence, and in particular to an embedded method and system for real-time road defect recognition based on lightweight CNN and attention mechanism, which is suitable for deployment on edge devices with limited computing resources. Background Technology

[0002] With the continuous expansion of my country's transportation infrastructure, road maintenance and management are facing increasingly severe challenges. Traditional manual inspection methods are not only inefficient and costly, but also pose safety hazards, making it difficult to meet the real-time and precision requirements of modern road maintenance. In recent years, automated detection technology based on computer vision has become a research hotspot, among which deep learning object detection algorithms, especially the YOLO series models, have shown great potential in road defect identification due to their high detection speed.

[0003] However, applying deep learning models to real-time road defect detection still faces significant technical bottlenecks. First, typical road defects such as cracks and potholes are usually small targets in images, with limited feature information and low pixel count. General-purpose target detection models are insufficient to perceive these targets, resulting in numerous false negatives and missed detections. Second, aerial or road images captured by drones or vehicle-mounted equipment have complex backgrounds, and the morphology, scale, and lighting conditions of defects vary greatly, placing extremely high demands on the robustness of the models. More importantly, real-world scenarios often require deploying detection algorithms on embedded terminal devices with limited computing resources, storage space, and power consumption. High-precision models typically have high computational complexity and a large number of parameters, making it difficult to meet the response speed requirements of real-time detection.

[0004] Therefore, developing a lightweight deep learning model that combines high detection accuracy, good robustness, and low computational complexity, and can be effectively deployed on embedded devices to achieve real-time road defect identification, has become a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a real-time road defect identification method and system with high detection accuracy, fast reasoning speed, and low resource consumption, which can be effectively deployed in embedded devices.

[0006] In a first aspect, embodiments of this application provide an embedded method for real-time road defect recognition based on lightweight CNN and attention mechanism, the method comprising:

[0007] Feature extraction is performed on the road image to be identified. Channel segmentation is used to divide the input feature map into two branches. In the first branch, a reconstructable convolution with multiple paths is applied to perform feature transformation. The feature maps output by the two branches are fused. Then, the channel dimension of the fused feature map is weighted by a channel attention mechanism.

[0008] The extracted features are input into a feature fusion network, where grouped convolutions are used for feature processing, and multi-scale features from different depths of the neural network are fused through a cross-stage connection structure based on grouping bottlenecks.

[0009] The fused multi-scale features are input into three prediction heads of different scales, which make predictions based on large, medium and small feature maps respectively, to detect road damage targets of different sizes in the road image.

[0010] During the model training phase, a composite loss function is used for optimization. The composite loss function includes at least a classification loss for handling class imbalance and a regression loss suitable for small target localization.

[0011] The trained model is pruned to remove redundant connections, and the pruned model is deployed on edge computing devices through an inference acceleration engine to perform real-time road damage identification and response to the input road image stream.

[0012] Secondly, embodiments of this application provide an embedded system for real-time road defect recognition based on lightweight CNN and attention mechanism, applied to the embedded method for real-time road defect recognition based on lightweight CNN and attention mechanism as described in the first aspect, the system comprising:

[0013] The lightweight feature extraction module is used to extract features from the road image to be identified. It uses channel segmentation to divide the input feature map into two branches. In the first branch, a reconstructable convolution with multiple paths is applied to perform feature transformation. The feature maps output by the two branches are fused, and then the channel dimension of the fused feature map is weighted by a channel attention mechanism.

[0014] The multi-scale feature fusion module is used to input the extracted features into the feature fusion network, where grouped convolutions are used for feature processing, and multi-scale features from different depths of the neural network are fused through a cross-stage connection structure based on grouping bottlenecks.

[0015] A multi-level prediction module is used to input the fused multi-scale features into three prediction heads of different scales. The prediction heads make predictions based on large, medium and small feature maps respectively, so as to detect the disease targets of different sizes in the road image.

[0016] The model optimization module is used to optimize the model during the training phase using a composite loss function, which includes at least a classification loss for handling class imbalance and a regression loss suitable for small target localization.

[0017] The model compression module prunes the trained model to remove redundant connections and deploys the pruned model on edge computing devices through an inference acceleration engine to perform real-time road image stream defect identification and response.

[0018] Thirdly, embodiments of this application provide an electronic device, including:

[0019] processor;

[0020] Memory used to store processor-executable instructions;

[0021] The processor is configured to implement the embedded method for real-time road defect recognition based on lightweight CNN and attention mechanism as described in the first aspect when executing the instructions.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the embedded method for real-time road defect recognition based on lightweight CNN and attention mechanism as described in the first aspect.

[0023] Based on the technical solution of this invention, its beneficial effects are mainly reflected in the following five aspects:

[0024] 1. Significantly Improved Detection Accuracy: By leveraging channel segmentation and reconfigurable convolutional structures in the lightweight feature extraction module, combined with a multi-scale feature fusion mechanism and a three-level prediction head optimized for different scales of defects, the model's ability to extract and locate features of small targets such as cracks and pits is effectively enhanced. The introduction of a composite loss function further synergistically optimizes classification accuracy and bounding box regression quality.

[0025] 2. Breakthrough in model efficiency: By adopting lightweight feature extraction modules and grouped convolutions in the feature fusion network, combined with multi-stage structured pruning, quantized perceptual training, and knowledge distillation techniques, the computational complexity and number of parameters of the model are significantly reduced while maintaining accuracy, achieving excellent lightweight performance.

[0026] 3. Excellent edge deployment capability: The optimized model, through operator fusion and memory optimization of the inference acceleration engine, combined with dynamic batch processing and multi-level caching mechanism, can achieve high frame rate real-time inference on embedded devices such as Jetson TX2, meeting the real-time requirements of road inspection.

[0027] 4. Enhanced environmental adaptability: Through improved channel attention mechanism and adaptive training strategies (including course learning, hard example mining and adversarial training), the robustness and generalization ability of the model under complex lighting, occlusion and diverse background conditions are significantly improved.

[0028] 5. High degree of technical integration and strong practicality: It integrates lightweight CNN architecture, attention mechanism, model compression technology and edge deployment scheme into a complete technical solution from algorithm innovation to engineering implementation, which has good prospects for industrial application. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of an embedded method for real-time road defect recognition based on lightweight CNN and attention mechanism, provided in an embodiment of this application.

[0030] Figure 2 This application provides an embedded system architecture diagram for real-time road defect recognition based on lightweight CNN and attention mechanism.

[0031] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0033] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0034] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] Example 1

[0036] Figure 1 This is a schematic diagram of an embedded method for real-time road defect recognition based on a lightweight CNN and attention mechanism, provided as an embodiment of this application. Figure 1 As shown, an embedded method for real-time road defect recognition based on lightweight CNN and attention mechanism includes:

[0037] S1. Feature extraction is performed on the road image to be identified. Channel segmentation divides the input feature map into two branches. In the first branch, a reconstructable convolution with multiple paths is applied for feature transformation. The feature maps output from the two branches are then fused, and a channel attention mechanism is used to weight the channel dimensions of the fused feature map. This is the foundational feature extraction layer of the entire model. Its core task is to extract rich and effective features from the input image with extremely low computational cost, especially for small targets like road defects. Specifically, channel segmentation distributes the computational task across two branches, reducing the computational load of a single branch. Reconstructable convolution enhances feature representation capabilities during training using a multi-path structure, and merges them into a single structure during inference to ensure speed, achieving powerful training and efficient inference. The channel attention mechanism automatically selects and strengthens feature channels crucial for defect identification, suppressing useless background information, thereby improving feature quality.

[0038] Specifically, in this embodiment, the feature extraction step includes:

[0039] The input feature map is uniformly divided into two sub-feature maps along the channel dimension; this reduces computational complexity and enables parallel feature processing. Assume the input feature map is... Where C is the number of channels, and H and W are the height and width, respectively. Channel segmentation divides it evenly into two feature maps: This operation halves the number of channels in subsequent convolution operations, according to the formula for convolution computation complexity. (where K is the kernel size,) The number of floating-point operations is a key indicator for measuring the computational complexity of a model or operation. This significantly reduces computational complexity (referring to the number of channels in the input / output feature maps). It lays the foundation for employing different strategies in the subsequent two branches.

[0040] The sub-feature maps of the first branch are processed. During the training phase, a reconstructable convolutional structure with three parallel branches is used: a 1×1 convolutional layer, a 3×3 convolutional layer, and an identity connection. The output features of each branch are summed and fused. During the inference phase, the multi-branch structure is reconstructed into a single 3×3 convolutional layer through parameter equivalence transformation. This enhances feature diversity during training and maintains high efficiency during inference. This is the core application of structural reparameterization technology. Specifically, during training (multi-branch enhancement): the three branches (1×1 convolution, 3×3 convolution, and identity connection) extract features from different receptive fields and levels, and their outputs are summed and fused. , The sum of the feature maps output from the three branches during the training phase integrates features from different receptive fields and levels. FInput feature map : Use a 1×1 convolution kernel to perform a convolution operation on F, extract local features and change the number of channels. We use a 3×3 convolution kernel to perform a convolution operation on F to extract features with a larger receptive field. Identity mapping, which directly outputs F itself while preserving the original features, greatly enriches feature representation and improves model capacity and performance. During inference (structural reorganization): multi-branch structures are merged into a single 3×3 convolution through parameter equivalence transformation. Specifically, 1×1 convolution kernels are padded to 3×3, and identity connections are transformed into a specific 3×3 convolution kernel. Finally, the weights and biases of all branches are summed and fused into an equivalent 3×3 convolutional layer weight. and bias .in, , The output feature map during the inference phase. This achieves the ideal of high performance during training and fast speed during inference, and is the key to balancing accuracy and efficiency.

[0041] The processed features from the first branch and the second branch are fused. Channel attention is applied to the fused feature map through global average pooling, followed by learning the correlation between channels via two fully connected layers. A sigmoid function is used to generate weight coefficients for each channel, and finally, these weight coefficients are multiplied channel-by-channel with the fused feature map. This adaptively calibrates the channel feature responses, highlighting important features and suppressing secondary features. Specifically, let the fused feature map be... The first step: compression. Global average pooling compresses the spatial information of each channel into a global descriptor. .in, `c` is the global descriptor (also known as the squeezed output) for the c-th channel. It is a scalar value representing the global average response intensity of all spatial location features on the c-th channel. `c` is the channel index of the feature map. Its value ranges from 1 to C (the total number of channels). `H` is the height of the feature map (in pixels or feature points). `W` is the width of the feature map (in pixels or feature points). It is the feature value of the input feature map X at the c-th channel and spatial position (i,j).

[0042] Step 2: Activation. The nonlinear relationships between channels are learned through two fully connected layers (forming a bottleneck structure), and weights are output. , where z is the compressed vector. and These are the weights of the fully connected layer. δσ is the ReLU activation function, and σ is the Sigmoid function. The resulting s is a channel weight vector between 0 and 1. z is the global descriptor vector obtained from the previous formula.

[0043] Step 3: Reweighting, multiplying the learned weights with the original feature map channel by channel: ,in, The feature map of the c-th channel after calibration (reweighting). It is the weight value in the weight vector s corresponding to the c-th channel. This is the feature map of the c-th channel of the original input. Based on the weights... The size is scaled for each channel of the original feature map X. Important feature channels are magnified ( (Approaching 1), unimportant feature channels are weakened ( (Close to 0). This process allows the model to autonomously learn which feature channels are more important for the current detection task, thereby significantly improving the feature representation ability.

[0044] Three steps constitute a highly efficient feature extraction unit: segmentation reduces computational load; reconfigurable convolution enhances feature representation without sacrificing inference speed; and channel attention refines and filters features. These three elements work together to ensure the lightweight feature extraction module delivers outstanding performance in both accuracy and efficiency.

[0045] Specifically, in this embodiment, the step of applying channel attention is implemented using an improved, efficient channel attention mechanism, which specifically includes:

[0046] After performing global average pooling on the fused feature map, a one-dimensional convolution is used to replace the two fully connected layers for inter-channel information interaction. The kernel size of the one-dimensional convolution is adaptively determined through a non-linear mapping of the number of channels; this significantly reduces the number of parameters and captures cross-channel interactions. This is a key lightweight improvement over classic channel attention modules (such as SENet). In traditional methods, the number of parameters in two fully connected layers (bottleneck structure) is... Where r is the reduction rate. When C is large, the number of parameters is enormous. However, a one-dimensional convolution (kernel size k) can be used to directly process the vector after global average pooling. z Its parameter count is only k×C. For adaptive kernel size, the kernel size k is not fixed, but is adaptively determined through a nonlinear mapping of the number of channels C, as shown in formula [example formula]. (Where b and γ are hyperparameters). This ensures that network layers with different numbers of channels can have a matching range of interactions. This step, while avoiding the side effects of dimensionality reduction, greatly reduces the number of parameters and computational cost of the attention module, making it more suitable for lightweight networks.

[0047] A channel grouping strategy is introduced during the weight generation process. Feature channels are divided into multiple groups, and attention weights are calculated for each group separately. This enhances the modeling of correlations between local channels, reduces computational complexity, and captures more refined intra-group channel correlations. All C channels are divided into G groups, each group having... Each group has 10 channels. Then, the one-dimensional convolutional attention mechanism described above is applied independently to each group. This has two advantages: 1. The computational cost is further reduced: the operation is changed from operating on the entire C-dimensional vector to operating on G channels. Operations on dimensional vectors have a computational complexity of... The magnitude was reduced to O( 2. Modeling more accurate local relationships: When learning attention, the model is forced to focus on closely related channels within a group first. This is usually more effective than directly modeling the global relationships of all channels, especially for different semantic parts of the feature map.

[0048] A batch normalization (BN) layer is added before the sigmoid activation function to stabilize the weight distribution during training, accelerate training convergence, and improve the stability of the attention weights. The BN layer normalizes the output of the one-dimensional convolution, making its mean 0 and variance 1. The formula is: ,in and These are the mean and variance of the current batch, and γ and β are learnable scaling and offset parameters. This step mitigates internal covariate bias, making the training process more stable. It pushes activation values ​​towards the sensitive region (rather than the saturation region) of the sigmoid function, resulting in larger gradients and accelerating model convergence. This makes the final distribution of the attention weights s more reasonable and stable.

[0049] The final generated attention weights are then weighted channel-by-channel with the fused feature map, and the weighted result is residually concatenated with the fused feature map to form a dual feature enhancement mechanism. This mechanism enhances important features while preserving the integrity of the original information and preventing information loss. Specifically, firstly, standard attention weighting is performed: Then, a residual connection is introduced to add the weighted features to the original input features: Even if the attention module makes a mistake (the weights of some channels are too low), the original feature information can still be preserved through the residual path, avoiding gradient vanishing and network degradation. Important features are amplified, while unimportant features remain largely unchanged. This forms a more robust and powerful feature enhancement mechanism.

[0050] These four steps together constitute an efficient, stable, and powerful lightweight attention module. It achieves lightweighting through one-dimensional convolution and grouping strategies, ensures training stability through batch normalization, and finally ensures information integrity through residual connections, thereby greatly improving the quality of feature extraction without significantly increasing computational costs.

[0051] S2. The extracted features are input into the feature fusion network, where grouped convolutions are used for feature processing, and multi-scale features from different depths of the neural network are fused through a cross-stage connection structure based on grouping bottlenecks. This is the model's information integration and enhancement layer. Its task is to effectively fuse feature maps of different depths (i.e., different scales) output from the basic feature extraction layer, so that the features at each level simultaneously contain rich semantic information (from deep layers) and precise location information (from shallow layers). Through grouped convolutions and cross-stage connections, the interaction and fusion of multi-scale features are achieved with lower computational cost. This step ensures that the model has a strong perception capability for both large-area mesh cracks (requiring deep semantic information) and fine cracks (requiring shallow location details).

[0052] Specifically, in this embodiment, the feature fusion network achieves multi-scale feature fusion through a cross-stage connection structure, specifically including:

[0053] In the main path, multiple concatenated grouped convolutional layers are used to process the input features. Each grouped convolutional layer is followed by batch normalization and activation functions to extract and transmit deep semantic features, perform deep and efficient feature transformations, and refine high-level semantic information.

[0054] Specifically, the input channels are divided into several groups, and convolution is performed within each group. The computational cost (FLOPs) is calculated as follows: , where g is the number of groups. Compared to standard convolution ( This significantly reduces computational and parameter requirements. By stacking multiple grouped convolutional layers, a lightweight deep network is constructed, progressively expanding the receptive field and extracting more abstract and semantic features. Batch normalization (BN) and activation functions ensure training stability and non-linear expressive power.

[0055] In the auxiliary path, the input features are adjusted for channel dimensions using 1×1 convolutions; the features output by the main path are then connected across layers with the features processed by the auxiliary path. The auxiliary path adjusts the channel dimensions of the input features using 1×1 convolutions and connects them across layers with the output features of the main path, preserving and transmitting the details and positional information of shallow features, and fusing them with deeper features. The 1×1 convolution primarily transforms the channel dimensions (upgrading or downgrading) to match the number of output channels of the auxiliary path with the main path for subsequent fusion. Its computation is lightweight. Cross-layer connections: This embodies the idea of ​​Feature Pyramid Network (FPN). It directly transmits shallow features (or input features) containing rich spatial details, which have not been processed by deep networks, to the fusion point. This effectively compensates for the small target positional information that may be lost in the main path due to multiple downsampling.

[0056] At cross-layer connections, a spatial attention mechanism is used to assign weights to features from the main path and auxiliary paths, selectively fusing feature information from different receptive fields; this achieves adaptive and intelligent feature fusion, rather than simple addition or splicing. This is the core innovation of this invention. It recognizes that the importance of the main path (deep semantic features) and auxiliary paths (shallow detail features) differs at different spatial locations. The fusion formula can be a weighted fusion: ,in, It is a main path feature. These are auxiliary path features. The model can learn autonomously—relying more on the semantic information of the main path for judgment in flat background areas, and relying more on the detailed information of the auxiliary path for accurate localization in disease edges and textured areas. This adaptive selection capability greatly improves the efficiency of feature fusion and the accuracy of the final detection.

[0057] In the grouped convolution process, a channel shuffling operation is employed to promote information interaction between different channel groups. This channel shuffling operation facilitates information interaction between different channel groups, avoids feature redundancy, and addresses the information isolation side effect of grouped convolution. Previously, each convolutional group only operated within its own group, resulting in no information exchange between groups, potentially leading to feature redundancy or decreased representational power. After grouped convolution, the output feature channels are systematically rearranged, interleaving the feature channels generated by different groups. This is equivalent to introducing cross-group information into the input of the next layer of grouped convolution without increasing computational cost, thus opening up information flow and ensuring the richness and diversity of features.

[0058] The selectively fused output features are added to the input features via skip connections to form a residual learning structure. Finally, the output features are added to the module input features via skip connections to form a residual learning structure, mitigating the vanishing gradient problem. This ensures effective gradient backpropagation, stabilizes deep network training, and preserves the original information flow. This is a direct application of the ResNet residual concept. Let the module input be X, and the complex transformation function of the entire cross-stage connection module be F(X). Output formula: (in (These are the features fused after all the aforementioned steps). During backpropagation, gradients can be directly propagated back through the shortcut of skip connections, making the network easy to train even if it is very deep. This ensures that even if the learning effect of module F is not ideal, at least the information of the input X is preserved, avoiding performance degradation. This allows the network to confidently stack more such modules to build a powerful feature fusion network.

[0059] These five steps collectively define an efficient, intelligent, and stable feature fusion module. It extracts features from both shallow and deep layers through a dual-path structure, achieves adaptive fusion through spatial attention, ensures information flow through channel shuffling, and finally guarantees training stability through residual connections. This combined approach perfectly supports the goal of achieving efficient fusion of multi-scale features as stated in claim 1, and is a key architecture for improving the detection performance of small-target diseases.

[0060] S3. The fused multi-scale features are input into three prediction heads of different scales. These prediction heads predict based on large, medium, and small feature maps respectively to detect road damage targets of different sizes in the road image. This is the final detection and output layer of the model. Its task is to specializedly detect damage targets of different sizes on feature maps of different resolutions based on the fused multi-scale features. Specifically, the small-scale prediction head (160×160) uses high-resolution feature maps to focus on detecting minute cracks and edges with extremely small pixel proportions. The medium-scale prediction head (80×80) balances semantics and details to detect medium-sized potholes. The large-scale prediction head (40×40) uses strong semantic information to detect large-area damage. This clearly defined design directly addresses the core difficulty of varying road damage scales, significantly improving the recall rate and localization accuracy for small targets.

[0061] Specifically, in this embodiment, the three prediction heads at different scales are optimized and coordinated in the following manner to establish a specialized detection pipeline for diseases at different scales:

[0062] For the 160×160 small-scale feature map, a first prediction head is configured to focus on detecting slender disease targets with crack and pit edges, utilizing its high spatial resolution. This feature map has the highest resolution, requires fewer downsampling passes through the network, and retains the richest spatial details and pixel-level information. Its function is to act as a microscope, specifically for capturing and locating tiny targets such as cracks and pit edges that occupy only a few dozen pixels in the image. This directly addresses the problem mentioned in the background technique that small target objects typically occupy only a few dozen pixels.

[0063] For an 80×80 mesoscale feature map, a second prediction head is configured to strike a balance between semantic information and spatial detail, focusing on identifying medium-sized potholes and localized damage. This feature map achieves a balance between semantic information and spatial detail. Its function is to act as a regular observer, identifying potholes and localized damage with medium size and certain macroscopic features.

[0064] A third prediction head is configured for the 40×40 large-scale feature map, utilizing its rich deep semantic information for overall judgment to detect large-area damage and network cracks. This feature map has been downsampled multiple times, resulting in the largest receptive field and containing the richest contextual and semantic information. Its function is to act as a wide-angle lens, judging from a macroscopic perspective what constitutes damage such as large-area damage and network cracks that require global information for accurate identification.

[0065] A decoupled design is employed for each prediction head, separating its classification and regression tasks. This mitigates the conflict between the two tasks, improving classification accuracy and localization precision. In traditional coupled detection heads, the same convolutional feature is used simultaneously to predict both class probabilities and bounding box coordinates. Since the needs of these two tasks may conflict (e.g., a feature contributes to classification but is noise for localization), they can interfere with each other. The decoupled head provides independent, small branch networks for each task. The classification branch focuses on learning feature patterns related to object categories; the regression branch focuses on learning feature shifts related to object position and shape. This design allows the model to learn more specific feature representations for different tasks, thereby independently and synchronously improving classification confidence and bounding box regression quality.

[0066] A feature interaction mechanism is established between prediction heads at three scales, achieving collaborative optimization of multi-scale features through top-down semantic transmission and bottom-up detail supplementation. This breaks down scale barriers, enabling cross-level semantic transmission and detail supplementation, enhancing feature representations at all scales. Top-down semantic transmission: Deep, high-semantic features (e.g., from a 40×40 layer) are upsampled and fused with shallow features (e.g., from an 80×80 layer). This injects powerful semantic information into the detail layer, helping small- and medium-scale prediction heads understand the global context of the target while retaining detail, reducing false detections. Bottom-up detail supplementation: Shallow, high-resolution features (e.g., from a 160×160 layer) are downsampled or transformed and fused with deep features. This feeds back fine positional details to the semantic layer, helping large-scale prediction heads refer to more accurate edge information during macroscopic judgments. This two-way interaction forms a closed-loop feature enhancement network, enabling each predictor to leverage its own scale advantages while also receiving information support from other scales, thereby comprehensively improving the model's detection performance at different scales and achieving a synergistic effect of 1+1+1>3.

[0067] S4. During the model training phase, a composite loss function is used for optimization. This composite loss function includes at least a classification loss to handle class imbalance and a regression loss suitable for small object localization. This is the model compression and acceleration phase. The trained model is slimmed down without significantly affecting accuracy. Redundant weights and connections are removed, significantly reducing the number of parameters and computational cost (FLOPs). This step is crucial for the successful deployment of the model on embedded devices, directly determining its size and operating efficiency.

[0068] Specifically, in this embodiment, the composite loss function used in the model training phase consists of a weighted average of classification loss, regression loss, and attention-aware loss, unifying the optimization objective and balancing the three tasks of classification, regression, and feature learning. Its definition is as follows:

[0069] .

[0070] Three independent loss functions are weighted by coefficients. This is combined into a single overall goal. This allows the model to simultaneously receive gradient signals from classification accuracy, localization accuracy, and feature focusing ability during training, guiding the network to learn an optimal state that performs well in all aspects. The weight coefficients can be adjusted according to different task priorities; for example, they can be increased later in training. To further enhance feature learning.

[0071] The classification loss employs a focus loss function to mitigate the imbalance in road damage categories. The classification branch also uses a focus loss function to address class imbalance, reshaping the standard cross-entropy loss so that model training focuses on samples that are difficult to classify (especially positive samples). Road images are predominantly background (negative samples), with damage (positive samples) making up a very small percentage, leading to severe class imbalance. The model tends to predict these as background. The focus loss addresses the extreme class imbalance between the foreground (damage) and the background (normal road surface). The formula is: ,in, For classification loss, an adaptive focus loss function is used. It is the model's predicted probability of the true class. It is a focusing parameter used to reduce the weight of easily classified samples. It is the category weight coefficient, used to balance the imbalance between positive and negative samples and between different disease categories.

[0072] The regression loss is based on normalized Wasserstein distance, used to enhance the model's sensitivity to the localization of small-sized disease targets; it also provides a more comprehensive bounding box similarity measure, optimizing the regression process. Regression loss formula: This provides a loss metric that is smoother and more sensitive to small-scale bounding box regression. For small targets like road defects, traditional IoU-based losses (such as CIoU) are highly unstable. Even small positional deviations can cause the IoU value to drop sharply or even to zero, leading to gradient vanishing and making optimization difficult. NWD (Normalized Wasserstein Distance), on the other hand, offers a more stable and sensitive loss metric. The core idea is to model the bounding box as a 2D Gaussian distribution and then calculate the Wasserstein distance between the two distributions. This distance is a smooth metric even when there is no overlap. (The formula...) Distance from the center point of the penalty. Difference between the width and height of the penalty.

[0073] For regression loss, a penalty is applied to the regression of the detection boxes (center point, width, and height) to amplify and smoothly feed back the error of small targets to the network. A normalized Wasserstein distance loss is used, wrapped in an exponential outer layer, so that as the NWD (i.e., the more similar the predicted box is to the ground truth box) increases, the error is reduced. The NWD tends towards 0; the smaller the NWD (lower the similarity), the lower the similarity. Tendency toward 1. To normalize the Wasserstein distance, here and These represent the center coordinates, width, and height of the predicted bounding box and the ground truth bounding box, respectively, where k is the scale normalization coefficient. Since the gradient is constant, this loss function is more sensitive and stable to the regression of small bounding boxes. This loss function provides a stable and effective gradient signal for the regression of small target bounding boxes, enabling the model to more accurately locate minute cracks and small pits, directly improving the efficiency of the regression branch.

[0074] The attention perception loss, calculated by comparing the predicted feature map with the attention heatmap generated from the ground truth annotations, guides the model to focus on disease-related areas. : ,in, and They represent the first The predicted feature map of each sample and the attention heatmap generated after Gaussian filtering of the ground truth segmentation mask are used as the loss term to guide the model to focus on image features more relevant to the diseased area; N: the number of samples in the batch. Pixel-level spatial attention supervision is introduced into the detection task to guide the model to focus on image regions related to the disease. The model may learn contextual features unrelated to the disease (such as road texture and shadows) to make judgments, which may affect its robustness in complex scenes. This is a realistic attention heatmap, generated by applying a Gaussian filter to a real segmentation mask (label). It produces a high response at the center of the diseased area and gradually decays towards the edges. To predict attention heatmaps, intermediate feature maps, typically derived from the network backbone or feature fusion components, are normalized. The L2 loss (mean squared error) is used to minimize the difference between the predicted heatmap and the actual heatmap. This loss acts as a powerful regularization term; it doesn't directly participate in classification and regression, but rather guides the network at the feature level, aligning its internal activations with the true spatial distribution of the disease. This forces the feature extraction module to actively focus on the disease itself rather than its surrounding environment, thereby learning more discriminative features and improving the model's interpretability and robustness in complex contexts.

[0075] The aforementioned composite loss function is a carefully designed, multi-pronged optimization strategy. It is not a simple aggregation of loss terms, but rather a collaborative effort where three distinct loss terms work together: Ensure the model is accurate (classifies correctly). Ensure the model frame is accurate (precise positioning), especially for small targets. Ensure the model focuses on the right areas (feature focusing) to improve feature quality. These three aspects work together... The organic combination of these technologies works together across the entire network from feature extraction to final prediction, comprehensively improving the model's performance in detecting and identifying road defects, especially small-target defects, from different dimensions.

[0076] S5. The trained model undergoes network pruning to remove redundant connections, and the pruned model is deployed on edge computing devices via an inference acceleration engine to perform real-time road image stream defect identification and response. This is the final implementation and application stage of the entire solution. The results of all the preceding steps are realized in a real-world scenario. Edge computing devices provide a practical, low-power computing platform. Inference acceleration engines (such as TensorRT) are used to optimize the model at a lower level, further improving inference speed. This step ultimately realizes the practical value of the technical solution, namely, achieving high frame rate, low latency real-time defect identification and alarm in real-world road inspection.

[0077] Specifically, in this embodiment, the model deployed on the edge computing device is a lightweight model jointly optimized by network pruning, quantization-aware training, and knowledge distillation, specifically including:

[0078] The first stage employs L1 norm-based channel pruning, sorting the output channels of the convolutional layers by importance, removing channels with contributions below a preset threshold, and recalibrating connections between adjacent layers; coarse-grained pruning is then performed to remove redundancy at the feature channel level. Importance is measured by calculating the sum of the absolute values ​​of the weights of each output channel in the convolutional layer (L1 norm) as an importance score. .in, Importance score of the c-th output channel : Number of input channels, K: Kernel size The c-th output channel, the ith input channel, and the position in the weight tensor. The weight values ​​are determined. Channels with smaller norms are considered to have lower contributions. Channels with importance scores below the threshold τ and their corresponding input channels in the next layer are removed.

[0079] Because pruning alters the output dimension, the number of input channels in the next convolutional layer needs to be adjusted synchronously to maintain network structure consistency. This is the most direct and efficient pruning method, significantly reducing model parameters and computational cost (FLOPs), laying the foundation for subsequent deployment in embedded devices. Pruning Decision: , where τ is a preset threshold ratio (e.g., 0.1).

[0080] The second stage involves sparsification of the attention mechanism module. This is achieved by controlling the sparsity of attention weights through a gating mechanism, removing attention heads that contribute little to feature selection, and performing fine-grained pruning to optimize the introduced attention module and remove internal redundancy. Attention modules (such as ECA and spatial attention) introduce additional computation while enhancing performance. Not all attention heads or weights are necessary. A learnable gating variable g is introduced along the computation path of the attention weights. This is achieved by adding an L1 regularization term to the loss function. This approach encourages most gating variables to approach zero, thus achieving structured sparsity. After training, attention heads or connections whose gating variables g are approximately zero are removed. This step refines the attention mechanism itself, removing ineffective attention and further reducing the computational overhead of attention while preserving its ability to enhance features.

[0081] The third stage implements inter-layer pruning. For cross-stage connection modules in the feature fusion network, it analyzes the feature redundancy of each path and removes auxiliary paths with low feature contributions. Structural pruning is then performed to optimize the complex module structure. Redundancy analysis assesses the similarity of output features between the main and auxiliary paths in the cross-stage connection module. If the output features of two paths are highly similar, redundancy is considered to exist. Path pruning, using sparsification methods similar to the second stage or by directly calculating path contributions, removes entire auxiliary paths with low feature contributions. This is the most aggressive pruning strategy, directly altering the network's macro-architecture, simplifying complex dual-path modules into single-path or other simpler structures, resulting in a significant reduction in computational cost.

[0082] Among them, path contribution assessment: ,in: Output characteristics of the main path and auxiliary paths It is the Frobenius norm.

[0083] Quantization-aware training is introduced during the pruning process, converting the model weights from FP32 precision to INT8 precision, and simulating the quantization effect during training to maintain model performance stability. Based on pruning, the model is further compressed and inference is accelerated by reducing numerical precision.

[0084] Convert network weights and activation values ​​from 32-bit floating-point numbers (FP32) to 8-bit integers (INT8). Theoretically, this can reduce the model size to about one-quarter, and integer operations are significantly faster than floating-point operations on hardware. Quantization-Aware Training (QAT): During forward inference training, the quantization process is simulated: converting an FP32 weight tensor W into an INT8 tensor. Weight quantization: , Where W: the original FP32 precision weight tensor. These are the weights that the model learns and updates during training. The maximum absolute value in the weight tensor W. This value determines the dynamic range of the weights. : The scaling factor for the weights. It is a scalar used to map the dynamic range of FP32 to the representation range of INT8. It is calculated by dividing the maximum value by 127 (because the maximum value of a positive INT8 number is 127). : Scaling operation. Divide the FP32 weight value by the scaling factor to map it to a range suitable for INT8. The rounding function rounds the scaled FP32 value to the nearest integer. This is the main source of precision loss during quantization. : Truncation function. Restricts the rounded value to the range of INT8 (-128 to 127). Ensures the final result is a valid INT8 value. : The quantized INT8 weight tensor. This is the weight that will ultimately be used for efficient inference. The high-precision FP32 weights are converted to the low-precision INT8 format while minimizing information loss.

[0085] Activation value quantification: , This set of formulas describes how to convert the FP32 output (activation value) tensor A of a network layer into an INT8 tensor. Its logic is similar to that of weighted quantization, but there are key differences. A : The original FP32 precision activation value tensor. This is the output of the intermediate layers of the network, such as the result after convolution and activation function. The maximum value in the activation tensor A. Due to the existence of activation functions (such as ReLU), activation values ​​are usually non-negative, so only the maximum value (not the absolute maximum value) is taken here. : Scaling factor for the activation value. Calculated by dividing the maximum value by 255 (because the range of unsigned INT8 is 0 to 255). : Scale the activation value. : Round the scaled activation value. : Restricts the value to the range of unsigned INT8 (0 to 255). : The quantized unsigned INT8 activation tensor. The weights W are static and can be calibrated and determined once after training. The activation value A is dynamic and depends on the input data; therefore, its scaling factor... Determining the value is more complex and usually requires running statistics on a batch of representative data (calibration set).

[0086] Quantization error modeling: This formula defines the quantization error, which is used to allow the model to perceive and adapt to the effects of quantization during training. Specifically, : Dequantized weights. (This refers to the INT8 weights.) Revert back to FP32 range. This operation is reversible without loss of precision because the fractional part was lost in the previous round operation. : The activation value after dequantization. The square of the L2 norm, also known as the mean square error, is used to measure the magnitude of quantization error. Total information loss. It quantifies the overall information loss caused by rounding operations.

[0087] In the forward propagation of training, we simulate the quantization process: FP32 -> Quantization -> INT8 -> Dequantization -> FP32. During backpropagation, the derivative of the round function is zero, and we use a pass-through estimator, i.e., directly passing the gradient, as if the round function were an identity mapping. By... As a regularization term added to the total loss function, the model actively adjusts its weight distribution while optimizing the main task (such as detection), making them more robust to quantization and thus minimizing the performance degradation caused by quantization. This set of formulas precisely defines the numerical conversion rules from FP32 to INT8 and guides model training through quantization error loss, ensuring that the INT8 model finally deployed on embedded devices maintains high accuracy.

[0088] After pruning, knowledge distillation is performed. The original unpruned model is used as the teacher model to guide the fine-tuning process of the pruned student model, recovering the accuracy loss caused by pruning. Knowledge transfer is utilized to restore and improve the performance of the lightweight student model. Teacher-Student Paradigm: The large, high-precision unpruned model (teacher) transfers its learned knowledge to the small, pruned model (student). Distillation Loss: The training objective of the student model is not only to fit the true labels but also to mimic the output of the teacher model. The total loss function is: ,in, This is task loss (such as the composite loss mentioned above). ), This is the distillation loss (such as KL divergence), expressed by the formula: Used to measure the logical values ​​output by students and teachers. The student model learns smoother decision boundaries and richer hidden knowledge from the teacher model, thus achieving or even surpassing the performance of the teacher model with a significantly reduced number of parameters, perfectly compensating for the accuracy loss caused by pruning and quantization.

[0089] The above process is a series of interconnected model compression systems. It prunes from coarse to fine (channels -> attention -> paths), introduces quantization to leverage hardware advantages, and finally uses knowledge distillation as a finishing step to restore model performance. This combined approach ensures that the final generated model is small, fast, and accurate, perfectly meeting the stringent requirements for real-time, high-precision road defect detection on embedded devices.

[0090] These five steps constitute a logically rigorous and progressively advanced complete technical solution: Steps S1-S3 collectively address the issue of detection accuracy, specifically optimizing the detection accuracy of small road defects through a series of carefully designed neural network structures. Step S4 addresses the issue of model size by pruning and compressing the model to make it suitable for embedded environments. Step S5 addresses the issue of processing speed by using edge deployment and acceleration technologies to ultimately meet real-time requirements. Together, these steps achieve the core objective of high-precision real-time detection with limited computing power.

[0091] Example 2

[0092] like Figure 2 As shown, this application provides an embedded system architecture diagram for real-time road defect recognition based on lightweight CNN and attention mechanism, which is applied to the embedded system for real-time road defect recognition based on lightweight CNN and attention mechanism as described in Embodiment 1. It includes a lightweight feature extraction module 11, a multi-scale feature fusion module 12, a multi-level prediction module 13, a model optimization module 14, and a model compression module 15.

[0093] The lightweight feature extraction module 11 is used to extract features from the road image to be identified. The input feature map is divided into two branches by channel segmentation. In the first branch, a reconstructable convolution with multiple paths is applied to perform feature transformation. The feature maps output by the two branches are fused. Then, the channel dimension of the fused feature map is weighted by a channel attention mechanism.

[0094] The multi-scale feature fusion module 12 is used to input the extracted features into the feature fusion network, wherein feature processing is performed using grouped convolutions, and multi-scale features from different depths of the neural network are fused through a cross-stage connection structure based on grouping bottlenecks.

[0095] The multi-level prediction module 13 is used to input the fused multi-scale features into three prediction heads of different scales. The prediction heads make predictions based on large, medium and small feature maps respectively, so as to detect disease targets of different sizes in the road image.

[0096] The model optimization module 14 is used to optimize the model during the training phase using a composite loss function, which includes at least a classification loss for handling class imbalance and a regression loss suitable for small target localization.

[0097] The model compression module 15 performs network pruning on the trained model to remove redundant connections, and deploys the pruned model on an edge computing device through an inference acceleration engine to perform real-time road image stream disease identification and response.

[0098] Figure 3This is an electronic device provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device includes at least the following components: processor 101 and memory 100, communication interface 103, and bus 102.

[0099] In this embodiment of the application, memory 100 is used to store executable instructions of processor 101, which, when configured to execute instructions, implements the method as described in the first aspect.

[0100] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0101] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0102] It should be noted that a portion of the electronic device described in the above embodiments can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0103] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.

[0104] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.

[0105] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0106] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. An embedded method for real-time road defect recognition based on lightweight CNN and attention mechanism, running on an edge computing device, characterized in that, Includes the following steps: Feature extraction is performed on the road image to be identified. Channel segmentation is used to divide the input feature map into two branches. In the first branch, a reconstructable convolution with multiple paths is applied to perform feature transformation. The feature maps output by the two branches are fused. Then, the channel dimension of the fused feature map is weighted by a channel attention mechanism. The extracted features are input into a feature fusion network, where grouped convolutions are used for feature processing, and multi-scale features from different depths of the neural network are fused through a cross-stage connection structure based on grouping bottlenecks. The fused multi-scale features are input into three prediction heads of different scales, which make predictions based on large, medium and small feature maps respectively, to detect road damage targets of different sizes in the road image. During the model training phase, a composite loss function is used for optimization. The composite loss function includes at least a classification loss for handling class imbalance and a regression loss suitable for small target localization. The trained model is pruned to remove redundant connections, and the pruned model is deployed on edge computing devices through an inference acceleration engine to perform real-time road image stream disease identification and response. The composite loss function used in the model training phase consists of a weighted average of classification loss, regression loss, and attention perception loss, as shown in the formula: ， in, For three independent loss functions, use weight coefficients; Wherein, the classification loss A focus loss function is used to alleviate the imbalance problem of road defect categories. The formula is as follows: ， in, It is the model's predicted probability of the true class. It is the focusing parameter. It is the category weight coefficient; The regression loss The normalized Wasserstein distance is used to enhance the model's sensitivity to the localization of small-sized disease targets. The formula is as follows: ， ， Where NWD is the normalized Wasserstein distance. and These represent the center coordinates, width, and height of the predicted and ground truth bounding boxes, respectively, with k being the scale normalization coefficient. It is a constant; The loss of attention perception By calculating the difference between the predicted feature map and the attention heatmap generated from the ground truth annotations, the model is guided to focus on disease-related areas. The formula is as follows: ， in, and They represent the first The attention heatmap generated by Gaussian filtering the predicted feature map of each sample and the actual segmentation mask, where N is the number of samples in the batch.

2. The embedded method for real-time road defect identification based on lightweight CNN and attention mechanism according to claim 1, characterized in that, The specific steps for feature extraction include: The input feature map is evenly divided into two sub-feature maps along the channel dimension; The sub-feature maps of the first branch are processed. During the training phase, a reconstructable convolutional structure with three parallel branches is adopted. The three parallel branches are a 1×1 convolutional layer, a 3×3 convolutional layer, and an identity connection, and the output features of each branch are summed and fused. During the inference phase, the multi-branch structure is reconstructed into a single 3×3 convolutional layer through parameter equivalence transformation. The processed features of the first branch are fused with the features of the second branch; Channel attention is applied to the fused feature map by global average pooling, followed by learning the correlation between channels through two fully connected layers, and the weight coefficients of each channel are generated using the Sigmoid function. Finally, the weight coefficients are multiplied channel by channel with the fused feature map.

3. The embedded method for real-time road defect identification based on lightweight CNN and attention mechanism according to claim 2, characterized in that, The step of applying channel attention is implemented using an improved, efficient channel attention mechanism, specifically including: After performing global average pooling on the fused feature map, a one-dimensional convolution is used to replace the two fully connected layers for inter-channel information interaction, wherein the kernel size of the one-dimensional convolution is adaptively determined through a non-linear mapping of the number of channels. In the weight generation process, a channel grouping strategy is introduced to divide the feature channels into multiple groups and calculate the attention weight of each group separately. A batch normalization layer is added before the Sigmoid activation function to stabilize the weight distribution during training. The final attention weights are then weighted channel-by-channel with the fused feature map, and the weighted result is then residually connected to the fused feature map.

4. The embedded method for real-time road defect identification based on lightweight CNN and attention mechanism according to claim 1, characterized in that, The feature fusion network achieves multi-scale feature fusion through a cross-stage connection structure, specifically including: In the main path, multiple grouped convolutional layers are used in series to process the input features, with each grouped convolutional layer followed by batch normalization and activation functions; In the auxiliary path, the input features are adjusted for channel dimensions using 1×1 convolution; Connect the features output by the main path with the features processed by the auxiliary path across layers; At the cross-layer connection, a spatial attention mechanism is used to assign weights to features from the main path and auxiliary path in order to selectively fuse feature information from different receptive fields. In the process of grouped convolution, channel shuffling is used to promote information exchange between different channel groups; The features output after selective fusion are added to the input features through skip connections to form a residual learning structure.

5. The embedded method for real-time road defect identification based on lightweight CNN and attention mechanism according to claim 1, characterized in that, The three prediction heads at different scales are optimized and coordinated in the following manner: For the small-scale feature map of 160×160, a first prediction head is configured to focus on detecting slender disease targets with cracks and pit edges by utilizing its high spatial resolution. For the 80×80 mesoscale feature map, a second prediction head is configured to achieve a balance between semantic information and spatial details, so as to focus on identifying medium-sized potholes and local damage. For the large-scale feature map of 40×40, a third prediction head is configured to make an overall judgment using its rich deep semantic information in order to detect large-area damage and network cracks. A decoupled design is adopted for each prediction head, separating its classification task from its regression task; A feature interaction mechanism is established between the prediction heads at three scales, and the collaborative optimization of multi-scale features is achieved through top-down semantic transmission and bottom-up detail supplementation.

6. The embedded method for real-time road defect identification based on lightweight CNN and attention mechanism according to claim 1, characterized in that, The model deployed on edge computing devices is a lightweight model jointly optimized through network pruning, quantization-aware training, and knowledge distillation, specifically including: The first stage employs L1 norm-based channel pruning, sorting the output channels of the convolutional layer by importance, removing channels with contributions below a preset threshold, and recalibrating the connections between adjacent layers. The second stage involves sparsifying the attention mechanism module by controlling the sparsity of attention weights through a gating mechanism and removing attention heads that contribute less to feature selection. The third stage implements inter-layer pruning, which analyzes the feature redundancy of each path for cross-stage connection modules in the feature fusion network and removes auxiliary paths with low feature contribution. Quantization-aware training is introduced during the pruning process to convert the model weights from FP32 precision to INT8 precision, and the quantization effect is simulated during training to maintain the stability of model performance. After pruning, knowledge distillation is performed, using the original unpruned model as the teacher model to guide the fine-tuning process of the student model after pruning, thus restoring the accuracy loss caused by pruning.

7. The embedded method for real-time road defect identification based on lightweight CNN and attention mechanism according to claim 6, characterized in that, The real-time disease identification and response are achieved through the following methods: The lightweight model is fused with operators and the computation graph is optimized using an inference acceleration engine. Dynamic batch processing technology is used to balance inference latency and throughput; And establish a priority response mechanism based on the severity of the disease.

8. The embedded method for real-time road defect identification based on lightweight CNN and attention mechanism according to claim 6, characterized in that, The lightweight model further employs an adaptive training strategy during the training phase. This strategy includes course learning, online hard example mining, and dynamically adjusting the weight coefficients of the composite loss function, specifically including: In the early stages of training, a course learning strategy was adopted. First, image samples containing obvious and typical diseases were used for training. Then, difficult samples containing subtle, complex and occluded diseases were gradually introduced to improve the model's feature learning ability. During model training, the weight coefficients in the loss function are dynamically adjusted based on the convergence of the prediction heads at each scale. In the early stages of training, the focus is on optimizing the regression loss to achieve rapid and stable localization, while in the later stages of training, the focus is on optimizing the classification loss and attention perception loss to improve discrimination accuracy. For small-scale prediction heads, an online hard example mining technique is used to select samples with low classification confidence or large localization error in each training batch for focused training, thereby enhancing the model's ability to identify difficult disease samples. During the model fine-tuning stage, adversarial example training techniques are used to generate difficult-to-identify disease images by adding small perturbations, thereby improving the model's robustness and generalization ability in complex environments.

9. An embedded system for real-time road defect recognition based on lightweight CNN and attention mechanism, applied to the embedded method for real-time road defect recognition based on lightweight CNN and attention mechanism as described in any one of claims 1 to 8, characterized in that, The system includes: The lightweight feature extraction module is used to extract features from the road image to be identified. It uses channel segmentation to divide the input feature map into two branches. In the first branch, a reconstructable convolution with multiple paths is applied to perform feature transformation. The feature maps output by the two branches are fused, and then the channel dimension of the fused feature map is weighted by a channel attention mechanism. The multi-scale feature fusion module is used to input the extracted features into the feature fusion network, where grouped convolutions are used for feature processing, and multi-scale features from different depths of the neural network are fused through a cross-stage connection structure based on grouping bottlenecks. A multi-level prediction module is used to input the fused multi-scale features into three prediction heads of different scales. The prediction heads make predictions based on large, medium and small feature maps respectively, so as to detect the disease targets of different sizes in the road image. The model optimization module is used to optimize the model during the training phase using a composite loss function, which includes at least a classification loss for handling class imbalance and a regression loss suitable for small target localization. The model compression module prunes the trained model to remove redundant connections and deploys the pruned model on edge computing devices through an inference acceleration engine to perform real-time road image stream defect identification and response.

Images