A highway disease identification model adaptive optimization iteration method

Abstract

This invention belongs to the field of image recognition technology, specifically relating to an adaptive optimization and iterative method for highway defect recognition models. At edge nodes, a lightweight defect feature extraction backbone network is used to extract local feature maps from the highway image. Based on the prediction confidence variance of different defect categories and the intersection-union matrix of the predicted bounding boxes, class-imbalanced feature vectors are generated and uploaded to the cloud. After receiving the vectors, the cloud calculates the gradient direction for each category. A gradient projection correction term for rare defect categories is added to the cross-entropy loss term to reduce the cosine value of the gradient angle between common and rare defect categories. The corrected loss function is used to update the parameters of the feature fusion classification head and then distributed to edge nodes for hot replacement. This scheme eliminates the cancellation interference of common defect gradients on rare defect gradients, overcoming the gradient submersion defect of rare defect samples in backpropagation under long-tailed distributions.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically relating to an adaptive optimization and iterative method for highway defect identification models. Background Technology

[0002] Highway defect identification models require continuous iterative updates during long-term operation. Current conventional iterative approaches employ either a centralized training architecture or an edge-based independent fine-tuning architecture. In the centralized training architecture, highway inspection equipment transmits all collected images back to a central server. The central server aggregates massive amounts of image data, calculates the cross-entropy loss based on the global dataset, performs full backpropagation updates on all network layer parameters of the model, and then distributes the updated complete model to the roadside equipment. In the edge-based independent fine-tuning architecture, after receiving newly acquired highway images, the roadside equipment directly calculates the loss function and modifies the model parameters on its local device. Both conventional approaches treat all defect categories equally, using a uniform cross-entropy function when calculating the loss value. The gradients of each category are superimposed during backpropagation, regardless of the number of samples per category.

[0003] Highway distress data exhibits an extreme long-tail characteristic in its natural distribution, with the number of images of normal pavement and common distresses such as potholes and transverse cracks far exceeding the number of images of rare distresses such as network cracks and micro-stripping. During the aforementioned centralized training or edge fine-tuning iterations, the massive number of normal and common distress samples generate a dominant gradient direction during backpropagation, causing the gradients generated by rare distress samples to be completely submerged during superposition. This leads to model parameters being updated towards fitting the common categories. This gradient-level conflict reduces the model's ability to extract features from rare distresses after iterations, making it impossible to achieve balanced learning of long-tailed distributed data within an edge-cloud collaborative computing power allocation framework. Summary of the Invention

[0004] The purpose of this invention is to provide an adaptive optimization and iterative method for highway defect identification models, which can effectively solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] An adaptive optimization iterative method for highway defect identification models includes:

[0007] At the edge nodes, a lightweight disease feature extraction backbone network is used to forward propagate real-time acquired highway images and extract local feature maps.

[0008] At the edge node, an imbalanced feature vector is generated based on the predicted confidence variance of different disease categories in the local feature map and the intersection-union matrix of the predicted bounding box, and the imbalanced feature vector is uploaded to the cloud.

[0009] The imbalanced feature vectors of the categories are received in the cloud, the gradient direction of each category in the current batch of data is calculated, and a loss function based on gradient conflict suppression is constructed. In the cross-entropy loss term, a gradient projection correction term for rare disease categories with low confidence and high variance is added to reduce the cosine value of the gradient angle between common disease categories and rare disease categories in backpropagation.

[0010] The modified loss function is used to update the parameters of the feature fusion classification head in the cloud, and the updated feature fusion classification head parameters are sent to the edge nodes for hot parameter replacement, completing one adaptive optimization iteration.

[0011] Preferably, at the edge node, an imbalanced feature vector is generated based on the predicted confidence variance of different disease categories in the local feature map and the intersection-union matrix of the predicted bounding box, including:

[0012] The local feature map is divided into grid regions of a preset size at the edge nodes. The prediction confidence of all pixels in each grid region for each disease category is calculated, and the variance of the prediction confidence in each grid region is calculated.

[0013] At the edge node, the intersection-union matrix of the predicted bounding boxes of different disease categories within the same grid area is obtained. The variance and the intersection-union matrix corresponding to each grid area are concatenated to generate the class imbalance feature vector.

[0014] Preferably, the class imbalance feature vector is received in the cloud, the gradient direction of each class in the current batch of data is calculated, and a loss function based on gradient conflict suppression is constructed, including:

[0015] Based on the variance and crossover ratio in the category imbalance feature vector, the samples in the current batch of data are divided into common disease category samples and rare disease category samples in the cloud.

[0016] The initial gradient directions of the common disease category samples and the rare disease category samples in the feature fusion classification head are calculated in the cloud.

[0017] In the cloud, the initial gradient direction of the rare disease category sample is used as the reference vector. The initial gradient direction of the common disease category sample is projected onto the orthogonal plane of the reference vector. The projected gradient vector is used to replace the initial gradient vector of the common disease category sample to generate the gradient projection correction term.

[0018] Preferably, the updated feature fusion classification header parameters are sent to the edge nodes for hot parameter replacement, including:

[0019] The edge node maintains a cache area for both old and new parameters, receives updated feature fusion classification header parameters from the cloud, and copies the currently running feature fusion classification header parameters to the cache area for both old and new parameters.

[0020] At the edge node, the received updated feature fusion classification head parameters are used to cover the fully connected layer weight parameters and convolutional layer bias parameters in the feature fusion classification head;

[0021] After the overwrite is completed, the historical parameter data in the old and new parameter cache areas are cleared, triggering the memory reclamation mechanism of the edge node to release the storage space of the old and new parameter cache areas.

[0022] Preferably, the lightweight disease feature extraction backbone network includes sequentially cascaded depthwise separable convolutional layers and channel attention layers;

[0023] The depthwise separable convolutional layer extracts spatial features from the highway image and outputs a spatial feature tensor.

[0024] The channel attention layer performs global average pooling on the spatial feature tensor to generate a one-dimensional channel descriptor. The one-dimensional channel descriptor is then transformed nonlinearly through two fully connected layers to output channel weight coefficients that match the dimension of the spatial feature tensor.

[0025] The local feature map is generated by multiplying the channel weight coefficients element-wise with the spatial feature tensor.

[0026] Preferably, before receiving the class-imbalanced feature vectors in the cloud and calculating the gradient direction of each class in the current batch of data, the method further includes:

[0027] The historical feature vector sliding window for each type of disease is maintained in the cloud, and the received category imbalance feature vector is input into the historical feature vector sliding window;

[0028] The Mahalanobis distance between the class imbalance feature vector and each historical feature vector in the sliding window of the historical feature vector is calculated in the cloud.

[0029] Extract class imbalance feature vectors whose Mahalanobis distance is greater than a preset distance threshold from the cloud, remove class imbalance feature vectors whose Mahalanobis distance is less than or equal to the preset distance threshold, and use the remaining class imbalance feature vectors after removal to perform subsequent gradient direction calculation steps.

[0030] Preferably, dividing the local feature map into grid regions of a preset size at the edge nodes includes:

[0031] A multi-scale feature pyramid is constructed at the edge node, the multi-scale feature pyramid containing multiple feature layers of different resolutions arranged sequentially from bottom to top;

[0032] Anchor points are generated on each feature layer at the edge nodes according to a preset step size. With each anchor point as the center, a grid area of ​​the preset size matching the resolution of the feature layer is divided on the corresponding feature layer.

[0033] The feature maps of overlapping grid regions in adjacent feature layers are fused at the pixel level to generate a fused grid region. Within the fused grid region, the variance of the prediction confidence and the intersection-union matrix are calculated.

[0034] Preferably, in the cloud, using the initial gradient direction of the rare disease category samples as a reference vector, the initial gradient direction of the common disease category samples is projected onto an orthogonal plane of the reference vector, including:

[0035] The cloud computes the dot product of the initial gradient vector of the common disease category sample and the reference vector, as well as the square of the magnitude of the reference vector;

[0036] Divide the dot product result by the square of the modulus to obtain the projection scalar coefficient;

[0037] Subtract the product of the projection scalar coefficient and the reference vector from the initial gradient vector of the common disease category samples to obtain the projection gradient vector perpendicular to the reference vector;

[0038] The projected gradient vector is multiplied by a preset scaling factor to generate the gradient projection correction term, which is ultimately used to replace the initial gradient vector of the common disease category samples.

[0039] Preferably, at the edge node, the received updated feature fusion classification head parameters are used to cover the fully connected layer weight parameters and convolutional layer bias parameters in the feature fusion classification head, including:

[0040] At the edge node, obtain the first weight matrix of the updated feature fusion classification head parameters and the second weight matrix of the fully connected layer weight parameters in the current feature fusion classification head;

[0041] Calculate the difference matrix between the first weight matrix and the second weight matrix;

[0042] Each element in the difference matrix is ​​multiplied by the exponential moving average coefficient, and the product is then superimposed onto the corresponding element position in the second weight matrix.

[0043] The superimposed second weight matrix is ​​stored as the weight parameters of the fully connected layer after the coverage is completed.

[0044] Preferably, after generating the local feature map by element-wise multiplying the channel weight coefficients with the spatial feature tensor, the method further includes:

[0045] The edge node receives a rare disease category indicator sent from the cloud, which is generated by the cloud based on the category imbalance feature vector.

[0046] At the edge node, the channel index number corresponding to the rare disease category in the local feature map is determined according to the rare disease category indicator.

[0047] At the edge node, the output value of the neuron node corresponding to the channel index number in the channel attention layer is set to a non-zero constant, and the output values ​​of other neuron nodes except the channel index number are set to zero;

[0048] The local feature map is regenerated using the output value after zeroing, and the regenerated local feature map is input into the feature fusion classification head.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] 1. This scheme generates class-imbalanced feature vectors at edge nodes based on the intersection-union matrix (IUCN) of predicted confidence variance and predicted bounding boxes. A loss function based on gradient conflict suppression is constructed in the cloud, utilizing a gradient projection correction term to reduce the cosine of the gradient angle between common and rare disease categories, thus eliminating the canceling interference of common disease gradients on rare disease gradients. This joint processing mechanism based on vector statistics and orthogonal gradient projection alters the model's parameter update trajectory under long-tailed data, enabling the model parameters to retain gradient components specific to rare disease categories during iteration. This overcomes the gradient overwhelming defect of rare disease samples during backpropagation, completing the learning of rare disease features.

[0051] 2. This solution decouples the model into a feature extraction part at the edge nodes and a classification head part in the cloud. It only uploads imbalanced feature vectors and updates only the classification head parameters, reducing the amount of data transmission between the edge and cloud and the computational overhead of the cloud. By maintaining a cache of old and new parameters at the edge nodes and using exponential moving average coefficients for hot parameter replacement, it avoids feature extraction anomalies caused by instantaneous changes in model parameters. The cloud filters feature vectors based on Mahalanobis distance, excluding redundant data that highly overlaps with historical features from participating in gradient calculation. Local feature maps are extracted through depthwise separable convolutional layers and channel attention layers. After receiving a rare disease category indicator, the output of the channel attention layer is selectively zeroed and replaced with a constant, reducing the interference of normal background features on the classification decision of rare diseases. Attached Figure Description

[0052] Figure 1 This is a flowchart of the adaptive optimization and iterative method for highway defect identification model of the present invention;

[0053] Figure 2 This is a flowchart illustrating the generation of class-imbalanced feature vectors according to the present invention;

[0054] Figure 3 The flowchart below illustrates the construction of a loss function based on gradient conflict suppression according to the present invention.

[0055] Figure 4 This is a flowchart of the hot replacement of edge node parameters in this invention;

[0056] Figure 5 This is a flowchart of the lightweight backbone network feature extraction and multi-scale mesh partitioning of the present invention;

[0057] Figure 6 This is a flowchart of the cloud-based historical feature filtering and edge channel selective zeroing process of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] Please refer to Figure 1This embodiment provides an adaptive optimization and iteration method for highway defect identification models, which is applied to a cloud-edge collaborative highway inspection scenario. The edge nodes are deployed on roadside inspection equipment or vehicle-mounted inspection terminals, and the cloud is deployed on a central server cluster. The identification model is decoupled into a lightweight defect feature extraction backbone network deployed on the edge nodes and a feature fusion classification head deployed on the edge nodes, whose parameters can be updated by the cloud. Based on this, the adaptive optimization and iteration of the model is completed.

[0060] In one embodiment, edge nodes acquire real-time highway images and perform preprocessing operations on them. This involves adjusting the tensor dimension of the highway image to match the size of the input layer of the lightweight disease feature extraction backbone network, and normalizing the pixel values ​​of the highway image to map them to a preset numerical range. After preprocessing, the edge nodes use the lightweight disease feature extraction backbone network to perform a forward propagation operation on the processed highway image, extracting the corresponding local feature map. Specifically, during the forward propagation operation, the highway image tensor sequentially passes through each level of convolutional and activation layers of the lightweight disease feature extraction backbone network, completing the feature transformation of spatial and channel dimensions. The output local feature map retains the core visual features of the disease targets in the highway image, such as texture, edges, and shape. The channel dimension of the local feature map matches the total number of disease categories, with each channel corresponding to the feature response of one type of disease target.

[0061] Furthermore, the edge node acquires the inference results output by the local feature map input feature fusion classification head. These results include the prediction confidence and predicted bounding box coordinates for different disease categories. The edge node generates an imbalanced feature vector based on the variance of the prediction confidence for different disease categories and the intersection-union (IU) matrix of the predicted bounding boxes in the local feature map. Specifically, for each disease category, the edge node calculates the prediction confidence of all pixels in the corresponding channel's local feature map and calculates the variance of the prediction confidence for that category. For every two different disease categories, the edge node extracts the predicted bounding box coordinates for those categories and calculates the IU between the two bounding boxes. The IU of all pairwise combinations of categories forms the IU matrix. The edge node then concatenates the one-dimensional vector composed of the variances of the prediction confidence for all disease categories with the one-dimensional vector obtained after flattening the IU matrix, generating the final imbalanced feature vector. The edge node uploads the generated imbalanced feature vector to the cloud via a pre-defined communication link. During the upload process, only the imbalanced feature vector is transmitted; the original highway image and the complete local feature map are not transmitted.

[0062] Furthermore, the cloud receives imbalanced feature vectors uploaded by edge nodes and aggregates these vectors from multiple edge nodes within the same time window into the current batch of data. The cloud calculates the gradient direction for each category in the current batch of data and constructs a loss function based on gradient conflict suppression. Specifically, the cloud distinguishes between common disease category samples and rare disease category samples in the current batch of data based on the variance and cross-union ratio (CUI) of the imbalanced feature vectors. It then calculates the initial gradient vectors for each category in the feature fusion classification head, where the initial gradient vectors are composed of the partial derivatives of the loss function with respect to the trainable parameters of the feature fusion classification head. The cloud adds a gradient projection correction term to the cross-entropy loss term for rare disease categories with low confidence and high variance. This term uses the initial gradient vector of the rare disease category samples as a reference and performs orthogonal projection processing on the initial gradient vectors of the common disease category samples. The projected gradient vectors replace the initial gradient vectors of the common disease category samples, thereby reducing the cosine of the gradient angle between the common and rare disease categories during backpropagation and suppressing conflicts between the gradients of the two categories.

[0063] Furthermore, the cloud performs backpropagation on the feature fusion classification head using the modified loss function to update its parameters. Specifically, based on the modified loss function, the cloud calculates the gradient of the loss value with respect to the weight parameters of the fully connected layers and the bias parameters of the convolutional layers in the feature fusion classification head. It then iteratively updates these parameters using a gradient descent algorithm. During the iteration, only the parameters of the feature fusion classification head are updated; the parameters of the lightweight disease feature extraction backbone network are not modified. After the parameter update is complete, the cloud distributes the updated feature fusion classification head parameters to the corresponding edge nodes. During the distribution process, only the trainable parameters of the feature fusion classification head are transmitted; the parameters of the lightweight backbone network are not transmitted.

[0064] Furthermore, the edge node receives the updated feature fusion classification head parameters from the cloud and performs a hot parameter replacement operation to complete an adaptive optimization iteration. Specifically, without interrupting the current highway image inference service, the edge node writes the updated parameters to the corresponding parameter storage address of the feature fusion classification head, replacing the original historical parameters. After the replacement is completed, the new parameters take effect immediately, and subsequently acquired highway images will use the updated feature fusion classification head to perform inference operations.

[0065] Table 1. Mapping Relationship Between Predictive Statistics for Different Disease Categories and Category Imbalance Feature Vectors

[0066]

[0067] The table above shows the sample distribution and predicted statistical characteristics of different disease categories in the current batch of data. Among them, the categories with higher predicted confidence variance and average bounding box intersection-union ratio values ​​correspond to rare disease categories. The corresponding feature vector dimension intervals are used for category division and gradient calculation in the cloud. The dimension intervals in the table define the storage locations of different statistics in the class imbalance feature vectors, ensuring that the cloud can accurately extract the statistical features of the corresponding categories.

[0068] This embodiment fully realizes the adaptive optimization and iteration of the model through edge-cloud collaboration. By extracting local features at the edge and generating and uploading imbalanced feature vectors, the amount of data transmitted between the edge and the cloud is reduced. By constructing a loss function based on gradient conflict suppression and updating the parameters of the feature fusion classification head in the cloud, the parameter update trajectory of the model under long-tailed data distribution is changed, avoiding the gradient of rare disease samples being submerged during backpropagation. Through hot parameter replacement operations at the edge, the model parameters are updated seamlessly, ensuring the continuity of highway disease identification services.

[0069] In one alternative embodiment, refer to Figure 2 and Figure 5 The lightweight disease feature extraction backbone network consists of cascaded depthwise separable convolutional layers and channel attention layers. Edge nodes acquire the preprocessed highway image tensor and input it into the depthwise separable convolutional layer to perform spatial feature extraction, outputting a spatial feature tensor. The depthwise separable convolutional layer is composed of cascaded depthwise convolution operations and pointwise convolution operations. The depthwise convolution operation performs convolution operations independently on each input channel to extract spatial features within each channel, while the pointwise convolution operation performs 1×1 convolution operations to achieve feature fusion between different channels. The operation process of the depthwise separable convolutional layer is defined by the following formula:

[0070]

[0071]

[0072] in, The input is a highway image tensor with dimension . , Image height, Image width, Input the number of channels; The kernel is a depthwise convolutional kernel with dimension . , The kernel size; This is a depthwise convolution operation; For pointwise convolution kernels, the dimension is , Number of output channels; This is a pointwise convolution operation; The output spatial feature tensor has dimension . , , These represent the height and width of the feature map after the convolution operation, respectively.

[0073] Furthermore, the edge nodes input the spatial feature tensor output by the depthwise separable convolutional layer into the channel attention layer, where channel-dimensional feature weighting is performed. The channel attention layer first performs global average pooling on the spatial feature tensor to generate a one-dimensional channel descriptor. The global average pooling operation is defined by the following formula:

[0074]

[0075] in, For the first The global average pooling result of the nth channel, i.e., the nth channel in the one-dimensional channel descriptor. One element; For the spatial feature tensor, the first The first channel, the first line, number The pixel values ​​of the column; This is the channel index number.

[0076] Furthermore, the channel attention layer performs a nonlinear transformation on the one-dimensional channel descriptor through two cascaded fully connected layers, outputting channel weight coefficients that match the dimension of the spatial feature tensor. The nonlinear transformation operation is defined by the following formula:

[0077]

[0078] in, For all The one-dimensional channel descriptor vector formed; This is the weight matrix of the first fully connected layer. The channel dimensionality reduction coefficient; It is the ReLU activation function; This is the weight matrix for the second fully connected layer; Use the Sigmoid activation function; Let be the output channel weight coefficient vector, where the th channel weight coefficient is... element Corresponding to the The weighting coefficients of each channel.

[0079] Furthermore, the channel attention layer performs element-wise multiplication of the output channel weight coefficients with the spatial feature tensor to generate a local feature map. The computation process is defined by the following formula:

[0080]

[0081] in, For the local feature map, the first Feature matrices of each channel; This is an element-wise multiplication operation.

[0082] Furthermore, the edge nodes construct a multi-scale feature pyramid, which contains multiple feature layers of different resolutions arranged sequentially from bottom to top. Each feature layer corresponds to the output of convolutional layers of different depths in the lightweight disease feature extraction backbone network. The resolution of the feature layers decreases sequentially from bottom to top, while the receptive field increases sequentially. The edge nodes generate anchor points on each feature layer according to a preset stride. Centered on each anchor point, a grid region of a preset size matching the feature layer resolution is divided on the corresponding feature layer. Specifically, high-resolution feature layers correspond to smaller grid region sizes and smaller anchor point strides to cover small-sized disease targets; low-resolution feature layers correspond to larger grid region sizes and larger anchor point strides to cover large-sized disease targets.

[0083] Furthermore, the edge nodes perform pixel-level feature map fusion on spatially overlapping grid regions in adjacent feature layers to generate fused grid regions. Specifically, the edge nodes perform bilinear interpolation upsampling on the feature maps of corresponding grid regions in the upper feature layer and spatially overlapping grid regions in the lower feature layer, and then add them element-wise to obtain a fused feature map. The fused feature map contains both high-resolution detailed features and low-resolution semantic features. Within the fused grid region, the edge nodes perform variance calculation of prediction confidence and obtain the intersection-over-union (IoU) matrix.

[0084] Specifically, the edge nodes statistically analyze the prediction confidence of all pixels within each fused grid region for each disease category, and calculate the variance of the prediction confidence within each fused grid region. The variance calculation process is defined by the following formula:

[0085]

[0086] in, For the first line, number The variance of prediction confidence corresponding to the column fusion grid region; This represents the total number of pixels within the merged grid area; For the first in this fused grid region The prediction confidence level of each pixel for the target disease category; This is the mean of the prediction confidence for all pixels within the fused grid area.

[0087] Furthermore, the edge nodes obtain the intersection-union ratio (CUI) matrix between predicted bounding boxes of different disease categories within the same fused grid region. The CUI calculation process is defined by the following formula:

[0088]

[0089] in, For the first Predicted bounding boxes for each disease category, For the first Predicted bounding boxes for each disease category; This represents the area of ​​the intersection region of the two bounding boxes; The area of ​​the union of the two bounding boxes; The intersection-union ratio (IUU) of the two bounding boxes, with a range of values ​​of 1000. Edge nodes construct a two-dimensional intersection-union (CUI) matrix by combining the cross-union ratios of all pairs of categories within the same fused grid region. A one-dimensional vector consisting of the variances of all fused grid regions is then concatenated with a one-dimensional vector obtained by flattening all CUI matrices to generate a class-imbalanced feature vector. The concatenation process is defined by the following formula:

[0090]

[0091] in, This generates class-imbalanced feature vectors; This is a one-dimensional vector consisting of the prediction confidence variances corresponding to all fused grid regions. The one-dimensional vector obtained by flattening all intersection-union matrices; This is a vector concatenation operation.

[0092] Table 2. Correspondence of Parameters for Each Layer of the Multi-Scale Feature Pyramid

[0093]

[0094] The table above shows the grid partitioning configuration of each feature layer in the multi-scale feature pyramid. Feature layers with different resolutions correspond to grid regions of different sizes and anchor point step sizes, achieving comprehensive coverage of disease targets at different scales. The fusion range of adjacent feature layers defines the level of feature fusion in the grid region, ensuring that the fused feature map contains both detailed features and semantic features, thus improving the accuracy of the calculation of prediction statistics.

[0095] This embodiment refines the process of extracting local feature maps and the logic for generating class-imbalanced feature vectors. A lightweight backbone network constructed with depthwise separable convolutional layers and channel attention layers achieves efficient feature extraction at edge nodes, reducing the computational overhead of edge nodes. Through grid partitioning of multi-scale feature pyramids and feature fusion of adjacent layers, comprehensive statistics of disease features at different scales are achieved, ensuring that class-imbalanced feature vectors can accurately represent the distribution differences of different disease categories, and providing reliable input data for gradient calculation and loss function construction in the cloud.

[0096] In yet another alternative embodiment, reference is made to... Figure 3 and Figure 6 The cloud maintains a sliding window of historical feature vectors for each type of disease. This sliding window has a fixed length and stores class-imbalanced feature vectors received within a preset time range. After receiving class-imbalanced feature vectors uploaded from edge nodes, the cloud inputs them into the historical feature vector sliding window and calculates the Mahalanobis distance between the current class-imbalanced feature vector and each historical feature vector in the sliding window. The Mahalanobis distance is defined by the following formula:

[0097]

[0098] in, The Mahalanobis distance; The historical feature vectors are the historical feature vectors within the sliding window. It is the mean vector of all historical feature vectors within the sliding window; It is the inverse of the covariance matrix of all historical eigenvectors within the sliding window.

[0099] Furthermore, the cloud-based system extracts class-imbalanced feature vectors with a Mahalanobis distance greater than a preset distance threshold, and removes class-imbalanced feature vectors with a Mahalanobis distance less than or equal to the preset distance threshold. The remaining class-imbalanced feature vectors are then used to perform subsequent gradient direction calculations. Specifically, class-imbalanced feature vectors with a Mahalanobis distance less than or equal to the preset distance threshold highly overlap with the distribution of historical feature vectors and are considered redundant data. Removing these feature vectors reduces the overhead of invalid gradient calculations and avoids interference from duplicate data in the parameter update process.

[0100] Furthermore, based on the variance and cross-union ratio (CUP) values ​​in the remaining class-imbalanced feature vectors, the cloud platform divides the samples in the current batch of data into common disease categories and rare disease categories. Specifically, the cloud platform sets variance thresholds and CUP thresholds, classifying categories with variance values ​​greater than the variance threshold and CUP values ​​greater than the CUP threshold as rare disease categories, and the corresponding samples are rare disease category samples; the remaining categories are classified as common disease categories, and the corresponding samples are common disease category samples.

[0101] Furthermore, the cloud-based system calculates the initial gradient directions for common disease category samples and rare disease category samples in the feature fusion classification head, respectively. Specifically, the cloud-based system constructs a basic cross-entropy loss function, the calculation of which is defined by the following formula:

[0102]

[0103] in, This represents the total number of samples in the current batch. This represents the total number of disease categories. For the first The sample is for the first Each category has a real label, with a value of 0 or 1; For the first The sample is for the first The prediction confidence scores for each category are calculated. The partial derivatives of the cross-entropy loss function with respect to the feature fusion classification head parameters are calculated in the cloud to obtain the initial gradient vectors corresponding to the common disease category samples. Initial gradient vector corresponding to rare disease category samples The direction of the initial gradient vector is the gradient direction of the corresponding class sample.

[0104] Furthermore, the cosine of the angle between the initial gradient vectors of common disease categories and rare disease categories is calculated in the cloud. The calculation process is defined by the following formula:

[0105]

[0106] in, Let cosine be the angle between the two initial gradient vectors; This is a vector dot product operation; The L2 norm of the vector is calculated. The closer the cosine of the angle is to 1, the closer the directions of the two gradient vectors are, and the smaller the gradient conflict; the closer the cosine of the angle is to -1, the more opposite the directions of the two gradient vectors are, the more serious the gradient conflict, and the easier it is for the gradient of rare disease samples to be submerged by the gradient of common disease samples.

[0107] Furthermore, the cloud platform uses the initial gradient direction of rare disease category samples as the reference vector and projects the initial gradient direction of common disease category samples onto the orthogonal plane of the reference vector to generate a gradient projection correction term. Specifically, the cloud platform first calculates the dot product of the initial gradient vector of the common disease category samples and the reference vector, as well as the square of the magnitude of the reference vector; divides the dot product by the square of the magnitude to obtain the projection scalar coefficient; subtracts the product of the projection scalar coefficient and the reference vector from the initial gradient vector of the common disease category samples to obtain the projected gradient vector perpendicular to the reference vector; multiplies the projected gradient vector by a preset scaling factor to generate the gradient projection correction term that is finally used to replace the initial gradient vector of the common disease category samples. The above calculation process is defined by the following formula:

[0108]

[0109]

[0110] in, The projection gradient vector; The square of the magnitude of the reference vector; This is the final gradient projection correction term; The preset scaling factor has a range of values. The projected gradient vector is orthogonal to the reference vector, and the cosine of the angle between them is 0, thus suppressing gradient conflicts between common and rare disease categories.

[0111] Furthermore, a total loss function based on gradient conflict suppression is constructed in the cloud, incorporating the gradient projection correction term into the cross-entropy loss term. The calculation process of the total loss function is defined by the following formula:

[0112]

[0113] in, This is the total loss function; This is the regularization coefficient, used to adjust the weights of the gradient projection correction term. The cloud-based system performs backpropagation using the total loss function, replacing the initial gradient vectors of common disease category samples with the gradient projection correction term to update the parameters of the feature fusion classification head.

[0114] Table 3 Comparison of Gradient Parameters for Common and Rare Diseases

[0115]

[0116] The table above shows the parameter changes of different categories of samples before and after gradient projection correction. The initial gradient angle cosine value is the cosine value of the angle between the gradient of common disease categories and the baseline gradient of rare disease categories. After projection, the gradient of common disease categories is orthogonal to the baseline gradient, and the angle cosine value drops to 0, thus suppressing gradient conflict. The scaling factor is used to adjust the magnitude of the gradient after projection, balance the parameter update amplitude of different categories of samples, and avoid the decline in model fitting ability caused by excessive parameter updates of common disease categories.

[0117] This embodiment refines the construction process of the gradient conflict suppression loss function in the cloud. By using a sliding window of historical feature vectors and Mahalanobis distance filtering, redundant feature vectors are eliminated, reducing the overhead of invalid gradient calculations. By introducing gradient orthogonal projection and gradient projection correction terms, the cosine value of the gradient angle between common and rare disease categories is reduced, suppressing gradient conflicts during backpropagation and preventing the gradients of rare disease samples from being submerged by the gradients of common disease samples. This allows the model parameters to retain gradient components specific to rare disease categories during iteration, enabling effective learning of rare disease features.

[0118] In yet another alternative embodiment, refer to Figure 4 Edge nodes maintain caches for both old and new parameters. These caches are pre-defined memory storage spaces used to store historical runtime parameters of the feature fusion classification head. When an edge node receives updated feature fusion classification head parameters from the cloud, it first performs a memory space check. If the available storage space in the caches is sufficient for parameter backup, the edge node copies the currently running feature fusion classification head parameters to the caches, completing the backup of historical parameters. During the backup process, the edge node's inference service continues to run, using the currently running parameters to process the real-time acquired highway images without interrupting the service flow.

[0119] Furthermore, the edge node utilizes the received updated feature fusion classification head parameters to overlay the fully connected layer weight parameters and convolutional layer bias parameters in the feature fusion classification head. Specifically, the edge node obtains the first weight matrix of the received updated feature fusion classification head parameters and the second weight matrix of the fully connected layer weight parameters in the current feature fusion classification head; calculates the difference matrix between the first and second weight matrices; multiplies each element in the difference matrix by an exponential moving average coefficient, and superimposes the product onto the corresponding element position in the second weight matrix; and stores the superimposed second weight matrix as the fully connected layer weight parameters after overlay. The above parameter update process is defined by the following formula:

[0120]

[0121] in, This is the weight parameter matrix of the fully connected layer after coverage is completed; This is the weight parameter matrix of the fully connected layer in the current feature fusion classification head; This is the updated first weight matrix distributed from the cloud. The coefficient of the exponential moving average has a range of values. By using the exponential moving average coefficient, the parameters are updated smoothly, avoiding feature distribution shifts and inference anomalies caused by sudden parameter jumps.

[0122] Furthermore, after parameter coverage is complete, the edge node performs parameter validity verification. It uses preset test samples to input the updated feature fusion classification head, obtains the inference results, and verifies their validity. Upon successful verification, the edge node clears historical parameter data from the old and new parameter caches, triggering its memory reclamation mechanism to release storage space in the old and new parameter caches, completing the entire hot parameter replacement process. If parameter anomalies are detected during verification, the edge node immediately writes the historical parameters stored in the old and new parameter caches back to the parameter storage address of the feature fusion classification head, rolling back to the parameter state before the update, ensuring the normal operation of the inference service.

[0123] Table 4. Feature Fusion Classification Header Parameter Hot Replacement Status Flow Table

[0124]

[0125] The table above shows the complete state transition process during parameter hot replacement. Through parameter backup, verification, and exception rollback mechanisms in the cache, the continuity of the inference service is ensured during parameter replacement, and service interruption caused by parameter anomalies is avoided. During the state transition, the inference service always remains running, realizing seamless updates of model parameters.

[0126] Furthermore, the edge nodes receive rare disease category indicators from the cloud. These indicators are generated by the cloud based on the class imbalance feature vector. After classifying rare diseases, the cloud encapsulates the corresponding category number into a rare disease category indicator and sends it to the corresponding edge nodes. Based on the rare disease category indicator, the edge nodes determine the channel index number corresponding to the rare disease category in the local feature map. Each channel index number corresponds to a disease category, and each channel index number corresponds to an independent neuron node in the channel attention layer.

[0127] Furthermore, the edge node sets the output value of the neuron node corresponding to the channel index in the channel attention layer to a preset non-zero constant, and sets the output values ​​of all other neurons nodes except for that channel index to zero; it then regenerates the local feature map using the zeroed-out output values, and inputs the regenerated local feature map into the feature fusion classification head. The process of regenerating the local feature map is defined by the following formula:

[0128]

[0129] in, For the regenerated local feature map; These are preset non-zero constants; This is the set of channel index numbers corresponding to rare disease categories. Through the above processing, only the feature responses of the channels corresponding to rare disease categories are retained, while the responses of other categories and background features are suppressed. This reduces the interference of normal background features and common disease features on the classification decision of rare diseases, and improves the model's ability to identify rare disease categories.

[0130] This embodiment refines the parameter hot replacement process of edge nodes and the logic for enhancing rare disease features. By maintaining the old and new parameter buffers and using the exponential moving average parameter update method, it achieves smooth updates of model parameters, avoids inference anomalies caused by instantaneous parameter jumps, and ensures the continuity of edge services. By selectively zeroing and replacing constants in the channel attention layer, it enhances the feature response of rare disease categories, reduces the interference of irrelevant features on classification decisions, further improves the model's recognition effect on rare disease categories, and completes the adaptive optimization iteration of the model.

Claims

1. An adaptive optimization iterative method for highway defect identification models, characterized in that, include: At the edge nodes, a lightweight disease feature extraction backbone network is used to forward propagate real-time acquired highway images and extract local feature maps. At the edge node, an imbalanced feature vector is generated based on the predicted confidence variance of different disease categories in the local feature map and the intersection-union matrix of the predicted bounding box, and the imbalanced feature vector is uploaded to the cloud. The imbalanced feature vectors of the categories are received in the cloud, the gradient direction of each category in the current batch of data is calculated, and a loss function based on gradient conflict suppression is constructed. In this function, a gradient projection correction term for low confidence and high variance rare disease categories is added to the cross-entropy loss term. The modified loss function is used to update the parameters of the feature fusion classification head in the cloud, and the updated feature fusion classification head parameters are sent to the edge nodes for hot parameter replacement, completing one adaptive optimization iteration.

2. The adaptive optimization iterative method for highway defect identification model according to claim 1, characterized in that, At the edge node, an imbalanced feature vector is generated based on the predicted confidence variance of different disease categories in the local feature map and the intersection-union matrix of the predicted bounding box, including: The local feature map is divided into grid regions of a preset size at the edge nodes. The prediction confidence of all pixels in each grid region for each disease category is calculated, and the variance of the prediction confidence in each grid region is calculated. At the edge node, the intersection-union matrix of the predicted bounding boxes of different disease categories within the same grid area is obtained. The variance and the intersection-union matrix corresponding to each grid area are concatenated to generate the class imbalance feature vector.

3. The adaptive optimization iterative method for highway defect identification model according to claim 1, characterized in that, The system receives the class-imbalanced feature vectors in the cloud, calculates the gradient direction of each class in the current batch of data, and constructs a loss function based on gradient conflict suppression, including: Based on the variance and crossover ratio in the category imbalance feature vector, the samples in the current batch of data are divided into common disease category samples and rare disease category samples in the cloud. The initial gradient directions of the common disease category samples and the rare disease category samples in the feature fusion classification head are calculated in the cloud. In the cloud, the initial gradient direction of the rare disease category sample is used as the reference vector. The initial gradient direction of the common disease category sample is projected onto the orthogonal plane of the reference vector. The projected gradient vector is used to replace the initial gradient vector of the common disease category sample to generate the gradient projection correction term.

4. The adaptive optimization iterative method for highway defect identification model according to claim 1, characterized in that, The updated feature fusion classification header parameters are sent to the edge nodes for hot parameter replacement, including: The edge node maintains a cache area for both old and new parameters, receives updated feature fusion classification header parameters from the cloud, and copies the currently running feature fusion classification header parameters to the cache area for both old and new parameters. At the edge node, the received updated feature fusion classification head parameters are used to cover the fully connected layer weight parameters and convolutional layer bias parameters in the feature fusion classification head; After the overwrite is completed, the historical parameter data in the old and new parameter cache areas are cleared, triggering the memory reclamation mechanism of the edge node to release the storage space of the old and new parameter cache areas.

5. The adaptive optimization iterative method for highway defect identification model according to claim 1, characterized in that, The lightweight disease feature extraction backbone network includes sequentially cascaded depthwise separable convolutional layers and channel attention layers; The depthwise separable convolutional layer extracts spatial features from the highway image and outputs a spatial feature tensor. The channel attention layer performs global average pooling on the spatial feature tensor to generate a one-dimensional channel descriptor. The one-dimensional channel descriptor is then transformed nonlinearly through two fully connected layers to output channel weight coefficients that match the dimension of the spatial feature tensor. The local feature map is generated by multiplying the channel weight coefficients element-wise with the spatial feature tensor.

6. The adaptive optimization iterative method for highway defect identification model according to claim 1, characterized in that, Before receiving the class-imbalanced feature vectors in the cloud and calculating the gradient direction of each class in the current batch of data, the process further includes: The historical feature vector sliding window for each type of disease is maintained in the cloud, and the received category imbalance feature vector is input into the historical feature vector sliding window; The Mahalanobis distance between the class imbalance feature vector and each historical feature vector in the sliding window of the historical feature vector is calculated in the cloud. Extract class imbalance feature vectors whose Mahalanobis distance is greater than a preset distance threshold from the cloud, remove class imbalance feature vectors whose Mahalanobis distance is less than or equal to the preset distance threshold, and use the remaining class imbalance feature vectors after removal to perform subsequent gradient direction calculation steps.

7. The adaptive optimization iterative method for highway defect identification model according to claim 2, characterized in that, The local feature map is divided into grid regions of a preset size at the edge nodes, including: A multi-scale feature pyramid is constructed at the edge node, the multi-scale feature pyramid containing multiple feature layers of different resolutions arranged sequentially from bottom to top; Anchor points are generated on each feature layer at the edge nodes according to a preset step size. With each anchor point as the center, a grid area of ​​the preset size matching the resolution of the feature layer is divided on the corresponding feature layer. The feature maps of overlapping grid regions in adjacent feature layers are fused at the pixel level to generate a fused grid region. Within the fused grid region, the variance of the prediction confidence and the intersection-union matrix are calculated.

8. The adaptive optimization iterative method for highway defect identification model according to claim 3, characterized in that, In the cloud, using the initial gradient direction of the rare disease category samples as a reference vector, the initial gradient direction of the common disease category samples is projected onto an orthogonal plane of the reference vector, including: The cloud computes the dot product of the initial gradient vector of the common disease category sample and the reference vector, as well as the square of the magnitude of the reference vector; Divide the dot product result by the square of the modulus to obtain the projection scalar coefficient; Subtract the product of the projection scalar coefficient and the reference vector from the initial gradient vector of the common disease category samples to obtain the projection gradient vector perpendicular to the reference vector; The projected gradient vector is multiplied by a preset scaling factor to generate the gradient projection correction term, which is ultimately used to replace the initial gradient vector of the common disease category samples.

9. The adaptive optimization iterative method for highway defect identification model according to claim 4, characterized in that, At the edge node, the received updated feature fusion classification head parameters are used to overwrite the fully connected layer weight parameters and convolutional layer bias parameters in the feature fusion classification head, including: At the edge node, obtain the first weight matrix of the updated feature fusion classification head parameters and the second weight matrix of the fully connected layer weight parameters in the current feature fusion classification head; Calculate the difference matrix between the first weight matrix and the second weight matrix; Each element in the difference matrix is ​​multiplied by the exponential moving average coefficient, and the product is then superimposed onto the corresponding element position in the second weight matrix. The superimposed second weight matrix is ​​stored as the weight parameters of the fully connected layer after the coverage is completed.

10. The adaptive optimization iterative method for highway defect identification model according to claim 5, characterized in that, After generating the local feature map by element-wise multiplying the channel weight coefficients with the spatial feature tensor, the method further includes: The edge node receives a rare disease category indicator sent from the cloud, which is generated by the cloud based on the category imbalance feature vector. At the edge node, the channel index number corresponding to the rare disease category in the local feature map is determined according to the rare disease category indicator. At the edge node, the output value of the neuron node corresponding to the channel index number in the channel attention layer is set to a non-zero constant, and the output values ​​of other neuron nodes except the channel index number are set to zero; The local feature map is regenerated using the output value after zeroing, and the regenerated local feature map is input into the feature fusion classification head.

Images