A lightweight instance segmentation model for highway construction scene and a construction method thereof

Abstract

The application relates to the field of computer vision and deep learning, and discloses a lightweight instance segmentation model for a highway construction scene and a construction method, which comprises the following steps: constructing a lightweight convolution unit and a residual feature extraction module by adopting a phantom convolution in a backbone network, so as to reduce the forward calculation and parameter storage of the network; connecting a space and channel collaborative attention mechanism to a neck network, so as to inhibit complex background interference; adopting a distance intersection over union loss function instead of a complete intersection over union loss in a detection head, so as to adapt to the boundary box regression of multi-form construction targets; and finally guiding a lightweight model to fit the feature distribution of a high-precision teacher network by using a mask generation distillation method. The application reduces the dependence of the model on hardware computing power, makes up for the segmentation precision loss caused by network compression, and meets the real-time data sensing demand of the construction site equipment.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, specifically to a lightweight instance segmentation model and construction method for highway construction scenarios. Background Technology

[0002] In intelligent construction scenarios for highway projects, instance segmentation models are commonly used to identify construction workers and machinery. These scenarios typically involve dynamic changes in lighting and cluttered backgrounds, and significant scale differences exist between different types of vehicles and workers, impacting the segmentation accuracy of general-purpose visual models. Furthermore, the actual deployment of these models often involves edge computing nodes or embedded hardware mounted on construction equipment such as road rollers, where computing power and power consumption are subject to strict limits. Conventional high-precision instance segmentation models, due to their large number of parameters and computational load, cannot meet the real-time inference and control requirements of on-site construction equipment.

[0003] To adapt to edge devices with limited computing power, existing technologies typically employ lightweight baseline models based on the YOLO framework. However, these models rely on standard convolutional operations to extract features in the backbone network stage, resulting in significant channel redundancy in the internal feature maps, leaving room for further computational compression at the lower levels. In the neck feature fusion stage, conventional feature enhancement modules struggle to handle both large-scale vehicle and small-scale personnel feature extraction at construction sites, exhibiting weak resistance to cluttered backgrounds. In the bounding box regression stage at the detection end, mainstream models generally use the complete intersection-union loss function for bounding box localization. Since this function includes a penalty term for the width-to-height ratio of the bounding box, excessive morphological constraints can hinder the model's regression convergence speed and affect the final localization accuracy when construction workers undergo posture changes (e.g., from standing to bending over) or when the observation perspective of construction vehicles shifts. Furthermore, further lightweight modifications to the network layers to reduce computational overhead inevitably lead to a direct loss in segmentation accuracy. Existing models lack corresponding knowledge transfer mechanisms during the training phase, making it difficult to guide lightweight networks to overcome performance bottlenecks caused by parameter reduction. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a lightweight instance segmentation model and construction method for highway construction scenarios. It solves the problems that conventional instance segmentation models cannot meet real-time inference requirements on highway construction edge devices due to limited computing power, and that the segmentation accuracy decreases when the model is lightweighted for complex backgrounds and variable targets.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The first aspect of this invention provides a lightweight instance segmentation model for highway construction scenarios, the model consisting of a backbone network, a neck network, and a detection head; The backbone network includes lightweight convolutional units constructed through phantom convolution modules and a lightweight residual feature extraction module; The P4 layer access space and channel collaborative attention mechanism for processing medium-scale targets in the neck network; The detection head is used to output target detection and instance segmentation results. The internal weight parameters of the model are obtained by training and updating based on the distance intersection-union loss function and the mask generation distillation method.

[0006] The second aspect of this invention provides a method for constructing a lightweight instance segmentation model for highway construction scenarios, applied to a lightweight instance segmentation model for highway construction scenarios provided in the first aspect of this invention, comprising the following steps: Lightweight improvements to the backbone network based on phantom convolution; Improvements to the neck network based on spatial and channel-based collaborative attention; Optimize the loss function of the detection head based on distance intersection and comparison; Knowledge transfer optimization is performed based on mask generation distillation to build a model.

[0007] In a preferred embodiment of the present invention, the lightweight improvement of the backbone network based on phantom convolution specifically includes: Extract the main feature information of the input data using a small number of standard convolution operations to generate a core feature map with fewer channels; Based on the core feature map, a corresponding redundant feature map is generated through a low-cost linear transformation operation. The core feature map and the generated redundant feature map are concatenated along the channel dimension to obtain the final output feature. All standard convolutional modules in the backbone network are replaced with phantom convolutional modules to form lightweight convolutional units.

[0008] Through the above improvements, the model uses partial standard convolution combined with low-cost linear transformation to generate feature maps of the target number of channels, replacing the complete standard convolution operation, thereby reducing the number of network parameters and computational cost in the feature extraction stage.

[0009] Furthermore, the lightweight improvement of the backbone network based on phantom convolution also includes: The phantom convolutional module is used to replace the standard convolutional layer inside the residual feature extraction module in the backbone network to construct a lightweight bottleneck structure. The bottleneck structure inside the residual feature extraction module in the backbone network is updated to the lightweight bottleneck structure to generate a lightweight residual feature extraction module. Short connections are built between network stages of different depths to fuse shallow features with deep features processed by the lightweight bottleneck structure across stages.

[0010] Through the above improvements, the network reduces model complexity while preserving the cross-stage transmission path of multi-layer features, thus avoiding gradient vanishing during deep network training iterations.

[0011] In a preferred embodiment of the present invention, the improvement of the neck network based on spatial and channel-based collaborative attention specifically includes: The spatial and channel collaborative attention mechanism is integrated into the P4 layer of the neck network for processing medium-scale targets; the medium-scale feature map output by the feature fusion module is received, and independent decomposition operations are performed on the medium-scale feature map in the height and width dimensions. Multi-scale lightweight convolution is used to extract and fuse spatial information of the two directional features after decomposition, generating a spatial attention map. The spatial attention map is then multiplied element-wise with the original input feature map to obtain a spatial enhancement feature map.

[0012] Furthermore, the improvement of the neck network based on spatial and channel collaborative attention also includes: Global information aggregation in the spatial dimension is performed on the spatial enhanced feature map to generate a global feature vector in the channel dimension; The global feature vector is progressively compressed and reduced in dimensionality through multi-layer linear mapping operations; The self-attention matrix is ​​calculated along the channel dimension of the compressed feature descriptor to generate the channel attention weight matrix. The channel attention weight matrix is ​​then multiplied element-wise with the spatial augmentation feature map to output the final recalibrated feature map.

[0013] Through the above improvements, the model performs spatial and channel-based collaborative computation on the mesoscale features of vehicle and personnel targets in the neck network. Spatial orientation decomposition is used to enhance the representation of target physical location information, channel dimensionality reduction recalibrates the weights of each channel feature, optimizes the network's response mechanism to high semantic value channel features, and suppresses interference from cluttered backgrounds in construction scenes.

[0014] In a preferred embodiment of the present invention, the optimization of the loss function of the detection head based on distance intersection and comparison specifically includes: Obtain the predicted bounding boxes and corresponding ground truth bounding boxes output by the detection head; Calculate the intersection-union ratio (IUU) between the predicted bounding box and the true bounding box; Calculate the square of the Euclidean distance between the center point coordinates of the predicted bounding box and the center point coordinates of the true bounding box; Calculate the square of the diagonal length of the smallest bounding rectangle that can enclose both the predicted bounding box and the true bounding box. The distance crossover ratio (CRO) loss value is calculated based on the CRO value, the square of the Euclidean distance between the center points, and the square of the diagonal length.

[0015] Through the above improvements, the model eliminates the need to calculate constraint parameters related to the aspect ratio of the bounding box. Addressing the morphological differences caused by the varying postures of construction workers and the changing field of view of vehicles in highway construction scenarios, the network regression directly relies on reducing the physical distance to the center point of the target bounding box, adapting to detection tasks of different object shapes and accelerating the loss convergence speed during training.

[0016] In a preferred embodiment of the present invention, the knowledge transfer optimization based on mask generation distillation specifically includes: Select the teacher model and the student model to be trained; Images of the same highway construction scene are input in batches into the teacher model and student model, and multi-scale feature maps are extracted respectively. A 1×1 convolution kernel is used to perform linear mapping calculation on the multi-scale feature map of the student model, so that the multi-scale feature map of the student model is completely aligned with the feature map of the teacher model at the same scale in terms of the number of feature channels.

[0017] Furthermore, the knowledge transfer optimization based on mask-generated distillation also includes: Generate a random mask matrix with the same spatial resolution size as the student model feature map after channel alignment; The student model feature map, after aligning the random mask matrix with the channel, is subjected to element-wise multiplication at spatially corresponding positions, and the pixel response information of the occluded area is discarded to obtain the incomplete feature map. The incomplete feature map after occlusion processing is input into the generator network for forward computation, and the complete feature map reconstructed by the generator network is output.

[0018] Furthermore, the knowledge transfer optimization based on mask-generated distillation also includes: Extract the reconstructed feature map output by the generator network and the feature map of the teacher model; The sum of squares of the numerical differences between the reconstructed feature map and the teacher model feature map at each feature channel and spatial pixel position is calculated, and the average value is obtained to obtain the mask generation distillation loss value. The total loss value is generated by summing the mask-generated distillation loss value with the target detection loss and instance segmentation loss, and then backpropagated to update the internal weight parameters of the student model.

[0019] Through the above improvements, the student model derives and reconstructs incomplete feature maps using a generative network. The training process uses the output features of the high-precision teacher model as a supervision benchmark, passing the spatial distribution and semantic association logic of the features extracted by the teacher model to the lightweight student model to compensate for the segmentation accuracy loss caused by backbone network compression.

[0020] This invention provides a lightweight instance segmentation model and construction method for highway construction scenarios. It offers the following advantages: This invention replaces the standard convolution operation in the backbone network with a phantom convolution module. It extracts core features through partial standard convolution and combines this with low-cost linear transformation to generate redundant feature maps. This significantly reduces the number of floating-point operations and parameter storage requirements during the model's forward inference process. The modified model has low hardware computing power requirements and can be directly deployed on edge computing or embedded devices mounted on construction equipment, meeting the real-time visual perception needs of engineering sites.

[0021] This invention embeds a spatial and channel collaborative attention mechanism in the middle of the neck network and uses the distance intersection-union (DIU) loss function as the regression optimization objective in the detection head. The collaborative attention mechanism effectively suppresses the feature response of cluttered backgrounds in the construction environment by utilizing spatial orientation decomposition and channel dimensionality reduction mapping operations. The DUI loss function relies on the physical distance of the center point to guide parameter updates, eliminating the original strict aspect ratio constraints and solving the problem of bounding box positioning deviation caused by the changing posture of construction personnel and the switching of vehicle observation perspectives.

[0022] This invention establishes a model knowledge transfer training framework using a mask-generated distillation method. During the training phase, a random mask is applied to the output features of the lightweight student model, and a generative network is used to reconstruct the features. The reconstructed features are then compared with the baseline features of the high-precision teacher model to calculate and provide feedback on the error. This process guides the lightweight network to fit the spatial distribution patterns of high-level semantic features, compensating for the loss of feature information caused by the significant compression of the backbone network, and ensuring instance segmentation accuracy without increasing the actual computational load. Attached Figure Description

[0023] Figure 1 This is a flowchart of the construction method of the present invention. Detailed Implementation

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] This invention uses the YOLOv11n instance segmentation model as the baseline model. This baseline model consists of a backbone network, a neck network, and a detection head.

[0026] The basic network structure of this baseline model includes standard convolutional modules, residual modules, fast spatial pyramid pooling modules, and parallel spatial attention convolutional blocks to achieve object detection and instance segmentation.

[0027] See attached document Figure 1 , Figure 1 This is a flowchart illustrating a lightweight instance segmentation model construction method for highway construction scenarios according to an embodiment of the present invention. The present invention provides a lightweight instance segmentation model construction method for highway construction scenarios, comprising the following steps: S10, a lightweight improvement to the backbone network based on phantom convolution; S20, an improvement to the neck network based on spatial and channel-based collaborative attention; S30, based on the optimization of the loss function of the distance intersection and comparison detection head; S40 optimizes knowledge transfer by generating distillation based on mask.

[0028] The specific technical solutions for each of the above model construction steps will be explained in detail below.

[0029] Regarding step S10 above, the process of lightweighting and improving the backbone network based on phantom convolution specifically includes the following steps: S101, Construct the Phantom Convolution Module. To address the issue of high feature map redundancy and computational complexity caused by standard convolution in the baseline model, a phantom convolution module is constructed to replace the original feature extraction method. Given an input feature map, a small number of standard convolution operations are used to extract the main feature information of the input data, generating a core feature map with fewer channels. Based on this core feature map, a corresponding redundant feature map is generated through a low-cost linear transformation operation. The core feature map and the generated redundant feature map are concatenated along the channel dimension to obtain the final output feature.

[0030] The process of generating the core feature map described above can be represented as follows: ; In the formula, Input feature map; The generated core feature map;

[0031] To generate the standard convolution weight matrix corresponding to the core feature map; This is a standard convolution operation. At this point, the core feature map... The number of channels is less than the total number of channels expected for output.

[0032] The operation of generating redundant feature maps based on core feature maps can be represented as: ; In the formula, Core feature map The first in Individual channel characteristics; To act on the first The first core channel feature A low-cost linear transformation function; This generates the corresponding redundant feature map. Through a pre-defined low-cost linear transformation function, the feature dimension can be expanded without increasing the number of parameters.

[0033] After generating the redundant feature maps, the core feature maps are then... With all generated redundant feature maps The feature maps are concatenated along the feature channel dimensions to output the total feature map.

[0034] For the process of splicing and combining multidimensional tensors in a specified channel dimension, those skilled in the art can use existing matrix splicing algorithms to achieve it. The tensor splicing operation is a well-known technology in this field and will not be described in detail here.

[0035] S102, replace the standard convolutional modules in the backbone network to form lightweight convolutional units. The original baseline model relied on standard convolutional modules when extracting image features from its backbone network structure. Replace all standard convolutional modules in the backbone network with the phantom convolutional modules constructed above.

[0036] This replacement operation can generate output feature maps of the same size and dimension using phantom convolution modules while keeping the layers of the backbone network feature extraction network unchanged, thereby reducing the parameter size and computational burden of the backbone network from the source.

[0037] S103, Reconstruct the baseline network to obtain a lightweight residual feature extraction module. The backbone network of the baseline model contains several residual modules, each containing a bottleneck structure for adjusting channels and feature extraction. Replace the standard convolutional layers inside the original bottleneck structure with the aforementioned phantom convolutional modules to construct a lightweight bottleneck structure. Update all bottleneck structures within the residual feature extraction module in the backbone network to the aforementioned lightweight bottleneck structure to generate the lightweight residual feature extraction module.

[0038] The lightweight residual feature extraction module described above continues to use and retain the original cross-stage connection feature transfer logic of the baseline network when running the extraction task. By constructing short connections between network stages of different depths, shallow features are fused with deep features processed by the lightweight bottleneck structure across stages, further compressing model complexity while avoiding the gradient vanishing phenomenon caused by deep network training.

[0039] For the specific branch routing and summation methods of cross-stage connection features, those skilled in the art can directly refer to the backbone network branch structure configuration of the existing baseline model. Its branch connection control is a well-known technology in the field and will not be elaborated here.

[0040] Regarding step S20 above, the process of improving the neck network based on spatial and channel collaborative attention specifically includes the following steps: S201, determine the embedding location of spatial and channel collaborative attention. The neck network is used to fuse multi-scale features extracted by the backbone network. In the neck network, layer P4, which processes medium-scale targets, is selected as the feature enhancement layer. The spatial and channel collaborative attention mechanism is integrated into layer P4, and its physical location is set between the output of the neck feature fusion module and the input of the detection and segmentation head. This setup is specifically designed for enhancing the medium-scale dimensions of engineering vehicles and construction workers in construction scenarios with large scale differences.

[0041] S202, Construct and execute shared multi-semantic spatial attention computation. Receive the mesoscale feature map output from the feature fusion module, and perform independent decomposition operations in the height and width dimensions of the feature map to obtain feature descriptors in the height and width directions respectively. Use multi-scale lightweight convolution to extract and fuse spatial information from the two decomposed features, generating the corresponding spatial attention map.

[0042] The above multi-semantic space attention calculation process can be represented as: ; ; ; In the formula, The input is a mesoscale feature map to the spatial and channel collaborative attention mechanism; For high-dimensional feature decomposition and pooling operations; For feature decomposition and pooling operations in the width dimension; For multi-scale lightweight convolution operation functions; The extracted height-direction spatial feature matrix; This is the extracted width-direction spatial feature matrix; This refers to matrix multiplication. For activation functions; This is the generated spatial attention map.

[0043] After calculating the spatial attention map, the spatial attention map is multiplied element-wise with the original input feature map to obtain the spatial enhancement feature map, thereby enhancing the expression of the spatial location information of the target region.

[0044] S203, Construct and execute progressive channel self-attention computation. Obtain the spatial enhanced feature map, aggregate global information in its spatial dimension to generate a global feature vector in the channel dimension. Set a phased compression ratio parameter, and progressively compress and reduce the dimensionality of the global feature vector through multi-layer linear mapping operations. Calculate the self-attention matrix along the channel dimension for the compressed feature descriptor, obtain the weight response coefficients of different feature channels, and complete the generation of the channel self-attention map.

[0045] The above progressive channel self-attention calculation and recalibration process can be expressed as: ; ; ; In the formula, This is the spatially augmented feature map after spatial augmentation processing; This is a function for global spatial information aggregation operations; to These are the parameters of the progressive dimensionality reduction mapping matrix for each stage; This is a channel feature descriptor after progressive compression. This is the normalized activation function used to compute the self-attention weights; The generated channel attention weight matrix; This is an element-wise multiplication operation; This is the final recalibrated feature map output to the detection segmentation head after channel feature recalibration.

[0046] For the specific implementation of the global aggregation of the above feature maps and the multi-layer linear mapping operation, those skilled in the art can use existing global average pooling technology and fully connected layer cascading method. The tensor dimensionality reduction and mapping calculation are well known technologies in this field and will not be described in detail here.

[0047] By executing S202 and S203 above, spatial and channel-dimensional collaborative modeling of the mesoscale features in the neck network is completed. Spatial attention calculation captures the physical location structure of the construction target, while progressive channel attention calculation filters feature channels with high semantic value. The combination of these two methods reduces the interference of cluttered backgrounds in the construction scene on target recognition and enhances the feature response of highway construction workers and engineering vehicles.

[0048] Regarding step S30 above, the process of optimizing the loss function of the detection head based on distance intersection-union comparison specifically includes the following steps: S301 replaces the original bounding box regression loss function in the detection head. The baseline model's detection head is configured by default with the full intersection-union (CIU) loss function to calculate the regression loss of the bounding box. Because the CIU loss function includes a strict penalty term on the aspect ratio of the detection box, in highway construction scenarios, construction workers exhibit changes in posture (standing, bending, etc.), and construction vehicles undergo multiple perspective transitions, resulting in significant differences in target shape. Continuing to use excessive aspect ratio constraints in this situation would lead to errors in regression localization. The detection head completely replaces the original CIU loss function with a distance-based CIU loss function, changing the regression optimization objective.

[0049] S302, Construct and calculate the center point distance constraint term. Obtain the predicted bounding box and the corresponding ground truth bounding box output by the detection head. Based on the traditional intersection-over-union (IoU) calculation, extract the center point coordinates of the predicted bounding box and the ground truth bounding box, and calculate the Euclidean distance between the two points.

[0050] Find the smallest bounding rectangle that can simultaneously enclose both the predicted and actual bounding boxes, and calculate the diagonal distance of this smallest bounding rectangle. Divide the square of the Euclidean distance between the center points by the square of the diagonal distance of the smallest bounding rectangle to generate a distance penalty term used to constrain the bounding box position.

[0051] The calculation process for the above distance intersection-union ratio loss can be expressed as follows: ; In the formula, To calculate the distance intersection-union ratio (DIU) loss value of the output; To predict the ratio of the area where the bounding box and the true bounding box overlap to the total area of ​​the two; To predict the coordinates of the center point of the bounding box; The coordinates of the center point of the actual bounding box; This is a function to calculate the Euclidean distance between the coordinates of two center points; The value is the diagonal length of the minimum bounding rectangle.

[0052] S303, perform direct regression constraint operation on the bounding box. Calculate the regression loss of the detection head according to the above formula. In this process, the calculation of constraint parameters for the aspect ratio of the bounding box is cancelled, and the network weight update is guided only by the intersection-union ratio parameter and the center point distance penalty term. By eliminating the excessive penalty in the aspect ratio dimension, the bounding box regression process of the detection head directly depends on the reduction of the physical distance to the center point of the target box, adapting to construction target detection tasks of different shapes.

[0053] The specific numerical solution method for the intersection-union ratio (IU / U) of the predicted and actual bounding boxes, as well as the geometric calculation logic for finding the coordinates of the minimum bounding rectangle of the bounding box, can be implemented by those skilled in the art using conventional bounding box geometric intersection algorithms. The calculation of the area ratio is a well-known technique in this field and will not be elaborated upon here. Based on the above improvements, the loss convergence speed of the detection head network during backpropagation is accelerated, and the final positioning accuracy of the bounding boxes of construction personnel and vehicles is improved.

[0054] Regarding step S40 above, the process of optimizing knowledge transfer based on mask generation distillation specifically includes the following steps: S401, configure the distillation model structure and extract multi-scale features from the network. A YOLOv11x network model with high detection accuracy is selected as the teacher model, and the model improved by the aforementioned network lightweighting is selected as the student model.

[0055] During the training phase, all network weight parameters of the teacher model were frozen. Identical highway construction scene images were batch-input into both the teacher and student models, and multi-scale feature maps were extracted from the network layers of both models. Due to the difference in model network size, the multi-scale feature maps output by the student model had fewer channel dimensions than the corresponding feature maps output by the teacher model.

[0056] S402, Perform feature channel alignment. Obtain the multi-scale feature map extracted by the student model. Perform linear mapping calculation on the feature map using a 1×1 convolutional kernel. By setting the number of output channels of the 1×1 convolutional layer to be equal to the number of channels of the teacher model's feature map, the dimensionality of the student model's feature information is expanded, so that it is completely aligned with the teacher model's feature map of the same scale in terms of the number of feature channels.

[0057] This alignment operation ensures that subsequent spatial pixel-level error comparison processes can be performed within the same data dimension.

[0058] S403, Perform feature masking and network reconstruction operations. For the channel-aligned student model feature map, generate a random mask matrix with the same spatial resolution size. The elements within this random mask matrix follow a binarized distribution. Perform element-wise multiplication of spatially corresponding positions between this random mask matrix and the channel-aligned student model feature map.

[0059] When the corresponding element in the mask matrix is ​​0, the corresponding pixel response information on the student model's feature map is occluded and discarded. A generative network consisting of several convolutional operation layers is constructed. The incomplete feature map after occlusion processing is input into this generative network. Information inference and forward computation are performed using the context pixel features of the unoccluded region, and the complete feature map reconstructed by the generative network is output.

[0060] S404, calculate the distillation loss and complete the model knowledge transfer. Use the multi-scale feature map output by the teacher model as the supervised comparison benchmark. Extract the reconstructed feature map output by the generator network and the teacher model feature map, calculating the numerical differences between the two at each feature channel and spatial pixel position. Calculate the sum of squares of the numerical differences at all corresponding pixel positions and take the average value. Use the final mean squared error value as the distillation loss for model training.

[0061] The process of feature alignment, mask occlusion, network reconstruction, and distillation loss calculation described above can be represented as follows: ; ; ; In the formula, Multi-scale feature maps extracted for student models; A 1×1 convolution mapping function to achieve feature channel alignment; This is the feature map of the student model after channel alignment. It is a random mask matrix generated based on a preset occlusion ratio; For matrix element-level multiplication operations; GG is the feature reconstruction mapping function of the generator network; To generate the reconstructed feature map output from the network's forward computation; Generate distillation loss values ​​for the calculated mask; , , These are the channel number dimension parameter, height dimension parameter, and width dimension parameter of the feature map, respectively; The feature map extracted for the teacher model in the first... The first channel, the first line, number The pixel value at the column position; To reconstruct the pixel values ​​at the corresponding dimensional coordinate positions of the feature map.

[0062] After obtaining the mask to generate the distillation loss value, it is numerically summed with the original target detection loss and instance segmentation loss of the student model to generate the total loss value used to guide the backpropagation of the student model. The internal weight parameters of the student model are iteratively updated using an optimization algorithm.

[0063] This mechanism forces the student model to use its extracted local features to fit the complete feature response of the teacher model with a large number of parameters, transferring the complex background suppression mechanism and high-level semantic feature expression knowledge contained in the teacher model to the lightweight student model.

[0064] The freezing control strategy for model parameter weights and the backpropagation rules for the mean squared error during the training process can be implemented by those skilled in the art using the internal compilation logic of existing deep learning frameworks. The gradient differentiation and backpropagation are well-known techniques in this field and will not be elaborated upon here. The distillation calculation process described above compensates for the parameter loss and accuracy degradation caused by the lightweight modification of the backbone and neck network structures.

[0065] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A lightweight instance segmentation model for highway construction scenarios, characterized in that, The model consists of a backbone network, a neck network, and a detection head; The backbone network includes lightweight convolutional units constructed through phantom convolution modules and a lightweight residual feature extraction module; The P4 layer access space and channel collaborative attention mechanism for processing medium-scale targets in the neck network; The detection head is used to output target detection and instance segmentation results. The internal weight parameters of the model are obtained by training and updating based on the distance intersection-union loss function and the mask generation distillation method.

2. A method for constructing a lightweight instance segmentation model for highway construction scenarios, applied to the lightweight instance segmentation model for highway construction scenarios as described in claim 1, characterized in that, Includes the following steps: Lightweight improvements to the backbone network based on phantom convolution; Improvements to the neck network based on spatial and channel-based collaborative attention; Optimize the loss function of the detection head based on distance intersection and comparison; Knowledge transfer optimization is performed based on mask generation distillation to construct a model.

3. The method for constructing a lightweight instance segmentation model for highway construction scenarios according to claim 2, characterized in that, The lightweight improvement of the backbone network based on phantom convolution specifically includes: Extract the main feature information of the input data using a small number of standard convolution operations to generate a core feature map with fewer channels; Based on the core feature map, a corresponding redundant feature map is generated through a low-cost linear transformation operation. The core feature map and the generated redundant feature map are concatenated along the channel dimension to obtain the final output feature. All standard convolutional modules in the backbone network are replaced with phantom convolutional modules to form lightweight convolutional units.

4. The method for constructing a lightweight instance segmentation model for highway construction scenarios according to claim 3, characterized in that, The lightweight improvement of the backbone network based on phantom convolution also includes: The phantom convolutional module is used to replace the standard convolutional layer inside the residual feature extraction module in the backbone network to construct a lightweight bottleneck structure. The bottleneck structure inside the residual feature extraction module in the backbone network is updated to the lightweight bottleneck structure to generate a lightweight residual feature extraction module. Short connections are built between network stages of different depths to fuse shallow features with deep features processed by the lightweight bottleneck structure across stages.

5. The method for constructing a lightweight instance segmentation model for highway construction scenarios according to claim 2, characterized in that, The improvement of the neck network based on spatial and channel collaborative attention specifically includes: The spatial and channel collaborative attention mechanism is integrated into the P4 layer of the neck network for processing medium-scale targets; the medium-scale feature map output by the feature fusion module is received, and independent decomposition operations are performed on the medium-scale feature map in the height and width dimensions. Multi-scale lightweight convolution is used to extract and fuse spatial information of the two directional features after decomposition, generating a spatial attention map. The spatial attention map is then multiplied element-wise with the original input feature map to obtain a spatial enhancement feature map.

6. The method for constructing a lightweight instance segmentation model for highway construction scenarios according to claim 5, characterized in that, The improvement to the neck network based on spatial and channel collaborative attention also includes: Global information aggregation in the spatial dimension is performed on the spatial enhanced feature map to generate a global feature vector in the channel dimension; The global feature vector is progressively compressed and reduced in dimensionality through multi-layer linear mapping operations; The self-attention matrix is ​​calculated along the channel dimension of the compressed feature descriptor to generate the channel attention weight matrix. The channel attention weight matrix is ​​then multiplied element-wise with the spatial augmentation feature map to output the final recalibrated feature map.

7. The method for constructing a lightweight instance segmentation model for highway construction scenarios according to claim 2, characterized in that, The optimization of the loss function of the detection head based on distance intersection-union comparison specifically includes: Obtain the predicted bounding boxes and corresponding ground truth bounding boxes output by the detection head; Calculate the intersection-union ratio (IUU) between the predicted bounding box and the true bounding box; Calculate the square of the Euclidean distance between the center point coordinates of the predicted bounding box and the center point coordinates of the true bounding box; Calculate the square of the diagonal length of the smallest bounding rectangle that can enclose both the predicted bounding box and the true bounding box. The distance crossover ratio (CRO) loss value is calculated based on the CRO value, the square of the Euclidean distance between the center points, and the square of the diagonal length.

8. The method for constructing a lightweight instance segmentation model for highway construction scenarios according to claim 2, characterized in that, The knowledge transfer optimization based on mask generation distillation specifically includes: Select the teacher model and the student model to be trained; Images of the same highway construction scene are input in batches into the teacher model and student model, and multi-scale feature maps are extracted respectively. A 1×1 convolution kernel is used to perform linear mapping calculation on the multi-scale feature map of the student model, so that the multi-scale feature map of the student model is completely aligned with the feature map of the teacher model at the same scale in terms of the number of feature channels.

9. The method for constructing a lightweight instance segmentation model for highway construction scenarios according to claim 8, characterized in that, The knowledge transfer optimization based on mask-generated distillation also includes: Generate a random mask matrix with the same spatial resolution size as the student model feature map after channel alignment; The student model feature map, after aligning the random mask matrix with the channel, is subjected to element-wise multiplication at spatially corresponding positions, and the pixel response information of the occluded area is discarded to obtain the incomplete feature map. The incomplete feature map after occlusion processing is input into the generator network for forward computation, and the complete feature map reconstructed by the generator network is output.

10. The method for constructing a lightweight instance segmentation model for highway construction scenarios according to claim 9, characterized in that, The knowledge transfer optimization based on mask-generated distillation also includes: Extract the reconstructed feature map output by the generator network and the feature map of the teacher model; Calculate the sum of squares of the numerical differences between the reconstructed feature map and the teacher model feature map at each feature channel and spatial pixel position, and take the average value to obtain the mask generation distillation loss value; The total loss value is generated by summing the mask-generated distillation loss value with the target detection loss and instance segmentation loss, and then backpropagated to update the internal weight parameters of the student model.

Images