A hardware-aware image segmentation model compression method

Through the image segmentation model compression method based on hardware perception, combined with structural re-parameterization and lightweight feature pyramid network, the detection head and feature extraction part are optimized, and the problem of high computational complexity of deep learning image segmentation algorithms on resource-constrained devices is solved. It realizes an efficient and high-precision image segmentation network, suitable for embedded devices.

CN118552739BActive Publication Date: 2024-11-08SHANDONG INST OF AEROSPACE ELECTRONICS TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411026792.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-08
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The existing deep learning image segmentation algorithm has high computational complexity on resource-constrained devices and cannot be directly applied to embedded devices and edge computing devices. The existing lightweight method ignores the changes in weight parameters, network structure and accuracy. The dynamic connection between the model cannot be unified in efficiency and accuracy of model sparseness.

Method used

Using hardware-aware image segmentation model compression method, the detection head and feature extraction part are optimized through structurally reparametric convolution neural networks and lightweight feature pyramid networks, combined with deep separable convolution and model distillation technology, and the detection head and feature extraction part are optimized, using directed Polar mask IoU loss function and knowledge distillation technology improve the operating efficiency and accuracy of the network.

Benefits of technology

While maintaining high precision, the complexity and computing consumption of the image segmentation network are significantly reduced, and are suitable for resource-constrained embedded devices, improving application capabilities on edge computing platforms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118552739B_ABST
    Figure CN118552739B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of remote sensing image processing, and specifically to a method for compressing an image segmentation model based on hardware perception; including obtaining a visible light image; extracting multi-scale features of the image using a lightweight backbone network; using a lightweight PAN network for multi-scale learning; lightweighting the multi-task head structure responsible for classification and mask regression branches, and selecting depth-separable convolution to replace ordinary convolution; using a directed polar mask IoU loss on the mask regression loss function to guide the detection network to learn target direction information; and using a knowledge distillation method during the network training process. The present invention uses strategies such as hardware perception backbone network construction, regression loss function improvement, and model distillation to reduce the computational complexity of the image segmentation network while ensuring the segmentation effect, and improve its running speed on an embedded platform, and can be applied to platforms with limited computing resources such as drones.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of remote sensing image processing, and in particular to an image segmentation model compression method based on hardware perception. Background Art

[0002] In the field of computer vision, image segmentation is one of the important and fundamental research directions, which aims to achieve category recognition and precise positioning of each instance target in the image. Specifically, image segmentation can not only accurately identify the instance category but also accurately segment the pixels in the image area where the instance is located.

[0003] Instance segmentation related technologies have played an important role in various fields. For example: in the field of autonomous driving, from the perspective of computer vision, road information is integrated into the vehicle assisted driving system through lightweight instance segmentation technology, and information such as pedestrians, vehicles, traffic lights and signs are comprehensively and intelligently judged; in the field of smart medical care, the introduction of instance segmentation technology can realize actual medical scenarios such as pathogen localization, lesion judgment, mutant cell and external morphology segmentation to assist in improving diagnostic efficiency and interpretation accuracy.

[0004] However, image segmentation algorithms usually require a lot of computing resources and storage space, which limits their application in resource-constrained environments. For example, embedded devices, mobile devices, and edge computing devices have limited computing resources and storage space. Traditional remote sensing image segmentation algorithms, such as convolutional neural network (CNN) models based on deep learning, have large parameters and high computational complexity and cannot be directly applied to these resource-constrained devices. Therefore, studying lightweight image segmentation algorithms has become the key to solving this problem.

[0005] However, in the current lightweight intelligent computing applications, the acceleration of deep learning models is still limited to models with smaller scale and fewer parameters. The existing deep network lightweight methods suitable for general hardware devices are mainly characterized by reducing parameter redundancy to facilitate network sparse representation, but ignore the dynamic connection between weight parameters, network structure and network accuracy changes, and lack the prediction of model compression quality under specific platforms, resulting in the inability to unify the high efficiency of model sparsification and the high accuracy of the network, which does not meet the requirements of network accuracy and processing power for intelligent processing on resource-constrained platforms. Therefore, it is necessary to construct a more efficient convolutional network lightweight and acceleration technology suitable for edge intelligent processing based on the dynamic properties of convolutional networks and the characteristics of typical hardware platforms. Summary of the invention

[0006] The present invention provides an image segmentation model compression method based on hardware perception, which aims to solve the compression and optimization problems of deep learning image segmentation networks under resource-constrained hardware platforms, and to improve the operating efficiency of image segmentation networks while maintaining accuracy.

[0007] To achieve the above object, the technical solution of the present invention is:

[0008] The present invention provides an image segmentation model compression method based on hardware perception, comprising the following steps:

[0009] Step 1: Get a high-resolution input image and divide the original image into several blocks with a pixel resolution of The slice image is then normalized by standard deviation and used as the input of the image segmentation and recognition network;

[0010] Step 2: Using the convolutional neural network based on structural re-parameterization as the backbone network, the three scale features of the target in the slice image are extracted, that is, the feature maps are obtained by downsampling 8, 16 and 32 times respectively. , and ;

[0011] Step 3: According to the generation rules of the lightweight PAN feature pyramid network, the feature map , and Perform fusion processing to obtain multi-scale features , and ;

[0012] Step 4: Lightweight the detection head responsible for classification and regression branches, and select depth-wise separable convolution to replace ordinary convolution;

[0013] Step 5: In order to improve the accuracy of the lightweight image segmentation network, the directional polarity mask IoU loss is used on the mask regression loss function to guide the detection network to learn the target direction information;

[0014] Step 6: Use the supervised information of the large model to guide the training process of the small model through model distillation technology. This method aims to significantly improve the performance and accuracy of the small model by extracting key knowledge from the large model and passing it to the small model. Through this process, the small model can achieve a performance level comparable to that of the large model while maintaining high efficiency, providing a more flexible and efficient solution for practical applications.

[0015] Further, the step 2 specifically comprises the following steps:

[0016] Step 2.1: In order to meet the deployment requirements of image segmentation tasks on resource-constrained platforms and speed up the inference speed while ensuring high accuracy, the RepVGG (Reparameterization Visual Geometry Group) network is introduced;

[0017] Step 2.2, during the training phase, the RepVGG network uses skip connections and Branches build multi-branch structures to improve network performance;

[0018] Step 2.3, in the inference stage, through structural reparameterization, the multi-branch network structure and parameters during model training are converted into a single-path network structure and parameters during inference, so that RepVGG can take into account both the advantages of multi-branch model training and the advantages of single-path architecture inference, fast memory saving.

[0019] Further, the step 3 specifically comprises the following steps:

[0020] Step 3.1, use the bidirectional path feature pyramid (PAN) for multi-scale learning, and use the CSP (Cross-Stage-Partial-connections) structure to connect and fuse adjacent feature maps. The CSP structure can achieve richer gradient combinations through dense local connections across stages;

[0021] Step 3.2: In order to reduce the computational complexity of the CSP-PAN network, the CSP-Block used in CSP-PAN is replaced by RepBlock, and the operators in the overall feature pyramid are adjusted. The purpose is to achieve efficient reasoning on hardware while maintaining good multi-scale feature fusion capabilities;

[0022] Further, the step 4 specifically comprises the following steps:

[0023] Step 4.1: In order to reduce the computational complexity of the detection head, The convolution kernel is replaced by Depthwise separable convolution. Compared with standard Compared with convolution, The depthwise separable convolution can expand the receptive field of the detector and improve the performance with less increase in the number of parameters.

[0024] Step 4.2: In order to further reduce the computational complexity of the detection head, the branch responsible for center regression in the benchmark detection algorithm FCOS is removed, leaving only the classification and regression branches;

[0025] Further, the step 5 specifically comprises the following steps:

[0026] Step 5.1, optimize the multi-task training loss function of the classic PolarMask algorithm so that when it faces a target with an angle, it can drive the segmentation model to learn the direction information of the target; training loss function Including classification loss function , Bounding box regression loss function , direction loss function and directional polarization mask IoU loss function ;in, and The same as the function in FCOS, using Focal Loss and GIoU loss respectively; the training loss function The definition is as follows:

[0027] (1.1)

[0028] in, and Represent the classification confidence and true label of the target respectively. and Represent the target true bounding box and the predicted bounding box output by the network, respectively. and They represent the predicted direction angle and the true instance direction angle respectively. Indicates the number of positive samples, parameter , and Represents the weight adjustment factor. is the indicator function, if the target true label , it means that the true label is not the background, that is .

[0029] Step 5.2, for the branch network structure design, the network adopts four The convolutional layer of the neural network is then used to smooth the feature map. Then a feature map with 256 channels is obtained. After that, the branch adds 6 Neural convolutional layer to transform features. Finally, a feature map consisting of 6 channels is obtained, each channel represents the probability that the direction of the pixel belongs to this angle interval. Assume is the predicted angle interval, is the direction of each pixel calculated from the true annotation, and the direction loss function is defined by the binary cross entropy (BCE) loss, as shown in the following formula.

[0030] (1.2)

[0031] Step 5.3, the present invention uses the directional polar mask IoU to optimize the mask regression. In polar coordinates, the standard IoU can be calculated by dividing the square integral of the smaller distance and the square integral of the larger distance. Assuming that the predicted target direction is consistent with the direction of the true labeled box of the high-resolution image in step 1, represents the predicted distance, represents the distance in the true label, Represents the number of sampling rays, which can be calculated by the following formula:

[0032] (1.3)

[0033] Simplify the above formula and convert it into discrete form, and simplify it accordingly to get the weighted polarity IoU, as shown in the following formula:

[0034] (1.4)

[0035] Further, the step 6 specifically comprises the following steps:

[0036] Step 6.1, use the basic model as the teacher model and the lightweight model as the student model;

[0037] Step 6.2, train the student model with the ultimate goal of knowledge distillation;

[0038] Step 6.3, transfer the intermediate representation of the segmentation model from the teacher model to the student model:

[0039] (1.5)

[0040] in, represents the overall training loss function during the model distillation process, is the instance segmentation network error of formula (1.1), is the distillation error of the teacher model and the student model transfer knowledge, is a hyperparameter. The expression is

[0041] (1.6)

[0042] in, and is the first The intermediate features of the layer, Indicates the number of network layers, is a The convolutional layer is used to map the features of the student model to the same number of channels in the corresponding features of the teacher model.

[0043] The beneficial effects achieved by the present invention are:

[0044] In response to the problems of limited computing power, storage and power consumption resources provided by edge computing platforms such as embedded ones, the present invention follows the basic detection process of "feature extraction backbone network-network prediction branch" and adopts a single-stage anchor-free frame structure in the selection of deep learning image instance segmentation framework, so as to minimize the contradiction between the complexity of deep learning calculations and the performance of edge computing platforms such as embedded ones.

[0045] First, in order to meet the real-time requirements of the algorithm and enhance the feature extraction capability of the network, a lightweight structured re-parameterized RepVGG backbone network is introduced to improve detection accuracy. By re-parameterizing the model inference stage, different network architectures are used in the network training and network inference stages, which greatly reduces the number of parameters. This allows the algorithm to be deployed on low-computing-power devices and better applied to practical scenarios.

[0046] Secondly, in the design of the feature pyramid network, the accuracy of feature extraction and the size of the model are taken into consideration, and the network is pruned and optimized in the later stage. Feature pyramid network (FPN) is a commonly used multi-scale feature learning method to solve the difficulty of large target space scale. Based on the pyramid shape of the convolutional neural network, FPN adds the feature fusion operation of adjacent layers to obtain multi-scale features with both resolution and semantic information. However, the FPN structure occupies a large amount of running memory and is not suitable for application in edge computing platforms such as embedded systems with limited performance. In order to reduce the computational complexity of the FPN structure and maintain good multi-scale detection capabilities, the present invention designs a lightweight PAN (Path Aggregation Network) network, which only contains two paths from top to bottom and from bottom to top, that is, it only contains the convolution after the backbone network feature extraction to complete the channel dimension alignment, and uses the interpolation algorithm to complete the up and down sampling, and realizes the fusion of multi-scale feature maps by addition.

[0047] Secondly, the multi-task head branch structure is lightweight: the shared weights are abandoned, and different convolution groups are still used to complete the feature extraction of each layer. At the same time, the original group normalization method is changed back to batch normalization, and the depth-separable convolution is selected to replace the ordinary convolution, and the number of convolution kernels and convolution channels in the regression classification process is reduced. The computational complexity is minimized to achieve a lightweight network model;

[0048] Afterwards, in the regression loss function, the directed polarity mask IoU loss is used on the mask regression loss function to guide the detection network to learn the target direction information; finally, in order to further enhance the detection accuracy of lightweight image segmentation, the idea of ​​knowledge distillation is used to improve its performance without increasing its computational consumption.

[0049] In order to design a lightweight image segmentation network that balances accuracy and speed, the present invention constructs a hardware-aware image segmentation model compression method. The advantages of the present invention are: in order to reduce the complexity of the image segmentation network, the method adopts a structurally re-parameterized backbone feature extraction network and a lightweight feature pyramid design, and at the same time, in order to compensate for the performance loss caused by network compression, a high polarity mask IoU regression positioning loss and a model distillation strategy are used to compensate for the accuracy loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0051] Figure 1 Shown is a schematic diagram of the grid structure for image instance segmentation of the method of the present invention.

[0052] Figure 2 Shown is a structural re-parameterization diagram of the RepVGG module of the method of the present invention.

[0053] Figure 3 The CSP-PAN network of the method of the present invention is shown.

[0054] Figure 4 Shown is the Rep-PAN lightweight feature pyramid. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0057] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, if the meaning of "and / or" appearing in the full text is to include three parallel schemes, taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0058] like Figure 1 As shown, the present invention discloses a hardware-aware image segmentation model compression method, which specifically includes the following steps:

[0059] Step 1: Use the image sensor to collect data and obtain a high-resolution image; in order to adapt to the input requirements of the image segmentation network, the original image is divided into several blocks with a pixel resolution of The slice image is then normalized by standard deviation and used as the input of the image segmentation and recognition network;

[0060] Step 2: If Figure 1 As shown in the figure, a convolutional neural network based on structural re-parameterization is used as the backbone network to extract the three scale features of the target in the slice image, that is, downsampling by 8, 16 and 32 times to obtain the feature maps respectively. , and ;

[0061] (2.1) In order to meet the deployment requirements of image segmentation tasks on resource-constrained platforms and speed up the inference speed while ensuring high accuracy, the RepVGG (Reparameterization Visual Geometry Group) network is introduced. It has the advantages of anti-gradient degradation and high training accuracy in the training stage, and high parallelism, memory saving, and fast inference speed in the inference stage.

[0062] (2.2), such as Figure 2 As shown in Figure 2, during the training phase, the RepVGG network uses skip connections and Branches build multi-branch structures to improve network performance;

[0063] (2.3), such as Figure 2As shown in the figure, during the inference phase, the multi-branch network structure and parameters during model training are converted into a single-path network structure and parameters during inference through structural reparameterization, so that RepVGG can take into account both the advantages of multi-branch model training and the advantages of single-path architecture inference speed and memory saving.

[0064] Step 3: According to the generation rules of the lightweight PAN feature pyramid network, the feature map , and Perform fusion processing to obtain multi-scale features , and , the specific sub-steps are:

[0065] (3.1), such as Figure 3 As shown in the figure, a bidirectional path feature pyramid (PAN) is used for multi-scale learning, and a CSP (Cross-Stage-Partial-connections) structure is used to connect and fuse adjacent feature maps. CSP is a connection strategy used in convolutional neural networks, especially in target detection tasks. The CSP structure can achieve richer gradient combinations through dense local connections across stages;

[0066] (3.2), such as Figure 4 As shown in Figure 1, in order to reduce the computational complexity of the CSP-PAN network, the CSP-Block used in CSP-PAN is replaced by RepBlock; in the inference stage, RepBlock is converted to a stack of 3×3 convolutional layers + ReLU activation functions (denoted as RepConv), as shown in Figure 1. Figure 2 As shown in the figure. Because 3×3 convolution has better optimization and computational density on both CPU and GPU, it can effectively utilize computing resources and increase inference speed while enhancing representation capabilities. After that, the width and depth of the 1×1 and 3×3 convolutions in the feature pyramid are adjusted to keep the feature channels and feature dimensions in one-to-one correspondence. The purpose of the above operations is to achieve efficient inference on hardware while maintaining good multi-scale feature fusion capabilities.

[0067] Step 4: Figure 1 As shown in the figure, the detection head responsible for classification and regression branches is lightweight, and depth-separable convolution is selected to replace ordinary convolution;

[0068] (4.1), in order to reduce the computational complexity of the detection head, the center of the detection head is The convolution kernel is replaced by Depthwise separable convolution. Compared with standard Compared with convolution, The depthwise separable convolution can expand the receptive field of the detector and improve the performance with less increase in the number of parameters.

[0069] (4.2), in order to further reduce the computational complexity of the detection head, the branch responsible for center regression in the benchmark detection algorithm FCOS is removed, leaving only the classification and regression branches;

[0070] Step 5: In order to improve the accuracy of the lightweight detection network, the Gauss-Wartherstein distance rotation regression positioning loss is used in the loss function to guide the detection network to learn the target direction information;

[0071] (5.1) The multi-task training loss function of the classic PolarMask algorithm is optimized so that when it faces a target with an angle, it can drive the segmentation model to learn the direction information of the target. Training loss function By classification loss function , Bounding box regression loss function , direction loss function and directional polarization mask IoU loss function Among them, and The same as the function in FCOS, using Focal Loss and GIoU loss respectively. The training loss function is The definition is as follows:

[0072] (1.1)

[0073] in, is the index of the pixel, and Represent the classification confidence and true label of the target respectively. and Represent the target true bounding box and the predicted bounding box output by the network, respectively. and They represent the predicted direction angle and the true instance direction angle respectively. Indicates the number of positive samples, parameter , and Represents the weight adjustment factor. is the indicator function, if the target true label , it means that the true label is not the background, that is .

[0074] (5.2), for the branch network structure design, the network adopts four The convolutional layer of the neural network is then used to smooth the feature map. Then a feature map with 256 channels is obtained. After that, the branch adds 6 Neural convolutional layer to transform features. Finally, a feature map consisting of 6 channels is obtained, each channel represents the probability that the direction of the pixel belongs to this angle interval. Assume is the predicted angle interval, is the direction of each pixel calculated from the true annotation, and the direction loss function is defined by the binary cross entropy (BCE) loss, as shown in the following formula.

[0075] (1.2)

[0076] in, Indicates Pixels, is the number of pixels.

[0077] (5.3), the present invention uses oriented polar mask IoU to optimize mask regression. In polar coordinates, the standard IoU can be calculated by dividing the square integral of the smaller distance and the square integral of the larger distance. Assuming that the predicted target direction is consistent with the direction of the true labeled box of the high-resolution image in step 1, represents the predicted distance, represents the distance in the true label, Represents the number of sampling rays, which can be calculated by the following formula:

[0078] (1.3)

[0079] in, Indicates sampling rays, is the number of sampling rays.

[0080] Simplify the above formula and convert it into discrete form, and simplify it accordingly to get the weighted polar IoU, then Polar IoULoss (i.e., in formula (5.1) ) is shown in the following formula:

[0081] (1.4)

[0082] Step 6: Use the supervised information of the large model to guide the training process of the small model through model distillation technology. This method aims to significantly improve the performance and accuracy of the small model by extracting key knowledge from the large model and passing it to the small model. Through this process, the small model can achieve a performance level comparable to that of the large model while maintaining high efficiency, providing a more flexible and efficient solution for practical applications.

[0083] (6.1), the basic model is used as the teacher model and the lightweight model is used as the student model;

[0084] (6.2), train the student model with the ultimate goal of knowledge distillation;

[0085] (6.3), transfer the intermediate representation of the segmentation model from the teacher model to the student model:

[0086] (1.5)

[0087] in, represents the overall training loss function during the model distillation process, is the instance segmentation network error of formula (1.1), is the distillation error of the teacher model and the student model transfer knowledge, is a hyperparameter. The expression is

[0088] (1.6)

[0089] in, and is the first The intermediate features of the layer, Indicates the number of network layers, is a The convolutional layer is used to map the features of the student model to the same number of channels in the corresponding features of the teacher model.

[0090] Through the above steps, a lightweight image segmentation network with high accuracy and high operating efficiency can be realized. This method reduces the complexity of the image segmentation algorithm, adopts a structurally re-parameterized backbone feature extraction network and a lightweight feature pyramid design, and in order to compensate for the performance loss caused by network compression, Gauss-Wasserstein distance rotation regression positioning loss and model distillation strategy are used to compensate for the accuracy loss.

[0091] The present invention provides a new method and technical approach for lightweight image segmentation under resource constraints, and has great practical application value.

[0092] The above descriptions are only optional embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A hardware-aware image segmentation model compression method, characterized in that: The following steps are involved: Step 1: Get a high-resolution input image and divide the original image into several blocks with a pixel resolution of The slice image is then normalized by standard deviation; Step 2: Using the convolutional neural network based on structural re-parameterization as the backbone network, the three scale features of the target in the slice image are extracted, that is, the feature maps are obtained by downsampling 8, 16 and 32 times respectively. , and ; Step 3: According to the generation rules of the lightweight PAN feature pyramid network, the feature map , and Perform fusion processing to obtain multi-scale features , and ; Step 4: Lightweight the head structure responsible for classification, center regression, and mask regression branches, and select depthwise separable convolution to replace ordinary convolution; Step 5: In order to improve the accuracy of the lightweight image segmentation network, the directional polarity mask IoU loss is used on the mask regression loss function to guide the detection network to learn the target direction information; Step 6: Through model distillation technology, the supervision information of the large model is used to guide the training of the small model, so as to extract key knowledge from the large model and pass it to the small model.

2. The hardware-aware image segmentation model compression method according to claim 1, characterized in that: The structure-re-parameterized convolutional neural network described in step 2 includes the following steps: Step 2.1: In order to meet the deployment requirements of image segmentation tasks on resource-constrained platforms and speed up the inference speed while ensuring high accuracy, the RepVGG network is introduced; Step 2.2, during the training phase, the RepVGG network uses skip connections and Branches build multi-branch structures; Step 2.3: In the inference phase, the multi-branch network structure and parameters during model training are converted into a single-path network structure and parameters during inference through structural reparameterization.

3. The hardware-aware image segmentation model compression method according to claim 1, characterized in that: The lightweight PAN feature pyramid network described in step 3 includes the following steps: Step 3.1, use the bidirectional path feature pyramid (PAN) for multi-scale learning, and use the CSP structure to connect and fuse adjacent feature maps; In step 3.2, in order to reduce the computational complexity of the CSP-PAN network, the CSP-Block used in CSP-PAN is replaced by RepBlock, and the operators in the overall feature pyramid are adjusted.

4. The hardware-aware image segmentation model compression method according to claim 1, characterized in that: Step 4 specifically includes the following steps: Step 4.1: In order to reduce the computational complexity of the head structure, the center of the head structure is The convolution kernel is replaced by Depthwise separable convolution; Step 4.2: In order to further reduce the amount of calculation for the head structure, the branch responsible for center regression in the benchmark detection algorithm PolarMask is removed, leaving only the classification and regression branches.

5. The hardware-aware image segmentation model compression method according to claim 1, characterized in that: Step 5 specifically includes the following steps: Step 5.1, optimize the multi-task training loss function of the classic PolarMask algorithm so that when it faces a target with an angle, it can drive the segmentation model to learn the direction information of the target; training loss function Including classification loss function , Bounding box regression loss function , direction loss function and directional polarization mask IoU loss function ;in, and The same as the function in FCOS, using Focal Loss and GIoU loss respectively; the training loss function The definition is as follows: (1.1) in, and Represent the classification confidence and true label of the target respectively. and Represent the target true bounding box and the predicted bounding box output by the network, respectively. and Represent the predicted direction angle and the real instance direction angle respectively; Indicates the number of positive samples, parameter , and represents the weight adjustment factor; is the indicator function, if the target true label , it means that the true label is not the background, that is ; Step 5.2, for the branch network structure design, the network adopts four The convolutional layer of the neural network is then used to smooth the feature map by performing a ReLU operation. A feature map with 256 channels is then obtained. After that, the branch adds 6 Neural convolution layer to transform features; finally a feature map consisting of 6 channels is obtained, each channel represents the probability that the direction of the pixel belongs to this angle interval; assuming is the predicted angle interval, The root truth annotation calculates the direction of each pixel, and the direction loss function is defined by the binary cross entropy (BCE) loss, as shown in the following formula: (1.2) Step 5.3, use the oriented polar mask IoU to optimize the mask regression; in polar coordinates, the standard IoU can be calculated by dividing the square integral of the smaller distance and the square integral of the larger distance; assuming that the predicted target direction is consistent with the direction of the true labeled box of the high-resolution image in step 1, represents the predicted distance, represents the distance in the true label, Represents the number of sampling rays, which can be calculated by the following formula: (1.3) Simplify the above formula and convert it into discrete form, and simplify it accordingly to get the weighted polarity IoU, as shown in the following formula: (1.4)。 6. The hardware-aware image segmentation model compression method according to claim 1, characterized in that: Step 6 specifically includes the following steps: Step 6.1, use the basic model as the teacher model and the lightweight model as the student model; Step 6.2, train the student model with the ultimate goal of knowledge distillation; Step 6.3, transfer the intermediate representation of the segmentation model from the teacher model to the student model: (1.5) in, represents the overall training loss function during the model distillation process, is the instance segmentation network error of formula (1.1), is the distillation error of the teacher model and the student model transfer knowledge, is a hyperparameter; The expression is: (1.6) in, and is the first The intermediate features of the layer, Indicates the number of network layers, is a The convolutional layer is used to map the features of the student model to the same number of channels in the corresponding features of the teacher model.