Lightweight model optimization method and system fusing cyclic convolution and re-parameterization

Abstract

The invention belongs to the technical field of embedded platform model optimization, and particularly relates to a lightweight model optimization method and system fusing cyclic convolution and re-parameterization. The method aims at solving the problems that an existing lightweight network is large in calculation amount, high in delay and poor in adaptability on resource-limited equipment. According to the method, a Metaformer framework is adopted as a basic framework; the method comprises the following steps: based on a MobileNet-V1 model, moving up a DW layer and an SE module, introducing batch normalized reparameterized jump connection, and constructing a shallow network module; global cyclic position sensing convolution is adopted, vertical and horizontal direction convolution is proposed, and a deep network module is constructed; and stacking the shallow and deep modules to construct a lightweight network model. Through high efficiency of local feature extraction, light weight of global feature capture, optimization of an attention mechanism and collaborative design of an architecture, the local feature extraction efficiency is improved, the global features are efficiently captured, the calculation overhead is reduced, and good balance between performance and efficiency is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of embedded platform model optimization, and particularly relates to a lightweight model optimization method and system fusing cyclic convolution and reparameterization. BACKGROUND

[0002] Traditional CNN (Convolutional Neural Network, CNN for short) gradually expands the receptive field by stacking local convolution operations, and its local feature extraction capability makes it achieve remarkable success in visual tasks. With the increasing demand for real-time performance and low power consumption of mobile devices, embedded systems and edge devices, the deployment of CNN on resource-constrained devices faces the challenges of high computational complexity and memory consumption. Many lightweight convolutional network architectures have emerged: MobileNet significantly reduces computational and parameter quantities through depthwise separable convolution and inverted residual structure; ShuffleNet optimizes memory bandwidth and computing resources using channel shuffle technology; GhostNet further reduces computational complexity by generating "pseudo feature maps" and reduces redundant calculations while maintaining accuracy. These lightweight architectures achieve a good balance between performance and efficiency on mobile devices, but they still have limitations in capturing global features and modeling long-distance dependencies. Researchers have proposed hybrid architectures that combine the advantages of CNN and ViT (Vision Transformer, ViT for short) to leverage the strengths of both. For example, CoAtNet combines convolution and self-attention mechanisms to achieve a unified local and global modeling capability; MobileFormer integrates Transformer modules with CNN modules through an efficient communication mechanism. However, these hybrid architectures, while improving performance, still have problems of insufficient computational efficiency and high latency, making it difficult to fully meet the needs of resource-constrained devices.

[0003] In the prior art, lightweight CNNs significantly reduce computational and parameter quantities through depthwise separable convolution and channel shuffle optimization techniques, improving the efficiency of local feature extraction. However, their receptive field expansion capability is limited, and they can only capture local context information, performing poorly in tasks that require modeling long-distance dependencies and global context relationships. The self-attention mechanism introduced by ViT can effectively capture long-distance dependencies and global features, but the computational complexity of self-attention is Especially in high-resolution images, the computational resource consumption and memory occupation increase significantly. Such computational characteristics make ViT have high inference delay and power consumption in embedded systems and edge devices, which is difficult to deploy. Existing lightweight network architectures do not fully utilize optimization methods such as structure reparameterization and channel shuffle in the design, resulting in that the computational overhead and delay optimization of the model in the inference stage cannot meet the expected. SUMMARY

[0004] The application provides a lightweight model optimization method and system fusing cyclic convolution and reparameterization, to solve the technical problems of existing lightweight network architectures in prior art, such as large amount of calculation, high delay and poor adaptability in resource-constrained embedded devices.

[0005] In order to achieve the above purpose, the application adopts the following technical solutions: A lightweight model optimization method fusing cyclic convolution and reparameterization, comprising the following steps: Token mixer and channel mixer of the Metaformer framework are used as the network basic architecture; Based on the MobileNet-V1 model, the DW (Depthwise Separable Convolution Layer) layer and the SE (Squeeze-and-Excitation Module) module of the MobileNet-V1 model are moved up in the token mixer of the Metaformer framework, and a reparameterizable skip connection of batch normalization is introduced into the MobileNet-V1 model to construct a shallow network module; Global cyclic position-aware convolution is used, and position-aware convolution in the vertical direction and the horizontal direction is performed in the channel mixer of the Metaformer framework to construct a deep network module; A lightweight network model is constructed by stacking a plurality of shallow network modules and deep network modules, and a lightweight image data classification processing is performed using the lightweight network model.

[0006] Based on the MobileNet-V1 model, the DW layer and the SE module of the MobileNet-V1 model are moved up in the token mixer structure of the Metaformer framework, and a reparameterizable skip connection of batch normalization is introduced into the MobileNet-V1 model to construct a shallow network module, specifically: the Metaformer framework includes a token mixer and a channel mixer, both the token mixer and the channel mixer use a residual structure, the DW layer and the SE module in the MobileNet-V1 model are moved up, the token mixer is The deep separable convolution of the depth separable convolution is introduced into the MobileNet-V1 model after moving the DW layer and the SE module, the re-parameterized jump connection with batch normalization is introduced, the batch normalization and the convolution are combined into one operation, and a deep network module is formed.

[0007] The vertical position-aware convolution is GCPA-V, and the horizontal position-aware convolution is GCPA-H. The receptive fields of the two position-aware convolutions GCPA-V and GCPA-H cover all pixels in the same column and the same row, respectively. When the feature extraction direction is the vertical direction, GCPA-V is used for convolution; when the feature extraction direction is the horizontal direction, GCPA-H is used for convolution. GCPA-H and GCPA-V are processed in parallel, and the features extracted in two directions are output. The features output by GCPA-H and GCPA-V are spliced and fused to generate a feature representation containing global information, and then global information is extracted from the input data.

[0008] When the feature extraction direction is the vertical direction, GCPA-V is used for convolution. GCPA-V adjusts the size of the position embedding according to the size of the input feature through a bilinear interpolation function, generates a series of position encoding vectors, and injects position information into each pixel. The position encoding vector generation formula is as follows:

[0009] wherein, is the position encoding vector, is the bilinear interpolation function, is the initial basic position embedding vector, is the number of pixels in the vertical direction, indicates that the row vector is transposed into a column vector.

[0010] The generated position encoding vector is copied times, and the copied position encoding vector is spliced in the horizontal direction using a vertical direction expansion function to obtain a position embedding matrix with a size of The vertical direction expansion function is as follows:

[0011] wherein, is the expanded position embedding matrix, is the expansion function in the vertical direction.

[0012] GCPA-V combines the input feature of the deep network module with the above-mentioned expanded position embedding matrix to form a position-aware input feature. The position-aware input feature generation formula is as follows:

[0013] wherein, is a position-aware input feature, is an input feature of a deep network module.

[0014] GCPA-V is based on the position-aware input feature generated above, the convolution kernel of GCPA-V adopts a cyclic convolution manner in the vertical spatial dimension, and outputs a fused vertical direction global information feature, and the calculation formula of the fused vertical direction global information feature is as follows:

[0015] wherein, is a fused vertical direction global information feature, is a cyclic convolution offset, is a vertical direction and a cyclic convolution offset corresponding weight, is a cyclic index, is a vertical coordinate of a certain position in an output feature map, is a vertical dimension size of an input feature map, is a horizontal coordinate of a certain position in a feature map.

[0016] The weight of the height of the vertical direction instance convolution kernel is generated by bilinear interpolation, and the weight matrix generation formula of all heights in the vertical direction instance convolution kernel is: wherein, is an initial basic weight vector, is a bilinear interpolation function, is a vertical direction and a cyclic convolution offset value range.

[0017] In the deep network module, an ECA (Efficient Channel Attention, channel-level attention mechanism) channel-level attention mechanism is introduced in the feedforward neural network part of the channel mixer, each channel of the deep network module input feature is globally pooled, spatial features are extracted, the spatial features extracted by each channel are compressed into a scalar, then the global features of each channel are mapped to the interval [0, 1] through a Sigmoid activation function, attention weights of each channel are generated, the size of the convolution kernel is dynamically selected, the scale of the convolution kernel is adjusted according to the number of channels, and finally the generated attention weights are applied to the original input feature map channel by channel, the feature importance of each channel is adjusted, and a weighted output feature map is generated.

[0018] The application discloses a lightweight model optimization system fusing a cyclic convolution and a reparameterization, comprising a basic architecture construction unit, a shallow network module construction unit, a deep network module construction unit and a lightweight network model construction unit. The basic architecture construction unit is used for adopting a token mixer and a channel mixer of a Metaformer framework as a network basic architecture. The shallow network module construction unit is used for moving a DW layer and an SE module of a MobileNet-V1 model upwards in the token mixer of the Metaformer framework and introducing a reparameterizable skip connection with batch normalization into the MobileNet-V1 model based on the MobileNet-V1 model to construct a shallow network module. The deep network module construction unit is used for adopting a global cyclic position-aware convolution to perform position-aware convolution in two directions of a vertical direction and a horizontal direction in the channel mixer of the Metaformer framework to construct a deep network module. The lightweight network model construction unit is used for stacking a plurality of shallow network modules and deep network modules to construct a lightweight network model and performing lightweight image data classification processing by using the lightweight network model.

[0019] Compared with the prior art, the application has the following beneficial effects: The lightweight model optimization method fusing the cyclic convolution and the reparameterization is proposed in the application, the shallow network module is based on the Metaformer framework and combines the optimization idea of the MobileNet-V1 model, the DW layer and the SE module are moved upwards, the reparameterizable skip connection with batch normalization is introduced, the efficiency of local feature extraction is realized, the DW layer and the SE module are moved upwards to form a depth separable convolution, the depth separable convolution itself has reduced the calculation amount through the separation of the channel-by-channel convolution and the 1x1 point convolution, the feature processing procedure is further optimized after the DW layer and the SE module are moved upwards, the combination of the local feature extraction and the channel attention mechanism is more efficient, and the redundant intermediate calculation is avoided.

[0020] In the lightweight model optimization method combining cyclic convolution and reparameterization, the deep network module adopts global cyclic position perception convolution, which compensates for the defects of limited receptive field and only capturing local information of traditional lightweight CNN through parallel convolution design in vertical and horizontal directions. Among them, through parallel convolution in vertical and horizontal directions, the receptive field of convolution covers the whole column and whole row of pixels of the input feature map, which can theoretically capture spatial dependence of any long distance, and solve the problem of weak long-distance dependence modeling ability caused by slow expansion of receptive field through stacking convolution layers in traditional CNN. Compared with the self-attention mechanism of ViT, the global cyclic position perception convolution in the application reduces the complexity of global feature extraction to linear through cyclic convolution index and parallel direction processing perpendicular to horizontal, greatly reduces the consumption of computing resources and memory occupation under high-resolution image, and is more suitable for low-latency requirements of embedded and edge devices.

[0021] Further, in the global cyclic position perception convolution, the introduced bilinear interpolation position coding solves the problem of position information loss in global feature modeling. It dynamically adjusts the size of position embedding through the bilinear interpolation function, so that the position coding can adapt to input feature maps of different sizes, ensuring that each spatial position carries accurate position information. The combination of position coding vector and input feature ensures that the global cyclic convolution does not ignore the spatial position correlation when capturing long-distance dependence, avoids the fuzzification of global features, improves the accuracy of feature representation, and is particularly important for position-sensitive visual tasks.

[0022] Further, the application introduces ECA channel-level attention mechanism in the construction of deep network module, which enhances the feature selection ability while avoiding additional computational burden. The ECA channel-level attention mechanism replaces the redundant calculation of the full connection layer in the traditional SE module by dynamically adjusting the size of the convolution kernel, and only generates channel attention weights through global pooling and lightweight convolution, which significantly reduces the computational overhead of the attention mechanism itself. The channel-by-channel weighting property of ECA channel-level attention mechanism can accurately filter important features and suppress redundant information, and cooperates with the global cyclic convolution to further improve the effectiveness of the global feature, reduces invalid calculation while ensuring accuracy, and solves the problem of "difficulty in balancing efficiency and performance" of traditional attention mechanism in lightweight network.

[0023] In the lightweight model optimization method combining cyclic convolution and reparameterization, the shallow network module and the deep network module are stacked to optimize the traditional lightweight model, wherein the shallow network module is used for local feature extraction, and the deep network module is used for global feature extraction, and the overall architecture of the optimized lightweight network model realizes the cooperative modeling of local details and global context. Compared with the CNN or the ViT model alone, the optimized lightweight network model of the application does not need a complex cross-module communication mechanism, and the design is more simple and the calculation efficiency is higher. Compared with the existing hybrid architecture, the application further reduces the delay while maintaining the performance through the lightweight design of reparameterization and cyclic convolution, and can better meet the strict demand of performance-efficiency balance of resource-limited devices. Specifically, the lightweight model optimization method combining cyclic convolution and reparameterization is suitable for embedded devices, mobile terminals, Internet of Things (IoT) nodes, intelligent monitoring cameras, automatic driving edge units and other scenes with limited computing power, memory and power consumption, especially for the low delay, high precision and low resource occupation demand of image classification tasks in these scenes. For the computing power and memory shortage of resource-limited devices, the shallow network module of the application adopts the technical means of depth separable convolution and reparameterizable jump connection, which can reduce the calculation amount and parameters; the deep network module reduces the global feature extraction complexity to linear through global cyclic convolution. In the application, the design of reparameterization combines the batch normalization and convolution in the inference stage of the lightweight model into a single operation, eliminating redundant calculation; the global cyclic convolution avoids the quadratic complexity of self-attention, which can greatly reduce the delay of real-time image classification tasks in actual video stream processing, and realize the smooth experience of image classification and recognition. In view of the weak global feature capture ability and low classification accuracy of small devices in practical application, the application realizes vertical and horizontal position perception convolution through global cyclic position perception convolution, realizes vertical and horizontal parallel cyclic convolution to cover the global receptive field, combines bilinear interpolation position coding to retain position information, and combines ECA channel-level attention mechanism to enhance feature selection, especially for the recognition accuracy of image edge global morphology is obviously improved, solves the misjudgment problem of only seeing local but not overall on small devices. In the adaptability of the model to different sizes of input, the existing technology generally needs to retrain the model due to the change of input feature map size, which leads to high model deployment cost and poor scene adaptability.This invention dynamically adjusts the position embedding and convolution kernel size through bilinear interpolation, achieving adaptive compatibility with different feature map sizes. For vertical position encoding, the initial basic position embedding vector is adjusted based on the height of the input feature map using a bilinear interpolation function to generate a position encoding vector adapted to the current feature map height. For vertical convolution kernels, the initial technical weight vector is also adjusted based on the feature map height using a bilinear interpolation function to generate a weight matrix adapted to the current height. The horizontal position embedding and convolution kernel adjustment use the same method. This technique eliminates the need for repeated training on input feature maps of different sizes; dynamic interpolation during the inference phase can adapt to inputs of any reasonable size. This significantly reduces the training cost of the model in multi-scenario deployments and avoids feature information loss due to forced scaling of the input size, thus significantly improving the model's scenario adaptability in practical applications. Attached Figure Description

[0024] Figure 1 The lightweight model architecture diagram of this invention that integrates recurrent convolution and reparameterization; Figure 2 : A schematic diagram of the shallow network module architecture in this invention; Figure 3 : A schematic diagram of the global recurrent position-aware convolution process in this invention; Figure 4 : A schematic diagram of the deep network module architecture in this invention; Figure 5 : A schematic diagram of the lightweight model optimization method that integrates recurrent convolution and reparameterization in this invention; Figure 6 : A schematic diagram of the lightweight model optimization system that integrates recurrent convolution and reparameterization in this invention. Detailed Implementation

[0025] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0026] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0027] like Figure 5 As shown, this embodiment proposes a lightweight model optimization method that integrates recurrent convolution and reparameterization, including the following steps: The token mixer and channel mixer of the Metaformer framework are used as the network infrastructure; Based on the MobileNet-V1 model, the DW layer and the SE module of the MobileNet-V1 model are moved up in the token mixer of the Metaformer framework, and the reparameterization jump connection with batch normalization is introduced into the MobileNet-V1 model to construct a shallow network module. The global cyclic position-aware convolution is adopted, and the position-aware convolution in the vertical direction and the horizontal direction is performed in the channel mixer of the Metaformer framework to construct a deep network module. The lightweight network model is constructed by stacking a plurality of shallow network modules and deep network modules, and the lightweight image data classification processing is performed by using the lightweight network model.

[0028] Based on the above steps of the lightweight model optimization method for fusing the cyclic convolution and the reparameterization, the following detailed description is provided as follows: The Metaformer framework is selected as the network basic architecture, and the Metaformer framework mainly comprises a token mixer (Token Mixer) and a channel mixer (Channel MLP). The token mixer and the channel mixer both use a residual structure to ensure that the gradient of the deep network module is effectively transmitted to the shallow network module, thereby improving the subsequent feature fusion efficiency.

[0029] Based on the MobileNet-V1 model, the lightweight characteristics of the MobileNet-V1 model are utilized to modify the structure of the key components in the MobileNet-V1 model. The key components in the MobileNet-V1 model include the DW layer and the SE module. The DW layer and the SE module are moved up, and then the token mixer in the Metaformer framework is formed into a deep separable convolution. Further, the reparameterization jump connection with batch normalization is introduced into the MobileNet-V1 model after the structure modification of the key components to construct a shallow network module. By combining multiple operations such as convolution and batch normalization into one operation, the redundant calculation can be reduced during inference. The shallow network module architecture is shown in Figure 2 , wherein Figure 2Fig. 2(a) is a structural diagram of the shallow network module maintaining the structure of the multi-branch stage in the training stage to improve the representation ability of the shallow network module, which mainly includes a main branch, a multi-branch over-parameterized structure, an SE module and a residual connection. The main branch is composed of a 3x3 depth convolution (3x3 DW) and a batch normalization (BN) and is responsible for basic spatial local feature extraction. The multi-branch is parallel to the main branch and includes K sub-paths similar to the main branch. Each sub-path includes convolution and batch normalization (BN) operations. Through multi-branch parallel learning, the parameter capacity during training is increased, the learning ability of the deep network module for detailed features is improved, and the training accuracy is improved. The SE module is located after the fusion of the main branch and the over-parameterized branch and is used to generate channel attention weights to dynamically adjust the feature importance of each channel. In the residual connection and the activation function, the outputs of the main branch, the over-parameterized branch and the SE module are added through the residual connection ( ), and the gradient vanishing problem is alleviated. Finally, the activation function (i.e., Act) is introduced to introduce nonlinearity to enhance the feature expression ability. Figure 2 Fig. 2(b) is a diagram of the fusion of convolution and batch normalization in the shallow network module in the reparameterization stage. After training, the parameters of all over-parameterized branches are combined into the main branch through the reparameterization technology. All over-parameterized branches are removed, and only the 3x3 depth convolution (3x3 DW) and BN of the main branch are reserved. The channel attention weights of the SE module are learned during training and directly act on the feature map during inference without additional training. The activation function (Act) is still reserved to ensure the nonlinear expression of the features. Figure 2 Fig. 2(c) is a structural diagram of the shallow network module degenerating into a single branch in the inference stage, which combines the core logic of the Metaformer architecture to realize zero additional overhead through the series process of normalization (Norm), token mixer (Token Mixer), residual connection ( ) and channel mixer (FFN).

[0030] In the deep network module, a global cyclic position-aware convolution is used to replace the depth separable convolution in the shallow network module into a depth direction position-aware convolution, which not only improves the efficiency of local feature extraction, but also solves the boundary effect problem in traditional convolution by introducing the mechanism of position-aware convolution, while expanding the global perception ability of the network. The global cyclic position-aware convolution process is shown in Fig. 3. First, the channel number C of the input feature map X is divided into two parts by a channel splitting module (such as ChannelSplit in Fig. 4), each part containing K channels. The K represents that the elements in the feature map are real numbers, Figure 3 Figure 3 ​​​​​​a height of the feature map, a width of the feature map, an input feature map is a real tensor with a dimension of , which contains feature information with a height of , a width of , and a number of channels of . The segmented features are independently processed along the horizontal and vertical directions. In the horizontal direction, first, the feature map is subjected to position encoding (Position Encoding-Horizontal, horizontal position encoding) to inject position information for each horizontal direction pixel in the feature map, so that the convolution kernel can perceive the position information in the horizontal direction during processing. The position-encoded feature is then processed by a circular convolution, which ensures that the convolution kernel has global perception ability in the horizontal direction through cyclic processing, i.e., even at the image edge, the convolution operation can completely capture the context information, ensuring that when processing pixels beyond the image boundary, the convolution kernel can return to the starting position of the image, avoiding information loss.

[0031] The global cyclic position-aware convolution is implemented by relying on position-aware convolutions in the vertical and horizontal directions, GCPA-V and GCPA-H, respectively. By using position-aware convolutions in the vertical and horizontal directions, the receptive field covers all pixels in the same column and the same row of the input feature, respectively, capturing features in different directions in the spatial domain, while the cyclic convolution mechanism is used to enhance the global receptive field of the network, thereby achieving global information extraction of the deep network module. Finally, the final feature map is output through channel dimension concatenation (Channel Concat). The architecture of the deep network module is shown in Figure 4 The deep network module receives the feature map from the shallow network, first adjusts the channel dimension through point-wise convolution (PW Conv) to provide an adaptive feature representation for subsequent directional convolution. After adjusting the channel dimension through point-wise convolution, the vertical and horizontal position-aware convolutions are processed in parallel, and then the output features of the parallel processing of the two direction position-aware convolutions are concatenated to generate a feature map of vertical + horizontal global information.

[0032] In the deep network module, the vertical position-aware convolution GCPA-V generates a series of encoding vectors to inject position information for each spatial position, thereby forming a position-aware input feature. Specifically as follows: The vertical position-aware convolution GCPA-V uses a bilinear interpolation function to adjust the size of the position embedding based on the magnitude of the input features, generating a series of position encoding vectors. The formula for generating the position encoding vectors is as follows:

[0033] in, For position encoding vectors, It is a bilinear interpolation function. As the initial base position embedding vector, This represents the number of pixels in the vertical direction. This means transpose a row vector into a column vector.

[0034] Copy the generated position encoding vector. Next, using a vertical expansion function, the copied position encoding vector is concatenated horizontally to obtain a result of size [value missing]. Position embedding matrix:

[0035] in, For the expanded position embedding matrix, This is an extension function in the vertical direction.

[0036] GCPA-V combines the input features of the deep network module with the above-mentioned extended position embedding matrix to form position-aware input features. The formula for generating the position-aware input features is as follows:

[0037] in, For location-aware input features. These are the input features for deep network modules.

[0038] Based on the location-aware input features generated above, GCPA-V uses a cyclical approach for the convolutional kernels in the spatial dimension to output features. The calculation formula for the output features is as follows:

[0039] in, For output features, This is the offset of the circular convolution. Vertical offset relative to the circular convolution The corresponding weights For circular index, To output the vertical coordinates of a specific location in the feature map, The vertical dimension of the input feature map. This represents the horizontal coordinate of a location in the feature map.

[0040] the vertical direction and the cyclic convolution offset corresponding weights The weight matrix of all heights in the instance convolution kernel of the vertical direction is generated by bilinear interpolation, and the generation formula is: wherein, is an initial basic weight vector, is a bilinear interpolation function, is the value range of the vertical direction and the cyclic convolution offset .

[0041] In a further preferred embodiment of the present embodiment, the feedforward neural network part of the channel mixer in the Metaformer framework in the deep network module introduces an ECA channel-level attention mechanism, as shown in Figure 4 The ECA channel-level attention mechanism corresponds to the dashed part in the figure, and the ECA channel-level attention mechanism performs average pooling (Avg pool) on the input features, so that the deep network module can perform global pooling on each channel of the input features, and compress the spatial features of each channel into a scalar. Then, according to the number of channels , the size of the convolution kernel is dynamically determined by the function, and the pooled features are interacted by multi-scale convolution, and the output is still dimensional. After convolution interaction, the global features of each channel are mapped to the interval [0, 1] by the Sigmoid activation function, and the attention weights of dimensional are generated, and the size of the convolution kernel is dynamically selected, and the scale of the convolution kernel is adjusted according to the number of channels. Finally, the generated attention weights are applied to the original input feature map channel by channel, and the feature importance of each channel is adjusted, thereby generating a weighted output feature map with stronger feature expression ability.

[0042] Figure 1 A lightweight network model is constructed by stacking a plurality of shallow network modules and deep network modules constructed as described above. The lightweight network model is specifically as shown in Figure 1 , in the initial stage of the network (i.e. the stem part in , the input end (Input) of the lightweight network model is combined by convolution and convolution to realize fast downsampling and preliminary feature extraction of the input image. In this process, the Figure 1 depthwise separable convolution is used to reduce the computational overhead while effectively extracting local information. In the shallow network module, the lightweight network model stacks a classical convolution block, as shown in , each block is composed of convolution and The depth separable convolution composition not only effectively reduces the parameter amount of the lightweight network model, but also enables the local features to be refined and enhanced layer by layer in the shallow stage. The channel rearrangement technology is also introduced in the shallow stage, which enhances the information interaction between channels, reduces channel redundancy, and improves the diversity of overall feature expression. With the increase of network depth, in the deep network module stage, such as the LGFR Deep Block in FIG. 1, Figure 1 The lightweight network model of the present application introduces the core innovative module-global cyclic position-aware convolution, instead of the traditional convolution block. Through the vertical direction position-aware convolution GCPA-V and the horizontal direction position-aware convolution GCPA-H, the ring convolution operation in the vertical direction and the horizontal direction enables the lightweight network model to have a larger global receptive field while maintaining the local convolution calculation efficiency, and can capture the global features in the image. At the same time, combined with the position encoding generated by the above-mentioned bilinear interpolation function, the spatial position awareness of the lightweight network model is enhanced, which is good at learning local features when dealing with complex visual tasks.

[0043] After all the convolution blocks are stacked, the lightweight network model of the present application uses a 1x1 convolution to further expand the channel number to 1280, and a pooling layer (such as the pooling in FIG. 1) is used to compress the spatial dimension of the feature map into a global feature vector through global average pooling. Finally, the final classification decision is made through a fully connected layer (i.e., FC in FIG. 1), to ensure that the global information can be effectively utilized in the classification stage. Figure 1 Figure 1

[0044] In order to further optimize the feature selection of the lightweight network model, the channel-level attention mechanism and the structure reparameterization technology are combined in the feedforward neural network stage of the lightweight network model: the channel-level attention mechanism can dynamically adjust according to the importance of each channel feature, thereby improving the accuracy of feature selection; the structure reparameterization technology improves the learning ability of features through multi-branch structure in the training stage, and simplifies the network through parameter reorganization in the inference stage, thereby reducing the calculation cost in the inference stage. In the entire deep stage, the channel number is gradually expanded from 64 to 128, 256, and finally to 512, to adapt to the expression of deep features.

[0045] Based on the lightweight network model constructed above, in order to make it suitable for resource-constrained environments such as embedded platforms, it is trained, verified and tested through a large amount of data.

[0046] ​​For image classification tasks, public image datasets such as Cifar10, Cifar100 and ImageNet are obtained, wherein Cifar10 and Cifar100 are composed of 10 classes and 100 classes of 60,000 images respectively, and 50,000 images are used for training and 10,000 images are used for testing. In order to further verify the performance of the model on large datasets, experiments are also carried out on the ImageNet dataset, which includes 1.3 million images from 1000 different object categories and 50,000 images. The dataset is standardized and data enhanced, and the preprocessed dataset is divided into training set, validation set and test set. The training set is used to train the lightweight network model, and the number of times is 300 epochs, and the batch size of the lightweight network model during training is set to 128. In this embodiment, the AdamW optimizer is used for training, and the momentum parameters and are set to 0.9 and 0.999 respectively, the weight decay is set to 0.05, and the initial learning rate is 4e-3. The learning rate adopts a cosine annealing strategy, and the parameter in the cosine annealing strategy is set to 300. In terms of image enhancement, the RandAugment method is used for data enhancement, and in addition, random horizontal flip and random cropping strategies are combined to further improve the generalization ability of the lightweight network model.

[0047] To further verify the effectiveness of the lightweight network model constructed in this embodiment in lightweight image classification tasks, the performance of multiple classic lightweight models on the CIFAR-100 dataset is compared, and the results are shown in Table 1 below. In the following model data comparison, the lightweight network model constructed by the present application is described as LGFRNet, and further detailed description. In this embodiment, the LGFRNet introduces the initial value 1 of the width scaling factor , , and by adjusting the network width, multiple models of different sizes are constructed to meet the inference requirements on various resource-constrained devices. In this embodiment, the lightweight network models used for comparison in the experiment are lightweight network models with a width scaling factor of 1.0 and a width scaling factor of 2.0, namely LGFRNet-1.0 and LGFRNet-2.0.

[0048] Table 1

[0049] As shown in Table 1 above, where Param is the parameter amount of each model, in millions (M), used to reflect the complexity of the reaction model, the smaller the parameter amount, the more lightweight the model; FLOPs is the number of floating point operations of each model, in billions (G), used to reflect the size of the calculation amount when the model is in inference, the smaller the FLOPs value, the faster the model runs on the device with limited computing resources; Top-1 Acc is the accuracy of the model prediction result in the image classification task, the first ranked category is the same as the real category, the unit is percentage (%), the higher the index, the better the accuracy of the model classification; Top-5 Acc is the accuracy of the model prediction result in the top five categories, which contains the real category, the unit is percentage (%), for the data set with more categories, the index can reflect the ability of the model to distinguish similar categories to a certain extent; Latency is the delay of the model when it is in inference, the unit is millisecond (ms), the lower the Latency value, the lower the delay, the faster the response speed of the model in actual application, and the user experience may be better. From Table 1, compared with RepViT-M1.0, LGFRNet-1.0 reduces the parameter amount by 28%, and the FLOPs reduces by 21.4%. In terms of accuracy, the Top-1 accuracy improves by 3.2%, and the Top-5 accuracy improves by 2.0%. At the same time, the inference delay is significantly reduced by 34.8%, proving that the lightweight design of the LGFRNet of the application is more suitable for low delay scenarios. From the experimental results, it can be seen that the LGFRNet series model of the application significantly reduces the parameter amount and the calculation amount while greatly improving the classification accuracy. Compared with multiple classic lightweight network, the parameter amount and floating point operation times FLOPs of the lightweight network model of the application are significantly reduced, which provides the possibility for wide application on edge devices and resource limited devices.

[0050] On the CIFAR-10 data set, the application also compares LGFRNet with other lightweight networks to evaluate the classification accuracy, parameter amount, calculation amount (FLOPs), and inference delay of key performance indicators. The experimental results are shown in Table 2 below, the LGFRNet series model maintains high classification accuracy while achieving lower parameter amount and inference delay. The parameter amount of LPARNet-1.0 is only 60.7% of that of MobileViT-small, and the floating point operation times FLOPs is reduced by 62.1%. The Top-1 accuracy of LPARNet-1.0 is 12.0% higher. At the same time, the inference delay is significantly reduced by 35.4%, fully embodying the lightweight design advantage of the LGFRNet series model.

[0051] Table 2

[0052] On the ImageNet dataset, detailed comparative experiments were conducted on LGFRNet and various classic lightweight networks. Models with similar parameter quantities and FLOPs were selected for comparison to comprehensively evaluate their performance in Top-1 accuracy, Top-5 accuracy, and inference delay. The experimental results are as shown in Table 3. The LGFRNet series lightweight network model of the present application exhibits obvious advantages in many aspects. Among them, RepViT-M1.1 is a pure CNN model designed based on a lightweight ViT, with a parameter quantity of 5.3M and FLOPs of 5.2G. The parameter quantity and floating-point operation times FLOPs of LPARNet-1.0 are reduced by 35.8% and 30.8%, respectively. In terms of performance, the Top-1 accuracy of LPARNet-1.0 is improved by 3.2%, the Top-5 accuracy is improved by 1.7%, and the inference delay is reduced by 11.6%. This result shows that LGFRNet further optimizes the computing performance of ViT in lightweight design.

[0053] Table 3

[0054] Further, in order to verify the contribution of the global cyclic position-aware convolution (GCPA-H and GCPA-W), channel shuffle, structural re-parameterization (SR), and ECA channel-level attention mechanism used in the present application to the performance improvement of the lightweight network model, the present application conducts an ablation experiment. LGFRNet-1.0x is selected for experiment on the CIFAR-100 dataset, with a batch size of 1 and an input image size of 128 128. In the ablation experiment, the present application removes each module and records the accuracy and delay time of the model to evaluate the influence of each module on the model. The experimental results show that removing any one module will cause the model to degrade in accuracy and delay. The specific results are shown in Table 4.

[0055] Table 4

[0056] In summary, the lightweight network model constructed in this invention uses MetaFormer as the external framework, decoupling the spatial dimension feature mixer and channel mixer. Addressing the problem that traditional static convolutional kernels cannot dynamically adjust according to content at different positions and struggle to capture changes in complex scenes, global convolutional kernels and circular convolutions are introduced to achieve awareness of the global structure. Simultaneously, an ECA channel-level attention mechanism is incorporated, generating dynamic convolutional kernels by adaptively adjusting the weights of each kernel, enabling dynamic adjustment of convolutional operations based on the input content. To ensure the model's lightweight nature and low latency, this invention's lightweight network model also incorporates various structural optimization strategies, including channel rearrangement and structural reparameterization of small convolutional kernels. In the performance evaluation experiments, LGFRNet-1.0 improved accuracy by 3.2% compared to RepViT-1.0, while significantly reducing latency and the number of parameters by 28%. Furthermore, compared to MobileNetV2-x1.4, this invention's lightweight network model achieves a 1.4x improvement in inference speed and a 6.3% improvement in accuracy. The lightweight network model of this invention also significantly reduces the number of floating-point operations (FLOPs), achieving a significant optimization and a balance between performance and computational complexity.

[0057] Example 2 Based on the lightweight model optimization method integrating recurrent convolution and reparameterization proposed in Example 1, this example proposes a lightweight model optimization system integrating recurrent convolution and reparameterization, such as... Figure 6 As shown, it includes basic architecture building units, shallow network module building units, deep network module building units, and lightweight network model building units; The infrastructure building unit is used to construct the network infrastructure using the Metaformer framework. It automatically loads the token mixer and channel mixer of the Metaformer framework. The token mixer and channel mixer are residual structures. Through their residual connections, the gradients of deep network modules are effectively transferred to shallow network modules, thereby improving the efficiency of subsequent feature fusion.

[0058] The shallow network module building unit is used to move the DW layer and SE module of the MobileNet-V1 model up in the token mixer of the Metaformer framework based on the MobileNet-V1 model, so that the token mixer is transformed into a 3×3 depthwise separable convolution, which enhances the ability to extract local spatial features. It also introduces batch-normalized reparameterizable skip connections into the MobileNet-V1 model to improve the representation ability of the shallow network module and builds the shallow network module. The shallow network module generates channel attention weights through the SE module, dynamically adjusts the channel importance, and finally outputs features through residual connections and activation functions. The deep network module construction unit is configured to adopt global cyclic position-aware convolution, perform position-aware convolution in two directions of a vertical direction and a horizontal direction in parallel in a channel mixer of the Metaformer framework, construct a deep network module, splice output features in the two directions of the vertical direction and the horizontal direction by the deep network module, and generate a feature map containing global spatial information; and introduce an ECA channel-level attention mechanism in a feedforward neural network of the channel mixer, compress channel features by average pooling, dynamically select a convolution kernel size to generate attention weights, and adjust feature importance channel by channel. The lightweight network model construction unit is configured to stack a plurality of shallow network modules and deep network modules to construct a lightweight network model, and perform lightweight image data classification processing by using the lightweight network model.

[0059] In addition, it should be understood that, although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the specification is described in this way only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that those skilled in the art can understand. The above is only to illustrate the technical idea of the present application, and cannot limit the protection scope of the present application, and any modification made on the basis of the technical solutions according to the technical idea of the present application falls within the protection scope of the claims of the present application.

Claims

1. A lightweight model optimization method fusing cyclic convolution and reparameterization, characterized in that, The method comprises the following steps: Token mixer and channel mixer of the Metaformer framework are used as a network infrastructure; Based on the MobileNet-V1 model, the DW layer and SE module of the MobileNet-V1 model are moved up in the token mixer of the Metaformer framework, and a re-parameterizable skip connection with batch normalization is introduced into the MobileNet-V1 model to construct a shallow network module; Global cyclic position-aware convolution is used in the channel mixer of the Metaformer framework to perform position-aware convolution in the vertical and horizontal directions to construct a deep network module; A lightweight network model is constructed by stacking a plurality of shallow network modules and deep network modules, and the lightweight network model is used for lightweight image data classification processing.

2. The method of claim 1, wherein, The MobileNet-V1 model is introduced into the batch normalization reparameterization skip connection in the token mixer structure of the Metaformer framework, the DW layer and the SE module of the MobileNet-V1 model are moved upwards, and a shallow network module is constructed, specifically: the Metaformer framework includes a token mixer and a channel mixer, both the token mixer and the channel mixer use a residual structure, the DW layer and the SE module in the MobileNet-V1 model are moved upwards, the token mixer is a depth separable convolution, the MobileNet-V1 model after the DW layer and the SE module are moved upwards is introduced into the batch normalization reparameterization skip connection, the batch normalization and the convolution are combined into one operation, and a deep network module is formed.

3. The method of claim 1, wherein, The vertical direction position-aware convolution is GCPA-V, and the horizontal direction position-aware convolution is GCPA-H. The receptive fields of the two kinds of position-aware convolution cover all the pixels in the same column and the same row of the input features, respectively. When the feature extraction direction is the vertical direction, GCPA-V is used for convolution. When the feature extraction direction is the horizontal direction, GCPA-H is used for convolution. GCPA-H and GCPA-V are processed in parallel to output the features extracted in the two directions. The features output by GCPA-H and GCPA-V are spliced and fused to generate a feature representation containing global information, and then global information is extracted from the input data.

4. The method of claim 3, wherein, When the feature extraction direction is the vertical direction, GCPA-V is used for convolution. GCPA-V adjusts the size of the position embedding according to the size of the input feature through a bilinear interpolation function to generate a series of position encoding vectors to inject position information for each pixel. The position encoding vector generation formula is as follows: wherein, is a position encoding vector, is a bilinear interpolation function, is an initial base position embedding vector, is a number of pixels in the vertical direction, denotes transposing a row vector to a column vector.

5. The method of claim 4, wherein, The generated position encoding vector is copied Next, the copied position encoding vector is spliced in the horizontal direction to obtain a position embedding matrix with a size of by using an expansion function in the vertical direction, and the expansion function in the vertical direction is as follows: wherein, is the extended position embedding matrix, is the vertical direction extension function.

6. The method of claim 5, wherein, GCPA-V combines the input features of the deep network module with the above-mentioned expanded position embedding matrix to form position-aware input features. The position-aware input feature generation formula is as follows: wherein, is a location-aware input feature, is an input feature for a deep network module.

7. The method of claim 6, wherein, Based on the above-mentioned generated position-aware input features, the convolution kernel of GCPA-V uses cyclic convolution in the vertical spatial dimension to output fused vertical direction global information features. The calculation formula of the fused vertical direction global information features is as follows: wherein, is a fused vertical global information feature, is a cyclic convolution offset, is a vertical direction and cyclic convolution offset corresponding weights, is a cyclic index, is a vertical coordinate of a position in the output feature map, is a vertical dimension size of the input feature map, is a horizontal coordinate of a position in the feature map.

8. The method of claim 7, wherein, The vertical direction and the cyclic convolution offset Corresponding weights The weight matrix of all heights in the vertical direction and the cyclic convolution offset is generated by bilinear interpolation, and the generation formula is: Wherein, is an initial basic weight vector, is a bilinear interpolation function, is the value range of the vertical direction and the cyclic convolution offset .

9. The method of claim 1, wherein, In the deep network module, the ECA channel-level attention mechanism is introduced in the feedforward neural network part of the channel mixer. The spatial features are extracted by global pooling for each channel of the input features of the deep network module. The spatial features extracted from each channel are compressed into a scalar. Then, the global features of each channel are mapped to the interval [0, 1] through the Sigmoid activation function to generate the attention weight of each channel. The size of the convolution kernel is dynamically selected, the scale of the convolution kernel is adjusted according to the number of channels, and finally the generated attention weight is applied to the original input feature map channel by channel to adjust the feature importance of each channel and generate a weighted output feature map.

10. A system for optimizing a lightweight model fusing recurrent convolution and reparameterization, based on the method for optimizing a lightweight model fusing recurrent convolution and reparameterization in any of claims 1 to 9, characterized in that, The base architecture construction unit, the shallow network module construction unit, the deep network module construction unit and the lightweight network model construction unit are included. The base architecture construction unit is configured to use a token mixer and a channel mixer of a Metaformer framework as a network base architecture. The shallow network module construction unit is configured to, based on a MobileNet-V1 model, move a DW layer and an SE module of the MobileNet-V1 model upward in the token mixer of the Metaformer framework, and introduce a reparameterizable skip connection of batch normalization into the MobileNet-V1 model to construct a shallow network module. The deep network module construction unit is configured to use a global cyclic position-aware convolution to perform position-aware convolution in two directions of a vertical direction and a horizontal direction in the channel mixer of the Metaformer framework to construct a deep network module. The lightweight network model construction unit is configured to stack a plurality of shallow network modules and deep network modules to construct a lightweight network model, and use the lightweight network model to perform lightweight image data classification processing.

Images