Dynamic weight compression method and device based on cross-modal transfer learning

Abstract

The embodiment of the application provides a dynamic weight compression method and device based on cross-modal transfer learning, relates to the technical field of cross-modal transfer learning and neural network structure mapping, and comprises the following steps: extracting a network structure and pre-training weight parameters from a two-dimensional convolutional neural network model, and constructing a two-dimensional model representation of structure and weight pairing by using the network structure and the pre-training weight parameters; performing static aggregation along a height dimension on the two-dimensional convolution kernel with weight information, introducing a learnable dynamic weight factor for weighting compression to obtain a one-dimensional convolution kernel with two-dimensional weight information; rewriting a weight-free structure layer according to a one-dimensional mapping rule and fusing the one-dimensional convolution kernel structure level to form a one-dimensional neural network model with two-dimensional weight information; and performing adaptive reconstruction on a target one-dimensional task by taking a main task loss, sparsity regularization acting on the dynamic weight factor and channel diversity regularization as a joint training target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cross-modal transfer learning and neural network structure mapping, and particularly relates to a dynamic weight compression method and device for transferring a two-dimensional convolutional neural network with rich pre-training information to a one-dimensional time series task. BACKGROUND

[0002] With the development of cross-modal transfer learning, more and more researches attempt to transfer the pre-trained two-dimensional model in computer vision to one-dimensional sensor tasks to alleviate the problem of insufficient labeled data. However, the existing methods face three challenges: first, the sensor task lacks a unified pre-training benchmark similar to ImageNet, and its data characteristics (such as multi-axis coupling and time series continuity) are obviously different from image semantics, which limits the transfer effect; second, the key parameters (such as compression rate and retention threshold) in the current transfer process mostly rely on manual setting, which is difficult to adapt to different sensor types and application scenarios, affecting the generalization; the "two-dimensional" processing (such as constructing time-frequency graph) for compatibility with two-dimensional models may destroy the original structure, introduce irrelevant image features, increase the risk of overfitting, and limit the further improvement of model performance; finally, the existing compression routes mostly face the volume reduction of the same modality, such as weight quantization and retraining with replacement of activation function to compensate for accuracy, or performing pruning, bit-plane index quantization or parallel decompression coding process on weights, the above technical routes do not involve mapping two-dimensional convolutional kernel structure to one-dimensional convolutional kernel, nor do they provide a data-dependent dynamic weighting mechanism for target one-dimensional data, nor do they provide a unified framework for time dimension rule mapping and structure-level fusion of weightless structure layers (pooling, normalization, full connection, Dropout), thus it is difficult to maintain structure deployability and performance stability in cross-modal transfer scenarios. SUMMARY

[0003] Therefore, the embodiments of the present application provide a dynamic weight compression method and device based on cross-modal transfer learning, which performs "static aggregation to dynamic weighting one-dimensionalization" on the weight structure, and performs time dimension rule mapping and structure-level fusion on the weightless structure layer, to solve the technical defects of the neural network model in the prior art for transferring two-dimensional pre-training information to a one-dimensional time series perception task in terms of structure adaptation, compression flexibility and weight redundancy, and improve deployability.

[0004] According to a first aspect of the embodiments of the present application, a dynamic weight compression method based on cross-modal transfer learning is provided, comprising:

[0005] extracting a network structure and pre-training weight parameters from a two-dimensional convolutional neural network model, and using the network structure and the pre-training weight parameters to construct a two-dimensional model representation of structure and weight pairing;

[0006] The two-dimensional model representation with weight parameters is executed static aggregation along the spatial height dimension to obtain intermediate weights, and a learnable dynamic weight factor is constructed based on the statistics of the target one-dimensional data, and the weights are dynamically weighted and compressed to generate a one-dimensional convolution kernel with weights.

[0007] A plurality of regularization terms are introduced for joint optimization. The main task loss and the plurality of regularization terms are jointly trained to construct the final training target of the one-dimensional neural network model, so as to realize effective cross-modal transfer and structural adaptive reconstruction in the training phase of the one-dimensional neural network model with two-dimensional pre-training information.

[0008] Preferably, the initialization two-dimensional model representation constructed by using the network structure and the pre-training weight parameters includes:

[0009] The kernel tensor of each convolution layer is one-to-one bound to the hierarchical and connection relationship in the network topology, so that the subsequent static aggregation and rule mapping are executed on the hierarchical pair.

[0010] Preferably, the one-dimensional neural network with two-dimensional pre-training information is obtained by executing static aggregation along the spatial height dimension of the two-dimensional convolution kernel structure layer in the two-dimensional model representation with weight parameters to obtain intermediate weights, constructing a learnable dynamic weight factor based on the statistics of the target one-dimensional data, and dynamically weighting and compressing the weights to generate a one-dimensional convolution kernel with weights, rewriting the weightless structure layer in the two-dimensional model representation into a one-dimensional structure according to a predefined one-dimensional mapping rule, and combining the one-dimensional convolution kernel with weights.

[0011] The dynamic weight factor is introduced , and the dynamic weight factor is used to perform static aggregation and dynamic factor weighting compression on the two-dimensional convolution kernel structure layer with weight parameters in the initialization two-dimensional model representation to obtain a one-dimensional convolution kernel model with weight parameters;

[0012] The weightless layer in the initialization two-dimensional model representation is regularized according to the kernel length, step length and statistical dimension of the one-dimensional time dimension, and is structurally fused with the one-dimensional convolution kernel to obtain a one-dimensional structure layer model without weight parameters;

[0013] The one-dimensional neural network model is obtained by fusing the one-dimensional convolution kernel model with weight parameters and the one-dimensional structure layer model without weight parameters.

[0014] Preferably, the step of mapping the unweighted parameter-free structural layer in the initial two-dimensional model representation according to structural rules to obtain an unweighted parameter-free one-dimensional structural layer model includes:

[0015] By mapping the pooling layer, normalization layer, Dropout layer, and fully connected layer in the unweighted parameter structure layer of the initial two-dimensional model representation according to the time dimension rule, a one-dimensional structure layer without weight parameters is obtained.

[0016] Preferably, the final training objective of introducing multiple regularization terms and jointly modeling the main task loss with the multiple regularization terms to construct the one-dimensional neural network model includes:

[0017] For dynamic weighting factors To enhance expressive complementarity, a sparsity regularization loss term is introduced to suppress redundant features from participating in the representation and a channel diversity regularization loss term is introduced to improve overall expressive power.

[0018] The main task loss is jointly modeled with the sparsity regularization loss and the channel diversity regularization loss to construct the final training objective of the one-dimensional neural network model used to guide model training.

[0019] Preferably, the final training objective of jointly modeling the main task loss with the sparsity regularization loss and the channel diversity regularization loss to construct the one-dimensional neural network model for guiding model training includes:

[0020] ;

[0021] Among them, the This refers to the final training objective; the stated This refers to the loss of the main task; the aforementioned This refers to the loss of the sparsity regularization term; the aforementioned This refers to the loss of the channel diversity regularization term.

[0022] According to a second aspect of the embodiments of this application, a dynamic weight compression device based on cross-modal transfer learning is provided, comprising:

[0023] The two-dimensional representation construction module is configured to obtain the network structure and its pre-trained weight parameters from the two-dimensional convolutional neural network model, and use the network structure and the pre-trained weight parameters to construct an initial two-dimensional representation with structure and weight pairing;

[0024] a one-dimensional kernel generation module configured to perform static aggregation on a two-dimensional convolution kernel structure layer with weight parameters in the two-dimensional model representation along a spatial height dimension to obtain intermediate weights, construct a learnable dynamic weight factor based on a statistic of target one-dimensional data, and perform dynamic weight compression on the weights to generate a one-dimensional convolution kernel with weights, rewrite a structure layer without weights in the two-dimensional model representation into a one-dimensional structure according to a predefined one-dimensional mapping rule, and obtain a one-dimensional neural network with two-dimensional pre-training information by combining the one-dimensional structure with the one-dimensional convolution kernel with weights;

[0025] a rule mapping and structure fusion module configured to introduce multiple regularization terms for joint optimization, construct a final training target of the one-dimensional neural network model by jointly training a main task loss and the multiple regularization terms, and implement effective cross-modal transfer and structure adaptive reconstruction in a training phase of the one-dimensional neural network model with two-dimensional pre-training information.

[0026] According to a third aspect of the embodiments of the present application, a computing device is provided, including:

[0027] a memory and a processor;

[0028] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the steps of any one of the dynamic weight compression methods based on cross-modal transfer learning.

[0029] According to a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores computer executable instructions, and the instructions are executed by a processor to implement the steps of any one of the dynamic weight compression methods based on cross-modal transfer learning.

[0030] The dynamic weight compression method based on cross-modal transfer learning provided by the embodiments of the present application extracts network structure and pre-training weight parameters from a two-dimensional convolutional neural network model, and constructs an initialized two-dimensional model representation of structure and weight pairing by using the network structure and the pre-training weight parameters. The initialized two-dimensional model representation is processed by dimension reduction to obtain a one-dimensional neural network model. Multiple regularization terms are introduced, and a final training target of the one-dimensional neural network model is constructed by jointly modeling a main task loss and the multiple regularization terms, so as to implement effective transfer and structure adaptive reconstruction in a training phase of the one-dimensional neural network model. The dynamic weight compression method based on cross-modal transfer learning provided by the embodiments of the present application introduces a learnable parameter-based The weight adjustment mechanism realizes dynamic weighted compression of the spatial dimension in the convolution kernel by constructing a structure-pluggable adaptive compression module without relying on manual rules or fixed compression rates, thereby improving the performance, flexibility and deployability of the cross-modal model conversion. Secondly, the structure of any pre-trained two-dimensional convolution model can be deconstructed and compressed, and the weight adjustment mechanism is introduced in the training stage The weight adjustment mechanism is jointly optimized with the main task, thereby compressing redundant structures while maintaining key feature expression, and significantly improving the performance and memory utilization in the one-dimensional sensor task. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flowchart of a dynamic weight compression method based on cross-modal transfer learning provided by an embodiment of the present application;

[0032] Figure 2 is a flowchart of a dynamic weight compression method based on cross-modal transfer learning provided by an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of a dynamic weight compression device based on cross-modal transfer learning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced in a variety of ways beyond the specific embodiments described herein without departing from the scope of the present application, and it is understood that similar modifications can be made by one skilled in the art without departing from the spirit of the present application, and therefore the present application is not limited to the specific implementation disclosed below.

[0035] The embodiment of the application provides a dynamic weight compression method and device based on cross-modal transfer learning, which comprises the following steps.

[0036] The embodiment of the application does not take weight quantization, activation function replacement, pruning index or compression as necessary steps. A plurality of regular terms are introduced, and the final training target of the one-dimensional neural network model is constructed by jointly modeling the main task loss and the plurality of regular terms, so that effective migration and structure adaptive reconstruction are realized in the one-dimensional neural network model training stage.

[0037] Figure 1 A flowchart of a dynamic weight compression method based on cross-modal transfer learning is shown, which comprises the following steps.

[0038] Step S101: extracting network layer structure and corresponding pre-training weight parameters from a pre-training two-dimensional convolutional neural network model as unified input for subsequent one-dimensionalization and rule mapping;

[0039] In an embodiment of the application, the construction of the structure and weight paired initialization two-dimensional model representation using the network structure and the pre-training weight parameters comprises:

[0040] The network structure and the corresponding pre-training weight parameters are bound one by one according to the layer level and connection relationship of the original network to form the structure and weight paired initialization two-dimensional representation.

[0041] Step S102: obtaining a two-dimensional pre-training information one-dimensional neural network by performing static aggregation on the two-dimensional convolution kernel structure layer with weight parameters in the two-dimensional model representation along the spatial height dimension to obtain intermediate weights, constructing a learnable dynamic weight factor based on the statistics of the target one-dimensional data, and performing dynamic weight compression on the weights to generate a one-dimensional convolution kernel with weights, rewriting the structure layer without weights in the two-dimensional model representation into a one-dimensional structure according to a predefined one-dimensional mapping rule, and combining the one-dimensional convolution kernel with weights to obtain the two-dimensional pre-training information one-dimensional neural network;

[0042] In an embodiment of the present application, the one-dimensional neural network model obtained by performing one-dimensional processing on the two-dimensional convolution kernel structure layer with weight parameters and the structure layer without weight parameters in the initialized two-dimensional model representation includes:

[0043] introducing a dynamic weight factor and using the dynamic weight factor performing static aggregation on the two-dimensional convolution kernel structure layer with weight parameters in the initialized two-dimensional model representation along the height dimension to obtain intermediate weights, then combining a learnable channel-level dynamic weight factor to complete channel weighting and one-dimensional processing, obtaining a one-dimensional convolution layer that can be directly deployed, and obtaining a one-dimensional convolution kernel model with weight parameters;

[0044] obtaining a one-dimensional structure layer model without weight parameters by mapping (pooling layer processing time window and stride; normalization layer accumulating along the time dimension according to statistics; fully connected layer mapping one-dimensional convolution with a kernel length of 1; Dropout layer independently sampling along the time dimension) the structure layer without weight parameters in the initialized two-dimensional model representation along the time dimension;

[0045] obtaining a one-dimensional neural network model by performing structure-level fusion processing on the one-dimensional convolution layer with weight parameters and the one-dimensional structure layer model without weight parameters.

[0046] Step S103: introducing multiple regularization terms for joint optimization, constructing the final training target of the one-dimensional neural network model by jointly training the main task loss and the multiple regularization terms, so as to realize effective cross-modal transfer and structure adaptive reconstruction in the training phase of the two-dimensional pre-training information one-dimensional neural network model.

[0047] In an embodiment of the present application, the dynamic weight factor is introduced, and the dynamic weight is used to perform weight compression processing on the two-dimensional convolution kernel structure layer with weight parameters in the initialized two-dimensional model representation to obtain a one-dimensional convolution kernel model with weight parameters, which includes:

[0048] The static compression aggregation weight is obtained by performing static compression processing on each convolution kernel in a height direction of a structure layer of a two-dimensional convolution kernel with a weight parameter in the initialized two-dimensional model.

[0049] A dynamic weight factor is introduced and the dynamic weight is utilized The static compression aggregation weight is subjected to weighted compression processing to obtain a one-dimensional convolution kernel model with a weight parameter.

[0050] In an embodiment of the present application, the mapping processing of the structure layer without a weight parameter in the initialized two-dimensional model representation according to a structure rule to obtain a one-dimensional structure layer model without a weight parameter comprises:

[0051] The mapping processing of the structure layer without a weight parameter in the initialized two-dimensional model representation according to a structure rule to obtain a one-dimensional structure layer model without a weight parameter comprises:

[0052] In an embodiment of the present application, the introduction of the plurality of regular terms and the joint modeling of the main task loss and the plurality of regular terms to construct the final training target of the one-dimensional neural network model comprises:

[0053] The dynamic weight factor is introduced The sparse regular term loss for suppressing redundant features from participating in representation and the channel diversity regular term loss for improving overall expression capability are introduced;

[0054] The joint modeling of the main task loss and the sparse regular term loss and the channel diversity regular term loss to construct the final training target of the one-dimensional neural network model for guiding model training.

[0055] In an embodiment of the present application, the joint modeling of the main task loss and the sparse regular term loss and the channel diversity regular term loss to construct the final training target of the one-dimensional neural network model for guiding model training comprises:

[0056] ;

[0057] wherein, the final training target is represented by L; the main task loss is represented by Lmain; the sparse regular term loss is represented by Lsparse; and the channel diversity regular term loss is represented by Lchannel. is referred to as the final training target; the is referred to as the main task loss; the is referred to as the sparse regular term loss; and the is referred to as the channel diversity regular term loss.

[0058] The dynamic weight compression scheme based on cross-modal transfer learning provided by the embodiments of the present application is suitable for a neural network model for migrating a two-dimensional convolutional neural network model to a one-dimensional time sequence perception task. In view of the problems of the prior art in structure adaptation, compression flexibility and weight redundancy, a weight adjustment mechanism based on learnable parameters is introduced . By constructing a structure-pluggable adaptive compression module, dynamic weighting compression of the spatial dimension in the convolution kernel is realized without relying on manual rules or fixed compression rates, and the performance, flexibility and deployability of cross-modal model conversion are improved. The method supports structure deconstruction and channel compression of any pre-trained two-dimensional convolutional model, and the weight adjustment mechanism is introduced in the training stage to jointly optimize the main task, so that the key feature expression is maintained while the redundant structure is compressed, and the performance and memory utilization in the one-dimensional sensor task are significantly improved.

[0059] In the embodiments, a model using a one-dimensional convolutional layer in a neural network is referred to as a one-dimensional model, and a task corresponding to the one-dimensional model is referred to as a one-dimensional task (time sequence classification, etc.). Since the prior art lacks rich data for one-dimensional tasks as much as two-dimensional tasks (image recognition), it is difficult for a one-dimensional model to have pre-trained weights with strong robustness as a two-dimensional model. The traditional method usually converts one-dimensional signal data into a frequency spectrum image, and then uses a two-dimensional model with pre-trained weights for recognition. There are problems of unlearnable information conversion, mismatched data structure and model architecture, and limited task transfer capability. The present method realizes dynamic compression of a two-dimensional convolution kernel through learnable weights, maintains the expression capability of the pre-trained model, realizes structure lightweight and task adaptability, and avoids the information loss and structure mismatch problems existing in the traditional frequency spectrum conversion method. Based on this, the embodiments of the present application provide a model training and inference method based on learnable compressed weights, as shown in Figure 2 , which includes

[0060] S10, two-dimensional model structure and pre-trained weight extraction;

[0061] The core goal of the operation is to extract the detailed structure definition and pre-trained weight parameters of a standard two-dimensional convolutional neural network (2D-CNN) from the two-dimensional convolutional neural network, to prepare for subsequent conversion to a one-dimensional structure and reuse of learned features. That is, the network structure and pre-trained weight parameters are extracted from an existing two-dimensional neural network to provide a basis for subsequent structure mapping and knowledge transfer, and to realize the reuse of high-level semantic features learned in the visual model. Specifically, the following is included:

[0062] S11: calling a two-dimensional convolutional neural network model with a general structure from a public model library; ​

[0063] A 2D-CNN model with general-purpose structure is called from a public model library (e.g., torchvision of PyTorch). This kind of model is composed of convolutional layers, activation functions, normalization layers, etc., and is widely used in two-dimensional tasks such as image processing. However, because its structure is based on modeling spatial features, it cannot be directly applied to one-dimensional time series or sensor data processing. That is, a 2D-CNN architecture (such as VGG, ResNet, MobileNet, etc.) with a pre-defined and implemented model architecture and parameters is loaded from a widely used deep learning library (such as torchvision.models in PyTorch). These 2D-CNN models are designed initially for processing image data and have shown excellent versatility in various image recognition tasks. These models are usually trained on a large and diverse image dataset such as ImageNet, which contains millions of pictures and thousands of categories. This training process enables the model to learn low-level features (such as edges, textures) and higher-level semantic features that are prevalent in images.

[0064] As a model called from the torchvision.models library, it itself has a general-purpose structure, that is, a two-dimensional convolutional neural network model is composed of convolutional layers (Conv2d), activation functions (such as ReLU), normalization layers (nn.BatchNorm2d), and pooling layers, etc. Standard components.

[0065] S12: Load the pre-trained weight parameters corresponding to the called model;

[0066] Load the pre-trained weights that match the structure of the called model, which are obtained by training the model structure on public image datasets (such as ImageNet), including convolution kernel parameters, bias terms, and normalization parameters, used to initialize the layers of the model.

[0067] After instantiating the 2D model structure, the purpose of this step is to load the weights that have been trained on a large-scale image dataset (e.g., ImageNet) that matches the structure. After instantiating the 2D CNN model structure, the configuration (model.weights) is used to decide whether to load the pre-trained weights on a large-scale dataset (e.g., ImageNet). When model.weights is specified as a specific identifier such as "IMAGENET1K_V1", the model automatically loads these pre-trained parameters, including convolution kernels, biases, and batch normalization layer statistics, which embody the general visual knowledge learned by the model from massive image data. If model.weights is set to None, the model will use the default weights initialized randomly. In short, this step aims to lay the foundation for subsequent training by selectively loading pre-trained weights, taking advantage of their powerful feature extraction capabilities learned on ImageNet.

[0068] S13: Construct an initialized model representation of structure and weight pairing;

[0069] Align and bind the invoked two-dimensional model structure with its corresponding pre-trained weights to form a structure and parameter consistent initialized model representation, providing a unified input for subsequent one-dimensional model construction and compression operations.

[0070] This step aims to construct a 2D model object in memory with completed structure and weight pairing, which will serve as the "source" for subsequent 1D conversion. The result of setup_model() execution is a torch.nn.module instance containing the complete 2D network structure and its pre-trained (or randomly initialized) weights.

[0071] Subsequently, this model object is passed to the model conversion function. Inside the conversion function, the 2D model submodules are accessed by iterating through them, and the module type (such as convolution layers, pooling layers, etc.) is determined using isinstance(). The structure information of the model is extracted. For specific layers, such as nn.Conv2d (two-dimensional convolution layer), the defined parameters (such as stride, padding, kernel_size) are further accessed. At the same time, the weight parameters are directly extracted from the 2D layer module.

[0072] Through the operations of S11 to S13, a 2D-CNN with a specific structure is instantiated from the model library, its pre-trained weights are loaded, and the complete representation of this model in memory is obtained. Subsequently, in the conversion process, the detailed structure information (layer type, layer parameter) and numerical weights of the model are accessed and extracted layer by layer, laying the foundation for constructing a new one-dimensional model structure and transferring the knowledge learned across modalities.

[0073] S20, Dynamic Weight Compression and One-Dimensional Convolutional Kernel Transformation;

[0074] By introducing learnable dynamic weights We perform weighted compression on the original two-dimensional convolution kernel to achieve dimensionality reduction mapping and construct a structurally equivalent one-dimensional convolution model, thus introducing high-quality representation capabilities for one-dimensional tasks.

[0075] The dynamic weight compression and one-dimensional convolution kernel conversion process aims to transform an existing two-dimensional convolution operation (including its structure definition and weights) into an equivalent and optimized one-dimensional convolution operation, enabling it to effectively process one-dimensional sequence data. Step S20 includes:

[0076] S21: Map the two-dimensional convolutional kernel structure to a one-dimensional convolutional form;

[0077] The two-dimensional convolutional layers in the model structure constructed by S10 undergo structure mapping processing. While maintaining the consistency of input / output channels and horizontal stride, the width dimension is preserved and the height dimension is compressed. Specifically, for the convolutional kernel shape ( A two-dimensional convolutional layer that retains the width dimension while keeping the number of input and output channels constant. and will be high Compressed to 1, the resulting shape is ( The transformation process uses a one-dimensional convolutional kernel. This transformation process semantically preserves the horizontal feature extraction capability of the original convolutional kernel, while enabling the network structure to adapt to one-dimensional input signals, such as time series or sensor data.

[0078] This initial step defines the basic structure of the one-dimensional convolutional layer. Instead of directly modifying the weights, it constructs a new one-dimensional convolutional layer with the same structure based on the two-dimensional model called from torchvision.models. A new, empty one-dimensional convolutional layer (containing only initialization parameters) is planned and instantiated based on the parameters of the original two-dimensional convolutional layer.

[0079] In the process of converting a 2D convolutional layer into a 1D convolutional layer, the first step is to determine the number of channels in the 1D convolutional layer. Based on the number of input and output channels of the original 2D convolutional layer, you can determine the number of input channels the new 1D convolutional layer will receive and the number of output channels it will produce. The number of channels in the new layer can be less than or equal to the number of channels in the original layer. This is achieved in PyTorch by defining the input and output channel parameters of `torch.nn.Conv1d`. Secondly, determine the width of the 1D convolutional kernel. Following step S21, the width dimension of the original 2D convolutional kernel is... The length of the new one-dimensional convolution kernel, i.e., the kernel size (kernel_size), is set to... . Then, other convolution parameters such as stride and padding of the one-dimensional convolution layer in the width direction will inherit the corresponding values of the original two-dimensional convolution layer in the width direction, respectively, and set the stride and padding parameters of torch.nn.Conv1d accordingly. After determining the parameters, the one-dimensional convolution layer can be instantiated, for example, new_conv1d = torch.nn.Conv1d(in_channels=input channel number, out_channels=output channel number, , stride=stride of the original 2D layer in the width direction, padding=padding of the original 2D layer in the width direction). At this time, the weights and biases of the new layer are still randomly initialized by default of the framework, which are to be replaced by the weights converted from the two-dimensional kernel in steps S22 and S23. For example, if the original 2D convolution layer is Conv2d(in_channels, out_channels, kernel_size=(5, 5), stride=(2, 2), padding=(3, 3)), the instantiated one-dimensional convolution layer parameters are Conv1d(in_channels, out_channels, kernel_size=5, stride=2, padding=3).

[0080] S22: Static aggregation of two-dimensional convolution kernel weights;

[0081] To realize the conversion of two-dimensional convolution kernel to one-dimensional structure, first, each convolution kernel is statically compressed along the height dimension. Specifically, for the original convolution kernel , the sum operation is performed along the height dimension , where is the two-dimensional convolution kernel weight, represents the number of output channels of the convolution kernel, represents the number of input channels, and are the dimensions of the convolution kernel weight in the height and width directions, respectively. The generated one-dimensional convolution kernel weight is , and its calculation method is:

[0082] ;

[0083] represents the static aggregation weight result of the compressed one-dimensional intermediate convolution kernel in the th column, represents the th row and the th column in the two-dimensional convolution kernel, the output channel and the input channel The weights at corresponding positions. This step preserves the structural information of the convolution kernel in the width direction, while effectively mapping the two-dimensional structure into a representation suitable for one-dimensional input tasks (such as time series or sensor data), providing a foundation for the subsequent dynamic adjustment stage S23.

[0084] In other words, the goal of step S22 is to transform the actual weight values ​​of the original two-dimensional convolutional kernel so that its dimension and shape match the target one-dimensional convolutional layer defined in S21, thereby adjusting the channel dimension.

[0085] The original two-dimensional convolution kernel weights are The shape is ( The weight shape of the target one-dimensional convolution kernel should be (). , ).

[0086] At this stage, the goal is to transform the spatial structure of the original two-dimensional convolutional kernel (typically...) For example, the kernel size is ) compressed into a one-dimensional structure (such as This is done to adapt to the form of a one-dimensional convolutional kernel. Specifically, a aggregation operation is performed on the height direction of the two-dimensional convolutional kernel, typically using summation or averaging, to preserve the feature structure in the width direction and remove the height dimension. Assume an original two-dimensional convolutional kernel is... The matrix contains the weights of the convolution kernel.

[0087] ;

[0088] To convert it into a one-dimensional convolution kernel (i.e., 1 × 3), we perform a summation or average operation on each column, that is, aggregate along the height direction, resulting in:

[0089] Compressed 2D weights = ;

[0090] Such a primitive The convolution kernel is compressed to The one-dimensional convolutional kernel preserves the structural features in the width direction. This compression not only reduces computational cost but also allows for the transfer of model weights from two-dimensional convolutional weights. Static compression to one-dimensional convolution It provides a structural foundation while retaining the lateral sensing capability of the convolution kernel.

[0091] S23: Constructing Learnable Dynamics Weight compression

[0092] Furthermore, learnable channel-level dynamic weighting factors are generated based on the statistics of the target one-dimensional data. , based on the weights obtained in S22, perform position-by-position scaling to construct a set of bases and the width of the convolution kernel The same size weight adjustment factor . For the output channel , input channel , width direction position , the initial value of the weight adjustment factor is defined as:

[0093] ;

[0094] Take the absolute value to represent the response strength of each position weight, without being affected by the positive and negative values in the weight. is a numerical stability term to prevent division by zero. To avoid falling into a parameter symmetric state at the beginning of training, a small amount of Gaussian noise is added to each initial value , enhancing learnability.

[0095] Finally, the weight adjustment factor is applied to the static aggregated weight to generate a one-dimensional convolution kernel with two-dimensional convolution kernel weights, and the final convolution expression for feature extraction is obtained in a position-by-position scaling manner:

[0096] ;

[0097] More specifically, after obtaining the static compressed , further introduce a learnable dynamic weight factor , which is element-by-element fine-tuned to better adapt to specific tasks. Specifically, each element of is equipped with a learnable scaling factor , and through a specific initialization and learning process, the final one-dimensional convolution kernel weight is the scaled product of and .

[0098] S231: Define parameters;

[0099] To achieve dynamic adjustability of one-dimensional convolution weights, a set of learnable scaling parameter tensors with the same shape as the static compressed weight is constructed. This scaling factor parameter will be used to adjust each convolution weight element-by-element after compression, improving the model's ability to adapt to different tasks. Assuming that the static compressed weight corresponding to a certain output channel and input channel is:

[0100] ;

[0101] ​​Assign an independent, learnable scaling factor to each element in the vector. Obtain the dynamic compression weights:

[0102] ;

[0103] Each of them All of these are learnable parameters.

[0104] S232: Initialization parameter;

[0105] Furthermore, in this embodiment, to achieve position-by-position dynamic adjustment of the static aggregated convolution weights, a set of compression weight tensors is introduced. Scaling factor parameter tensors of the same shape The scaling vector is initialized based on the relative magnitude of the intermediate weights and a numerical normalization strategy. This parameter will participate in model training as a learnable parameter in the neural network, and will be given an initial value with numerical distribution guidance during the initialization phase, and then adaptively updated during training.

[0106] In this step The parameters are not initialized randomly or with constant padding. Instead, initial values ​​are generated based on the relative magnitudes of the static convolution weights within the same channel, through numerical normalization and logarithmic mapping, giving them a certain degree of structural sensitivity. This initialization strategy does not rely on external data and is entirely based on existing static weights. Internal weight distribution characteristics. For example, suppose the convolution weights after static compression are:

[0107] ;

[0108] First, calculate the scale value of each position within the channel, then take the logarithm to obtain the initial scaling parameters:

[0109] ;

[0110] in To prevent small constants with a denominator of zero, This initialization strategy can be a constant offset or a slight perturbation to prevent slow convergence in the early stages of training. It does not rely on external input, but is calculated solely based on existing static weight tensors, reflecting the importance of different positions to the overall channel output. This helps the model capture the spatial distribution characteristics of the convolutional kernel more quickly in the early stages of training. After initialization... Registered as network parameters, they are automatically updated with gradients during training, enabling learnable adjustment of each position of the convolutional kernel.

[0111] S233: Based on Generate the final one-dimensional convolution kernel weights ;

[0112] After the initialization of the scaling factor parameter is completed, the parameter is further multiplied by the static compressed convolutional weight to generate a final weight tensor that can be used for one-dimensional convolution;

[0113] Specifically, for a convolutional weight with an output channel , an input channel , and a width direction , the calculation of the final weight is as follows:

[0114] ;

[0115] The operation is an element-wise point multiplication, where is the learnable scaling coefficient at the th position in the width direction; is the initial weight obtained by static compression; and the one-dimensional convolutional weight that can be directly used for convolution operation.

[0116] S30, structure mapping of the weightless layer and construction of the one-dimensional model;

[0117] All weightless modules (such as pooling, normalization, flattening, etc.) in the original two-dimensional model are mapped to one-dimensional form according to the structural rules, ensuring that the overall model still has functional equivalence after dimension transformation, and realizing the construction of a complete one-dimensional neural network. After completing the core convolutional layer weight conversion, the goal of S30 is to map all auxiliary modules (such as pooling layers, normalization layers, Dropout layers, and finally flattening and full connection adaptation) in the original two-dimensional model that are usually considered to have no independent training weight to the corresponding one-dimensional form according to the structural rules. This step is crucial to ensure that the overall model still has basic functional equivalence after dimension transformation, and finally realizes a complete and executable one-dimensional neural network. The one-dimensional structure transformation method of non-parametric layers is as follows:

[0118] S31: Pooling layer conversion;

[0119] Replace the two-dimensional max-pooling and average-pooling operations with corresponding one-dimensional pooling operations. The kernel and stride of two-dimensional pooling are mapped to the time window length and time stride according to the equivalence principle, which completes the down-sampling in the sequence direction and maintains the feature compression capability. During the migration of the two-dimensional model to the one-dimensional model, the original two-dimensional pooling layer (MaxPool2d or AvgPool2d) should be converted to the corresponding one-dimensional pooling layer (such as MaxPool1d or AvgPool1d). The core is to keep the width direction parameter of the original pooling kernel as a one-dimensional parameter, and strictly keep the original kernel_size, stride, padding, dilation, and ceil_mode parameters consistent. For example, if the original structure is MaxPool2d(kernel_size=(3, 3), stride=(2,2), padding=(1, 1), dilation=1, ceil_mode=0), it is converted to MaxPool1d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=0). Only the width direction parameter is retained (the height dimension is ignored), and the remaining parameters are completely reused to ensure that the network compression behavior is consistent after conversion.

[0120] S32: Normalization layer conversion;

[0121] Replace the two-dimensional batch normalization module with a one-dimensional batch normalization form to complete the standardization processing in the channel dimension and maintain the training stability. For BatchNorm normalization operations, replace BatchNorm2d in the two-dimensional model with BatchNorm1d while keeping all normalization parameters completely consistent, including num_features (i.e., the number of channels), eps (numerical stability constant), momentum (momentum coefficient), affine (whether to enable learnable scaling and bias), track_running_stats (whether to track training statistics), etc. For example, if the original normalization layer is BatchNorm2d(num_features=64, eps=1e-5, momentum=0.1, affine=1, track_running_stats=1), then after conversion, BatchNorm1d(num_features=64, eps=1e-5, momentum=0.1, affine=1, track_running_stats=1) should be used to ensure that the numerical behavior of the normalization calculation is completely preserved in the one-dimensional model.

[0122] S33: Dropout layer conversion;

[0123] Convert the spatial Dropout operation to the Dropout in the sequence direction to adapt to the distribution characteristics of one-dimensional input features. For the Dropout layer, the conversion principle is to keep its random dropout behavior completely consistent with the parameters and only adjust the adaptive dimensions. The nn.Dropout2d or nn.Dropout in the two-dimensional model can be uniformly used as nn.Dropout (or nn.Dropout1d, the same behavior) in the one-dimensional model, keeping the dropout probability p and the inplace parameter consistent. For example, if the original layer is nn.Dropout2d(p=0.3, inplace=1), the converted layer is nn.Dropout(p=0.3, inplace=1) to ensure that the dropout rate and the calculation process do not change during training.

[0124] S34: Full connection structure adaptation;

[0125] The input size of the original full connection structure is reconstructed and matched so that it can directly accept the one-dimensional sequence representation after the one-dimensional pooling and flattening. For the linear layer (i.e., Linear), the key is to flatten the input tensor according to the equivalent logic and keep in_features and out_features completely consistent. If the output dimension before flattening in the original two-dimensional structure is ( ), and the linear layer structure is Linear(in_features=C×H×W, out_features=K), in the one-dimensional model, the equivalent one-dimensional flattening method should be used to convert (B, C, L) to (B, C×L) so that the input feature number is still C×L=C×H×W, and then the Linear(in_features=C×L, out_features=K) structure remains unchanged. For example, if the original is Linear(512, 10), as long as it is flattened to 512, it can be directly reused.

[0126] Through these detailed structure mapping steps S31 to S34, various auxiliary operation modules in the original two-dimensional model are smoothly converted into their functional counterparts in the one-dimensional field. This ensures that the entire data processing flow from input to output is still complete and logically consistent after dimension conversion, and finally builds a functional one-dimensional neural network model.

[0127] S40, regular driven structure adaptation and knowledge transfer optimization mechanism.

[0128] In the model training stage, in order to realize the effective migration and structural adaptive reconstruction of two-dimensional pre-training knowledge in one-dimensional tasks, a dynamic optimization mechanism driven by regularization is introduced. By dynamically introducing multiple structural constraint regularization terms in the forward propagation or backward propagation process, the structural indicators such as feature sparsity and channel diversity are jointly optimized, and the model weights are continuously compressed and adjusted without changing the network topology, so as to complete the adaptive reconstruction of the parameter layer, improve the model stability, generalization ability and migration performance on the premise of maintaining the expression ability.

[0129] After the structure construction and weight (including learnable dynamic parameters) setting of the one-dimensional model are completed in the S10-S30 stage, the core goal of the S40 stage is to jointly optimize the model (especially the learnable α weight) by introducing a specific regularization mechanism and combining the main task loss during the actual training iteration of the model. This is to promote the more effective migration of two-dimensional pre-training knowledge to one-dimensional tasks without changing the structure of the constructed one-dimensional model, and to realize the adaptive structural adjustment and reconstruction of the model parameter layer.

[0130] Further, after compressing the two-dimensional neural network structure and weight into the one-dimensional model, multiple regularization constraints are introduced in the training process of the dynamic adjustment mechanism, and are jointly optimized with the main task target.

[0131] S41: Regularization constraint mechanism;

[0132] In order to improve the stability and expression controllability of the compressed model structure, and guide the weight learning to a more beneficial mode, multiple regularization terms for the weight are introduced as auxiliary loss functions and main tasks to jointly optimize the dynamic weight factor. The sparsity constraint is added to suppress redundancy, and the diversity constraint is added to reduce correlation. The weight is a tunable hyperparameter.

[0133] The characteristics of the weight are constrained by adding an additional loss term, such as encouraging sparsity or diversity. In the deep learning operation process, a sparsity regularization term ( ) is introduced to encourage the weight to be sparse, that is, to let the model learn to identify and reduce the influence of those original two-dimensional features (aggregated by the static aggregation in S22, and then adjusted by the adjustment) that contribute little to the current one-dimensional task, so as to suppress the expression of redundant parameters or features. This loss term is usually based on the sum of the absolute values of the weight, and the formula is expressed as follows:

[0134] ;

[0135] where is the weight hyper-parameter of this regularizer, denotes the summation of all weight elements.

[0136] Further, a channel diversity regularizer ( ) is introduced to encourage the difference of weights among different channels, so as to improve the overall feature expression ability of the model and avoid learning too similar compression patterns by different channels. By increasing this item loss, it can encourage the corresponding to different channels to learn inconsistent weight patterns, prevent the compression structure from being too single, and thus improve the adaptability of the model to multi-scale features under complex tasks. The formula is expressed as follows:

[0137] ;

[0138] where is the weight hyper-parameter, denotes the variance of weights across channels, denotes the number of input channels, denotes the statistical variance function, which calculates the statistical variance of the average value vector of multiple input channels, used to measure whether the distribution of these channels is consistent. By collecting all related parameters, calculating their certain statistical characteristics (for example, the average value of weights corresponding to each output channel) in the channel dimension, and then calculating the variance of these statistical characteristics, multiplied by the weight . If the model learns that some channels should be preserved and some channels should be suppressed, the distribution appears obvious difference, the variance becomes larger, the value is higher.

[0139] S42: Total loss function construction;

[0140] Further, the loss of the model on the main task is combined with each item regularization loss defined in S41 to construct the final overall optimization target for guiding the model training. A composite loss function is created, which not only focuses on the performance of the model on the prediction task, but also optimizes the structural characteristics of the α weight. Calculate the main task loss ( ), for the current batch of input data of the model, perform complete forward propagation (including the process of dynamically generating one-dimensional convolution kernel in S20), get the prediction output, and calculate the main loss between the task type (such as classification, regression, etc.) and the real label.

[0141] According to the method described in S41, calculate​ and Then, the main task loss is weighted and summed with all regularization loss terms, that is, the main task loss is summed with... The regularization terms are jointly modeled to construct the final training objective:

[0142] ;

[0143] in, , These are the weighting factors for each regularization term, which are adjusted according to the task scenario.

[0144] In the deep learning framework, in each training step, the following is calculated separately: , and These are several scalar values, and then they are added together to get... .

[0145] S43: Model training process;

[0146] After completing the model structure compression and dynamic adjustment mechanism construction, a standard end-to-end training process was used for overall optimization. First, the one-dimensional model structure and... Weight parameters are defined; subsequently, in each training round, forward propagation is performed to dynamically generate one-dimensional convolutional kernels, complete feature extraction, and calculate the main task loss and various regularization terms; then, backpropagation is used for joint optimization. The convolutional kernel weights are adjusted until the overall model achieves an optimal balance between task performance and structural compression. This training process is highly compatible with traditional deep model training, requires no additional structural modifications, and has good feasibility and scalability.

[0147] In other words, after completing the model structure construction (S10–S30) and defining the total loss function (S42), a standard end-to-end training process is used to train the entire one-dimensional model (including its components). The weight parameters are optimized. The training process includes structure initialization and parameter preparation, training data iteration, loss function construction, backpropagation and parameter update. First, static aggregated weights are completed according to steps S21 to S23. The construction of the shape and initialization of the scaling factor consistent with its shape. This serves as a learnable parameter in the model for generating dynamic convolutional kernels. Subsequently, the training dataset undergoes multiple epochs. In each epoch, a forward propagation is performed on each batch of data, feeding the input data into the constructed one-dimensional model structure, dynamically... Under the control mechanism, based on the current Values ​​and static compression weights The final one-dimensional convolution kernel weights are calculated. , for convolution operation and feature extraction. Then, the main task loss between the task output and the real label is calculated , and the regularization constraint loss is calculated synchronously, including the sparsity regularization term and the channel diversity regularization term , to build the total loss function by weighted combination . According to the total loss, the error back propagation operation is performed to calculate the gradient of and other network parameters, and the parameters are updated using the optimizer (such as SGD, Adam). After each iteration, according to the performance of the model on the validation set or the convergence characteristics of the structure distribution, it is judged whether the preset termination condition is met, and finally the one-dimensional convolution model is obtained after structure adjustment and task adaptability optimization.

[0148] Through the above dynamic training mechanism, the invention realizes the unified optimization of structure compression parameters and task feature learning, and does not need to introduce new training framework or additional structure modification in the model training process, which has good compatibility, deployability and generalizability. Compared with the traditional training method which only focuses on performance, the invention simultaneously guides the sparsity and channel diversity of the convolution structure, so that the scaling factor has stable distribution, task responsiveness and structure sensitivity, which can be continuously optimized in the model training, thereby dynamically generating one-dimensional convolution kernel weights with task adaptability. This process not only preserves the representation ability of the original two-dimensional model, but also makes the structure compressed model have higher structural interpretability and reasoning efficiency when executing specific one-dimensional tasks (such as speech, text, time series, etc.), realizing the goal of migrating two-dimensional knowledge to one-dimensional model structure self-adaptive reconstruction, and forming a more optimal structure compression expression.

[0149] An embodiment of the present application provides a structural schematic diagram of a dynamic weight compression device based on cross-modal transfer learning. The device comprises: a two-dimensional representation construction module configured to obtain a network structure and pre-training weight parameters from a two-dimensional convolutional neural network model, and construct an initialized two-dimensional representation of structure and weight pairing by using the network structure and the pre-training weight parameters; a one-dimensional kernel generation module configured to perform static aggregation on a two-dimensional convolution kernel structure layer with weight parameters in the two-dimensional model representation along a spatial height dimension to obtain intermediate weights, construct a learnable dynamic weight factor based on a statistical quantity of target one-dimensional data, and generate a weighted one-dimensional convolution kernel by dynamically weighting and compressing the weights, rewrite a two-dimensional structure layer without weight in the two-dimensional model representation into a one-dimensional structure according to a predefined one-dimensional mapping rule, and obtain a one-dimensional neural network with two-dimensional pre-training information by combining the one-dimensional structure with the weighted one-dimensional convolution kernel; and a rule mapping and structure fusion module configured to introduce multiple regularization terms for joint optimization, construct a final training target of the one-dimensional neural network model by jointly training a main task loss and the multiple regularization terms, and realize effective cross-modal transfer and structure adaptive reconstruction in a training stage of the one-dimensional neural network model with two-dimensional pre-training information.

[0150] Figure 3 A structural schematic diagram of a dynamic weight compression device based on cross-modal transfer learning provided by an embodiment of the present application is shown. As shown in the figure, the device comprises a two-dimensional representation construction module, a one-dimensional kernel generation module, and a rule mapping and structure fusion module. Figure 3The method comprises the following steps: a two-dimensional representation construction module is configured to obtain a network structure and pre-training weight parameters from a two-dimensional convolutional neural network model, and to bind the structure and the weight parameters one by one in a hierarchical and connection relationship to construct an initialized two-dimensional representation of the structure and the weight parameters; a one-dimensional kernel generation module is configured to perform static aggregation along a height direction on a two-dimensional convolution kernel with weight parameters in the initialized two-dimensional representation, and to perform channel-level weighting and one-dimensional projection according to a learnable dynamic weight factor generated based on target one-dimensional data, so as to obtain a one-dimensional convolution kernel; the dynamic weight factor is updated jointly with network parameters in a training process, and the generation process does not require weight quantization or activation function replacement as a necessary step; a rule mapping and structure fusion module is configured to map a structure layer without weight in the initialized two-dimensional representation according to a one-dimensional rule and to perform structure-level fusion with the one-dimensional convolution kernel, wherein the one-dimensional rule at least comprises: kernel size and step length of a two-dimensional pooling layer are mapped into time window length and time step length according to an area and step equivalent principle, statistics of a batch normalization layer are accumulated and updated along a time dimension, a fully connected layer is replaced with a one-dimensional convolution with a kernel length of 1, and Dropout is independently sampled in a time dimension; a training and optimization module is configured to optimize the one-dimensional neural network on a target task by using a joint training target, the joint training target comprising a sparsity constraint and a channel diversity constraint of the dynamic weight factor and a main task loss, so as to realize cross-modal effective migration and structure adaptive reconstruction; the training and deployment process does not include weight pruning, bit plane index encoding and Viterbi-based compression or decompression processes.

[0151] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details and limit the application to the specific embodiments described. Obviously, according to the content of the embodiments of the present application, many modifications and changes can be made. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present application, so that those skilled in the art can well understand and utilize the present application. The present application is limited by the claims and their entire scope and equivalents.

Claims

1. A dynamic weight compression method based on cross-modal transfer learning, characterized in that, The method comprises the steps of: extracting network structure and pre-training weight parameters from a two-dimensional convolutional neural network model, and constructing a two-dimensional model representation with paired structure and weight by using the network structure and the pre-training weight parameters; by performing static aggregation on the two-dimensional convolution kernel structure layer with weight parameters in the two-dimensional model representation along the spatial height dimension to obtain intermediate weights, and constructing a learnable dynamic weight factor based on the statistics of the target one-dimensional data and dynamically weighting and compressing the weights to generate a weighted one-dimensional convolution kernel, rewriting the structure layer without weight in the two-dimensional model representation into a one-dimensional structure according to a pre-defined one-dimensional mapping rule, and combining it with the weighted one-dimensional convolution kernel to obtain a one-dimensional neural network with two-dimensional pre-training information, comprising: Introducing dynamic weight factor And using the dynamic weight factor The structure layer of the two-dimensional convolution kernel with weight parameters in the initialized two-dimensional model representation is first subjected to height dimension static aggregation and then subjected to dynamic factor weighting compression to obtain a one-dimensional convolution kernel model with weight parameters. by performing static compression processing on each convolution kernel of the two-dimensional convolution kernel structure layer with weight parameters in the initialized two-dimensional model in the height direction, and projecting the original spatial dimension to the time dimension as the projection basis, a static compressed aggregated weight is obtained; According to the target one-dimensional neural network model, a learnable dynamic weight factor is generated Channel-level weighting and one-dimensional projection are performed on the intermediate weight to obtain a one-dimensional convolution kernel; the dynamic weight factor In the training process, it is updated jointly with other network parameters by regularizing and rewriting the weightless layer in the initialized two-dimensional model representation according to the kernel length, step length and statistical dimension of the one-dimensional time dimension, and performing structure-level fusion with the one-dimensional convolution kernel, a one-dimensional structure layer model without weight parameters is obtained; by fusing the one-dimensional convolution kernel model with weight parameters and the one-dimensional structure layer model without weight parameters, a one-dimensional neural network model is obtained; introducing multiple regularization terms for joint optimization, training the main task loss and the multiple regularization terms jointly to construct the final training target of the one-dimensional neural network model, so as to realize effective cross-modal transfer and structure adaptive reconstruction in the training stage of the one-dimensional neural network model with two-dimensional pre-training information.

2. The method of claim 1, wherein, The use of the network structure and the pre-training weight parameters to construct the initialized two-dimensional model representation with paired structure and weight comprises: by one-to-one binding and corresponding the kernel tensor of each convolution layer with its hierarchical and connection relationship in the network topology, the subsequent static aggregation and regular mapping are performed on the paired structure with consistent hierarchy.

3. The method of claim 1, wherein, The mapping processing of the weightless parameter structure layer in the initialized two-dimensional model representation to obtain the one-dimensional structure layer model without weight parameters comprises: by mapping and processing the pooling layer, the normalization layer, the Dropout layer and the fully connected layer in the weightless parameter structure layer in the initialized two-dimensional model representation according to the structure rule, and performing structure-level fusion with the one-dimensional convolution kernel, a one-dimensional structure layer model without weight parameters is obtained.

4. The method of claim 1, wherein, The training adopts a joint target, and the main task loss and the multiple regularization terms are jointly modeled to construct the final training target of the one-dimensional neural network model, comprising: Dynamic weight factor The sparsity constraint is applied for restraining the sparsity regular term loss of redundant features participating in representation and the channel diversity regular term loss for improving overall expression ability. jointly modeling the main task loss and the sparsity regularization term loss and the channel diversity regularization term loss to construct the final training target of the one-dimensional neural network model for guiding model training.

5. The method of claim 4, wherein, The joint modeling of the main task loss and the sparsity regularization term loss and the channel diversity regularization term loss to construct the final training target of the one-dimensional neural network model for guiding model training comprises: ； Wherein, the refers to the final training target; the refers to the main task loss; the is a sparse regular term loss containing a dynamic weight factor ; and the is a channel diversity regular term loss containing a dynamic weight factor .

6. A dynamic weight compression device based on cross-modal transfer learning, characterized in that, comprising: The two-dimensional representation construction module is configured to obtain a network structure and pre-training weight parameters from a two-dimensional convolutional neural network model, and construct an initialized two-dimensional representation with a structure and weight pair by using the network structure and the pre-training weight parameters. The one-dimensional kernel generation module is configured to perform static aggregation on a two-dimensional convolution kernel structure layer with weight parameters in the two-dimensional model representation along a spatial height dimension to obtain intermediate weights, construct a learnable dynamic weight factor based on a statistical quantity of target one-dimensional data, and generate a weighted one-dimensional convolution kernel with weights by dynamically weighting and compressing the weights, rewrite a weightless structure layer in the two-dimensional model representation into a one-dimensional structure according to a predefined one-dimensional mapping rule, and obtain a one-dimensional neural network with two-dimensional pre-training information by combining the one-dimensional structure with the weighted one-dimensional convolution kernel, including: Introducing dynamic weight factor And using the dynamic weight factor The structure layer of the two-dimensional convolution kernel with weight parameters in the initialized two-dimensional model representation is first subjected to height dimension static aggregation and then subjected to dynamic factor weighting compression to obtain a one-dimensional convolution kernel model with weight parameters. The static compression and aggregation weight is obtained by performing static compression processing on each convolution kernel of the two-dimensional convolution kernel structure layer with weight parameters in the initialized two-dimensional model in the height direction, and projecting the original spatial dimension to the time dimension as a projection basis. According to the target one-dimensional neural network model, a learnable dynamic weight factor is generated Channel-level weighting and one-dimensional projection are performed on the intermediate weight to obtain a one-dimensional convolution kernel; the dynamic weight factor In the training process, it is updated jointly with other network parameters The weightless parameter one-dimensional structure layer model is obtained by regularizing and rewriting the weightless layer in the initialized two-dimensional model representation according to the kernel length, step length, and statistical dimension of the one-dimensional time dimension, and performing structure-level fusion with the one-dimensional convolution kernel. The one-dimensional neural network model is obtained by fusing the one-dimensional convolution kernel model with weight parameters and the weightless parameter one-dimensional structure layer model. The regular mapping and structure fusion module is configured to introduce multiple regularization terms for joint optimization, construct a final training target of the one-dimensional neural network model by jointly training the main task loss and the multiple regularization terms, and realize effective cross-modal transfer and structure adaptive reconstruction in the training stage of the one-dimensional neural network model with two-dimensional pre-training information.

7. A computing device, comprising: including: a memory and a processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions, which implement the steps of the dynamic weight compression method based on cross-modal transfer learning of any one of claims 1 to 5 when executed by the processor.

8. A computer-readable storage medium, characterized in that, The memory has computer executable instructions stored therein, which implement the steps of the dynamic weight compression method based on cross-modal transfer learning of any one of claims 1 to 5 when executed by the processor.

Images