Methods, devices, electronic equipment, and storage media for constructing early stroke diagnosis models

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for constructing an early stroke diagnosis model. Applied to the field of model construction, during the transformer training process, based on the activation matrix and current width of the transformer's feedforward network, the feedforward network's channel reduction or expansion is performed to adjust the feedforward network's width. This reduces the number of parameters and memory usage without sacrificing model performance, thereby reducing the computational resource overhead of the early stroke diagnosis system and enhancing its adaptability.

Description

Technical Field

[0001] This invention relates to the field of model building technology, specifically to a method, apparatus, electronic device, and storage medium for constructing an early diagnosis model of stroke. Background Technology

[0002] In recent years, early stroke diagnosis systems based on multimodal large models have become an important technological approach for medical image analysis and clinical decision support. This early stroke diagnosis system integrates various heterogeneous data (such as CT / MRI images, electronic medical record texts, etc.) and uniformly inputs them into a transformer architecture to improve the accuracy of early stroke warning.

[0003] However, existing early stroke diagnosis systems have a large number of model parameters, which significantly impacts computational resource consumption and inference latency, resulting in problems such as high resource consumption and insufficient adaptability. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, apparatus, electronic device and storage medium for constructing an early stroke diagnosis model, in order to solve the problems of high resource consumption and insufficient adaptability of existing early stroke diagnosis systems.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] The first aspect of this invention discloses a method for constructing an early diagnosis model for stroke, the method comprising:

[0007] Extracting multimodal features from stroke-related sample data;

[0008] The multimodal features are input into the transformer, and the transformer is trained using a preset loss function.

[0009] Periodically obtain the activation matrix and current width of the feedforward network of the transformer in the current training round;

[0010] According to the activation matrix and the current width, the feedforward network is subjected to channel reduction or channel expansion to adjust the width of the feedforward network;

[0011] When the transformer meets the preset training completion conditions, an early stroke diagnosis model is obtained.

[0012] Preferably, adjusting the width of the feedforward network by performing channel reduction or channel expansion according to the activation matrix and the current width includes:

[0013] The rank estimate of the activation matrix is ​​calculated using singular value decomposition.

[0014] If the product of the first superpartition's rank estimate is less than the current width, channel reduction is applied to the feedforward network to adjust its width.

[0015] If the product of the second superpartition and the rank estimate is greater than the current width, the feedforward network is channel-expanded to adjust the width of the feedforward network.

[0016] Preferably, channel reduction of the feedforward network to adjust its width includes:

[0017] Reshape the activation matrix into a higher-order tensor;

[0018] The higher-order tensors are subjected to Tucker decomposition or Tensor-Train decomposition to obtain tensor decomposition results.

[0019] Based on the rank estimate and the tensor decomposition results, the channels of the feedforward network are pruned to adjust the width of the feedforward network.

[0020] Preferably, based on the rank estimate and the tensor decomposition result, pruning the channels of the feedforward network to adjust the width of the feedforward network includes:

[0021] Based on the rank estimate and the tensor decomposition result, the first N singular vectors are selected in descending order of eigenvalues.

[0022] Remove the feature vectors in the feedforward network that correspond to the unselected singular vectors to adjust the width of the feedforward network.

[0023] Preferably, channel expansion of the feedforward network to adjust its width includes:

[0024] Add k channels to the feedforward network, set the gating mask to off, and initialize the newly added channels to adjust the width of the feedforward network, where k is calculated based on the current width, the second hyperparameter, and the rank estimate.

[0025] A second aspect of this invention discloses an apparatus for constructing an early diagnosis model of stroke, the apparatus comprising:

[0026] Extraction unit, used to extract multimodal features from stroke-related sample data;

[0027] The training unit is used to input the multimodal features into the transformer and train the transformer using a preset loss function.

[0028] The first acquisition unit is used to periodically acquire the activation matrix and current width of the feedforward network of the transformer in the current training round;

[0029] An adjustment unit is used to adjust the width of the feedforward network by reducing or expanding the channels according to the activation matrix and the current width.

[0030] The second acquisition unit is used to obtain an early stroke diagnosis model when the transformer meets the preset training completion conditions.

[0031] Preferably, the adjustment unit includes:

[0032] A computational subunit is used to compute a rank estimate of the activation matrix using singular value decomposition;

[0033] A reduction subunit is used to reduce the channel width of the feedforward network if the product of the first superpartition and the rank estimate is less than the current width.

[0034] An extended subunit is configured to perform channel expansion on the feedforward network to adjust the width of the feedforward network if the product of the second superpartition and the rank estimate is greater than the current width.

[0035] Preferably, the reduction subunit is specifically used for: reshaping the activation matrix into a higher-order tensor; performing Tucker decomposition or Tensor-Train decomposition on the higher-order tensor to obtain tensor decomposition results; and pruning the channels of the feedforward network to adjust the width of the feedforward network based on the rank estimate and the tensor decomposition results.

[0036] A third aspect of this invention discloses an electronic device, comprising: a processor and a memory, wherein the processor and the memory are connected via a communication bus; wherein the processor is used to call and execute a program stored in the memory; and the memory is used to store the program, which is used to implement the method for constructing an early diagnosis model of stroke disclosed in the first aspect of this invention.

[0037] The fourth aspect of this invention discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for constructing an early stroke diagnosis model disclosed in the first aspect of this invention.

[0038] Based on the above embodiments of the present invention, a method, apparatus, electronic device, and storage medium for constructing an early stroke diagnosis model are provided. The method involves: extracting multimodal features from stroke-related sample data; inputting the multimodal features into a transformer and training the transformer using a preset loss function; periodically obtaining the activation matrix and current width of the feedforward network in the current training round; adjusting the width of the feedforward network by reducing or expanding channels according to the activation matrix and current width; and obtaining the early stroke diagnosis model when the transformer meets preset training completion conditions. This scheme dynamically adjusts the width of the feedforward network based on the activation matrix during transformer training, reducing the number of parameters and memory usage without sacrificing model performance, thereby reducing the computational resource overhead of the early stroke diagnosis system and enhancing its adaptability. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a method for constructing an early stroke diagnosis model according to an embodiment of the present invention;

[0041] Figure 2 A flowchart for adjusting the width of the feedforward network provided in an embodiment of the present invention;

[0042] Figure 3 This is a structural block diagram of a device for constructing an early diagnosis model of stroke, provided in an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0045] To better understand the following content, some terms will be explained first.

[0046] SVD: Singular Value Decomposition.

[0047] Randomized SVD: A high-efficiency matrix factorization algorithm that uses random sampling to approximate the singular value decomposition.

[0048] Transformer: A neural network architecture based on self-attention mechanism.

[0049] Epoch: Training round.

[0050] FFN: Feedforward Network, the basic structure of neural networks, where information is passed unidirectionally from the input layer through hidden layers to the output layer, without recurrent or feedback connections. It is also the name of a layer in the Transformer, specifically the layer with the largest number of parameters.

[0051] PCA-Pruner: PCA pruner is a technique that uses principal component analysis (PCA) to compress models by pruning neural network parameters while retaining important feature dimensions.

[0052] CT / MRI: Computed Tomography / Magnetic Resonance Imaging.

[0053] Tucker decomposition and Tensor-Train decomposition: decomposition methods for higher-order tensors that represent the original tensor as the product of the core tensor and the factor matrix of each dimension.

[0054] Early stop: The early stopping method terminates training prematurely when the performance on the validation set no longer improves, thus preventing the model from overfitting.

[0055] Xavier initialization: A method for initializing neural network weights, which adjusts the initial weight range based on the number of input and output neurons to promote training stability.

[0056] In recent years, early stroke diagnosis systems based on multimodal large models have become an important technological approach for medical image analysis and clinical decision support. This early stroke diagnosis system integrates various heterogeneous data (such as CT / MRI images, electronic medical record texts, vital sign curves, voice consultation records, etc.) and inputs them uniformly into a transformer architecture to improve the accuracy of early stroke warning.

[0057] However, existing early stroke diagnosis systems have a large number of model parameters, especially the width d of the intermediate layers of the feedforward network (FFN). ff The impact on computing resource consumption and inference latency is particularly significant, mainly in the following aspects:

[0058] High resource overhead: FFN has the largest number of parameters in large multimodal models based on the transformer architecture. A relatively large fixed d ff Both the training and inference phases require high computing power and large video memory, which significantly increases the system deployment cost and brings real-time risks.

[0059] Insufficient adaptability: Different patient samples vary greatly in the complexity of multimodal features. The parameter requirements for image-text feature interaction and alignment differ at different network depths. The FFN requirements in some shallow transformers are significantly different from those in deeper corresponding layers, but the relationship is not linear and therefore difficult to adjust manually.

[0060] High real-time requirements in clinical settings: Early diagnosis scenarios have strict requirements for inference response speed, and fixed-width models are difficult to balance accuracy and latency.

[0061] It should be noted that d ff It is the width value of FFN in the transformer of the large model, such as 1024 or 2048.

[0062] Research has revealed the following specific details regarding some existing technical solutions and their limitations:

[0063] 1. Offline capacity pruning: Methods such as PCA-Pruner perform PCA dimensionality reduction analysis on the model activation matrix after training to retain static principal component channels and reduce model size.

[0064] Limitations: Single optimization lacks an iterative process, ignoring dynamic feature changes during training, resulting in limited improvement. It struggles to adapt to the changing importance of each modality under multimodal inputs over time, and relying solely on PCA for judgment leads to poor interpretability and a low upper limit for iterative optimization.

[0065] 2. Gated dynamic width adjustment: A learnable gating mechanism is introduced into the network, which enables some channels to automatically adjust their weights during training.

[0066] Limitations: Gating is primarily optimized based on gradients or regularization terms, making it difficult to accurately identify redundant components in the multimodal activation space. It also lacks interpretability, failing to explicitly analyze the redundancy of key matrices, resulting in a low optimization ceiling. Furthermore, it doesn't dynamically adjust the gating width during training; it only changes the weights without altering the model structure or reducing inference time and speed.

[0067] 3. One-time low-rank decomposition approximation: Tensor-Train or Tucker decomposition is used to compress the parameters of the pre-trained model.

[0068] Limitations: The rank value after decomposition is fixed and the model capacity is fixed, requiring manual specification of hyperparameters. Optimization is complex and is also a static optimization after training, without considering the dynamic iteration process.

[0069] To address the limitations of the existing technologies, this solution proposes a method, device, electronic equipment, and storage medium for constructing an early stroke diagnosis model. During transformer training, the width of the feedforward network is dynamically adjusted based on the activation matrix, reducing the number of parameters and memory usage without sacrificing model performance. This reduces the computational resource overhead of the early stroke diagnosis system and enhances its adaptability.

[0070] See Figure 1 The flowchart illustrates a method for constructing an early stroke diagnosis model according to an embodiment of the present invention. The method includes:

[0071] Step S101: Extract multimodal features from stroke-related sample data.

[0072] In the specific implementation step S101, sample data related to stroke is acquired. This sample data includes at least: text data (such as patient consultation voice transcripts), image data (such as facial images), and voice data (such as patient consultation voice).

[0073] Multimodal features are extracted from the acquired sample data. These multimodal features include at least: text embedding features, spatial attention features, and emotional state vector features (such as voice emotion embedding vectors).

[0074] Specifically, for the text data in the sample data, the text data is input into a pre-trained language model to extract text embedding features;

[0075] For the image data in the sample data, the image data is encoded into spatial attention features using ViT;

[0076] For the speech data in the sample data, the speech features of the speech data are extracted into emotional state vector features using wav2vec2 or EmotionNet.

[0077] Step S102: Input the multimodal features into the transformer and train the transformer using a preset loss function.

[0078] In the specific implementation step S102, the multimodal features extracted from the sample data are input into the transformer of the large model (the multimodal features form a fused multimodal sequence input), and the transformer is trained by combining the preset loss function.

[0079] It should be noted that during the training of the transformer, the input data consists of multimodal features such as text embedding features, spatial attention features, and emotional state vector features, and the output is a risk grading label for early stroke.

[0080] In some specific embodiments, the loss function used in this scheme to train the transformer is shown in formula (1).

[0081] (1);

[0082] In formula (1), L total Let L be the total loss function. task For stroke, the standard classification loss function is d. ff Let r(A) be the hidden layer dimension (i.e., the current width) of the current feedforward network (FFN), and r(A) be the rank estimate of the current sampled activation matrix A (also called the estimated rank, the rank reflects the dimension of effective information, i.e. the actual required width). The model gradient (a measure of the current model's training stability). , and To control the hyperparameters that influence the weights of various factors.

[0083] It should be noted that, , and Set according to the actual situation, for example , , .

[0084] The hyperparameters are used to correlate the width of the FFN with the rank of the activation matrix, and the difference between the number of linearly independent vectors in the FFN and the activation matrix is ​​used as a penalty term. The larger the number of linearly independent vectors, the larger the rank of the matrix, and the wider the FFN is suitable; conversely, the smaller the number of linearly independent vectors, the more suitable it is to reduce the width.

[0085] Based on the original task loss, a structure-driven term and a training stability term are introduced to form the loss function given by the above formula (1). This ensures that while pursuing the maximization of task performance, the model also takes into account the matching of structural resources and the stability of the training process, and guides the model to dynamically evolve its structural parameters during the training phase.

[0086] Step S103: Periodically obtain the activation matrix and current width of the transformer's feedforward network in the current training round.

[0087] It should be noted that during the training process, a fixed training period (such as every 5 epochs) is set to sample the activation matrix of the intermediate layer of the feedforward network, dynamically analyze its information redundancy and effective expressive ability, and thus adjust the width of the feedforward network.

[0088] In the specific implementation of step S103, during the training process, the activation matrix (also known as the activation tensor or activation data) of the transformer's feedforward network in the current training epoch is periodically obtained according to the training cycle, as well as the current width d of the feedforward network in the current training epoch. ff The current width can be read directly.

[0089] For example, when the training cycle is set to 5 training epochs, the activation matrix of the feedforward network in the current training epoch and the current width of the feedforward network in the current training epoch are obtained every 5 training epochs.

[0090] Specifically, the implementation of obtaining the activation matrix is ​​as follows: obtain the activation output of each layer of the transformer's feedforward network in the current training epoch, and use each layer as the activation matrix. The downsampled activation matrix in the current training epoch can be denoted as A. R is a real number, and B is the batch size (i.e., batch, the number of samples fed in one training iteration).

[0091] For example, in a certain training epoch, if batch=64 and hidden_dim=2048, then the dimension of the activation matrix output by this feedforward network is 64*2048.

[0092] Step S104: Adjust the width of the feedforward network by reducing or expanding the channels according to the activation matrix and the current width.

[0093] In the specific implementation step S104, the feedforward network is reduced or expanded according to the obtained activation matrix and the current width, thereby adjusting the width of the feedforward network.

[0094] In other words, a width update operation (adjusting the width of the feedforward network) is performed every training cycle.

[0095] For example, when the training cycle is set to 5 training rounds, a width update operation is performed every 5 training rounds.

[0096] Based on the above steps, the AdamW optimizer is used to update the parameters, and the impact of the structural adjustment on performance is recorded according to the loss curve after each update.

[0097] Step S105: When the transformer meets the preset training completion conditions, the stroke early diagnosis model is obtained.

[0098] In the specific implementation of step S105, steps S103 and S104 are repeated during transformer training to dynamically adjust the width of the feedforward network. After the transformer training reaches the training completion condition, the structure of all modules of the transformer is frozen to obtain the early diagnosis model of stroke.

[0099] For example, once the transformer training reaches the early stop criterion, the structure of all modules of the transformer is frozen, thus obtaining an early stroke diagnosis model.

[0100] It should be noted that large models typically consist of multiple transformers. Training this transformer using this approach yields a stroke early diagnosis model based on a multimodal large model. The stroke early diagnosis model is then exported as a static graphical model and deployed.

[0101] In this embodiment of the invention, during the transformer training process, based on the activation matrix and current width of the transformer's feedforward network, the feedforward network is subjected to channel reduction or channel expansion to adjust the width of the feedforward network. This reduces the number of parameters and memory usage without sacrificing model performance, thereby reducing the computational resource overhead of the early stroke diagnosis system and enhancing its adaptability.

[0102] Regarding the above embodiments of the present invention Figure 1 The adjustment of the feedforward network width involved in step S104 is described in [reference needed]. Figure 2 The flowchart illustrating the adjustment of the width of the feedforward network according to an embodiment of the present invention includes the following steps:

[0103] Step S201: Calculate the rank estimate of the activation matrix using singular value decomposition.

[0104] In the specific implementation step S201, singular value decomposition is performed on the activation matrix A downsampled in the current training round to calculate the rank estimate r(A) of the activation matrix.

[0105] For example, Randomized SVD is used to perform an approximate singular value decomposition on the activation matrix, thereby estimating the rank estimate r(A) of the activation matrix to measure the number of linearly independent features.

[0106] The calculated rank estimate can be used to determine whether to reduce or expand the width of the feedforward network, as detailed in subsequent steps S202 and S203.

[0107] Step S202: If the product of the first superpartition rank estimates is less than the current width, perform channel reduction on the feedforward network to adjust the width of the feedforward network.

[0108] In the specific implementation step S202, if the product of the first hyperparameter μ1 and the rank estimate r(A) is less than the current width d ff This indicates that the feedforward network has significant redundancy and needs to be narrowed. In this case, channel reduction is performed on the feedforward network to adjust its width.

[0109] In other words, if μ1*r(A) < d ff Channel reduction is performed on the feedforward network to adjust its width. During the channel reduction process, it is possible to determine which feature vectors need to be retained and which feature vectors need to be removed based on the main feature values.

[0110] In some specific embodiments, the channel reduction of the feedforward network is implemented as follows: the activation matrix is ​​reshaped into a higher-order tensor; the higher-order tensor is decomposed by Tucker or Tensor-Train to obtain the tensor decomposition result (which includes eigenvalues ​​sorted from largest to smallest); based on the rank estimate and the tensor decomposition result, the channels of the feedforward network are pruned to adjust the width of the feedforward network.

[0111] More specifically, if μ1*r(A) < d ff This indicates that the feedforward network has significant redundancy. For the activation matrix containing multi-dimensional modal features, the activation matrix is ​​reshaped into a higher-order tensor according to the batch, channel, and attention head dimensions.

[0112] The high-order tensor is subjected to Tucker decomposition or Tensor-Train decomposition to extract directional redundancy indices, thereby obtaining tensor decomposition results, which can be used to guide fine-grained structural trimming.

[0113] Based on the rank estimate and tensor decomposition results, the principal component directions are selected to be retained, thereby pruning the channels of the feedforward network.

[0114] In other specific embodiments, the specific implementation of pruning the channels of the feedforward network based on the rank estimate and tensor decomposition results is as follows: based on the rank estimate and tensor decomposition results, select the top N singular vectors in descending order of eigenvalues; remove the eigenvectors corresponding to the "unselected singular vectors" in the feedforward network to adjust the width of the feedforward network.

[0115] Specifically, the process of pruning the channels of the feedforward network mainly includes two parts: "principal component selection" and "channel removal".

[0116] Principal component selection: Select the first N singular vectors; where N is selected by accumulating the eigenvalues ​​of the corresponding vectors from largest to smallest until the sum of the accumulated eigenvalues ​​reaches 90% of the sum of all eigenvalues, which is the minimum N.

[0117] Channel Removal: Remove the feature vectors of the feedforward network corresponding to the singular vectors that were not selected, thereby completing the channel reduction of the feedforward network. Here, "singular vectors that were not selected" are the right singular vectors (corresponding to smaller singular values) that were not selected into the first N principal components after SVD decomposition.

[0118] By pruning the channels of the feedforward network in the above manner, the number of channels in the feedforward network is reduced to d. new =δ+r(A).

[0119] Where δ is the buffer amount, that is, the increase in magnitude, which can be a fixed step size, such as 32 or 64.

[0120] d new This reduces the width of the feedforward network after the channel is closed (i.e., the new width).

[0121] It should be noted that d new Subject to global constraints, d new =max(d min , δ+r(A)), d min This is the lower boundary limit value for the width of the feedforward network.

[0122] Similarly, d new Less than d max d max This represents the upper boundary limit value for the width of the feedforward network. That is, the width of the feedforward network is limited by (d...). min , d max This upper and lower boundary limit avoids training instability caused by violent structural oscillations, and the width is truncated if it exceeds the upper and lower boundaries.

[0123] The change in the width of the feedforward network (also called the magnitude of change, denoted as Δd) is also constrained; the change in width equals the width d of the feedforward network. ff The single adjustment amount, Δd∈[Δmin, Δmax], where Δmin is the minimum change in width and Δmax is the maximum change in width.

[0124] Step S203: If the product of the second superpartition rank estimates is greater than the current width, perform channel expansion on the feedforward network to adjust the width of the feedforward network.

[0125] In the specific implementation step S203, if the product of the second hyperparameter μ2 and the rank estimate r(A) is greater than the current width d ff This indicates that the structure's capacity is insufficient (actual demand significantly exceeds the current capacity). In this case, channel expansion is performed on the feedforward network to adjust its width. The width of the feedforward network after channel expansion is also affected by "(d min , d max The constraints are “Δd∈[Δmin, Δmax]” and “Δd∈[Δmin, Δmax]”.

[0126] In other words, if μ2*r(A) > d ff Channel expansion is performed on the feedforward network to adjust its width.

[0127] In some specific implementations, the specific way to implement channels in the feedforward network is as follows: add k channels to the feedforward network, set the gating mask to off and initialize the newly added channels to adjust the width of the feedforward network, where k is calculated based on the current width, the second hyperparameter and the rank estimate.

[0128] Specifically, if μ2*r(A) > d ff Add k channels to the feedforward network, k = μ2 * r(A) - d now , where d now This is the current width of the feedforward network at this moment; the newly added channels are initialized using the Xavier method (or the historical activation variance can be used for initialization), and the gating mask is set to off.

[0129] It should be noted that "setting the gating mask to off" essentially sets the initial value to 0.1 so that the newly added channel output is equivalent to 10% weakly blended with the original output, thus avoiding sudden activation that could disrupt the model's stability.

[0130] This solution uses a gating mask to gradually activate newly added channels. The specific content of the gating mask is as follows:

[0131] g j ∈{R}, R is a real number (a one-dimensional number), and the initial value is g.j =0.5, adopting a soft and hard masking hybrid mechanism, the gated output is defined as the content shown in formula (2).

[0132] (2);

[0133] In formula (2), g j It is a scalar of the original gate mask (learnable parameters, basic control variables, optimized by gradient descent). Set as the lower bound threshold for the soft mask (e.g., 0.1, to prevent the gradient from vanishing due to an excessively small gate value). This is the upper threshold. For g j The parameters are truncated by taking the upper and lower bounds (to prevent the value range from exceeding the expected range).

[0134] The gated output given by formula (2) above is only used in the channel expansion stage, specifically for: output fusion of newly added channels, gradient backpropagation during model optimization, and as g... j Enhance and gradually increase the contribution of newly added channels.

[0135] The above embodiments of the present invention Figure 2 This section provides information regarding adjusting the width of the feedforward network.

[0136] As can be seen from the above embodiments, the main innovation of this solution lies in:

[0137] Innovation Point 1: Online Periodic Width Adjustment Mechanism

[0138] During training, the activation matrix of the intermediate layers of the feedforward network is collected at fixed training epochs (e.g., every 5 epochs). The hidden dimension (current width) of the layer is evaluated and dynamically adjusted in real time. By continuously iterating and updating within the training epochs, the model's expressive power and computational resources are balanced, thereby improving convergence efficiency and inference performance.

[0139] This update method has the following characteristics: it is non-static and iteratively optimizable, unlike traditional offline pruning methods; it supports bidirectional width adjustment, which can both shrink and expand, improving the structure's adaptability; the width adjustment operation effectively controls computation and memory costs while keeping the main functions unchanged, ensuring stable model convergence; the adjustment is based on feature analysis of the matrix, rather than on experience or fixed thresholds.

[0140] Innovation Point Two: Multimodal Rank Estimation Evaluation

[0141] Matrix rank estimation: The rank estimate r (numerical rank) is calculated directly on the sampled activation matrix A or approximated by Randomized SVD, reflecting the number of current linearly independent components.

[0142] Higher-order tensor decomposition: The activation matrix is ​​reshaped into higher-order tensors according to batch, channel or attention head dimensions, and the rank information of each mode is obtained by Tucker decomposition or Tensor-Train decomposition, and the redundancy in each direction is measured in fine granularity.

[0143] The rank estimation results mentioned above provide quantitative support for subsequent structural adjustments, significantly improving adaptability and computational efficiency.

[0144] Innovation Point 3: Adaptive Objective Function-Driven Approach

[0145] In traditional model training, the network structure is usually statically preset, which makes it difficult to dynamically adjust according to the complexity of the actual task and the characteristics of the samples, easily leading to a waste of computing resources or insufficient model expression.

[0146] This solution proposes an adaptive objective function-driven mechanism that integrates task performance, structural requirements, and training stability. An innovative loss function, Equation (1), is proposed for this scenario. This mechanism, by introducing structural adjustment terms and stability constraints, enables the model to dynamically adjust the dimensions of intermediate layers during training, achieving synergistic optimization of resource utilization efficiency and model performance.

[0147] Innovation Point 4: Width Update and Mask Design

[0148] For reduction: Based on the matrix factorization results and the subspaces of each modality's rank direction, the corresponding weights are projected or pruned. The essence of this reduction operation is: finding the principal axis direction of the feature space through singular value decomposition, projecting the FFN weights onto this low-dimensional subspace, thereby achieving structured pruning.

[0149] For expansion: Initialize the new dimension based on historical activation variance or Xavier, and fine-tune it using a learnable binary / soft-gated mask, through upper and lower bounds (d min ,d max Constraints are used to prevent violent shaking.

[0150] In summary, this approach defines a structure-driven adaptive objective function and dynamically changes the model structure during training, enabling the FFN dimension in the Transformer to be periodically adjusted according to the model's expressive needs. By introducing a rank estimation guiding term and a gradient stabilization term on top of the main task loss, it balances performance, efficiency, and stability, making it suitable for resource-constrained multimodal stroke early screening systems.

[0151] Therefore, this solution has the following beneficial effects:

[0152] Benefit 1: Improves model inference efficiency and reduces resource consumption.

[0153] By periodically evaluating and reconstructing the width of the FFN in the Transformer, inefficient neuron channels can be effectively eliminated. This reduces the number of parameters and memory usage by about 30% without sacrificing model performance, thereby improving inference speed. It is suitable for deployment in hospital terminal equipment and telemedicine scenarios.

[0154] Benefit 2: Enhances the model's structural expressiveness and adaptability:

[0155] By introducing a structural activation sparsity metric and using tensor rank estimation and gradient-driven dynamic masking mechanisms, combined with a controlled loss function, the Transformer structure can be grown and compressed on demand during training, improving the model's adaptability to multimodal input variations. This is particularly suitable for modeling tasks with small sample medical data.

[0156] Thirdly, it supports the deep fusion of heterogeneous multimodal medical data:

[0157] By integrating medical data modalities such as voice, text, and images, and constructing a cross-modal collaborative attention module under the spatial attention mechanism, the early warning accuracy of complex diseases such as stroke is significantly improved, solving the problems of traditional models' strong dependence on a single modality and insufficient feature extraction capabilities.

[0158] Fourthly, it possesses good versatility and scalability:

[0159] The structure compression and width control method of this solution does not depend on specific task annotation rules or network structure, and can be directly embedded into a variety of mainstream large model systems, making it easy to promote and apply in other disease early screening, image-assisted diagnosis or public health monitoring scenarios.

[0160] Fifthly, it assists in achieving earlier and more intelligent stroke screening:

[0161] Based on clinical practice, a deployable early stroke diagnosis model is constructed, providing various users with a "low-cost, high-precision, and non-invasive" early stroke screening and warning method.

[0162] Corresponding to the method for constructing an early stroke diagnosis model provided in the above embodiments of the present invention, see also... Figure 3 The present invention also provides a structural block diagram of a device for constructing an early diagnosis model of stroke, the device comprising: an extraction unit 100, a training unit 200, a first acquisition unit 300, an adjustment unit 400, and a second acquisition unit 500.

[0163] Extraction unit 100 is used to extract multimodal features from stroke-related sample data.

[0164] Training unit 200 is used to input multimodal features into the transformer and train the transformer using a preset loss function.

[0165] The first acquisition unit 300 is used to periodically acquire the activation matrix and current width of the transformer's feedforward network in the current training round.

[0166] The adjustment unit 400 is used to adjust the width of the feedforward network by reducing or expanding the channels according to the activation matrix and the current width.

[0167] The second acquisition unit 500 is used to obtain an early stroke diagnosis model when the transformer meets the preset training completion conditions.

[0168] In this embodiment of the invention, during the transformer training process, based on the activation matrix and current width of the transformer's feedforward network, the feedforward network is subjected to channel reduction or channel expansion to adjust the width of the feedforward network. This reduces the number of parameters and memory usage without sacrificing model performance, thereby reducing the computational resource overhead of the early stroke diagnosis system and enhancing its adaptability.

[0169] Preferred, combined Figure 3 The adjustment unit 400, as shown, includes a calculation subunit, a reduction subunit, and an expansion subunit. The execution principle of each subunit is as follows:

[0170] The computational subunit is used to compute the rank estimate of the activation matrix using singular value decomposition.

[0171] The reduction sub-unit is used to reduce the channel width of the feedforward network if the product of the first superpartition rank estimate is less than the current width.

[0172] In specific implementation, the reduction sub-unit is used to: reshape the activation matrix into a higher-order tensor; perform Tucker decomposition or Tensor-Train decomposition on the higher-order tensor to obtain the tensor decomposition result; and prune the channels of the feedforward network to adjust the width of the feedforward network based on the rank estimate and the tensor decomposition result.

[0173] In a more specific implementation, the method for pruning the channels of the feedforward network is as follows: based on the rank estimate and tensor decomposition results, select the top N singular vectors in descending order of eigenvalues; remove the eigenvectors in the feedforward network corresponding to the unselected singular vectors to adjust the width of the feedforward network.

[0174] An extended sub-unit is used to perform channel expansion on the feedforward network to adjust the width of the feedforward network if the product of the second superpartition rank estimates is greater than the current width.

[0175] In the specific implementation, the extended sub-unit is used to: add k channels to the feedforward network, set the gating mask to off, and initialize the newly added channels to adjust the width of the feedforward network, where k is calculated based on the current width, the second hyperparameter, and the rank estimate.

[0176] Preferably, the present invention also provides an electronic device, including: a processor and a memory, the processor and the memory being connected via a communication bus; wherein, the processor is used to call and execute a program stored in the memory; the memory is used to store the program, the program being used to implement the method for constructing the early diagnosis model of stroke provided in the above method embodiments.

[0177] Preferably, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for constructing an early stroke diagnosis model provided in the above-described method embodiments.

[0178] In summary, the embodiments of the present invention provide a method, apparatus, electronic device and storage medium for constructing an early stroke diagnosis model. During the transformer training process, the width of the feedforward network is dynamically adjusted based on the activation matrix, reducing the number of parameters and memory usage without sacrificing model performance, thereby reducing the computational resource overhead of the early stroke diagnosis system and enhancing its adaptability.

[0179] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0180] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0181] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing an early diagnosis model for stroke, characterized in that, The method includes: Extracting multimodal features from stroke-related sample data; The multimodal features are input into the transformer, and the transformer is trained using a preset loss function. Periodically obtain the activation matrix and current width of the feedforward network of the transformer in the current training round; According to the activation matrix and the current width, the feedforward network is subjected to channel reduction or channel expansion to adjust the width of the feedforward network; When the transformer meets the preset training completion conditions, an early stroke diagnosis model is obtained. Adjusting the width of the feedforward network by performing channel reduction or channel expansion based on the activation matrix and the current width, including: The rank estimate of the activation matrix is ​​calculated using singular value decomposition. If the product of the first superpartition's rank estimate is less than the current width, channel reduction is applied to the feedforward network to adjust its width. If the product of the second superpartition and the rank estimate is greater than the current width, the feedforward network is channel-expanded to adjust the width of the feedforward network.

2. The method according to claim 1, characterized in that, Channel reduction of the feedforward network to adjust its width includes: Reshape the activation matrix into a higher-order tensor; The higher-order tensors are subjected to Tucker decomposition or Tensor-Train decomposition to obtain tensor decomposition results. Based on the rank estimate and the tensor decomposition results, the channels of the feedforward network are pruned to adjust the width of the feedforward network.

3. The method according to claim 2, characterized in that, Based on the rank estimate and the tensor decomposition result, the channels of the feedforward network are pruned to adjust the width of the feedforward network, including: Based on the rank estimate and the tensor decomposition result, the first N singular vectors are selected in descending order of eigenvalues. Remove the feature vectors in the feedforward network that correspond to the unselected singular vectors to adjust the width of the feedforward network.

4. The method according to claim 1, characterized in that, Channel expansion of the feedforward network to adjust its width includes: Add k channels to the feedforward network, set the gating mask to off, and initialize the newly added channels to adjust the width of the feedforward network, where k is calculated based on the current width, the second hyperparameter, and the rank estimate.

5. A device for constructing an early diagnosis model of stroke, characterized in that, The device includes: Extraction unit, used to extract multimodal features from stroke-related sample data; The training unit is used to input the multimodal features into the transformer and train the transformer using a preset loss function. The first acquisition unit is used to periodically acquire the activation matrix and current width of the feedforward network of the transformer in the current training round; An adjustment unit is used to adjust the width of the feedforward network by reducing or expanding the channels according to the activation matrix and the current width. The second acquisition unit is used to obtain an early stroke diagnosis model when the transformer meets the preset training completion conditions. The adjustment unit includes: A computational subunit is used to compute a rank estimate of the activation matrix using singular value decomposition; A reduction subunit is used to reduce the channel width of the feedforward network if the product of the first superpartition and the rank estimate is less than the current width. An extended subunit is configured to perform channel expansion on the feedforward network to adjust the width of the feedforward network if the product of the second superpartition and the rank estimate is greater than the current width.

6. The apparatus according to claim 5, characterized in that, The reduction subunit is specifically used to: reshape the activation matrix into a higher-order tensor; and perform Tucker decomposition or Tensor-Train decomposition on the higher-order tensor to obtain the tensor decomposition result. Based on the rank estimate and the tensor decomposition results, the channels of the feedforward network are pruned to adjust the width of the feedforward network.

7. An electronic device, characterized in that, include: A processor and a memory are connected via a communication bus; wherein the processor is used to call and execute a program stored in the memory; The memory is used to store a program for implementing the method for constructing an early stroke diagnosis model as described in any one of claims 1-4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for constructing an early stroke diagnosis model as described in any one of claims 1-4.

Images