Ultrasound image learning model training and image new lesion region identification method and device

Abstract

The application relates to the technical field of deep learning, and discloses an ultrasonic image learning model training and image new lesion region identification method and device. The method comprises the following steps: first, acquiring a training ultrasonic image data set and a historical ultrasonic image data set, training a historical model, and obtaining a training model; inputting the historical ultrasonic image data set into the training model and the historical model, obtaining an updated activation tensor and a historical activation tensor, and then obtaining a cost matrix; rearranging parameters of the historical model according to the cost matrix to obtain an alignment model; fusing the parameters of the training model and the parameters of the alignment model through an effective fusion coefficient to obtain a fusion model; and finally, constraining the training process of the fusion model through a preset cross-entropy loss function to obtain an ultrasonic image learning model. Through joint training of the model and construction of the cost matrix, the reliability of model fusion is improved; through fusion of the model, different network layers are automatically fused, and the learning ability of the model is improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method and apparatus for training an ultrasound image learning model and identifying new lesion regions in images. Background Technology

[0002] Continuous learning refers to the process by which a model, when faced with a sequential stream of tasks, not only learns and masters new tasks but also tries not to forget the knowledge of old tasks that it has already learned. Catastrophic forgetting refers to the phenomenon that the predictive performance of a deep neural network on old data distributions drops sharply after it learns new data distributions.

[0003] With the widespread application of continuous learning technology in the field of lesion area identification in medical ultrasound imaging, new diseases, new lesion subtypes, and cross-device imaging data are constantly emerging in clinical scenarios, requiring models to have incremental learning capabilities to continuously adapt to new diagnostic needs.

[0004] Existing learning models generally suffer from catastrophic forgetting when learning new tasks, meaning that the diagnostic performance of previous tasks significantly degrades after learning new knowledge, failing to meet the high reliability requirements of clinical diagnosis of old diseases. Current mainstream continuous learning solutions mainly fall into three categories: replay-based solutions, regularization constraints, and structural isolation. Replay-based solutions are limited by medical data privacy compliance, making large-scale application in hospital settings difficult. Solutions based on parameter regularization and knowledge distillation show significant forgetting accumulation when task characteristics differ greatly, making it difficult to maintain long-term stable performance on old tasks. Structural isolation-based solutions experience a continuous expansion of parameters as the number of tasks increases, leading to rising deployment and maintenance costs. Existing learning model training methods inherently suffer from a difficulty in balancing stability and plasticity (learning ability), failing to simultaneously achieve high-fidelity retention of old knowledge and efficient learning of new knowledge.

[0005] Therefore, a new training method for ultrasound image learning models is needed to improve the reliability of ultrasound image learning models while enhancing their learning ability. Summary of the Invention

[0006] This invention provides a method and apparatus for training an ultrasound image learning model and identifying new lesion regions in images, which improves the reliability of the ultrasound image learning model while enhancing its learning ability.

[0007] The first aspect of this invention discloses a method for training an ultrasound image learning model, the method comprising: Acquire historical ultrasound image datasets and training ultrasound image datasets; the ultrasound image datasets include first-type lesion regions and second-type lesion regions; The training model is obtained by training a preset historical model using the training ultrasound image dataset and the historical ultrasound image dataset; the first type of lesion region is the lesion region that can be identified by the historical model, and the second type of lesion region is the lesion region that cannot be identified by the historical model. The historical ultrasound image dataset is input into the training model to obtain the updated activation tensor; the historical ultrasound image dataset is input into the historical model to obtain the historical activation tensor. Based on the updated activation tensor and the historical activation tensor, a cost matrix is ​​obtained; based on the cost matrix, the parameters of the historical model are rearranged to obtain an aligned model. The parameters of the training model and the alignment model are fused using an effective fusion coefficient to obtain a fused model; the effective fusion coefficient is used to represent the weight ratio of the corresponding parameters of the alignment model and the training model. The training process of the fusion model is constrained by a preset cross-entropy loss function to obtain a trained ultrasound image learning model; the ultrasound image learning model is used for incremental learning of the identification of new lesion regions in ultrasound images.

[0008] As an optional implementation, in a first aspect of the invention, obtaining the cost matrix based on the updated activation tensor and the historical activation tensor includes: Based on the historical activation tensor, the historical discrete probability measure of neurons in the historical model is obtained; based on the updated activation tensor, the updated discrete probability measure of neurons in the training model is obtained. Based on the historical discrete probability measure and the updated discrete probability measure, the functional difference cost is obtained; based on the functional difference cost, the cost matrix is ​​obtained. The specific formula for calculating the cost of the functional difference is as follows:

[0009] In the above formula, For the historical model in the first The first in the network layer The neuron and the training model in the th... The first in the network layer The functional differences between individual neurons have generational value; ) represents the covariance. Standard deviation For the historical model in the first In the network layer, the _th The average discrete probability measure of a neuron To train the model in the first The first in the network layer The average discrete probability measure of a neuron It is a tiny constant.

[0010] As an optional implementation, in the first aspect of the present invention, rearranging the parameters of the historical model according to the cost matrix to obtain the aligned model includes: The cost matrix is ​​calculated using the Hungarian algorithm to obtain the optimal permutation matrix; the parameters of the historical model are rearranged using the optimal permutation matrix to obtain the aligned model.

[0011] As an optional implementation, in the first aspect of the present invention, fusing the parameters of the trained model and the parameters of the aligned model using effective fusion coefficients to obtain a fused model includes: By using effective fusion coefficients, the parameters of the trained model and the aligned model are fused to obtain fusion model coefficients; based on the fusion model coefficients, the fusion model is obtained. The effective fusion coefficient is obtained in the following way: The preset learnable fusion control variables are used through The function is mapped to obtain the effective fusion coefficients; the learnable fusion control variable is used to allow the model to automatically learn the optimal fusion ratio of new and old features based on the data. The fusion model coefficients are obtained using the following formula:

[0012] In the above formula, For training the model and aligning the model The coefficients of the fusion model for each network layer For training the model and aligning the model, the first The effective fusion coefficient of each network layer For the training model's first Parameters of each network layer For the alignment model's first The parameters of each network layer.

[0013] As an optional implementation, in the first aspect of the present invention, constraining the training process of the fusion model by a preset cross-entropy loss function to obtain a trained ultrasound image learning model includes: The historical ultrasound image dataset is input into the fusion model, and the cross-entropy surrogate loss function value is calculated using the cross-entropy loss function. The loss gradient is obtained based on the cross-entropy surrogate loss function value. The learnable fusion control variable is updated using the loss gradient to obtain the optimized learnable fusion control variable. Based on the optimized learnable fusion control variables, optimized fusion parameters are obtained; based on the optimized fusion parameters, an optimized model is obtained. The historical ultrasound image dataset is input into the optimization model for one forward propagation to obtain the ultrasound image learning model.

[0014] As an optional implementation, in the first aspect of the invention, the historical ultrasound image dataset is obtained in the following manner: A historical ultrasound image dataset is obtained through a replay buffer; the replay buffer is a storage space for storing the dataset used when the historical model was trained for historical tasks. And, the The specific formula for calculating the function is as follows:

[0015] In the above formula, For training the model and aligning the model, the first The effective fusion coefficient of each network layer For the first Learnable fusion control variables for each network layer.

[0016] A second aspect of this invention discloses a method for identifying new lesion regions in an image, the method comprising: When a second type of lesion region is identified, the ultrasound image learning model training method described in any of the first aspects of the present invention is used to obtain an updated ultrasound image learning model based on the second type of lesion region; the ultrasound image learning model is used to identify the second type of lesion region in the ultrasound image.

[0017] A third aspect of the present invention discloses an ultrasound image learning model training device, the device comprising: The image data acquisition module is used to acquire historical ultrasound image datasets and training ultrasound image datasets; the ultrasound image datasets include first-type lesion regions and second-type lesion regions. The model preliminary training module is used to train a preset historical model using the training ultrasound image dataset and the historical ultrasound image dataset to obtain a training model; the first type of lesion region is a lesion region that can be identified by the historical model, and the second type of lesion region is a lesion region that cannot be identified by the historical model. The model alignment training module is used to input the historical ultrasound image dataset into the training model to obtain the updated activation tensor; input the historical ultrasound image dataset into the historical model to obtain the historical activation tensor; obtain the cost matrix based on the updated activation tensor and the historical activation tensor; and rearrange the parameters of the historical model based on the cost matrix to obtain the alignment model. The model fusion training module is used to fuse the parameters of the training model and the alignment model through an effective fusion coefficient to obtain a fused model; the effective fusion coefficient is used to represent the weight ratio of the corresponding parameters of the alignment model and the training model; the training process of the fused model is constrained by a preset cross-entropy loss function to obtain a trained ultrasound image learning model; the ultrasound image learning model is used for incremental learning of the identification of new lesion regions in ultrasound images.

[0018] A fourth aspect of the present invention discloses an image new lesion region recognition device, the device comprising: The image new lesion region learning module is used to obtain an updated ultrasound image learning model based on the second type of lesion region when the existence of a second type of lesion region is determined. The ultrasound image learning model is used to identify the second type of lesion region in the ultrasound image.

[0019] A fifth aspect of the present invention discloses an apparatus comprising a memory and a processor, the apparatus comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the ultrasound image learning model training method disclosed in the first aspect of the present invention or the image new lesion region identification method disclosed in the second aspect of the present invention.

[0020] Compared with the prior art, the present invention has the following beneficial effects: This invention first acquires a training ultrasound image dataset and a historical ultrasound image dataset, trains a historical model, and obtains a training model; then inputs the historical ultrasound image dataset into the training model and the historical model to obtain an updated activation tensor and a historical activation tensor, thereby obtaining a cost matrix; based on the cost matrix, the parameters of the historical model are rearranged to obtain an aligned model; the parameters of the training model and the aligned model are fused using an effective fusion coefficient to obtain a fused model; finally, the training process of the fused model is constrained by a preset cross-entropy loss function to obtain an ultrasound image learning model.

[0021] By jointly training the model, we not only improve the learning ability of new tasks, but also provide functional features based on real sample responses for subsequent optimal transmission alignment, rather than relying solely on the parameter values ​​themselves; the subsequent alignment and fusion are more in line with the actual functional similarity of neurons, thus improving the reliability of model fusion.

[0022] By constructing a cost matrix, the problem of inconsistent parameter coordinates caused by the symmetry of neuron arrangement in deep neural networks can be eliminated. This avoids high loss barriers or feature destruction when directly interpolating unaligned parameters, enabling new and old models to have better compatibility in the parameter space and improving the reliability of the model.

[0023] By fusing the models, a fused model is obtained, which allows different network layers to automatically select a fusion ratio that is more biased towards the historical model or the current task model based on their own stability, thereby improving the model's learning ability. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a training method for an ultrasound image learning model disclosed in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for identifying new lesion regions in an image, as disclosed in an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of an ultrasound image learning model training device disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an image new lesion region recognition device disclosed in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a device including a memory and a processor disclosed in an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or system that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or systems.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] This invention provides a method and apparatus for training an ultrasound image learning model and identifying new lesion regions in images, which improves both the reliability and learning ability of the ultrasound image learning model. These are described in detail below.

[0030] Example 1 Please see Figure 1 , Figure 1 This is a schematic flowchart of an ultrasound image learning model training method disclosed in an embodiment of the present invention. Figure 1 The described ultrasound image learning model training method can be applied to an ultrasound image learning model training device, which can be integrated into a cloud server or a local server; this embodiment of the invention is not limited thereto. Figure 1 As shown, the ultrasound image learning model training method may include the following operations: Step 101: Obtain the historical ultrasound image dataset and the training ultrasound image dataset.

[0031] In this embodiment of the invention, the ultrasound image dataset may include a first type of lesion region and a second type of lesion region, or it may only include the first type of lesion region or only include the second type of lesion region. For example, a historical ultrasound image dataset may only include the first type of lesion region, or a historical ultrasound image dataset may include both the first and second types of lesion regions; a training ultrasound image dataset may only include the second type of lesion region, or a training ultrasound image dataset may include both the first and second types of lesion regions. The first type of lesion region is used to describe lesion regions that can be identified by the historical model, and the second type of lesion region is used to describe lesion regions that cannot be identified by the historical model.

[0032] Among them, the first type of lesion area and the second type of lesion area can be two different lesion areas; the first type of lesion area can be a thyroid nodule lesion area, a thyroid tumor lesion area, a breast nodule lesion area, a breast tumor lesion area, a kidney nodule lesion area, etc.; the second type of lesion area can also be a thyroid nodule lesion area, a thyroid tumor lesion area, a breast nodule lesion area, a breast tumor lesion area, a kidney nodule lesion area, etc.

[0033] In this embodiment of the invention, the historical ultrasound image dataset can be an ultrasound image dataset that has been learned by the model and can identify the first type of lesion area; the training ultrasound image dataset can be an ultrasound image dataset that has not been learned by the model and cannot identify the second type of lesion area.

[0034] Step 102: Train the preset historical model using the training ultrasound image dataset and the historical ultrasound image dataset to obtain the training model.

[0035] In this embodiment of the invention, the preset historical model architecture can be a Transformer architecture or a ResNet-18 architecture, etc., as the initial model architecture. The training process can be as follows: for the t-th task, the cached historical model... As initialization parameters, combined with the training ultrasound image dataset (i.e., the current task data) The model is trained using a standard gradient descent algorithm on a dataset of historical ultrasound images (i.e., historical data in the replay buffer M) and a historical ultrasound image dataset (i.e., historical data in the replay buffer M). This results in a highly plastic training model (i.e., the new model). This training model is used to retain the ability to identify first-type lesion regions, while learning and retaining the ability to identify second-type lesion regions.

[0036] The replay buffer, also known as the experience replay pool or memory buffer, is a core data module used in continuous learning (incremental learning) to mitigate catastrophic forgetting. It is a dedicated storage space used to retain training sample data from past tasks. When learning new tasks, it is mixed with current task data to participate in training, allowing the model to master new knowledge without losing the ability to learn from old tasks.

[0037] As those skilled in the art will understand, gradient descent is a parameter optimization algorithm in the field of deep learning. Essentially, it iteratively updates model parameters along the negative gradient direction of the loss function, allowing the loss function value to continuously decrease, and ultimately finding the parameter combination that optimizes the model's prediction performance.

[0038] Step 103: Input the historical ultrasound image dataset into the training model to obtain the updated activation tensor; input the historical ultrasound image dataset into the historical model to obtain the historical activation tensor; obtain the cost matrix based on the updated activation tensor and the historical activation tensor; rearrange the parameters of the historical model based on the cost matrix to obtain the aligned model.

[0039] In this embodiment of the invention, the activation tensor can be obtained by randomly acquiring a batch of historical ultrasound image datasets from the replay buffer M and inputting them into the historical model. and training model Extracting historical models and training model The Middle Flattened activation tensor of each network layer and ;in, It can be a historical activation tensor. This could be updating the activation tensor.

[0040] In this embodiment of the invention, the cost matrix can be obtained by first extracting the first activation from the historical activation tensor. We obtain a subtensor from all the data of each neuron; we flatten this subtensor to obtain a one-dimensional sequence of activation values; we normalize the activation values ​​to convert them into historical discrete probability measures. Then we perform the same operation on the updated activation tensor to obtain the updated discrete probability measure.

[0041] Then this The discrete probability measures of all neurons in each network layer are calculated to determine the historical model. and training model No. The average historical discrete probability measure and the average update discrete probability measure of each network layer are used to calculate the historical model. and training model The covariance and standard deviation between the two are used to calculate the historical model. and training model The different functional differences in cost are calculated; finally, the calculated functional difference cost is arranged in rows and columns to form a two-dimensional matrix, which is the... The cost matrix of each network layer.

[0042] Furthermore, the cost matrix can also be obtained by extracting the activation vectors of each neuron or channel in the same network layer on the surrogate sample from the historical activation tensor, and extracting the activation vectors of each neuron or channel in the corresponding network layer of the training model on the surrogate sample from the updated activation tensor; treating each neuron or channel in the same network layer as a discrete point in the optimal transmission matching, and assigning the same weight to each neuron or channel in the same layer; using the activation vector of each neuron or channel on the surrogate sample as the functional representation of the corresponding discrete point; based on the historical model, the cost matrix can be obtained by extracting the activation vectors of each neuron or channel on the surrogate sample from the historical activation tensor as a functional representation of the corresponding discrete point; and by extracting the activation vectors of each neuron or channel on the surrogate sample from the historical activation tensor as a functional representation of the corresponding discrete point. The activation vector of the nth neuron or channel is related to the nth neuron or channel in the training model. The Pearson correlation coefficient between the activation vectors of each neuron or channel is used to calculate the functional difference cost between them. The functional difference costs of each neuron or channel in the same network layer are arranged in rows and columns to obtain the cost matrix of the network layer.

[0043] In this embodiment of the invention, the process of rearranging the parameters of the historical model can be as follows: the cost matrix is ​​calculated using the Hungarian algorithm to obtain the optimal permutation matrix; the parameters of the historical model are then rearranged using the optimal permutation matrix to obtain an aligned model. This aligned model is used to adjust the parameter coordinate system of the historical model so that its position is consistent with the corresponding position of the parameter coordinate system of the training model.

[0044] Those skilled in the art will understand that the Hungarian algorithm is a combinatorial optimization algorithm for finding the minimum weight perfect matching in a bipartite graph. In this technical solution, it is executed by inputting the first... The cost matrix of each network layer outputs the historical model. and training model Finding the correspondence with the minimum total matching cost between neurons, that is, finding the globally optimal "pairing of neurons with the most similar functions", will enable precise alignment for subsequent parameter / feature fusion.

[0045] Furthermore, after calculating the optimal permutation matrix using the Hungarian algorithm, the order of neurons (channels) in the training model is used as a basis to reorder the historical model's... The parameters of each network layer are reordered according to their functional matching relationship, so that the neurons of the historical model and the training model at the same index position after the reordering correspond one-to-one. This ensures that the subsequent layer-by-layer parameter weighted fusion is a fusion of parameters with the same function, rather than a mixture of random positions. The reordering only adjusts the order of the parameters and does not modify the parameter values, so it does not change the function of the historical model or the performance of old knowledge.

[0046] Step 104: Fuse the parameters of the training model and the alignment model using effective fusion coefficients to obtain a fused model; constrain the training process of the fused model using a preset cross-entropy loss function to obtain a trained ultrasound image learning model.

[0047] In this embodiment of the invention, the ultrasound image learning model is used to incrementally learn the ability to identify unidentified lesion areas in ultrasound images.

[0048] In this embodiment of the invention, the effective fusion coefficient is used to represent the weight ratio of the output features of the alignment model and the training model; the ultrasound image learning model is used to incrementally learn the identification of new lesion regions in ultrasound images.

[0049] In this embodiment of the invention, the fusion model can be obtained by fusing the parameters of the training model and the alignment model using preset effective fusion coefficients to obtain fusion model coefficients; and by obtaining the fusion model based on the fusion model coefficients. The effective fusion coefficients can be used to represent the weight ratio of the parameters of the alignment model and the parameters of the training model.

[0050] The effective fusion coefficients can be obtained by mapping a pre-defined learnable fusion control variable using the Sigmoid function. The learnable fusion control variable is used to allow the model to automatically learn the optimal fusion ratio of new and old features based on the data. It is not the original neural network's convolutional kernel, fully connected weights, or BN parameters, but rather a low-dimensional fusion control parameter introduced additionally during the model fusion stage. It allows the model to automatically learn the optimal fusion ratio of new and old features for that layer based on the data. The learnable fusion control variable itself can take any real value and does not need to be manually restricted to between 0 and 1.

[0051] The specific formula for calculating the Sigmoid function is as follows:

[0052] In the above formula, For the first The effective fusion coefficient of each network layer For the first Learnable fusion control variables for each network layer. Mapped lie in Within the range, it can be used as the first Effective fusion coefficients for each network layer. Initialization. hour, This indicates that initially, the historical model and the trained model each account for half of the parameters. The Sigmoid function is used to transform learnable fusion control variables into computable fusion proportions.

[0053] In this embodiment of the invention, those skilled in the art will understand that the cross-entropy loss function is used to measure the difference between the model's predicted probability distribution and the true label distribution. The larger the difference, the higher the loss value. During training, the model will update its parameters in reverse to reduce this loss.

[0054] Furthermore, the process of constraining the training process of the fusion model using a preset cross-entropy loss function can be as follows: Input the historical ultrasound image dataset into the fusion model, calculate the cross-entropy surrogate loss function value using the cross-entropy loss function, and then obtain the loss gradient based on the cross-entropy surrogate loss function value. Freeze the large-scale basic network parameters of the training model and the alignment model, and use the chain rule to strictly backpropagate the loss gradient to the low-dimensional learnable fusion control variables, updating the learnable fusion control variables to obtain optimized learnable fusion control variables. Freeze the basic network parameters of the training model and the alignment model, and update the learnable fusion control variables only based on the loss gradient. Map the optimized learnable fusion control variables using the Sigmoid function to obtain optimized fusion parameters. Apply the converged optimized fusion parameters to fuse the model to obtain the final optimized model for the current task. Finally, set the fused model to training mode, input the historical ultrasound image dataset into the optimized model for a forward propagation, and force the BN layer to re-estimate the moving mean and variance to repair the activation variance collapse caused by model interpolation fusion, thus obtaining the trained ultrasound image learning model.

[0055] As can be seen, in this embodiment of the invention, the training model is obtained by first acquiring a training ultrasound image dataset and a historical ultrasound image dataset, training a historical model, and then inputting the historical ultrasound image dataset into the training model and the historical model to obtain the updated activation tensor and the historical activation tensor, thereby obtaining the cost matrix; the parameters of the historical model are rearranged according to the cost matrix to obtain the aligned model; the parameters of the training model and the aligned model are fused using an effective fusion coefficient to obtain the fused model; finally, the training process of the fused model is constrained by a preset cross-entropy loss function to obtain the ultrasound image learning model.

[0056] By jointly training the model, we not only improve the learning ability of new tasks, but also provide functional features based on real sample responses for subsequent optimal transmission alignment, rather than relying solely on the parameter values ​​themselves; the subsequent alignment and fusion are more in line with the actual functional similarity of neurons, thus improving the reliability of model fusion.

[0057] By constructing a cost matrix, the problem of inconsistent parameter coordinates caused by the symmetry of neuron arrangement in deep neural networks can be eliminated. This avoids high loss barriers or feature destruction when directly interpolating unaligned parameters, enabling new and old models to have better compatibility in the parameter space and improving the reliability of the model.

[0058] By fusing the models, a fused model is obtained, which allows different network layers to automatically select a fusion ratio that is more biased towards the historical model or the current task model based on their own stability, thereby improving the model's learning ability.

[0059] In an optional embodiment, the cost matrix obtained based on the updated activation tensor and the historical activation tensor may include: Based on the historical activation tensor, the historical discrete probability measure of neurons in the historical model is obtained; based on the updated activation tensor, the updated discrete probability measure of neurons in the trained model is obtained. Based on the historical discrete probability measure and the updated discrete probability measure, the cost value of functional differences is obtained; based on the cost value of functional differences, the cost matrix is ​​obtained.

[0060] In this optional embodiment, the activation tensor can be obtained by randomly acquiring N historical ultrasound image datasets from the replay buffer M and inputting them into the historical model. and training model Extracting historical models and training model The Middle Flattened activation tensor of each network layer and ;in, It can be the historical model number. The historical activation tensor of each network layer It can be the training model number. The update activation tensor of each network layer.

[0061] In this optional embodiment, the cost matrix can be obtained by first extracting the first activation from the historical activation tensor. We obtain a subtensor from all the data of each neuron; we flatten this subtensor to obtain a one-dimensional sequence of activation values; we normalize the activation values ​​to convert them into historical discrete probability measures. Then we perform the same operation on the updated activation tensor to obtain the updated discrete probability measure.

[0062] Then the first The discrete probability measures of all neurons in each network layer are calculated to determine the historical model. No. Average historical discrete probability measure and training model of each network layer No. The average update discrete probability measure of each network layer; calculate the historical model. and training model The covariance and standard deviation between the two are used to calculate the historical model. and training model Between The cost difference between different functions of each network layer; and the cost difference is used to construct the cost matrix, thereby calculating the optimal permutation matrix, which can be used to transform the historical model. The output channels, input channels, and associated parameters of the corresponding layers are reordered for consistency to align the trained model. This reduces the barrier to feature destruction, performance degradation, and interpolation loss caused by neuron misalignment. The two-dimensional matrix formed by arranging the calculated functional difference costs in rows and columns is the [missing information]. The cost matrix of each network layer is calculated. Finally, the cost matrix of the entire model is calculated by calculating the cost matrix of all network layers. The cost matrices of the entire model are independent of each other, and each cost matrix is ​​processed separately for the network layer to which it belongs.

[0063] Furthermore, the cost matrix can also be obtained by extracting the activation vectors of each neuron or channel in the same network layer on the surrogate sample from the historical activation tensor, and extracting the activation vectors of each neuron or channel in the corresponding network layer of the training model on the surrogate sample from the updated activation tensor; treating each neuron or channel in the same network layer as a discrete point in the optimal transmission matching, and assigning the same weight to each neuron or channel in the same layer; using the activation vector of each neuron or channel on the surrogate sample as the functional representation of the corresponding discrete point; based on the historical model, the cost matrix can be obtained by extracting the activation vectors of each neuron or channel on the surrogate sample from the historical activation tensor as a functional representation of the corresponding discrete point; and by extracting the activation vectors of each neuron or channel on the surrogate sample from the historical activation tensor as a functional representation of the corresponding discrete point. The activation vector of the nth neuron or channel is related to the nth neuron or channel in the training model. The Pearson correlation coefficient between the activation vectors of each neuron or channel is used to calculate the functional difference cost between them. The functional difference costs of each neuron or channel in the same network layer are arranged in rows and columns to obtain the cost matrix of the network layer.

[0064] For example, by randomly obtaining N historical ultrasound image datasets (where N is a non-zero positive integer) from the replay buffer M, the original activation value sequence of the historical activation tensor is calculated as follows: Normalize the activation values ​​to convert them into historical discrete probability measures. Similarly, we can obtain the training model's... In the network layer, the _th Update discrete probability measure of each neuron ; Then calculate the average of the probability measure sequence:

[0065] in, It is the average of historical discrete probability measures. To update the average of the discrete probability measure; Then calculate the covariance. This is used to measure the direction and degree of the interaction between two variables:

[0066] Calculate the standard deviation It is used to measure the fluctuation range of a variable itself:

[0067]

[0068] In the above formula, The standard deviation is the corresponding historical model. This represents the standard deviation corresponding to the training model.

[0069] Finally, the specific formula for calculating the cost of functional differences is as follows:

[0070] In the above formula, For the historical model in the first The first in the network layer The neuron and the training model in the th... The first in the network layer The functional differences between individual neurons have generational value; ) represents the covariance. Standard deviation For the historical model in the first In the network layer, the _th The average discrete probability measure of a neuron To train the model in the first The first in the network layer The average discrete probability measure of a neuron It is a tiny constant.

[0071] By sorting all the calculated functional difference costs, we obtain the [number]th [unit]. The cost matrix of each network layer is calculated, and then the same method is used to calculate the cost matrix of the entire model.

[0072] As can be seen, in this optional embodiment, by calculating the historical discrete probability measure of neurons in the historical model and the updated discrete probability measure of neurons in the training model, the functional difference cost between the historical model and the training model can be calculated, thereby constructing a cost matrix. By constructing the cost matrix, the optimal permutation matrix can be calculated using the Hungarian algorithm, thus aligning the parameters of the historical model with the parameters of the training model. This reduces feature destruction, performance degradation, and interpolation loss barriers caused by neuron misalignment, thereby improving the reliability of the model.

[0073] In another alternative embodiment, the parameters of the historical model are rearranged according to the cost matrix to obtain an aligned model, including: The cost matrix is ​​calculated using the Hungarian algorithm to obtain the optimal permutation matrix; the parameters of the historical model are rearranged using the optimal permutation matrix to obtain the aligned model.

[0074] In this optional embodiment, those skilled in the art will understand that the Hungarian algorithm is a combinatorial optimization algorithm for finding the minimum weight perfect matching in a bipartite graph. In this technical solution, it is executed by inputting the first... The cost matrix of each network layer outputs the historical model. and training model The correspondence with the minimum total matching cost between neurons is to find the globally optimal "pairing of the most functionally similar neurons" scheme (that is, the optimal permutation matrix), which is used for precise alignment in subsequent parameter / feature fusion.

[0075] In this optional embodiment, the obtained optimal permutation matrix can be used to refine the historical model. The output channels, input channels, and associated parameters of the corresponding layers are reordered in a consistent manner (to align the training model). ), to obtain the alignment model after geometric alignment Adjust the coordinate system of the historical model to match that of the training model. Use a coordinate system that is as consistent as possible. This way, when performing linear fusion later, parameters at the same location are more likely to represent the same or similar functional features, thereby reducing feature destruction, performance degradation, and interpolation loss barriers caused by misaligned neuron arrangement.

[0076] In this optional embodiment, the process of rearranging the parameters of the historical model using the optimal permutation matrix can be as follows: based on the order of neurons (channels) in the new model, rearrange the parameters of the historical model... The parameters of each network layer are reordered according to their functional matching relationships, ensuring that the functions of new and old neurons at the same index position correspond one-to-one after the reordering. This guarantees that subsequent layer-by-layer parameter weighted fusion is a "fusion of parameters with the same function," rather than a mixture of random positions. The reordering only adjusts the order of parameters, without modifying their values, so it does not change the functionality of the historical model or the performance of old knowledge.

[0077] As can be seen, in this optional embodiment, the optimal permutation matrix is ​​calculated using the Hungarian algorithm, and then the neuron parameters of the historical model are rearranged and aligned using the optimal permutation matrix to obtain the aligned model. This ensures that the functions of the old and new neurons at the same index position correspond one-to-one after rearrangement, guaranteeing that subsequent layer-by-layer parameter weighting fusion is a "fusion of parameters with the same function," rather than a mixture of random positions, thereby improving the reliability of the model.

[0078] In another optional embodiment, the parameters of the trained model and the parameters of the aligned model are fused using effective fusion coefficients to obtain a fused model, including: By using effective fusion coefficients, the parameters of the trained model and the aligned model are fused to obtain fusion model coefficients; based on the fusion model coefficients, the fusion model is obtained.

[0079] In this optional embodiment, the effective fusion coefficient can be obtained in the following way: The preset learnable fusion control variables are used through The function is mapped to obtain effective fusion coefficients; while learnable fusion control variables can be used to allow the model to automatically learn the optimal fusion ratio of new and old features based on the data. It is aimed at the network. The learnable auxiliary variables set for each network layer can be set on each neural network layer of the aligned model, each neural network layer of the trained model, or each neural network layer of both the aligned and trained models. It is not the convolutional kernel, fully connected weights, or BN parameters of the original neural network, but a low-dimensional fusion control parameter introduced in the model fusion stage, which allows the model to automatically learn the optimal fusion ratio of new and old features for that layer based on the data. It can take any real value and does not need to be manually restricted to between 0 and 1.

[0080] What those skilled in the art will understand is that The function is a sigmoid nonlinear activation function in deep learning. Its core function is to compress any real number input into an open interval (0,1).

[0081] In this optional embodiment, The specific formula for calculating the function is as follows:

[0082] In the above formula, For training the model and aligning the model, the first The effective fusion coefficient of each network layer For the first Learnable fusion control variables for each network layer. Initialization. hour, This indicates that initially, the historical model and the training model each account for half of the parameters.

[0083] In this optional embodiment, the fusion model coefficients can be obtained using the following formula:

[0084] In the above formula, To train the model and align the model in the first The coefficients of the fusion model for each network layer For training the model and aligning the model, the first The effective fusion coefficient of each network layer For the training model's first Parameters of each network layer For the alignment model's first The parameters of each network layer.

[0085] As can be seen, in this optional embodiment, the preset learnable fusion control variables are used... The function is mapped to obtain effective fusion coefficients; the effective fusion coefficients are used to fuse the parameters of the training model and the alignment model to obtain fusion model coefficients, and finally the fusion model is obtained; by setting learnable fusion control variables, the fusion model coefficients obtained avoid the "one-size-fits-all" problem of using a globally fixed fusion ratio in traditional model fusion, so that different network layers can automatically select a fusion ratio that is more biased towards the historical model or the training model according to their own stability and plasticity requirements. That is, shallow layers usually need more stability to retain general low-level features, while deep layers usually need more plasticity to adapt to new task semantics. Layer-by-layer adaptive fusion can better meet the hierarchical heterogeneity of deep networks and improve the model's ability to learn new things.

[0086] In another optional embodiment, the training process of the fusion model is constrained by a preset cross-entropy loss function to obtain a trained ultrasound image learning model, including: The historical ultrasound image dataset is input into the fusion model, and the cross-entropy surrogate loss function value is calculated using the cross-entropy loss function. The loss gradient is obtained based on the cross-entropy surrogate loss function value. The learnable fusion control variables are updated using the loss gradient to obtain the optimized learnable fusion control variables. Based on the optimized learnable fusion control variables, the optimized fusion parameters are obtained; based on the optimized fusion parameters, the optimized model is obtained. The historical ultrasound image dataset is input into the optimization model for one forward propagation to obtain the ultrasound image learning model.

[0087] In this optional embodiment, those skilled in the art will understand that the cross-entropy loss function is used to quantify the difference between the predicted probability distribution of the model output and the true label distribution, and is the core optimization objective of the gradient descent algorithm.

[0088] In this optional embodiment, the formula for calculating the cross-entropy proxy loss function value can be:

[0089] In the above formula, The cross-entropy proxy loss function value is used for the replay buffer M; The mathematical expectation operator represents the calculation of the average loss of a single sample by randomly sampling samples from the replay buffer M. Representative input sample Add real tags The old task data distribution follows the buffer M; ( The loss function is the single-sample loss function, also known as the cross-entropy loss function. For the fusion model to input The predicted output; the model parameters are the fused parameters. ,and Learnable fusion control variables generate.

[0090] In this optional embodiment, the specific formula for calculating the loss gradient based on the cross-entropy proxy loss function value can be as follows: =

[0091] In the above formula, For the fusion model in the first The loss gradient of each network layer For training the model and aligning the model, the first The effective fusion coefficient of each network layer For training the model and aligning the model, the first Learnable fusion control variables for each network layer The cross-entropy proxy loss function value, To train the model and align the model in the first The first network layer The fusion model coefficients of individual neurons; To align the model in the first The first network layer The parameters of each neuron; To train the model in the first The first network layer The parameters of each neuron; This parameter represents the difference between the aligned model and the trained model. The total loss gradient is obtained by calculating the gradients of all layers. The algorithm uses a chain rule to backpropagate the loss change of the fusion model in the buffer to the learnable fusion control variable of each layer. This allows the system to automatically determine whether increasing or decreasing the weights of the historical model at that layer is beneficial for reducing the proxy loss, thus enabling the use of optimizers such as Adam to automatically update the data. .

[0092] In this optional embodiment, the specific process of updating the learnable fusion control variable using the loss gradient to obtain the optimized learnable fusion control variable is as follows: The goal of this approach is not to retrain the entire neural network, but rather to align the model. With the training model Between two fixed endpoints, the optimal fusion ratio for each layer is learned. Therefore, the alignment model is performed during the fusion phase. and training model Treating them as fixed constants, only optimizing learnable fusion control variables; transforming the original network optimization problem containing millions or even tens of millions of parameters into one containing only... The low-dimensional optimization problem of multi-level fusion parameters, among which This refers to the number of network functional layers or modules. This significantly reduces memory usage and computational overhead, and avoids reintroducing parameter-level gradient conflicts between current task data and historical data during the fusion phase. This allows the proposed solution to be used as a post-hoc module in existing continuous learning methods such as ER, DER++, and ER-ACE without altering their training processes. The formula for calculating the optimized learnable fusion control variable is:

[0093] In the above formula, To optimize learnable fusion control variables, The cross-entropy proxy loss function value; The learnable fusion control variables belong to the L-dimensional real space; using the previously derived gradient loss formula, backpropagation is used to calculate each learnable fusion control variable. The gradient loss is used to progressively update the parameters until the loss converges, ultimately yielding an optimization that can learnably fuse control variables to obtain an approximate optimal solution. By updating the fusion control variables in reverse, it is possible to learn them without saving or re-accessing all historical task data. The existing replay buffer can be used to provide optimization signals for the fusion coefficients. By minimizing the proxy loss, the system can automatically determine whether each layer should retain more historical model parameters or adopt more current task model parameters, thus achieving an adaptive balance between stability and adaptability.

[0094] In this optional embodiment, after obtaining the optimized learnable fusion control variables, the optimized fusion parameters are calculated using the fusion model coefficient formula mentioned above, and the aligned model and the training model are fused to obtain the optimized model.

[0095] In this optional embodiment, inputting the historical ultrasound image dataset into the optimization model for a forward propagation can be done by setting the fused optimization model to training mode, extracting the already trained historical ultrasound image dataset from the input buffer M, performing a forward propagation, and forcing the BN layer to re-estimate the moving mean and variance, thereby repairing the activation variance collapse phenomenon caused by model interpolation fusion, and finally obtaining the trained ultrasound image learning model.

[0096] Furthermore, since parameter interpolation can cause activation distribution shifts or activation variance collapse, re-estimating the BN statistics can match the internal statistics of the fused model with the fused parameters, thereby correcting the activation distribution of the fused network and improving the inference stability of the final model. This solution only requires caching one additional copy of the historical model. Furthermore, the historical model is updated to the current fusion model after each task is completed. Therefore, its structured storage overhead does not increase linearly with the number of tasks.

[0097] As can be seen, in this optional embodiment, the loss gradient is obtained by calculating the cross-entropy proxy loss function value, and the learnable fusion control variable is updated using the loss gradient value. This does not require saving or re-accessing all historical task data, thereby improving the model's learning ability and speed. By performing another forward propagation on the optimized model, the BN statistic can be re-estimated so that the internal statistics of the fused model match the fusion parameters, thereby correcting the activation distribution of the fused network and improving the reliability of the model.

[0098] In yet another optional embodiment, the historical ultrasound image dataset is obtained in the following manner: The historical ultrasound image dataset is obtained through the replay buffer; the replay buffer is the storage space for storing the dataset used when the historical model was trained for historical tasks. as well as, The specific formula for calculating the function is as follows:

[0099] In the above formula, For training the model and aligning the model, the first The effective fusion coefficient of each network layer For the first Learnable fusion control variables for each network layer.

[0100] In this optional embodiment, the replay buffer can be a dedicated storage space used to retain some training samples from past tasks. These samples are then mixed with current task data during training to learn new tasks, allowing the model to acquire new knowledge without losing its ability to perform older tasks. After the training ultrasound image dataset has been trained, the trained ultrasound image dataset enters the replay buffer, becoming the historical ultrasound image dataset.

[0101] As can be seen, in this optional embodiment, by setting a replay buffer, the model can learn new knowledge without losing the ability to perform old tasks, thereby improving the model's learning ability.

[0102] Example 2 Please see Figure 2 , Figure 2 This is a schematic flowchart of a method for identifying new lesion regions in an image, as disclosed in an embodiment of the present invention. Figure 2 The described method for identifying new lesion regions in images can be applied to a device for identifying new lesion regions in images. This device can be integrated into a cloud server or a local server; the embodiments of this invention are not limited thereto. Figure 2 As shown, the method for identifying new lesion regions in an image may include the following operations: Step 201: When a second type of lesion region is identified, the ultrasound image learning model training method is used to obtain an updated ultrasound image learning model based on the second type of lesion region.

[0103] In this embodiment of the invention, the ultrasound image learning model training method may be any of the ultrasound image learning model training methods in Embodiment 1 of the invention; the ultrasound image learning model may be used to identify the second type of lesion region in the ultrasound image.

[0104] In this embodiment of the invention, the first type of lesion area can be a lesion area that has been identified in the ultrasound image, and the second type of lesion area can be a lesion area that has not been identified in the ultrasound image; the first type of lesion area and the second type of lesion area are different from each other. For example, the first type of lesion area can be a thyroid nodule area, and the second type of lesion area can be a breast nodule lesion area.

[0105] In this embodiment of the invention, when it is determined that there is no unidentified second type of lesion area and only an ultrasound image of the identified first type of lesion area exists, the existing ultrasound image learning model is directly used to identify the first type of lesion area.

[0106] As can be seen, in this embodiment of the invention, when it is determined that there is an unidentified second type of lesion area, the ultrasound image learning model training method is used to identify and train the second type of lesion area, thereby obtaining an ultrasound image learning model that can identify the second lesion area in the ultrasound image, and improving the learning compatibility of the model.

[0107] Example 3 Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an ultrasound image learning model training device disclosed in an embodiment of the present invention. The device may include: The image data acquisition module 301 is used to acquire historical ultrasound image datasets and training ultrasound image datasets; the ultrasound image datasets include first-class lesion regions and second-class lesion regions.

[0108] The model preliminary training module 302 is used to train a preset historical model by using a training ultrasound image dataset and a historical ultrasound image dataset to obtain a training model; the first type of lesion region is the lesion region that can be identified by the historical model, and the second type of lesion region is the lesion region that cannot be identified by the historical model.

[0109] The model alignment training module 303 is used to input the historical ultrasound image dataset into the training model to obtain the updated activation tensor; input the historical ultrasound image dataset into the historical model to obtain the historical activation tensor; obtain the cost matrix based on the updated activation tensor and the historical activation tensor; and rearrange the parameters of the historical model based on the cost matrix to obtain the aligned model.

[0110] The model fusion training module 304 is used to fuse the parameters of the training model and the alignment model through an effective fusion coefficient to obtain a fused model. The effective fusion coefficient is used to represent the weight ratio of the corresponding parameters of the alignment model and the training model. The training process of the fused model is constrained by a preset cross-entropy loss function to obtain a trained ultrasound image learning model. The ultrasound image learning model is used to incrementally learn the recognition of new lesion regions in ultrasound images.

[0111] As can be seen, in this embodiment of the invention, the training model is obtained by first acquiring a training ultrasound image dataset and a historical ultrasound image dataset, training a historical model, and then inputting the historical ultrasound image dataset into the training model and the historical model to obtain the updated activation tensor and the historical activation tensor, thereby obtaining the cost matrix; the parameters of the historical model are rearranged according to the cost matrix to obtain the aligned model; the parameters of the training model and the aligned model are fused using an effective fusion coefficient to obtain the fused model; finally, the training process of the fused model is constrained by a preset cross-entropy loss function to obtain the ultrasound image learning model.

[0112] By jointly training the model, we not only improve the learning ability of new tasks, but also provide functional features based on real sample responses for subsequent optimal transmission alignment, rather than relying solely on the parameter values ​​themselves; the subsequent alignment and fusion are more in line with the actual functional similarity of neurons, thus improving the reliability of model fusion.

[0113] By constructing a cost matrix, the problem of inconsistent parameter coordinates caused by the symmetry of neuron arrangement in deep neural networks can be eliminated. This avoids high loss barriers or feature destruction when directly interpolating unaligned parameters, enabling new and old models to have better compatibility in the parameter space and improving the reliability of the model.

[0114] By fusing the models, a fused model is obtained, which allows different network layers to automatically select a fusion ratio that is more biased towards the historical model or the current task model based on their own stability, thereby improving the model's learning ability.

[0115] Example 4 Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an image new lesion region identification device disclosed in an embodiment of the present invention. The device may include: The image new lesion region learning module 401 is used to obtain an updated ultrasound image learning model based on the second type of lesion region when the existence of a second type of lesion region is determined.

[0116] As can be seen, in this embodiment of the invention, when it is determined that there is an unidentified second type of lesion area, the ultrasound image learning model training method is used to identify and train the second type of lesion area, thereby obtaining an ultrasound image learning model that can identify the second lesion area in the ultrasound image, and improving the learning compatibility of the model.

[0117] Example 5 Please see Figure 5 , Figure 5 This is a schematic diagram of a device including a memory and a processor, as disclosed in an embodiment of the present invention. Figure 5 As shown, the device may include: Memory 501 storing executable program code; Processor 502 coupled to memory 501; The processor 502 calls the executable program code stored in the memory 501 to execute some or all of the steps in the ultrasound image learning model training method described in Embodiment 1 of the present invention or the image new lesion area identification method described in Embodiment 2.

[0118] Example 6 This invention applies the ultrasound image learning model training method described in Embodiment 1 to an adaptive layer-by-layer model fusion method for continuous learning. The specific process of this method is as follows: In existing technologies, model fusion has the following drawbacks: (1) Plasticity bottleneck of Experience Replay (ER): In continuous learning, in order to alleviate catastrophic forgetting, most existing mainstream methods adopt experience replay technology, that is, to maintain a buffer in memory to store historical task samples and perform joint training when learning new tasks. However, when the storage budget increases, serious parameter-level gradient conflicts will occur between new task data and historical data, which will limit the optimization trajectory of the model. At this time, although the stability of the model is improved, the ability to learn new knowledge (plasticity) is significantly reduced, resulting in a plasticity bottleneck.

[0119] (2) Limitations of existing model merging methods: To bypass gradient conflicts in the data space, existing techniques have begun to attempt to perform model merging directly in the parameter space (e.g., interpolating historical models with new models). However, these methods generally suffer from two major drawbacks: First, deep neural networks typically exhibit permutation symmetries, and direct interpolation can lead to severe feature destruction; second, existing merging methods (such as Git-Re-Basin and CLEWI) mostly use globally uniform fixed scalar parameters for merging. This "one-size-fits-all" approach ignores the unique hierarchical characteristics of deep neural networks: shallow networks may require high stability to preserve the generalization features of the lower layers, while deep networks may require high plasticity to adapt to the semantics of new tasks. Furthermore, manually using heuristic search to find the optimal merging ratio layer by layer is computationally infeasible.

[0120] The core inventive point (innovation) of this invention lies in proposing a post-hoc parameter space intervention architecture, specifically including: (1) Geometric alignment mechanism based on optimal transmission: Before model fusion, the activation features of the new and old models are extracted, a cost matrix based on Pearson correlation coefficient is constructed, and the optimal transmission allocation is solved to align the neurons of the new and old models and eliminate the interpolation barrier caused by the arrangement symmetry.

[0121] (2) Gradient-driven layer-by-layer differentiable fusion mechanism: This breaks the limitation of traditional fusion using a global fixed scalar. This invention introduces layer-by-layer learnable parameters with extremely low dimensionality (only the same as the number of network layers), uses the data in the replay buffer as a surrogate distribution, and optimizes the fusion coefficients of each layer end-to-end through backpropagation.

[0122] (3) The scheme successfully decouples the "stability-plasticity balance" and the "parameter-level gradient conflict", and realizes the adaptive exploration of the characteristics of each layer of the network with minimal computational overhead.

[0123] The overall technical solution of this invention consists of three main processing stages: Step 1: Training the current task model and extracting proxy features For the t-th task, using the cached historical model To initialize parameters, combine current task data. Using historical data in the replay buffer M, a new model with high plasticity is trained through standard gradient descent algorithm. Then, a batch of data is randomly sampled from the replay buffer M and input into the historical model. and new models Extract the first one from the middle. Flattening activation tensor of the layer and Regarding the first For each task, we will first follow the standard continuous learning process and utilize the data from the current task. With replay buffer The current task model is obtained by training on historical samples. This allows them to fully learn the knowledge of the new task. Simultaneously, from the replay buffer... A batch of proxy samples were sampled and input into the historical model. Compared with the current task model This process extracts activation features from each layer, providing a basis for subsequently determining the functional correspondence between neurons or channels in the old and new models. This step retains the learning ability of conventional experience replay methods for new tasks while providing functional features based on real sample responses for subsequent optimal transmission alignment, rather than relying solely on parameter values. Therefore, subsequent alignment and fusion are more consistent with the actual functional similarity of neurons, improving the reliability of model fusion.

[0124] Step 2: Feature geometry alignment based on optimal transmission (1) Constructing the cost matrix: Treating neurons as discrete probability measures, and calculating the neuron's cost matrix based on the scaling-invariant Pearson correlation coefficient. With neurons Cost of functional differences between them:

[0125] in, The historical model is represented by the first Layer The activation vector of a neuron or channel on a surrogate sample. Indicates the current task model's... Layer The activation vector of a neuron or channel on a surrogate sample. To prevent small constants from being divided by zero, for( Covariance ( The standard deviation is denoted as . This step eliminates the inconsistency in parameter coordinates caused by the symmetry of neuron arrangement in deep neural networks, avoids high loss barriers or feature destruction when directly interpolating unaligned parameters, and enables better fusion of old and new models in the parameter space.

[0126] (2) Solving for the optimal permutation matrix: The above matrix... As input to the Hungarian algorithm, the optimal permutation matrix that minimizes the total transmission cost is solved. .

[0127] (3) Parameter rearrangement: using Historical models The corresponding layer parameters are rearranged to obtain the geometrically aligned historical model. .

[0128] Step 3: Layer-by-layer differentiable adaptive fusion (1) Parametric fusion coefficients: for each functional layer of the network Define an unbounded real scalar And it is mapped to the effective fusion range of (0,1) using the Sigmoid function: =Sigmoid Here, "unbounded real scalar" refers to the scalar with respect to the network's first... Each functional layer has learnable auxiliary variables. These are not the convolutional kernels, fully connected weights, or batch normalization (BN) parameters of the original neural network, but rather low-dimensional fusion control parameters introduced additionally during the model fusion stage. These parameters allow the model to automatically learn the optimal ratio of new to old features fusion for that layer based on the data. (2) Constructing a fusion model representation: The first The parameters after layer fusion are explicitly expressed as: +

[0129] This step avoids the "one-size-fits-all" problem of using a globally fixed fusion ratio in traditional model fusion, allowing different network layers to automatically select a fusion ratio that favors the historical model or the current task model based on their own stability-plasticity requirements. Shallow layers typically require more stability to retain general low-level features, while deeper layers typically require more plasticity to adapt to new task semantics. Layer-by-layer adaptive fusion better suits the hierarchical heterogeneity of deep networks. In the "proxy goal optimization" and "calculating and updating low-dimensional gradients," this scheme updates each layer separately through backpropagation based on the proxy loss on the replay buffer. Therefore, the fusion ratio of different layers is not manually specified, nor is it globally uniform, but is automatically learned by the model based on the proxy data.

[0130] (3) Proxy goal optimization: Treat the replay buffer M as a globally jointly distributed proxy. Input the data from M into the fusion model and calculate the cross-entropy proxy loss. Furthermore, the optimal layer-by-layer fusion parameters are:

[0131] In the above formula, The cross-entropy proxy loss function value is used for the replay buffer M; The mathematical expectation operator represents the calculation of the average loss of a single sample by randomly sampling samples from the replay buffer M. Representative input sample Add real tags The old task data distribution follows the buffer M; ( The loss function is the single-sample loss function, also known as the cross-entropy loss function. For the fusion model to input The predicted output; the model parameters are the fused parameters. ,and Learnable fusion control variables generate.

[0132] This step does not require saving or re-accessing all historical task data; it only needs to utilize the existing replay buffer to provide optimization signals for the fusion coefficients. By minimizing the proxy loss, the system can automatically determine whether each layer should retain more historical model parameters or adopt more current task model parameters, thereby achieving an adaptive balance between stability and flexibility.

[0133] (4) Calculate and update the low-dimensional gradient: freeze the massive base network weights. and Using the chain rule, the gradient of the loss is rigorously backpropagated to the low-dimensional learnable vector w: =

[0134] Use an optimizer (such as Adam) to perform fast iterative updates on w with a small number of epochs.

[0135] (5) Apply the fusion weight parameters and output: Apply the converged fusion weight parameters. The fusion model yields the final output model for the current task. .

[0136] Step 4: Batch Normalization (BN) Statistics Repair The fused model is set to training mode, and the samples in the input buffer M are propagated forward once to force the BN layer to re-estimate the moving mean and variance, thereby repairing the activation variance collapse phenomenon caused by model interpolation fusion.

[0137] The advantages of the embodiments of the present invention are as follows: (1) Breaking the plasticity bottleneck: This invention alleviates the physical gradient conflict during joint training of data by intervening in the parameter space. Even with a sufficient memory budget, it can achieve a win-win improvement in stability and plasticity, and the forgetting rate is significantly reduced in multiple benchmark tests (Seq-CIFAR-100, Seq-Tiny-Image Net).

[0138] (2) Fully respecting the hierarchical heterogeneity of deep neural networks: Compared with the traditional global single interpolation coefficient, this method gives each layer the ability to autonomously find the optimal weights. Shallow layers tend to retain historical features, while deep layers tend to integrate new features, which significantly improves the generalization ability of the model.

[0139] (3) Low computational overhead and plug-and-play characteristics: This invention reduces the dimensionality of the massive model parameter optimization problem to an optimization problem of a fusion vector w containing only a few hundred parameters (layers), greatly reducing memory usage and computation time. Simultaneously, as a post-hoc mechanism, this method can be seamlessly integrated with existing empirical replay methods (such as ER, DER++, ER-ACE). Parameterized fusion coefficients and low-dimensional gradient updates in layer-by-layer differentiable adaptive fusion. Due to the freezing of this scheme... and Only optimize the length. fusion vector Therefore, the computational cost and GPU memory usage are significantly lower than retraining the entire model or optimizing all network parameters. The "plug-and-play" characteristic primarily corresponds to the post-hoc design flow of this solution. Specifically, existing experience-based replay methods such as ER, DER++, and ER-ACE can still complete the current task training in the original way; this solution only adds three post-processing steps after the training is complete: "geometric alignment—layer-by-layer fusion—BN statistic repair," to obtain the final model. Therefore, this solution does not rely on specific replay loss functions, buffer sampling strategies, or training algorithms, and can be used as a post-processing module to integrate with various continuous learning baseline methods. Furthermore, this solution only requires caching one additional copy of the historical model. Furthermore, the historical model is updated to the current fusion model after each task is completed. Therefore, its structured storage overhead does not increase linearly with the number of tasks.

[0140] The specific implementation examples are as follows: Example 1: Application of Incremental Learning Based on ResNet-18 Architecture 1. System Initialization: In image classification continuous learning tasks (such as Seq-CIFAR-100), the system is equipped with a ResNet-18 model and allocated a capacity of [missing information]. The image replay buffer.

[0141] 2. Task Loop and Baseline Update: The system receives image data for the t-th task. Combining this data with older images from the buffer, a new model is trained using the SGD optimizer. .

[0142] 3. Feature Alignment Execution: The optimal transport module is invoked to extract features from each convolutional and fully connected layer of the ResNet-18. The Pearson cost matrix is ​​calculated using a formula, and the permutation matrix is ​​obtained using the Hungarian algorithm. For the old model... The weight channels are physically swapped to obtain .

[0143] 4. Layer-by-layer optimization: Initialize a zero-based vector w of length L (corresponding to the number of learnable layers in ResNet-18). Use the Adam optimizer with a learning rate of 0.1. Run several epochs on the proxy buffer. In the forward propagation of each batch, dynamically weight the vectors according to Sigmoid(w). and Obtain the instantaneous parameters. Update them through error backpropagation. .

[0144] 5. Model solidification and statistical repair: After the application optimization is completed. The final weighted fusion model is then constructed. Subsequently, by inputting 256 images from the buffer, the momentum update mechanism of the Batch Normalization (BN) layer inside the model is triggered to correct the variance distribution and complete the model iteration for the current task.

[0145] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0146] Finally, it should be noted that the above embodiments are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training an ultrasound image learning model, characterized in that, The method includes: Acquire historical ultrasound image datasets and training ultrasound image datasets; the ultrasound image datasets include first-type lesion regions and second-type lesion regions; The training model is obtained by training a preset historical model using the training ultrasound image dataset and the historical ultrasound image dataset; the first type of lesion region is the lesion region that can be identified by the historical model, and the second type of lesion region is the lesion region that cannot be identified by the historical model. The historical ultrasound image dataset is input into the training model to obtain the updated activation tensor; the historical ultrasound image dataset is input into the historical model to obtain the historical activation tensor. Based on the updated activation tensor and the historical activation tensor, a cost matrix is ​​obtained; based on the cost matrix, the parameters of the historical model are rearranged to obtain an aligned model. The parameters of the training model and the alignment model are fused using an effective fusion coefficient to obtain a fused model; the effective fusion coefficient is used to represent the weight ratio of the corresponding parameters of the alignment model and the training model. The training process of the fusion model is constrained by a preset cross-entropy loss function to obtain a trained ultrasound image learning model; the ultrasound image learning model is used for incremental learning of the identification of new lesion regions in ultrasound images.

2. The ultrasound image learning model training method according to claim 1, characterized in that, The step of obtaining the cost matrix based on the updated activation tensor and the historical activation tensor includes: Based on the historical activation tensor, the historical discrete probability measure of neurons in the historical model is obtained; based on the updated activation tensor, the updated discrete probability measure of neurons in the training model is obtained. Based on the historical discrete probability measure and the updated discrete probability measure, the functional difference cost is obtained; based on the functional difference cost, the cost matrix is ​​obtained. The specific formula for calculating the cost of the functional difference is as follows: In the above formula, For the historical model in the first The first in the network layer The neuron and the training model in the th... The first in the network layer The functional differences between individual neurons have generational value; ) represents the covariance. Standard deviation For the historical model in the first In the network layer, the _th The average discrete probability measure of a neuron To train the model in the first The first in the network layer The average discrete probability measure of a neuron It is a tiny constant.

3. The ultrasound image learning model training method according to claim 2, characterized in that, The step of rearranging the parameters of the historical model according to the cost matrix to obtain the aligned model includes: The cost matrix is ​​calculated using the Hungarian algorithm to obtain the optimal permutation matrix; the parameters of the historical model are rearranged using the optimal permutation matrix to obtain the aligned model.

4. The ultrasound image learning model training method according to claim 3, characterized in that, The step of fusing the parameters of the trained model and the aligned model using effective fusion coefficients to obtain a fused model includes: By using effective fusion coefficients, the parameters of the trained model and the aligned model are fused to obtain fusion model coefficients; based on the fusion model coefficients, the fusion model is obtained. The effective fusion coefficient is obtained in the following way: The preset learnable fusion control variables are used through The function is mapped to obtain the effective fusion coefficients; the learnable fusion control variable is used to allow the model to automatically learn the optimal fusion ratio of new and old features based on the data. The fusion model coefficients are obtained using the following formula: In the above formula, For training the model and aligning the model The coefficients of the fusion model for each network layer For training the model and aligning the model, the first The effective fusion coefficient of each network layer For the training model's first Parameters of each network layer For the alignment model's first The parameters of each network layer.

5. The ultrasound image learning model training method according to claim 4, characterized in that, The step of constraining the training process of the fusion model using a preset cross-entropy loss function to obtain a trained ultrasound image learning model includes: The historical ultrasound image dataset is input into the fusion model, and the cross-entropy surrogate loss function value is calculated using the cross-entropy loss function. The loss gradient is obtained based on the cross-entropy surrogate loss function value. The learnable fusion control variable is updated using the loss gradient to obtain the optimized learnable fusion control variable. Based on the optimized learnable fusion control variables, optimized fusion parameters are obtained; based on the optimized fusion parameters, an optimized model is obtained. The historical ultrasound image dataset is input into the optimization model for one forward propagation to obtain the trained ultrasound image learning model.

6. The ultrasound image learning model training method according to claim 1, characterized in that, The historical ultrasound image dataset was obtained through the following methods: A historical ultrasound image dataset is obtained through a replay buffer; the replay buffer is a storage space for storing the dataset used when the historical model was trained for historical tasks. And, the The specific formula for calculating the function is as follows: In the above formula, For training the model and aligning the model, the first The effective fusion coefficient of each network layer For the first Learnable fusion control variables for each network layer.

7. A method for identifying new lesion regions in an image, characterized in that, The method includes: When a second type of lesion region is identified, the ultrasound image learning model training method according to any one of claims 1-6 is used to obtain an updated ultrasound image learning model based on the second type of lesion region; the ultrasound image learning model is used to identify the second type of lesion region in the ultrasound image.

8. A training device for an ultrasound image learning model, characterized in that, The device includes: The image data acquisition module is used to acquire historical ultrasound image datasets and training ultrasound image datasets; the ultrasound image datasets include first-type lesion regions and second-type lesion regions. The model preliminary training module is used to train a preset historical model using the training ultrasound image dataset and the historical ultrasound image dataset to obtain a training model; the first type of lesion region is a lesion region that can be identified by the historical model, and the second type of lesion region is a lesion region that cannot be identified by the historical model. The model alignment training module is used to input the historical ultrasound image dataset into the training model to obtain the updated activation tensor; input the historical ultrasound image dataset into the historical model to obtain the historical activation tensor; obtain the cost matrix based on the updated activation tensor and the historical activation tensor; and rearrange the parameters of the historical model based on the cost matrix to obtain the alignment model. The model fusion training module is used to fuse the parameters of the training model and the alignment model through an effective fusion coefficient to obtain a fused model; the effective fusion coefficient is used to represent the weight ratio of the corresponding parameters of the alignment model and the training model; the training process of the fused model is constrained by a preset cross-entropy loss function to obtain a trained ultrasound image learning model; the ultrasound image learning model is used for incremental learning of the identification of new lesion regions in ultrasound images.

9. A device for identifying new lesion regions in images, characterized in that, The device includes: The image new lesion region learning module is used to obtain an updated ultrasound image learning model based on the second type of lesion region when the existence of a second type of lesion region is determined, using the ultrasound image learning model training method according to any one of claims 1-6; the ultrasound image learning model is used to identify the second type of lesion region in the ultrasound image.

10. An apparatus comprising a memory and a processor, characterized in that, The device includes: A memory storing executable program code, and a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the ultrasound image learning model training method as described in any one of claims 1-6 or the image new lesion region identification method as described in claim 7.

Images