A gradient optimization method and system for multitask radiology report generation

Abstract

The application discloses a kind of gradient optimization method and system of multitask radiology report generation, it is related to medical information technology field, including, obtaining medical training data set containing image, label and report text, construct and include visual encoder, clinical constraint auxiliary task branch and text decoder Multi-task report generation model;Firstly, the failure mechanism of linear scalarization is studied from the perspective of gradient dynamics, and it is formalized as the "double dilemma" of drift term bias and diffusion term attenuation using the SDE framework, thereby revealing the geometric root cause of suboptimality in RRG multitask optimization, and proposing CAME-Grad, a gradient optimization algorithm designed specifically for multitask RRG. As an optimizer independent of the backbone network, it can be integrated into various model architectures in a plug-and-play manner. On the MIMIC-CXR and IU X-Ray datasets, CAME-Grad was extensively evaluated across eight representative RRG methods, and the results showed that its average clinical performance improved by 2.3% and 1.9%, respectively.

Description

Technical Field

[0001] This invention relates to the field of medical information technology, and in particular to a gradient optimization method and system for generating multi-task radiology reports. Background Technology

[0002] Automated radiology report generation has become a research hotspot in assisted diagnosis. Existing methods mostly adopt an encoder-decoder architecture, using convolutional neural networks or visual Transformers to extract image features and combining them with long short-term memory networks or Transformer decoders to generate natural language descriptions. In order to improve the diagnostic accuracy of the model, multi-task learning paradigms have been widely introduced, which optimize both disease classification and report generation tasks by sharing the underlying feature encoder.

[0003] Existing technologies typically employ a simple linear weighted summation strategy to aggregate gradients. This coarse-grained approach is prone to causing gradient updates for one task to cancel out effective learning signals for another, leading to difficulties in model convergence or mediocre performance on both tasks. Medical imaging data generally suffers from severe long-tail distribution problems, with rare lesion samples being scarce. Existing gradient update mechanisms lack the ability to dynamically perceive sample frequency, causing models to tend to fit common lesions during training and exhibit weak gradient responses to rare lesions, resulting in extremely high false negative rates. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a gradient optimization method for multi-task radiology report generation to address the problems of model convergence difficulties or mediocre performance on both tasks. Medical imaging data generally suffers from severe long-tail distribution and rare lesion samples are scarce. Existing gradient update mechanisms lack dynamic perception of sample frequency, causing the model to tend to fit common lesions during training and have a weak gradient response to rare lesions, resulting in an extremely high rate of missed diagnoses.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a gradient optimization method for multi-task radiology report generation, which includes acquiring a medical training dataset containing images, labels and report text, and constructing a multi-task report generation model including a visual encoder, a clinical constraint-assisted task branch and a text decoder;

[0008] Input the dataset into the multi-task report generation model to extract features and generate prediction results. Calculate the independent loss gradients for each task and aggregate them into the original joint gradient and average gradient.

[0009] The optimal weights are obtained by solving the dual problem within the trust region centered on the average gradient, thereby calculating the geometric correction gradient to eliminate destructive interference.

[0010] The target amplitude is calculated based on the magnitude of the original joint gradient and the preset gain factor, and the geometric correction gradient is scaled and compensated to obtain the kinetic energy enhancement gradient.

[0011] Using a preset fusion coefficient, the kinetic enhancement gradient and the original joint gradient scaled by the gain factor are linearly interpolated to obtain the semantic preservation gradient, and the model parameters are updated accordingly until convergence is obtained to obtain the trained multi-task report generation model.

[0012] The image to be analyzed is input into the trained multi-task report generation model, which outputs a radiological report.

[0013] As a preferred embodiment of the gradient optimization method for multi-task radiology report generation described in this invention, the specific steps for constructing a multi-task report generation model comprising a visual encoder, a label prediction branch, and a text decoder are as follows:

[0014] The visual encoder adopts a deep convolutional neural network architecture or a visual Transformer architecture to extract spatial semantic information through multi-layer convolution and pooling operations or through a self-attention mechanism, and output a high-dimensional feature tensor.

[0015] The clinical constraint-assisted task branch includes a fully connected layer network, which maps the high-dimensional feature tensor to the disease category space, generates a predicted probability distribution vector, and inputs it together with the real disease label into the cross-entropy loss function to calculate the label task loss value.

[0016] The clinical constraint assistance task branch includes an image-text alignment module or a retrieval enhancement module, used to calculate the corresponding alignment loss or retrieval loss;

[0017] The text decoder adopts a Transformer architecture based on a self-attention mechanism. It uses the high-dimensional feature tensor as the initial context state and predicts diagnostic text terms one by one through autoregression to generate a diagnostic text sequence. This sequence is then input into the language model loss function along with the real report text to calculate the text task loss value.

[0018] As a preferred embodiment of the gradient optimization method for multi-task radiology report generation described in this invention, the steps of inputting the dataset into the multi-task report generation model to extract features and generate prediction results, calculating the independent loss gradients for each task, and aggregating them into the original joint gradient and the average gradient are as follows:

[0019] Execute the backpropagation algorithm to calculate the partial derivatives of the loss value of the main task and the loss value of each clinical constraint auxiliary task with respect to the shared parameters, and obtain the independent gradient vector of the main task and the independent gradient vector of each auxiliary task.

[0020] Based on the preset static task weights, the independent gradient vectors of all tasks are weighted and summed to generate the original joint gradient vector.

[0021] The average gradient is obtained by performing an arithmetic mean operation on the independent gradient vectors of all tasks. A trust region is constructed based on the average gradient, and the dual problem is solved to obtain the optimal correction direction without destructive interference.

[0022] As a preferred embodiment of the gradient optimization method for multi-task radiological report generation described in this invention, the specific steps of obtaining the optimal weights by solving the dual problem within the trust region centered on the average gradient, thereby calculating the geometric correction gradient to eliminate destructive interference, are as follows:

[0023] Execute the backpropagation algorithm to calculate the independent gradient vectors of the main task and each clinical constraint auxiliary task with respect to the shared parameters;

[0024] The average gradient is obtained by calculating the arithmetic mean of all independent gradient vectors. And calculate the original joint gradient based on the preset weights;

[0025] With the average gradient Construct a trust region around the center, and within this region construct a constrained optimization master problem aimed at maximizing the worst-case local improvement;

[0026] By introducing dual variables, the primal problem is transformed into a dual problem, and the optimal weights are obtained by solving it. Then the weighted gradient is calculated. ;

[0027] Using the optimal weight and weighted gradient The optimal correction direction vector can be directly calculated using a closed-form solution. ;

[0028] The optimal correction direction vector It is directly used as the update direction input for subsequent stages after eliminating gradient interference.

[0029] As a preferred embodiment of the gradient optimization method for generating multi-task radiological reports according to the present invention, the specific steps are as follows: The target amplitude is calculated based on the magnitude of the original joint gradient and a preset gain factor, and the geometric correction gradient is scaled and compensated using this to obtain the kinetic enhancement gradient.

[0030] Set the gain adjustment hyperparameter to control the baseline level of the overall compensation intensity;

[0031] The target amplitude is calculated by multiplying a preset gain factor by the original joint gradient magnitude.

[0032] Based on the original joint gradient vector Calculate the target amplitude using the modulus. ;

[0033] Using the target amplitude For geometrically corrected gradient vector Perform proportional scaling compensation to obtain the kinetic energy enhancement gradient vector. .

[0034] As a preferred embodiment of the gradient optimization method for generating multi-task radiological reports according to the present invention, the step of linearly interpolating the kinetic enhancement gradient and the original joint gradient scaled by the gain factor using a preset fusion coefficient to obtain the semantically preserved gradient is as follows:

[0035] Setting scalar fusion hyperparameters This is used to control the linear interpolation ratio of the gradient between the "kinetic enhancement gradient" and the "scaled original joint gradient";

[0036] By using simple weighted fusion, while preserving the high-energy gradient features after geometric correction and amplitude recovery, some original joint gradient information is introduced to maintain the stability of the semantic space and avoid semantic drift caused by over-correction.

[0037] Perform an adaptive linear fusion operation to enhance the kinetic gradient vector. With the gain coefficient Scaled original joint gradient vector The gradients are fused to obtain the final semantically preserved gradient vector. ;

[0038] when When the gradient approaches 0, it mainly depends on the kinetic enhancement gradient, and the model focuses on breaking through local minima to capture the characteristics of rare lesions;

[0039] when When the gradient approaches 1, the gradient mainly regresses to the scaled original joint gradient, and the model focuses on maintaining the semantic consistency and stability of the original multi-task learning.

[0040] By adjusting The value of achieves a flexible balance between "feature exploration capability" and "semantic preservation capability" during gradient update.

[0041] As a preferred embodiment of the gradient optimization method for multi-task radiology report generation described in this invention, the specific steps of inputting the image to be analyzed into the trained multi-task report generation model and outputting a radiology report are as follows:

[0042] The adaptive moment estimation optimizer is invoked to calculate the update amount of all learnable parameters in the visual encoder, label prediction branch and text decoder based on the semantically preserved gradient vector, and to perform iterative update operations to drive the model loss function to converge to the global optimum.

[0043] Monitor the clinical efficacy F1 score on the validation set, and when the clinical efficacy reaches its optimum, save and solidify the model parameters;

[0044] During the application phase, unknown radiological image data to be analyzed is collected and input into the trained multi-task report generation model;

[0045] The model automatically performs forward propagation, the visual encoder extracts image features, the label prediction branch outputs the disease classification probability, and the text decoder generates natural language diagnostic descriptions based on the features.

[0046] By integrating classification results with text descriptions, a standard-format radiology report containing lesion localization, property analysis, and diagnostic suggestions is generated, achieving end-to-end intelligent assisted diagnostic output.

[0047] In a second aspect, the present invention provides a gradient optimization system for generating multi-task radiological reports, including a data construction module, a basic gradient calculation module, a geometric correction module, a kinetic enhancement module, a semantic fusion module, and a report generation module;

[0048] The data construction module is used to acquire a medical training dataset containing images, labels, and report text, and to construct a multi-task report generation model that includes a visual encoder, a clinical constraint-assisted task branch, and a text decoder.

[0049] The basic gradient calculation module is used to input the dataset into the multi-task report generation model to extract features and generate prediction results, calculate the independent loss gradient of each task, and aggregate them into the original joint gradient and average gradient.

[0050] The geometric correction module is used to obtain the optimal weight by solving the dual problem within the trust region centered on the average gradient, thereby calculating the geometric correction gradient to eliminate destructive interference.

[0051] The kinetic energy enhancement module is used to calculate the target amplitude based on the magnitude of the original joint gradient and a preset gain factor, and to scale and compensate the geometric correction gradient to obtain the kinetic energy enhancement gradient.

[0052] The semantic fusion module is used to linearly interpolate the kinetic enhancement gradient and the original joint gradient scaled by the gain factor using a preset fusion coefficient, and fuse them to obtain the semantically preserved gradient, and update the model parameters accordingly until convergence to obtain the trained multi-task report generation model.

[0053] The report generation module is used to input the image to be analyzed into the trained multi-task report generation model and output a radiological report.

[0054] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the gradient optimization method for generating multi-task radiological reports as described in the first aspect of the present invention.

[0055] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the gradient optimization method for generating multi-task radiological reports as described in the first aspect of the present invention.

[0056] The beneficial effects of this invention are as follows: By constructing a multi-task report generation model that includes a visual encoder, a label prediction branch, and a text decoder, the optimal weights are solved by directly constructing a dual problem within the trust region centered on the average gradient, thereby unconditionally purifying conflicting gradients into geometrically corrected gradients pointing to the Pareto optimal consensus region; by combining the magnitude of the original joint gradient with a preset fixed gain factor to construct a target amplitude, a kinetic enhancement gradient is generated to compensate for the optimization dynamics; finally, adaptive gradient linear fusion is achieved through preset fusion coefficients, effectively overcoming the problem of missed diagnosis of rare diseases caused by imbalanced medical image data. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 A flowchart of the gradient optimization method for generating multi-task radiology reports in the first embodiment;

[0059] Figure 2 A schematic diagram of the gradient optimization system for generating multi-task radiology reports in the first embodiment;

[0060] Figure 3 This is a schematic diagram of the CAME-Grad optimizer architecture and processing flow for multi-task radiology report generation in the second embodiment.

[0061] Figure 4 A schematic diagram comparing the "double dilemma" in multi-task optimization in the second embodiment and the principle by which the CAME-Grad optimizer solves this dilemma;

[0062] Figure 5 This is a qualitative comparison of the second embodiment with the existing baseline model on the test set, along with a visualization of the attention map. Detailed Implementation

[0063] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0064] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0065] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0066] Example 1, referring to Figures 1-2 This is the first embodiment of the present invention, which provides a gradient optimization method for generating multi-task radiology reports, including the following steps:

[0067] S1. Obtain a medical training dataset containing images, labels, and report text, and construct a multi-task report generation model that includes a visual encoder, a clinical constraint-assisted task branch, and a text decoder.

[0068] Furthermore, the visual encoder employs a deep convolutional neural network architecture or a visual Transformer architecture to extract spatial semantic information through multi-layer convolutional operations and pooling operations or through a self-attention mechanism, and outputs a high-dimensional feature tensor.

[0069] The clinical constraint-assisted task branch includes a fully connected layer network, which maps high-dimensional feature tensors to the disease category space, generates a predicted probability distribution vector, and inputs it together with the real disease labels into the cross-entropy loss function to calculate the label task loss value.

[0070] The clinical constraint-assisted task branch includes an image-text alignment module or a retrieval enhancement module, which are used to calculate the corresponding alignment loss or retrieval loss;

[0071] The text decoder adopts a Transformer architecture based on a self-attention mechanism. It uses a high-dimensional feature tensor as the initial context state and predicts diagnostic text terms one by one through autoregression to generate a diagnostic text sequence. This sequence is then input into the language model loss function along with the real report text to calculate the text task loss value.

[0072] It should be noted that the deep reuse of image features was achieved through the visual encoder, which effectively reduced the number of model parameters and improved the feature extraction efficiency. At the same time, the parallel architecture design of the label prediction branch and the text decoder enabled the model to learn the discrete classification boundary of the disease and the continuous semantic description logic at the same time, laying a solid foundation for the unified feature representation for the subsequent collaborative optimization of multi-task gradients and avoiding the information loss problem caused by traditional serial processing.

[0073] S2. Input the dataset into the multi-task report generation model to extract features and generate prediction results. Calculate the independent loss gradients for each task and aggregate them into the original joint gradient and average gradient.

[0074] Furthermore, the backpropagation algorithm is executed to calculate the partial derivatives of the loss value of the main task and the loss value of each clinical constraint auxiliary task with respect to the shared parameters, thereby obtaining the independent gradient vector of the main task and the independent gradient vector of each auxiliary task.

[0075] Based on the preset static task weights, the independent gradient vectors of all tasks are weighted and summed to generate the original joint gradient vector.

[0076] The average gradient is obtained by performing an arithmetic mean operation on the independent gradient vectors of all tasks. A trust region is constructed based on the average gradient, and the dual problem is solved to obtain the optimal correction direction without destructive interference.

[0077] It should be noted that this algorithm performs unconditional direction correction in each iteration to ensure that the optimization process is strictly constrained within the Pareto optimal consensus region of each task throughout the entire training cycle. This effectively eliminates the negative and destructive interference between the pursuit of clear decision boundaries in classification tasks and the pursuit of semantic coherence in generation tasks, and significantly improves the stability of the training process.

[0078] S3. Within the trust region centered on the average gradient, the optimal weights are obtained by solving the dual problem, thereby calculating the geometric correction gradient to eliminate destructive interference.

[0079] Furthermore, the backpropagation algorithm is executed to calculate the independent gradient vectors of the main task and each clinical constraint auxiliary task relative to the shared parameters;

[0080] The average gradient is obtained by calculating the arithmetic mean of all independent gradient vectors. And calculate the original joint gradient based on the preset weights;

[0081] average gradient Construct a trust region around the center, and within this region construct a constrained optimization master problem aimed at maximizing the worst-case local improvement;

[0082] Introducing dual variables transforms the primal problem into a dual problem, and solving this problem yields the optimal weights. Then the weighted gradient is calculated. ;

[0083] Using optimal weights and weighted gradient The optimal correction direction vector can be directly calculated using a closed-form solution. ;

[0084] The optimal correction direction vector It is directly used as the update direction input for subsequent stages after eliminating gradient interference.

[0085] It should be noted that by constructing an optimization problem within a trust region centered on the average gradient, this algorithm cleverly separates conflicting gradient components from the update direction, forcing gradient updates to point towards the "Pareto optimal" consensus region of each task. This mechanism obtains the optimal weights by solving the dual problem, achieving accurate identification and cancellation of destructive interference terms. Thus, while eliminating directional conflicts between tasks, it ensures that the optimization trajectory is always within the geometrically effective tangent space, achieving geometric purification of the gradient space.

[0086] S4. Calculate the target amplitude based on the magnitude of the original joint gradient and the preset gain factor, and use this to scale and compensate the geometric correction gradient to obtain the kinetic energy enhancement gradient.

[0087] Furthermore, a gain adjustment hyperparameter is set to control the baseline level of the overall compensation intensity;

[0088] The target amplitude is calculated by multiplying a preset gain factor by the original joint gradient magnitude.

[0089] Based on the original joint gradient vector Calculate the target amplitude using the modulus. ;

[0090] Using the target amplitude For geometrically corrected gradient vector Perform proportional scaling compensation to obtain the kinetic energy enhancement gradient vector. .

[0091] It should be noted that, in response to the long-tail distribution problem commonly found in medical images, an innovative kinetic energy injection mechanism is introduced. By amplifying the magnitude of the original joint gradient through a preset fixed gain factor, this magnitude is used as the target amplitude to scale and compensate the geometric correction gradient. While restoring the amplitude decay caused by geometric constraints, additional exploratory kinetic energy is injected, forcing the model to naturally escape sharp local minima during training, effectively endowing the model with the ability to capture rare and subtle lesions.

[0092] S5. Using the preset fusion coefficients, the kinetic enhancement gradient and the original joint gradient scaled by the gain factor are linearly interpolated to obtain the semantic preservation gradient, and the model parameters are updated accordingly until convergence is obtained to obtain the trained multi-task report generation model.

[0093] Furthermore, we set scalar fusion hyperparameters. This is used to control the linear interpolation ratio of the gradient between the "kinetic enhancement gradient" and the "scaled original joint gradient";

[0094] By using simple weighted fusion, while preserving the high-energy gradient features after geometric correction and amplitude recovery, some original joint gradient information is introduced to maintain the stability of the semantic space and avoid semantic drift caused by over-correction.

[0095] Perform an adaptive linear fusion operation to enhance the kinetic gradient vector. With the gain coefficient Scaled original joint gradient vector The gradients are fused to obtain the final semantically preserved gradient vector. ;

[0096] when When the gradient approaches 0, it mainly depends on the kinetic enhancement gradient, and the model focuses on breaking through local minima to capture the characteristics of rare lesions;

[0097] when When the gradient approaches 1, the gradient mainly regresses to the scaled original joint gradient, and the model focuses on maintaining the semantic consistency and stability of the original multi-task learning.

[0098] By adjusting The value of achieves a flexible balance between "feature exploration capability" and "semantic preservation capability" during gradient update.

[0099] It should be noted that by introducing statically preset scalar fusion coefficients, a stable trade-off between the "theoretically optimal gradient" and the "task-specific inductive bias" is achieved. This not only utilizes the reinforcing effect of kinetic gradients on rare disease exploration, but also retains some scaled original joint gradient information to maintain linguistic fluency and logical consistency. This effectively suppresses the report hallucination phenomenon caused by overcorrection of classification indicators, ensuring the clinical usability of the final model.

[0100] S6. Input the image to be analyzed into the trained multi-task report generation model and output a radiological report.

[0101] Furthermore, the adaptive moment estimation optimizer is invoked to calculate the update amount of all learnable parameters in the visual encoder, label prediction branch and text decoder based on the semantically preserved gradient vector, and to perform iterative update operations to drive the model loss function to converge to the global optimum.

[0102] Monitor the clinical efficacy F1 score on the validation set, and when the clinical efficacy reaches its optimum, save and solidify the model parameters;

[0103] During the application phase, unknown radiological image data to be analyzed is collected and input into the trained multi-task report generation model;

[0104] The model automatically performs forward propagation, the visual encoder extracts image features, the label prediction branch outputs the disease classification probability, and the text decoder generates natural language diagnostic descriptions based on the features.

[0105] By integrating classification results with text descriptions, a standard-format radiology report containing lesion localization, property analysis, and diagnostic suggestions is generated, achieving end-to-end intelligent assisted diagnostic output.

[0106] It should be noted that a closed loop from model training to clinical application has been completed. By solidifying the model parameters that have undergone multi-stage gradient optimization, the efficiency and robustness of the inference stage are ensured. The output radiology report not only contains high-precision disease classification results, but also has natural language descriptions that conform to doctors' reading habits. It achieves an organic unity of qualitative analysis and quantitative description, which greatly reduces the workload of radiologists and provides auxiliary diagnostic support for primary healthcare institutions. It has clinical application value and social benefits.

[0107] This embodiment also provides a gradient optimization system for generating multi-task radiological reports, including: a data construction module, a basic gradient calculation module, a geometric correction module, a kinetic enhancement module, a semantic fusion module, and a report generation module;

[0108] The data construction module is used to acquire a medical training dataset containing images, labels, and report text, and to build a multi-task report generation model that includes a visual encoder, a clinical constraint-assisted task branch, and a text decoder.

[0109] The basic gradient calculation module is used to input the dataset into the multi-task report generation model to extract features and generate prediction results, calculate the independent loss gradient of each task, and aggregate them into the original joint gradient and average gradient.

[0110] The geometric correction module is used to obtain the optimal weights by solving the dual problem within the trust region centered on the average gradient, thereby calculating the geometric correction gradient to eliminate destructive interference.

[0111] The kinetic energy enhancement module is used to calculate the target amplitude based on the magnitude of the original joint gradient and the preset gain factor, and to scale and compensate the geometric correction gradient to obtain the kinetic energy enhancement gradient.

[0112] The semantic fusion module is used to linearly interpolate the kinetic enhancement gradient and the original joint gradient scaled by the gain factor using preset fusion coefficients, and fuse them to obtain the semantically preserved gradient. The model parameters are then updated accordingly until convergence is achieved to obtain the trained multi-task report generation model.

[0113] The report generation module is used to input the images to be analyzed into the trained multi-task report generation model and output a radiological report.

[0114] This embodiment also provides a computer device applicable to the gradient optimization method for generating multi-task radiology reports, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the gradient optimization method for generating multi-task radiology reports as proposed in the above embodiment.

[0115] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0116] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the gradient optimization method for generating multi-task radiological reports as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0117] Example 2, refer to Figures 3-5 This is a second embodiment of the present invention, which provides a gradient optimization method for multi-task radiological report generation based on dynamic compensation, including the following steps:

[0118] Step 1: Forward Propagation and Basic Gradient Calculation in Multi-Task Networks

[0119] Construct a multi-task radiology report generation model, which includes a main task (report generation task, indexed as ). )and Each auxiliary task (clinically constrained task, such as disease classification, image-text alignment, or search enhancement, etc., is denoted as [index name]). ).

[0120] 2. Sample a batch of medical images and corresponding report data from the dataset and input them into the model, then calculate the loss function for the main task. and the loss function of each auxiliary task .

[0121] For shared parameters Perform backpropagation to obtain the gradient vector for each task.

[0122] Based on preset static task weights Calculate the original joint gradient and the average gradient of all task gradients :

[0123] ;

[0124] Step Two: Phase One (S1) – Direction Adjustment for Conflict Avoidance

[0125] To ensure geometric validity for all tasks and eliminate "drift term bias" caused by destructive interference, this invention uses the average gradient... Within the central trust region, find an update direction vector that maximizes the worst-case local improvement. .

[0126] Construct the constrained master optimization problem:

[0127] ;

[0128] in, To control the hyperparameter of the trust region radius, constraints are imposed to limit the extent to which the update vector deviates from the average gradient to ensure optimization stability.

[0129] Introducing dual variables Then, the dual problem is solved, and the optimal weights are obtained by projective gradient descent. .

[0130] Based on the obtained optimal weights, the optimal correction direction vector is recovered and calculated through closed-form solution. :

[0131] ;

[0132] Step 3: Phase Two (S2) – Amplification-enhanced Energy Injection

[0133] Constraint optimization during the orientation correction process can compress the magnitude of the gradient. To counteract this magnitude collapse and reverse the "diffusion term decay," this invention introduces an magnitude enhancement mechanism to actively inject escape kinetic energy.

[0134] Calculate the target amplitude First, restore the magnitude to the baseline level of the original joint gradient. Based on this, a gain factor is introduced. Further enhancements were made by injecting additional exploration noise.

[0135] The correction direction vector obtained in stage one The amplitude is restored and enhanced to the target amplitude level, resulting in the energy-enhanced gradient. :

[0136] ;

[0137] in, To prevent division by zero of extremely small constants.

[0138] Step 4: Phase 3 (S3) – Adaptive Gradient Fusion

[0139] Purely mathematical corrections may over-orthogonalize gradients, leading to the loss of weak but important feature signals (such as long-tailed vocabulary gradients). To preserve the inductive bias of a specific task, this invention designs an adaptive fusion mechanism.

[0140] Introducing fusion coefficient At the theoretically optimal gradient (the gradient after energy enhancement) ) and the original joint gradient after amplitude enhancement ( Linear interpolation is performed between ).

[0141] The fused gradient used for parameter update is calculated. :

[0142] ;

[0143] Step 5: Model Parameter Update

[0144] Using the set learning rate and the gradient obtained by final fusion Shared parameters for multi-task radiology report generation model Update:

[0145] ;

[0146] Repeat steps one through five until the model converges.

[0147] Device Corresponding Items:

[0148] This invention also provides a plug-and-play multi-task gradient optimizer (device) corresponding to the above method, characterized in that it includes the following functional modules:

[0149] 1. Basic gradient calculation module: It is used to receive the report generation main task gradient and the gradients of each clinical constraint auxiliary task generated by the forward propagation of the multi-task network, and calculate the original joint gradient and the average gradient.

[0150] 2. Geometric Correction Module: Used to solve the dual problem within the trust region centered on the average gradient to obtain the geometric correction gradient that maximizes the worst-case local improvement, thereby eliminating destructive interference between gradients.

[0151] 3. Kinetic Energy Enhancement Module: This module calculates the target amplitude based on the magnitude of the original joint gradient and a preset gain factor, and uses this to scale and compensate the geometric correction gradient to obtain the kinetic energy enhancement gradient, thereby injecting escape kinetic energy into the optimization process.

[0152] 4. Semantic Fusion Module: This module uses preset fusion coefficients to linearly interpolate the kinetic enhancement gradient with the original joint gradient scaled by the gain factor, and outputs the final semantically preserved gradient for parameter updates in the backbone network.

[0153] Compared with existing technologies, it has the following advantages and positive effects:

[0154] 1. Overcoming the "Double Dilemma" and Establishing Stable Gradient Optimization Dynamics: This invention, for the first time, solves the optimization failure problem in multi-task radiological report generation from the perspective of gradient dynamics. Destructive interference is eliminated through direction correction, establishing geometric validity within the manifold tangent space, thereby correcting the "drift term bias." Simultaneously, by restoring and enhancing the gradient magnitude, the "diffusion term decay" is reversed, injecting necessary escape kinetic energy into the optimization process.

[0155] 2. Driving the model to escape local optima and accurately capture rare pathological details: Existing linear scalarization strategies easily lead to gradient energy depletion, causing the model to get trapped in sharp local optima. This invention actively injects exploratory noise through an amplitude enhancement mechanism, driving the model to shift towards a smoother minimum. Qualitative results show that this invention successfully guides the model to focus on rare but crucial pathological features in images (such as lesions that would otherwise be missed by the baseline model), reducing the risk of clinical misdiagnosis and greatly improving the diagnostic reliability of the generated radiology report.

[0156] 3. Achieving a dynamic balance between language smoothness and clinical diagnostic accuracy: Simple mathematical direction correction may lead to the loss of weak but important feature signals for specific tasks. The adaptive gradient fusion mechanism of this invention can dynamically balance the theoretically optimal direction with the inductive bias of specific tasks, effectively preventing the catastrophic forgetting of semantic features. This allows the model to strictly adhere to clinical constraints while generating fluent natural language reports, improving overall diagnostic accuracy.

[0157] 4. Excellent versatility and "plug-and-play" characteristics: The method and apparatus (CAME-Grad) provided by this invention are designed as a general-purpose optimizer independent of the backbone network. It can directly replace the traditional linear scalarization strategy and achieve plug-and-play integration without modifying any core multi-task network architecture.

[0158] 5. Consistent and robust clinical efficacy improvements on mainstream datasets: Experiments show that, as a general-purpose optimizer, this invention delivers substantial and consistent performance improvements across eight different representative multi-task radiology report generation methods. On the MIMIC-CXR and IU X-Ray public datasets, this invention improves overall clinical efficacy by an average of 2.4% and 1.9%, respectively. Furthermore, this invention demonstrates strong generalization ability and robustness even in data-scarce and cross-domain zero-shot inference scenarios.

[0159] In summary, this invention addresses the challenge of missed diagnoses of rare diseases caused by imbalanced medical imaging data by: constructing a multi-task report generation model comprising a visual encoder, a label prediction branch, and a text decoder; directly solving the dual problem within the trust region centered on the average gradient to obtain the optimal weights; unconditionally purifying conflicting gradients into geometrically corrected gradients pointing to the Pareto optimal consensus region; constructing a target amplitude by combining the magnitude of the original joint gradient with a preset fixed gain factor; generating a kinetic enhancement gradient to compensate for optimization dynamics; and finally achieving adaptive gradient linear fusion through preset fusion coefficients.

[0160] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A gradient optimization method for multi-task radiology report generation, characterized in that: include: Obtain a medical training dataset containing images, labels, and report text, and construct a multi-task report generation model that includes a visual encoder, a clinical constraint-assisted task branch, and a text decoder; Input the dataset into the multi-task report generation model to extract features and generate prediction results. Calculate the independent loss gradients for each task and aggregate them into the original joint gradient and average gradient. The optimal weights are obtained by solving the dual problem within the trust region centered on the average gradient, thereby calculating the geometric correction gradient to eliminate destructive interference. The target amplitude is calculated based on the magnitude of the original joint gradient and the preset gain factor, and the geometric correction gradient is scaled and compensated to obtain the kinetic energy enhancement gradient. Using a preset fusion coefficient, the kinetic enhancement gradient and the original joint gradient scaled by the gain factor are linearly interpolated to obtain the semantic preservation gradient, and the model parameters are updated accordingly until convergence is obtained to obtain the trained multi-task report generation model. The image to be analyzed is input into the trained multi-task report generation model, which outputs a radiological report.

2. The gradient optimization method for generating multi-task radiology reports as described in claim 1, characterized in that: The specific steps for constructing a multi-task report generation model that includes a visual encoder, a label prediction branch, and a text decoder are as follows: The visual encoder adopts a deep convolutional neural network architecture or a visual Transformer architecture to extract spatial semantic information through multi-layer convolution and pooling operations or through a self-attention mechanism, and output a high-dimensional feature tensor. The clinical constraint-assisted task branch includes a fully connected layer network, which maps the high-dimensional feature tensor to the disease category space, generates a predicted probability distribution vector, and inputs it together with the real disease label into the cross-entropy loss function to calculate the label task loss value. The clinical constraint assistance task branch includes an image-text alignment module or a retrieval enhancement module, used to calculate the corresponding alignment loss or retrieval loss; The text decoder adopts a Transformer architecture based on a self-attention mechanism. It uses the high-dimensional feature tensor as the initial context state and predicts diagnostic text terms one by one through autoregression to generate a diagnostic text sequence. This sequence is then input into the language model loss function along with the real report text to calculate the text task loss value.

3. The gradient optimization method for generating multi-task radiology reports as described in claim 2, characterized in that: The steps for inputting the dataset into the multi-task report generation model to extract features and generate prediction results, calculating the independent loss gradients for each task, and aggregating them into the original joint gradient and average gradient are as follows: Execute the backpropagation algorithm to calculate the partial derivatives of the loss value of the main task and the loss value of each clinical constraint auxiliary task with respect to the shared parameters, and obtain the independent gradient vector of the main task and the independent gradient vector of each auxiliary task. Based on the preset static task weights, the independent gradient vectors of all tasks are weighted and summed to generate the original joint gradient vector. The average gradient is obtained by performing an arithmetic mean operation on the independent gradient vectors of all tasks. A trust region is constructed based on the average gradient, and the dual problem is solved to obtain the optimal correction direction without destructive interference.

4. The gradient optimization method for generating multi-task radiology reports as described in claim 3, characterized in that: The steps for obtaining the optimal weights by solving the dual problem within the trust region centered on the average gradient, and then calculating the geometric correction gradient to eliminate destructive interference, are as follows: Execute the backpropagation algorithm to calculate the independent gradient vectors of the main task and each clinical constraint auxiliary task with respect to the shared parameters; The average gradient is obtained by calculating the arithmetic mean of all independent gradient vectors. And calculate the original joint gradient based on the preset weights; With the average gradient Construct a trust region around the center, and within this region construct a constrained optimization master problem aimed at maximizing the worst-case local improvement; By introducing dual variables, the primal problem is transformed into a dual problem, and the optimal weights are obtained by solving it. Then the weighted gradient is calculated. ; Using the optimal weight and weighted gradient The optimal correction direction vector can be directly calculated using a closed-form solution. ; The optimal correction direction vector It is directly used as the update direction input for subsequent stages after eliminating gradient interference.

5. The gradient optimization method for generating multi-task radiology reports as described in claim 4, characterized in that: The target amplitude is calculated based on the magnitude of the original joint gradient and a preset gain factor, and the geometric correction gradient is scaled and compensated to obtain the kinetic energy enhancement gradient. The specific steps are as follows: Set the gain adjustment hyperparameter to control the baseline level of the overall compensation intensity; The target amplitude is calculated by multiplying a preset gain factor by the original joint gradient magnitude. Based on the original joint gradient vector Calculate the target amplitude using the modulus. ; Using the target amplitude For geometrically corrected gradient vector Perform proportional scaling compensation to obtain the kinetic energy enhancement gradient vector. .

6. The gradient optimization method for generating multi-task radiology reports as described in claim 5, characterized in that: The step involves using a preset fusion coefficient to linearly interpolate the kinetic enhancement gradient and the original joint gradient scaled by the gain factor, and fusing them to obtain the semantically preserved gradient. The specific steps are as follows: Setting scalar fusion hyperparameters This is used to control the linear interpolation ratio of the gradient between the "kinetic enhancement gradient" and the "scaled original joint gradient"; By using simple weighted fusion, while preserving the high-energy gradient features after geometric correction and amplitude recovery, some original joint gradient information is introduced to maintain the stability of the semantic space and avoid semantic drift caused by over-correction. Perform an adaptive linear fusion operation to enhance the kinetic gradient vector. With the gain coefficient Scaled original joint gradient vector The gradients are fused to obtain the final semantically preserved gradient vector. ; when When the gradient approaches 0, it mainly depends on the kinetic enhancement gradient, and the model focuses on breaking through local minima to capture the characteristics of rare lesions; when When the gradient approaches 1, the gradient mainly regresses to the scaled original joint gradient, and the model focuses on maintaining the semantic consistency and stability of the original multi-task learning. By adjusting The value of achieves a flexible balance between "feature exploration capability" and "semantic preservation capability" during gradient update.

7. The gradient optimization method for generating multi-task radiology reports as described in claim 6, characterized in that: The specific steps for inputting the image to be analyzed into the trained multi-task report generation model and outputting a radiological report are as follows: The adaptive moment estimation optimizer is invoked to calculate the update amount of all learnable parameters in the visual encoder, label prediction branch and text decoder based on the semantically preserved gradient vector, and to perform iterative update operations to drive the model loss function to converge to the global optimum. Monitor the clinical efficacy F1 score on the validation set, and when the clinical efficacy reaches its optimum, save and solidify the model parameters; During the application phase, unknown radiological image data to be analyzed is collected and input into the trained multi-task report generation model; The model automatically performs forward propagation, the visual encoder extracts image features, the label prediction branch outputs the disease classification probability, and the text decoder generates natural language diagnostic descriptions based on the features. By integrating classification results with text descriptions, a standard-format radiology report containing lesion localization, property analysis, and diagnostic suggestions is generated, achieving end-to-end intelligent assisted diagnostic output.

8. A gradient optimization system for multi-task radiology report generation, based on the gradient optimization method for multi-task radiology report generation according to any one of claims 1 to 7, characterized in that: Including data building modules, The module includes a basic gradient calculation module, a geometric correction module, a kinetic energy enhancement module, a semantic fusion module, and a report generation module. The data construction module is used to acquire a medical training dataset containing images, labels, and report text, and to construct a multi-task report generation model that includes a visual encoder, a clinical constraint-assisted task branch, and a text decoder. The basic gradient calculation module is used to input the dataset into the multi-task report generation model to extract features and generate prediction results, calculate the independent loss gradient of each task, and aggregate them into the original joint gradient and average gradient. The geometric correction module is used to obtain the optimal weight by solving the dual problem within the trust region centered on the average gradient, thereby calculating the geometric correction gradient to eliminate destructive interference. The kinetic energy enhancement module is used to calculate the target amplitude based on the magnitude of the original joint gradient and a preset gain factor, and to scale and compensate the geometric correction gradient to obtain the kinetic energy enhancement gradient. The semantic fusion module is used to linearly interpolate the kinetic enhancement gradient and the original joint gradient scaled by the gain factor using a preset fusion coefficient, and fuse them to obtain the semantically preserved gradient, and update the model parameters accordingly until convergence to obtain the trained multi-task report generation model. The report generation module is used to input the image to be analyzed into the trained multi-task report generation model and output a radiological report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the gradient optimization method for generating multi-task radiological reports as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the gradient optimization method for generating multi-task radiological reports as described in any one of claims 1 to 7.

Images