A method and system for intelligent Q&A and guideline updates for radiotherapy knowledge with feedback optimization mechanisms

Abstract

This invention relates to the intersection of artificial intelligence and medical technology, and discloses a method and system for intelligent question answering and guideline updating of radiotherapy knowledge with a feedback optimization mechanism. The method includes: S1: constructing a knowledge base with a structured index; S2: receiving user queries, retrieving semantically relevant knowledge units through a retrieval model optimized by deep contrastive learning, and sorting them; generating response content consistent with radiotherapy guidelines through a generative model combined with learnable cross-modal output tokens and a local optimizer; S3: monitoring update information from external authoritative radiotherapy guidelines, automatically capturing and extracting key update points; filtering and sorting the updated content, and integrating valid updates into the knowledge base after confirmation; S4: collecting feedback data from users and experts, iteratively optimizing the retrieval model and the generative model to achieve a closed-loop improvement in system performance. This invention enables more accurate radiotherapy knowledge question answering, dynamic guideline updates, and continuous optimization of system performance.

Description

Technical Field

[0001] This invention relates to the intersection of artificial intelligence and medical technology, specifically to a method and system for intelligent question answering and guideline updating of radiotherapy knowledge with a feedback optimization mechanism. Background Technology

[0002] With the rapid development of radiation oncology, radiotherapy has become one of the core methods of cancer treatment. Its knowledge system covers multiple disciplines such as radiation physics, radiation biology, and anatomy, and needs to keep up with the continuous updates of clinical guidelines. Currently, authoritative organizations such as the American Society of Radiation Oncology (ASTRO), the European Society of Radiation Therapy and Oncology (ESTRO), the National Comprehensive Cancer Network (NCCN), and the Chinese Society of Clinical Oncology (CSCO) regularly publish or revise clinical guidelines for radiotherapy, promoting the iterative upgrading of treatment plans, dosage standards, and evidence-based approaches.

[0003] The existing radiotherapy knowledge Q&A and guide application system has the following key technical defects:

[0004] 1. Insufficient accuracy of knowledge representation: Traditional systems often rely on single text semantic vectors to build knowledge bases, failing to effectively integrate cross-modal information from radiotherapy guidelines and medical images such as computed tomography (CT) scans, and lacking standardized prompt embedding generation mechanisms, resulting in limited accuracy of knowledge retrieval and generation;

[0005] 2. Unreasonable distribution of semantic vector space: Existing semantic vector generation methods rely heavily on a large amount of labeled data. In unsupervised scenarios, it is difficult to achieve effective aggregation of similar knowledge units and widen the distance between dissimilar units, resulting in low semantic matching accuracy during retrieval.

[0006] 3. AI “hallucination” suppression effect is poor: The generative model lacks an effective cross-modal constraint mechanism and is prone to generating fictitious information that deviates from the guidelines (such as incorrect dose parameters or non-existent treatment plans), which poses serious safety risks in the field of high-risk radiotherapy;

[0007] 4. Delayed update of guidelines: The guidelines are mostly updated manually on a regular basis, which makes it impossible to quickly capture the revision dynamics of authoritative guidelines. Furthermore, there is a lack of a precise screening and structured integration mechanism for the updated content, resulting in a disconnect between systematic knowledge and clinical practice.

[0008] 5. The model lacks continuous optimization capabilities: It lacks multi-dimensional feedback collection and targeted iteration mechanisms, making it difficult to correct retrieval biases and generation errors through user interaction and expert review of data, and the system performance cannot adapt to clinical needs in the long term. Summary of the Invention

[0009] This invention addresses the shortcomings of existing technologies by proposing a method and system for intelligent question-and-answer and guideline updates for radiotherapy knowledge with a feedback optimization mechanism. Through the construction of a structured knowledge base, the design of an integrated "retrieval-generation" module, and a dynamic update and feedback optimization mechanism, it accurately solves the core problems in existing technologies such as the lag in knowledge timeliness, AI "illusion," and insufficient information reliability, thereby achieving more accurate radiotherapy knowledge question-and-answer, dynamic guideline updates, and continuous optimization of system performance.

[0010] To achieve the above objectives, the following technical solution is adopted:

[0011] In a first aspect, embodiments of the present invention provide a method for intelligent question-and-answer and guideline updating of radiotherapy knowledge with a feedback optimization mechanism, the method comprising:

[0012] S1: Construct a knowledge base that integrates radiotherapy guideline text with corresponding medical images and generates a structured index through cross-modal semantic feature extraction and semantic prompt embedding steps;

[0013] S2: Receive user queries, match knowledge units related to the query semantics in the knowledge base using a retrieval model optimized by deep contrastive learning, and sort the matching results; generate response content consistent with radiotherapy guidelines by combining a generative model with learnable cross-modal output tokens and a local optimizer.

[0014] S3: Monitor the update information of external authoritative radiotherapy guidelines, automatically capture and extract the key update points; filter and sort the updated content through semantic relevance calculation, and after confirmation, integrate the valid updated content into the knowledge base and update the corresponding semantic vector index synchronously.

[0015] S4: Collect multi-dimensional feedback data from users and experts on the system output, and iteratively optimize the retrieval model and generation model based on the multi-dimensional feedback data to achieve a closed-loop improvement in system performance.

[0016] Furthermore, in S1, the construction of the knowledge base includes a cross-modal semantic feature extraction step, specifically:

[0017] The radiotherapy guideline text and corresponding medical images were encoded separately to obtain linguistic and visual features;

[0018] Feature fusion is achieved through a two-stage progressive attention mechanism:

[0019] In the first stage, bidirectional multi-head cross-attention calculation is performed on the language features and visual features to obtain preliminary fused text-visual features for text querying vision and visual-text features for visual querying text; wherein, the projection transformation in the attention calculation is implemented through 3×3 depth convolution.

[0020] In the second stage, the language features and visual features are concatenated and processed by convolution to generate intermediate fusion features. The intermediate fusion features are used as attention queries and cross-attention calculations are performed with the text-visual features and the visual-text features respectively to obtain two deepened fusion features.

[0021] By fusing the two enhanced fusion features and processing them through a feedforward network, a cross-modal semantic feature that integrates textual semantics and image spatial information is output.

[0022] Furthermore, in S1, the construction of the knowledge base also includes a semantic prompt embedding generation step, specifically:

[0023] The cross-modal semantic features are fused with multi-scale visual features extracted from the corresponding medical images through a bidirectional feature pyramid network.

[0024] The fused features are normalized and subjected to attention processing, and then a basic semantic embedding is generated through a multi-head attention mechanism.

[0025] The basic semantic embeddings are processed in the following branches: the first branch generates dense cue embeddings containing fine-grained information through adaptive pooling and convolutional layer processing; the second branch generates sparse cue embeddings focusing on core semantic nodes through processing by a multilayer perceptron network; the dense cue embeddings and sparse cue embeddings will serve as the structured semantic representations of the corresponding knowledge units in the knowledge base.

[0026] Furthermore, in S2, the retrieval process includes:

[0027] A bidirectional long short-term memory network is used to parse user query intent, extract key constraints such as guideline version, cancer type and stage, treatment method and imaging features, and automatically supplement implicit search dimensions for fuzzy queries;

[0028] A deep contrastive learning-based framework is used to optimize the spatial distribution of semantic vectors in the knowledge base. By using a standardized temperature-scaling cross-entropy loss function, semantically similar knowledge units are clustered in the vector space, while semantically dissimilar units are kept apart.

[0029] Candidate knowledge units are quickly coarsely screened from the knowledge base based on cosine similarity. Multidimensional input features including semantic matching degree, knowledge timeliness, application frequency and cross-modal relevance are constructed. The candidate knowledge unit set is finely sorted by a multilayer perceptron, and the most relevant core knowledge units are output as the input of the generation module.

[0030] Furthermore, in S2, the generation of response content consistent with radiotherapy guidelines by combining a generative model with learnable cross-modal output tokens and a local optimizer specifically includes:

[0031] A generative framework is constructed based on a generative pre-trained transformation model with pre-trained weights in the field of fusion radiotherapy, serving as the generative model;

[0032] The sparse and dense cue embeddings corresponding to the retrieved candidate knowledge units, along with the learnable cross-modal output tokens, are concatenated with the original output token of the generative model and input together into the mask decoder. Simultaneously, the dense cue embeddings are fused with the features of the corresponding medical images as auxiliary inputs to the mask decoder. The mask decoder then generates the original mask.

[0033] The updated cross-modal output token and the cross-modal semantic features corresponding to the knowledge unit are subjected to nonlinear transformation by a local optimizer, and then a dot product operation is performed to generate an optimized mask. The original mask and the optimized mask are fused to obtain the final generation constraint mask used for the constraint generation process.

[0034] Based on the generated constraint mask, user identity category, and historical query records, generate and output response content adapted to the current user.

[0035] Furthermore, in S3, the filtering and sorting of updated content through semantic relevance calculation adopts a similarity ranking strategy, specifically including the following process:

[0036] The system regularly accesses the websites of authoritative radiotherapy guideline publishers via web crawlers and automatically identifies and extracts updated guideline information using natural language processing models.

[0037] The similarity ranking strategy is as follows: calculate the semantic similarity between the updated content to be integrated and the existing related knowledge in the knowledge base, and prioritize and extract the core points of the updated content based on the similarity score; the valid updated content confirmed by experts is automatically integrated into the knowledge base, and the corresponding semantic index and vector representation are updated.

[0038] Furthermore, in S4, the multi-dimensional feedback data includes feedback from at least two of the following dimensions:

[0039] Users directly rate the system's output in terms of accuracy, usability, and comprehensibility, and provide text corrections.

[0040] The system infers potential satisfaction information by analyzing user interaction data.

[0041] Professional review comments, ratings, and corrections from radiation oncology experts on the system's output content.

[0042] Furthermore, in S4, the model iterative optimization includes:

[0043] Retrieval model optimization: Add the retrieval error samples identified in the multidimensional feedback data to the training set to fine-tune the parameters of the semantic vector generation model and the retrieval ranking model;

[0044] Generative model optimization: The generative model is incrementally trained using the correct answers corrected by experts as positive samples and the incorrect answers from the original system as negative samples, while its internal cross-modal constraint parameters are updated synchronously.

[0045] Furthermore, in S4, the training of the generative model adopts a combined loss function, which includes: a basic generation loss consisting of a weighted sum of focus loss and dice loss, a refined generation loss based on optimized mask calculation, and an auxiliary loss consisting of mean absolute error loss and cross-entropy loss; the combined loss function is a weighted sum of the basic generation loss, the refined generation loss, and the auxiliary loss.

[0046] Secondly, embodiments of the present invention also provide a radiotherapy knowledge intelligent question-and-answer and guideline update system with a feedback optimization mechanism, the system comprising:

[0047] The structured knowledge base module is used to build a knowledge base that integrates radiotherapy guideline text with corresponding medical images and forms a structured index through cross-modal semantic feature extraction and semantic prompt embedding.

[0048] The retrieval-generation integrated module receives user queries, matches semantically relevant knowledge units in the knowledge base using a retrieval model optimized through deep contrastive learning, and sorts the matching results. It then generates response content consistent with radiotherapy guidelines by combining a learnable cross-modal output token and a local optimizer with a generative model.

[0049] The dynamic update module is used to monitor the update information of external authoritative radiotherapy guidelines, automatically capture and extract the key update points; filter and sort the updated content through semantic relevance calculation, and after confirmation, integrate the valid updated content into the knowledge base and update the corresponding semantic vector index synchronously.

[0050] The feedback optimization module is used to collect multi-dimensional feedback data from users and experts on the system output, and to iteratively optimize the retrieval model and the generation model based on the multi-dimensional feedback data, so as to achieve a closed-loop improvement in system performance.

[0051] Compared with the prior art, the present invention achieves the following beneficial effects:

[0052] 1. Significantly Improved Knowledge Representation Accuracy: The standardized sparse / dense cue embeddings automatically generated by the Semantic Prompt Projector effectively integrate the semantics of radiotherapy text and image spatial features, making the semantic representation of knowledge units more targeted. This solves the problem of the lack of standardized constraints in traditional knowledge representation. By fusing cross-modal semantic features and multi-scale image features, and through processing such as Bidirectional Feature Pyramid Network (BiFPN), Batch Normalization (BN), and Adaptive Pooling, sparse cue embeddings focusing on core semantics are generated. ) and dense cue embedding containing fine-grained information ( This provides precise semantic constraints for subsequent retrieval and generation, laying a precise foundation and effectively improving the accuracy of cross-modal knowledge association recognition;

[0053] 2. Superior Semantic Retrieval Efficiency and Accuracy: The semantic vector optimization framework (DeCLUTR) based on deep contrastive learning optimizes the semantic vector space distribution through a standardized temperature-scaled cross-entropy loss function (NT-Xent), addressing the issue of low semantic matching accuracy in unsupervised scenarios. It utilizes cosine similarity to quantify the correlation between knowledge units and strengthens the vector clustering of similar knowledge units through a loss function, improving semantic matching efficiency during retrieval. Combined with a two-stage strategy of "coarse retrieval + fine ranking," the retrieval accuracy is increased to over 95%.

[0054] 3. Effective suppression of AI “illusion” generation: The learnable cross-modal output token and local optimizer form a dual constraint, combined with the hard constraints of guideline text and image features, resulting in a high degree of consistency between the generated content and authoritative guidelines, completely avoiding risks such as fictitious treatment plans and incorrect dosage parameters;

[0055] 4. Significantly improved timeliness of guide updates: Through web crawler monitoring and similarity ranking self-updating strategy, the key points of guide updates are quickly captured, accurately filtered and automatically integrated, ensuring the timeliness of knowledge, shortening the guide update response time, significantly improving efficiency compared to manual updates, and accurately extracting core update points to avoid redundant information interference.

[0056] 5. Continuous adaptive optimization of system performance: The multi-dimensional feedback and iterative optimization mechanism enables the model to specifically correct problems such as retrieval omissions and generation biases. As feedback data accumulates, the retrieval model and generation model are continuously fine-tuned through explicit, implicit and professional feedback data to achieve closed-loop optimization of system performance. The retrieval accuracy and generation consistency are continuously improved, and the ability to adapt to different cancer types and different user scenarios is constantly enhanced.

[0057] 6. Wide range of applications and strong practicality: It is suitable for professional diagnosis and treatment inquiries by clinicians, knowledge learning needs of medical students, and popular consultations by patients' families. It also supports historical version tracking and related knowledge recommendations, balancing professionalism and ease of use, and providing efficient support for knowledge dissemination and clinical practice in the field of radiotherapy.

[0058] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0059] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the invention. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0060] Figure 1 This is a schematic diagram of the intelligent Q&A and guideline update method for radiotherapy knowledge with a feedback optimization mechanism provided in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of the overall system architecture and data flow according to an embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of a semantic vector optimization framework based on deep contrastive learning according to an embodiment of the present invention;

[0063] Figure 4 This is a flowchart of the dynamic update and feedback optimization closed loop of an embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram of the modules of the intelligent Q&A and guide update system for radiotherapy knowledge with a feedback optimization mechanism provided in an embodiment of the present invention;

[0065] Figure 6 This is a schematic diagram of the system functions according to an embodiment of the present invention;

[0066] Figure 7 This is a performance comparison chart of the algorithm of this invention with three traditional algorithms: GPT, Sentence-BERT, and BiLSTM. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0069] like Figure 1 and 2 As shown, the intelligent question-and-answer and guideline update method for radiotherapy knowledge with a feedback optimization mechanism includes:

[0070] S1: Construct a knowledge base that integrates radiotherapy guideline text with corresponding medical images and generates a structured index through cross-modal semantic feature extraction and semantic prompt embedding steps;

[0071] Step S1 is used for the construction and preprocessing of the structured knowledge base, and includes the following steps:

[0072] Step S1.1: Integration of authoritative data from multiple sources

[0073] Based on the clinical guidelines for radiotherapy published by authoritative organizations such as the American Society of Radiation Therapy (ASTRO), the European Society of Radiation Therapy and Oncology (ESTRO), the National Comprehensive Cancer Network (NCCN), the American Society of Clinical Oncology (ASCO), and the Chinese Society of Clinical Oncology (CSCO), and supplemented by core journal articles, treatment guidelines, clinical pathology data standards, and computed tomography (CT) images related to diagnosis and treatment, a comprehensive data source system covering multiple cancer types, stages, and modalities is established.

[0074] Step S1.2: Cross-modal semantic feature extraction

[0075] Step S1.2 is the cross-modal semantic feature extraction step for constructing the knowledge base. Specifically, it involves encoding the radiotherapy guideline text and corresponding medical images to obtain linguistic and visual features. Feature fusion is then performed through a two-stage progressive attention mechanism: In the first stage, bidirectional multi-head cross-attention calculation is performed on the linguistic and visual features to obtain preliminary fused text-visual features (text querying vision) and visual-text features (visual querying text). The projection transformation in the attention calculation is implemented using a 3×3 depthwise convolution. In the second stage, the original linguistic and visual features are concatenated and processed by convolution to generate intermediate fused features. These intermediate fused features are used as attention queries and cross-attention calculations are performed with both the text-visual and visual-text features to obtain two deepened fused features. The two deepened fused features are then fused and processed by a feedforward network to output cross-modal semantic features that integrate text semantics and image spatial information. The specific implementation is as follows:

[0076] We employ a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model from the field of radiotherapy to automatically extract core entities such as cancer type names, treatment techniques, dose parameters, and image feature correlation indicators, and construct a radiotherapy-specific entity dictionary.

[0077] A two-stage, progressive cross-attention cross-modal semantic interaction mechanism is proposed to associate the guideline text description with corresponding CT image features. Medical images are defined. (in (representing the depth, height, and width of the image, respectively) and the corresponding text description. ( (The text length is specified here). After the text is padded with sentence start markers ([SOS]) and sentence end markers ([EOS]), it is projected into a text embedding sequence. Images and text are input into their respective encoders to obtain visual features. and language features .

[0078] Radiotherapy-related medical imaging data (such as CT images) are in the form of a three-dimensional matrix, covering imaging information of the tumor and surrounding tissues. Medical Imaging dimensional space, Represents the real number field. These correspond to the depth, height, and width dimensions of the image, respectively. The depth dimension of medical imaging refers to the number of slices in the image, which corresponds to the layer thickness of CT images in radiotherapy. The height dimension of a medical image refers to the number of pixels in the vertical direction of a single image slice. The width dimension of a medical image refers to the number of pixels in the horizontal direction of a single image slice. The radiotherapy guideline text description data corresponding to medical images is in one-dimensional sequence form, covering core information such as cancer type, treatment plan, and dosage parameters. Text description dimensional space, Indicates the length of the text sequence (i.e., the number of characters or words). Text description The length of the text refers to the total number of characters or words contained in the text. [SOS]: Sentence start marker, used to identify the beginning of the radiotherapy guideline text description, assisting the model in recognizing text boundaries. [EOS]: Sentence end marker, used to identify the end of the radiotherapy guideline text description, assisting the model in fully capturing the text's semantics. The normalized text embedding sequence generated after being supplemented with [SOS] and [EOS] tags is used for subsequent encoder processing. Text embedding sequence dimensional space, For the dimensions of text embedding, Corresponding original text length Add the length of the two markers. The dimension of text embedding, i.e. the dimension of the feature vector obtained after the text is projected, is used to quantify the semantic information of the text. Medical Imaging The visual feature vectors obtained after encoder processing cover spatial feature information such as image texture, contour, and tumor location. Text embedding sequence The language feature vectors obtained after encoder processing cover textual feature information such as semantics and logical relationships in the radiotherapy guidelines.

[0079] The first layer achieves preliminary modality fusion through multi-head cross-attention computation, employing... Depth-convolutional projection replaces direct linear projection to capture local information; the calculation formula is as follows:

[0080]

[0081] : Multi-head cross-attention function, used to achieve cross-modal fusion of visual and linguistic features. The first parameter is projected as a query vector, and the second parameter is projected as a key vector and a value vector. The output features of the first-level cross-attention operation are obtained through language features. For query, visual features It is obtained by key-value operations, realizing the semantic guidance of language on vision. The output features of the first-level cross-attention operation are obtained through visual features. For query, language features It is obtained by key-value operations, realizing the visual supplementation of language features. Depthwise convolutional projection: a feature processing method used to replace direct linear projection, through... The depth convolution kernel of a certain size extracts local features, enhancing the capture of the association between radiotherapy-related image details and text keywords.

[0082] The second level will and After splicing, it is processed by a convolutional layer to obtain , and then For query and , The formula for cross-attention calculation is as follows:

[0083]

[0084] Finally, the summation and feedforward network (FF) output cross-modal semantic features (CS-Features). This forms a multimodal knowledge unit consisting of "text + image".

[0085] Feature concatenation function, used to combine visual features With language features By stitching together the original feature information of the two modalities, the features are integrated. : Convolution operation function, used to perform dimensionality fusion and redundant information filtering on the concatenated features, and output more compact fused features. The intermediate output features of the second-level feature processing are obtained by... and The concatenated result is used to perform a convolution operation to obtain the query vector for subsequent cross-attention operations. The output features of the second-level cross-attention operation are obtained by using... For query, The results are obtained through key-value operations, which deepens cross-modal semantic fusion. The output features of the second-level cross-attention operation are obtained by using... For query, The key-value operations are used to optimize cross-modal feature consistency. Feedforward Network is used to perform non-linear transformations and dimensional adjustments on the features output by the second-level cross-attention layer, thereby enhancing the feature representation capability. The final output of cross-modal semantic features (CS-Features) integrates the semantics of radiotherapy guidelines text with the spatial features of medical images, forming a core representation of a multimodal knowledge unit of "text + image".

[0086] In summary, the two-stage fusion process described above achieves a transition from initial association to deep semantic integration: First, bidirectional cross-attention guides text and image features to achieve initial alignment; then, the original features are fused with the initially aligned features to generate a more comprehensive intermediate representation, which is used as a benchmark to perform secondary calibration and refinement of the initial alignment results, ultimately outputting a unified semantic representation that accurately reflects the correspondence between "what the text describes" and "what the image shows".

[0087] Step S1.3: Semantic hint embedding generation

[0088] Step S1.3 is the semantic cue embedding generation step for constructing the knowledge base. Specifically, it involves fusing cross-modal semantic features with multi-scale visual features extracted from corresponding medical images through a bidirectional feature pyramid network; normalizing and attention-based processing of the fused features, followed by generating basic semantic embeddings through a multi-head attention mechanism; and then processing the basic semantic embeddings in the following branches: the first branch generates dense cue embeddings containing fine-grained information through adaptive pooling and convolutional layer processing; the second branch generates sparse cue embeddings focusing on core semantic nodes through a multilayer perceptron network. The dense and sparse cue embeddings will serve as the structured semantic representations of the corresponding knowledge units in the knowledge base. The specific implementation is as follows:

[0089] The Semantic Prompt Projector proposed in this embodiment is a key module for realizing the transformation from raw features to standardized prompt embeddings. It is used to convert cross-modal semantic features and multi-scale image features into standardized prompt embeddings, providing accurate semantic constraints for subsequent retrieval and generation. Its design purpose is to transform semantically rich text-image fusion features, i.e., cross-modal semantic features... Image multi-scale features containing spatial information of different granularities (as opposed to image encoders) Deep fusion is then performed. Specifically, a bidirectional feature pyramid network (BiFPN) structure is first used to obtain the aforementioned multi-scale information. Bidirectional Feature Pyramid Network (BiFPN) is a multi-scale feature fusion structure that uses a bidirectional propagation path, both top-down and bottom-up. For example, it first fused high-level semantic features... With high-resolution features Integration, and then step-by-step integration with , , Feature aggregation and propagation are performed through a top-down upsampling path and a bottom-up downsampling path, ultimately outputting an enhanced multimodal feature that combines global semantics and local details. To achieve four-scale image features and The deep integration of features enhances the consistency of cross-modal features, laying the foundation for the subsequent generation of discriminative cue embeddings.

[0090] Cross-modal semantic features, which integrate the semantics of radiotherapy guideline text and the spatial features of medical images, are the core representations and the basic input for semantic prompt generation. The four-scale visual feature set output by the image encoder corresponds to image features at different resolutions, covering multi-level information from global contours to local details (such as tumor edges and tissue textures). The first-scale feature of the image encoder corresponds to the low-resolution global image feature, capturing the overall location of the tumor and the information related to its anatomical structure. The second-scale feature of the image encoder corresponds to the low-to-medium resolution feature, which enhances the spatial relationship representation between the tumor and surrounding tissues. The third-scale feature of the image encoder corresponds to medium-to-high resolution features, focusing on the contour and morphological features of the tumor region. The fourth-scale feature of the image encoder corresponds to high-resolution features, capturing fine information such as the details and textures of the lunar tumor and the sharpness of its edges. The multimodal features fused by BiFPN integrate visual information at different scales and cross-modal semantic information, providing rich input for subsequent semantic embedding generation.

[0091] Batch Normalization (BN) balances visual features with Then, semantic embeddings are generated through multi-head cross-attention. Overlay position embedding And obtained through convolutional layer processing .

[0092] Batch Normalization (BN): A feature standardization operation used to balance visual features with... The numerical distribution reduces feature shift caused by modal differences, thereby improving model training stability and feature fusion performance. Semantic embedding vectors are obtained by semantically refining the features processed by BN through a multi-head cross-attention mechanism, accurately mapping the core semantic information related to radiotherapy (such as cancer type, treatment dose, and image feature association). Positional embedding is used to supplement the spatial location information of features, help the model distinguish knowledge units at different levels and locations, and improve the positional sensitivity of semantic matching. Enhanced semantic embedding, which combines positional embedding with convolutional layer processing, further integrates positional and semantic information through convolutional operations, thereby optimizing the expressive power of features.

[0093] Through adaptive pooling and convolutional layers, a two-layer multilayer perceptron, ) respectively generate dense prompt embeddings, ) and Sparse Prompt Embedding, The annotation is performed using a hierarchical structure of "cancer type - stage - treatment plan - adverse reactions - imaging features". This hierarchical structure is a knowledge annotation rule defined in this invention and is used to annotate... and Structured annotation ensures that the embedded hints are consistent with the organizational logic of the knowledge base, thereby improving the accuracy of retrieval and matching.

[0094] Adaptive pooling (ada-pooling): A dynamic pooling operation that adaptively adjusts the pooling window size based on the dimensionality and semantic importance of the input features. It achieves feature dimensionality reduction while preserving key semantic information, thus avoiding the loss of core information related to radiotherapy.

[0095] Convolutional layer (Conv): Feature extraction and dimension adjustment module, used for... Local feature enhancement and dimensionality adaptation are performed to provide adaptive input for dense / sparse cue embedding generation.

[0096] Two-layer multilayer perceptron ): A neural network module with two hidden layers, which realizes semantic mapping and dimensional transformation of features through nonlinear transformation, adapting to the generation requirements of prompt embedding.

[0097] Dense Prompt Embedding ): High-density semantic cue representation, containing rich fine-grained semantic information in the field of radiotherapy (such as specific treatment dose parameters and details of adverse reaction management), to enhance the accuracy of generated content.

[0098] Sparse Prompt Embedding Low-density key semantic cue representation focuses on core semantic nodes in the field of radiotherapy (such as cancer type names and core treatment plans), providing key semantic constraints for the generation process to improve the efficiency of retrieval and generation.

[0099] Step S1.4: Knowledge Vector Index Construction

[0100] The Sentence-BERT bidirectional encoder representation transformer model is used to convert structured knowledge units into high-dimensional semantic vectors, and a hybrid structure of "semantic vector index + keyword index" is constructed. The index weight allocation is optimized by the Extreme Gradient Boosting (XGBoost) algorithm to improve retrieval efficiency.

[0101] S2: Receive user queries, match knowledge units related to the query semantics in the knowledge base using a retrieval model optimized by deep contrastive learning, and sort the matching results; generate response content consistent with radiotherapy guidelines by combining a generative model with learnable cross-modal output tokens and a local optimizer.

[0102] Step S2 is used to build a deep learning-driven "retrieval-generation" integrated module to generate response content, including the following steps:

[0103] Step S2.1: Construct a precise retrieval mechanism

[0104] The retrieval mechanism in step S2.1 is as follows: A bidirectional long short-term memory network is used to parse the user's query intent, extracting key constraints such as guideline version, cancer stage, treatment method, and imaging features, and automatically supplementing implicit retrieval dimensions for fuzzy queries; a deep contrastive learning-based framework is used to optimize the spatial distribution of semantic vectors in the knowledge base, utilizing a standardized temperature-scaled cross-entropy loss function to cluster semantically similar knowledge units in the vector space and keep semantically dissimilar units apart; a candidate knowledge unit set is quickly coarsely screened from the knowledge base based on cosine similarity; a multidimensional input feature is constructed, including semantic matching degree, knowledge timeliness, application frequency, and cross-modal relevance; the candidate knowledge unit set is finely sorted using a multilayer perceptron, and the most relevant core knowledge unit is output as the input to the generation module. The specific implementation is as follows:

[0105] A bidirectional long short-term memory (BiLSTM) network is used to parse user query intent and extract key constraints such as guideline version, cancer stage, treatment method, and imaging features. For fuzzy queries, implicit dimensions are automatically added (such as default association with the latest guideline version and mainstream treatment plan) to achieve targeted retrieval.

[0106] Figure 3 This is a schematic diagram of a semantic vector optimization framework based on deep contrastive learning, according to an embodiment of the present invention. Figure 3 As shown, this invention proposes a semantic vector optimization framework based on deep contrastive learning (DeCLUTR), which is an unsupervised semantic optimization module designed in this invention to optimize the semantic vector space distribution of knowledge units in the radiotherapy field and improve the semantic matching accuracy during retrieval. The semantic vector space distribution is optimized by using a standardized temperature-scaled cross-entropy loss function (NT-Xent), which enhances the vector clustering of similar knowledge units and widens the vector distance between dissimilar units, thereby optimizing the semantic space structure. The loss function is defined as:

[0107]

[0108] : Cosine similarity calculation function, used to quantify the semantic matching degree of any two radiotherapy knowledge semantic vectors. The output value of the NT-Xent loss function measures the degree of optimization of the semantic vector space distribution. The smaller the loss value, the better the vector clustering effect of similar knowledge units, the greater the vector distance of dissimilar units, and the higher the retrieval matching accuracy. The natural logarithm function is used to perform a nonlinear transformation on the loss term, enhancing the sensitivity of the loss value to vector distribution bias and making the model more likely to identify unreasonable distributions in the semantic vector space. The exponential function is used to perform numerical transformation on the results of cosine similarity normalization, amplifying the differences in similarity and enhancing the loss function's ability to perceive the degree of semantic association. The semantic vectors of knowledge units in the field of radiotherapy to be optimized (such as vector representations of a radiotherapy guideline segment or image-related knowledge) serve as "anchor vectors" in comparative learning and are the core reference benchmarks for similarity calculation. :and Semantic vectors of radiotherapy knowledge units with similar semantics (such as treatment plan knowledge vectors for the same cancer type and stage, and handling rule vectors for the same adverse reaction) serve as "positive sample vectors" in contrastive learning to strengthen the aggregation of vectors of similar knowledge. :and Semantic vectors of radiotherapy domain knowledge units with semantic dissimilarity (such as treatment knowledge vectors for different cancer types, irrelevant diagnostic and treatment guidelines, and rules for handling different adverse reactions), where As a "negative sample vector" in contrastive learning, it is used to widen the vector distance between dissimilar knowledge. Temperature over-parameter, with values ​​of This is used to adjust the weight distribution of cosine similarity, controlling the degree of clustering in the semantic vector space. Smaller... It will amplify the differences in similarity, making the vector space distribution sparser; larger This will reduce the similarity difference and make the vector space distribution denser. This invention verifies through experiments that this value can optimally balance clustering and discriminability. Anchor vector With positive sample vector The cosine similarity quantifies the degree of semantic association between the two; the larger the value, the more similar the semantics of the corresponding knowledge units. Needle vector With negative sample vector The cosine similarity quantifies the degree of semantic association between the two; the smaller the value, the more distant the semantics of the corresponding knowledge units. The similarity between the anchor vector and the positive sample vector, after exponential transformation and temperature adjustment, is used as the numerator of the loss function to strengthen the association weight between the positive sample and the anchor vector. The summation of the similarity between all negative sample vectors and anchor vectors after exponential transformation is used to construct the denominator of the loss function. The ratio of this summation to the numerator reflects the discriminative power between the anchor vector and the positive and negative sample vectors.

[0109] A two-stage retrieval strategy of "coarse semantic vector retrieval + fine ranking by multilayer perceptron (MLP)" is adopted: the first stage quickly filters the top 50 relevant knowledge units by calculating cosine similarity; the second stage takes "semantic matching degree between query and knowledge unit, knowledge unit update time, clinical application frequency, and cross-modal feature relevance" as input features, and outputs accurate ranking results through MLP model to select the top 10 core knowledge units as input for the generation module.

[0110] Step S2.2: Controllable Generation Strategy

[0111] The controllable generation strategy in step S2.2 involves generating response content consistent with radiotherapy guidelines by combining a generative model with learnable cross-modal output tokens and a local optimizer. Specifically, this includes: constructing a generative framework based on a generative pre-trained transformation model incorporating pre-trained weights from the radiotherapy domain; concatenating the sparse and dense cue embeddings corresponding to the retrieved candidate knowledge units, along with the learnable cross-modal output tokens, with the original output tokens of the generative model, and inputting them into the mask decoder; simultaneously, fusing the dense cue embeddings with the features of the corresponding medical images as auxiliary input to the mask decoder; generating an original mask based on the above inputs; performing a dot product operation on the updated cross-modal output tokens and the cross-modal semantic features corresponding to the knowledge units using a local optimizer to generate an optimized mask; fusing the original mask and the optimized mask to obtain the final generative constraint mask used to constrain the generation process; and generating and outputting response content adapted to the current user based on the generative constraint mask, user identity category, and historical query records. The specific implementation is as follows:

[0112] Based on the Generative Pre-trained Transformer (GPT) series of models, this framework incorporates weights from pre-trained BERT models in the field of radiotherapy to enhance the accuracy of medical terminology and image association descriptions. It possesses powerful natural language generation capabilities and is used to output fluent and grammatically logical radiotherapy knowledge Q&A content.

[0113] Bidirectional Encoder Representations from Transformers (BERT): A language model pre-trained with data from the radiotherapy field, whose weights are incorporated into the GPT framework to enhance the accuracy of radiotherapy terminology (such as dose parameters and treatment technique names) and image-related descriptions (such as the correspondence between tumor image features and treatment plans).

[0114] The proposal includes learnable cross-modal output tokens. The enhancement technique, specifically the generative constraint enhancement module designed in this invention, captures cross-modal semantic associations through learnable tokens, providing accurate multimodal knowledge constraints for the generation process. Specifically, it embeds sparse hints corresponding to the retrieved multimodal knowledge units. Dense prompts embedded With the model's original output token , After concatenation, the data is input into the mask decoder. Image embedding and The fused material is used as another input to the decoder to generate the original mask. ( (For mask decoder).

[0115] It can learn cross-modal output tokens to carry cross-modal semantic information of radiotherapy guideline text and medical images, and work with the model's original output tokens to optimize the generated content. The model's original output token is the basic output unit of the GPT framework, carrying the fundamental semantic logic of the generated content. : Multi-vector concatenation function, used to concatenate multiple vectors. By dimensionally concatenating and integrating basic semantics, cross-modal constraints, and core semantic prompts, a joint input for the decoder is formed. The input vector of the mask decoder is... It is pieced together and contains the complete semantic constraint information required for generation. Image embedding vectors are obtained from medical images (such as CT images) through encoder processing, carrying spatial features and anatomical structure information of the image. Feature fusion operations (element-wise addition) are used to embed images. Embedded with dense prompts This involves fusing visual features with fine-grained semantic features to achieve collaborative constraints. The mask decoder... ): The core decoding unit of the generation module, receiving and Two types of inputs, the output is the original mask used to constrain the generated content. The original mask, generated by the mask decoder, is used to initially limit the semantic scope of the generated content and avoid deviating from the knowledge boundaries of the radiotherapy field.

[0116] Design a local refiner to update the cross-modal output token. and Each layer is processed by a three-layer multilayer perceptron. After processing, a dot product operation is performed to generate an optimized mask. Finally, by summing the logarithms, we obtain... To suppress the generation of AI "illusions" (such as fictitious treatment plans and incorrect dosage parameters).

[0117] The Local Refiner is a generation detail optimization module designed in this invention, used to correct local deviations in the original mask and improve the detail accuracy of the generated content. The updated cross-modal output token, after being iterated by the mask decoder, further strengthens the cross-modal semantic association of the output token. Cross-modal semantic features integrate the core semantic representations of radiotherapy guidelines and medical images, providing a foundation for local optimization. Three-layer multilayer perceptron... ): A neural network module containing three hidden layers, used for... and Nonlinear transformations and dimensionality adaptations are performed to enhance the semantic expressive power of features. ⊙: Dot product operation, used for... and Element-level multiplication is performed to focus on the common information between cross-modal semantics and the updated token, generating accurate local optimization signals. Optimized mask, obtained through dot product operation, is used to correct the original mask. Local deviations (such as errors in dose parameter descriptions and errors in image feature correlation). Logarithmic summation is used to convert the original mask into a logarithmic summation operation. With optimized mask The logarithmic results are summed, and the preliminary constraints and local optimization information are combined to form the final generative constraint mask. The final constraint mask is used to constrain the generation process, and is used to strictly constrain the generation process of the GPT model.

[0118] Mean pooling is used to convert the encoder output into a fixed-length embedding. The professional depth and expression of the answers are automatically adjusted according to different user profiles (clinical doctors, medical students, patients' families), and related knowledge is recommended in combination with historical query records.

[0119] Mean pooling: A feature aggregation operation used to convert high-dimensional features from the encoder output into fixed-length semantic embeddings, ensuring dimensionality consistency of the generated inputs while preserving core semantic information.

[0120] User profiles: The system defines three core user groups (clinical doctors, medical students, and patients' families) to trigger adaptation adjustments to the generated content (such as outputting professional dosage details and evidence-based basis on the doctor's end, and outputting simplified treatment process instructions on the patient's end).

[0121] Historical query records: Users' past radiotherapy knowledge query data, used to uncover potential needs and recommend related knowledge (such as automatically recommending "pre-radiotherapy imaging assessment" and "post-radiotherapy adverse reaction care" after querying "laryngeal cancer radiotherapy plan").

[0122] S3: Monitor the update information of external authoritative radiotherapy guidelines, automatically capture and extract the key update points; filter and sort the updated content through semantic relevance calculation, and after confirmation, integrate the valid updated content into the knowledge base and update the corresponding semantic vector index synchronously.

[0123] Step S3 is used to build a dynamic update mechanism for the guidelines. Figure 4 This is a flowchart illustrating the dynamic update and feedback optimization closed-loop process of an embodiment of the present invention. Figure 4 As shown, the guidelines are dynamically updated: Updated content is filtered and sorted using semantic relevance calculations, employing a similarity-based ranking strategy. This involves periodically accessing the websites of pre-defined authoritative radiotherapy guideline publishing institutions via web crawlers, and automatically identifying and extracting update information using natural language processing models. The similarity-based ranking strategy calculates the semantic similarity between the updated content to be integrated and existing related knowledge in the knowledge base, and prioritizes and extracts core points based on the similarity score. Valid updated content, after expert review and confirmation, is automatically integrated into the knowledge base, and the corresponding semantic index and vector representation are updated. The specific implementation is as follows:

[0124] A web crawler monitoring channel is established, which, combined with a natural language processing model, regularly crawls guideline update announcements from authoritative organizations' official websites, automatically identifying new version releases or revisions of older versions. The web crawler (WebCrawler) is an information monitoring tool built in this invention, used to regularly access the official websites of authoritative organizations such as the American Society of Radiation Therapy (ASTRO), the European Society of Radiation Therapy and Oncology (ESTRO), the National Comprehensive Cancer Network (NCCN), the American Society of Clinical Oncology (ASCO), and the Chinese Society of Clinical Oncology (CSCO), crawling update announcements and new version content of radiotherapy clinical guidelines. The natural language processing model (NLP) is used to parse the information crawled by the web crawler, automatically identifying key update events such as new guideline releases and revisions of older versions, and extracting core metadata such as update time and scope.

[0125] A similarity-based ranking self-updating strategy is proposed as a knowledge base update optimization strategy. By calculating the similarity scores between knowledge units, the strategy achieves accurate screening and ranking of updated content, ensuring that high-value update points are prioritized for inclusion in the knowledge base.

[0126] Calculate the similarity score for each knowledge unit:

[0127]

[0128] : No. The similarity score of each radiotherapy knowledge unit is used to quantify the degree of semantic association between the knowledge unit and other knowledge units. The higher the score, the broader the core semantic coverage of the unit, and the higher the update priority. The total number of radiotherapy knowledge units involved in similarity calculation, covering related knowledge units in the same cancer type and the same scenario in the knowledge base (such as a set of knowledge units on treatment plans for different stages of a certain cancer type). The index variable for the knowledge unit, with a value range of 1 to... Used to iterate over all elements except the first one. All other knowledge units outside of this unit. : Constraints to ensure the calculation of the first When scoring the similarity of individual knowledge units, similarity calculations are not performed with themselves to avoid invalid results. : No. Each radiotherapy knowledge unit is a standardized knowledge fragment after structured processing (such as a text + image-related knowledge unit about "radiotherapy dosage for stage III laryngeal cancer" in a certain guideline). : No. A radiotherapy knowledge unit, and Belonging to the same related knowledge set, used with Perform a similarity comparison. Cosine similarity calculation function, used for quantization. and The degree of semantic matching; the larger the value, the closer the semantic relationship between the two. Similarity summation operation, calculating the first similarity... The sum of the cosine similarities between a knowledge unit and all other knowledge units in the set yields the comprehensive similarity score for that unit. .

[0129] After sorting by score, extract key update points such as treatment plan adjustments, dosage standard changes, evidence-based updates, and image feature association rule optimizations, and generate an update summary.

[0130] After review and confirmation by radiation oncology experts, the corresponding knowledge units, semantic vectors, and index structures in the knowledge base are automatically updated. The update timeliness threshold is set to 72 hours to ensure synchronization with the latest guidelines and to retain historical versions for retrospective querying.

[0131] S4: Collect multi-dimensional feedback data from users and experts on the system output, and iteratively optimize the retrieval model and generation model based on the multi-dimensional feedback data to achieve a closed-loop improvement in system performance.

[0132] Step S4 is used to implement feedback optimization. Figure 4 This is a flowchart illustrating the dynamic update and feedback optimization closed-loop process of an embodiment of the present invention. Figure 4As shown, feedback optimization specifically includes the following steps:

[0133] S4.1: Multi-dimensional Feedback Collection and Analysis

[0134] The design incorporates multi-dimensional feedback, using explicit, implicit, and professional feedback data to continuously fine-tune the retrieval and generation models.

[0135] Explicit Feedback: Set up a three-level rating button for "Accuracy, Usability, and Understandability" and a text correction entry to support users in submitting specific opinions (such as factual errors, version errors, and image correlation deviations).

[0136] Implicit feedback: Capture potential satisfaction by analyzing user interaction behavior (dwell time, click behavior, repeat queries, frequency of knowledge unit usage) through model analysis;

[0137] Professional feedback: Access to the radiation oncology expert review channel to obtain professional ratings, correction opinions, and multimodal knowledge association optimization suggestions from radiation oncology experts on the system's output content.

[0138] Step S4.2: Iterative Model Optimization

[0139] Step S4.2, model iterative optimization, includes: retrieval model optimization: adding retrieval error samples identified in the multidimensional feedback data to the training set for fine-tuning the parameters of the semantic vector generation model and the retrieval ranking model; generation model optimization: using the correct answers corrected by experts as positive samples and the original incorrect answers from the system as negative samples, incrementally training the generation model and simultaneously updating its internal cross-modal constraint parameters. Specifically, it includes the following steps:

[0140] Step S4.2.1: Optimization of the retrieval model

[0141] The system collects multi-dimensional feedback information, including explicit user ratings, text correction opinions, implicit interaction behavior data, and expert review opinions. Samples of "retrieval omissions" and "association errors" from the multi-dimensional feedback data are incorporated into the training set. The semantic vector generation parameters of the sentence bidirectional encoder representation converter (Sentence-BERT model) and the contrastive learning parameters of the semantic vector optimization framework based on deep contrastive learning (DeCLUTR) are fine-tuned to optimize the weights of the retrieval ranking multilayer perceptron (MLP) model.

[0142] "Retrieval Omission" Samples: Samples in the feedback data marked as "the user's query contains relevant knowledge but the system did not retrieve it" (e.g., a user queries "stage IV lung cancer radiotherapy combination regimen," the knowledge base contains the corresponding guideline content but it was not retrieved and output). "Association Error" Samples: Samples in the feedback data marked as "the system's retrieval results do not semantically match the user's query" (e.g., a user queries "laryngeal cancer radiotherapy dosage," the system retrieves content related to esophageal cancer radiotherapy). Training Set: A sample set used for model optimization, supplemented with "retrieval omission" and "association error" samples to strengthen the model's learning of high-error scenarios. Sentence-BERT (Bidirectional Encoder Representation Transformer): A model used to generate semantic vectors for radiotherapy knowledge units. By fine-tuning its semantic vector generation parameters (such as attention weights and hidden layer output dimensions), the representation accuracy of semantic vectors is improved. Deep Contrast Learning-Based Semantic Vector Optimization Framework (DeCLUTR): A framework used to optimize the spatial distribution of semantic vectors. By fine-tuning its contrastive learning parameters, the vector clustering of similar knowledge units is further strengthened. Multilayer Perceptron (MLP): A model used for ranking retrieval results. By optimizing its weight parameters, the weight ratio of features such as semantic matching degree, update time, and clinical application frequency is adjusted to improve the accuracy of ranking results.

[0143] Step S4.2.2: Generative Model Optimization

[0144] Using the expert-corrected answers as positive samples and the original incorrect answers as negative samples, the generative model is incrementally pre-trained, and the cross-modal output tokens are updated simultaneously. The parameters are related to the local optimizer; the loss function uses a combination of focal loss and dice loss, i.e. The weight ratio Simultaneously, mean absolute error (MAE) loss is introduced for intersection-to-union (IoU) prediction, and cross-entropy (CE) loss is used for target detection. The overall loss is:

[0145]

[0146] The corrected responses from experts: The corrected responses obtained by radiation oncology experts after correcting the system's incorrect responses (such as correcting the system's fictitious "radiotherapy dose for laryngeal cancer" to the standard dose specified in the guidelines) are used as positive samples for model training.

[0147] Original incorrect answers: Answers generated by the system that do not meet the guidelines, contain "illusions," or have expression biases are used as negative samples for model training to help the model learn to avoid incorrect generation patterns.

[0148] Incremental pre-training: Based on the original pre-trained model, additional training is performed using newly added positive and negative samples, without retraining the entire model, thus improving optimization efficiency.

[0149] Learnable cross-modal output tokens ): A token carrying cross-modal semantic constraints updates its parameters during incremental pre-training, enhancing the capture of information related to radiotherapy text and images.

[0150] Local Refiner Parameters: Parameters used in the local optimizer to refine generation details (e.g., ... The weights and activation function parameters are updated to improve the ability to correct generation errors.

[0151] Focal loss ): This loss function is used to address the imbalance of generated sample classes. It assigns higher weights to hard-to-classify samples (such as rare cancer treatment plans) to improve the model's generation accuracy for complex scenarios.

[0152] Dice loss ): A loss function used to measure the overlap between generated content and the real answer (expert-corrected answer), reinforcing the factual consistency of the generated content.

[0153] The basic generation loss is obtained by weighted summation of the focus loss and the dice loss, as shown in the formula: It is used to constrain the overall accuracy of the generated content.

[0154] and The weighting coefficients and weighting ratios are: To balance the impact of the two types of losses.

[0155] Mean Absolute Error (MAE) Loss : A loss function used for Intersection over Union (IoU) prediction, which measures the absolute deviation between the predicted and actual IoU values ​​of the generated content and the true answer, thereby improving the boundary accuracy of the generated content.

[0156] Intersection over Union (IoU): A metric used to quantify the degree of overlap between generated content and the actual answer. A higher value indicates a higher degree of overlap and better generation accuracy.

[0157] Cross-entropy (CE) loss ( ): A loss function used for object detection, used here to constrain the accuracy of core entities (such as cancer type name, dosage parameters, treatment techniques) in the generated content, and to avoid entity recognition errors.

[0158] The optimized generation loss is calculated based on the optimized mask output by the local optimizer and is used to enhance the detail accuracy of the generated content.

[0159] Learnable weighting coefficients are used to adjust... The proportion of loss is adjusted by training to adaptively optimize the overall loss balance.

[0160] The overall loss function of the generative model comprehensively constrains the overall accuracy, detail precision, and entity correctness of the generated content.

[0161] like Figure 5 and Figure 6 As shown, the intelligent Q&A and guideline update system for radiotherapy knowledge, equipped with a feedback optimization mechanism, includes:

[0162] The structured knowledge base module 210 is used to construct a knowledge base that integrates radiotherapy guideline text with corresponding medical images and forms a structured index through cross-modal semantic feature extraction and semantic prompt embedding generation steps;

[0163] The retrieval-generation integrated module 220 is used to receive user queries, match knowledge units related to the query semantics in the knowledge base through a retrieval model optimized by deep contrastive learning, and sort the matching results; and generate response content consistent with the radiotherapy guidelines by combining a generative model with a learnable cross-modal output token and a local optimizer.

[0164] The dynamic update module 230 is used to monitor the update information of external authoritative radiotherapy guidelines, automatically capture and extract the key update points; filter and sort the updated content through semantic relevance calculation, and after confirmation, integrate the valid updated content into the knowledge base and update the corresponding semantic vector index synchronously.

[0165] The feedback optimization module 240 is used to collect multi-dimensional feedback data from users and experts on the system output, and to iteratively optimize the retrieval model and the generation model based on the multi-dimensional feedback data to achieve a closed-loop improvement in system performance.

[0166] The intelligent Q&A and guide update system for radiotherapy knowledge with feedback optimization mechanism provided in the embodiments of the present invention can execute the intelligent Q&A and guide update system for radiotherapy knowledge with feedback optimization mechanism provided in any of the embodiments of the present invention. It has the corresponding functions and beneficial effects of executing the method. For details, please refer to the relevant operations of the foregoing method embodiments, which will not be repeated here.

[0167] To verify the effectiveness of the method described in this embodiment of the invention, a comparative experiment was conducted. The experiment constructed a test set based on a publicly available tumor image-report dataset, containing 1000 text-image pairs covering different cancer types and stages of radiotherapy cases. Retrieval accuracy was defined as the percentage of queries where at least one standard answer was found in the top 10 results returned by the system. The consistency of the generated content was assessed by three senior radiation oncology experts using a double-blind method. The experts comprehensively evaluated the content based on factors such as factual accuracy, professionalism of expression, and reasonableness of association with images, and then gave a consistency percentage score (0-100%), which was averaged. The baseline systems for comparison included: a generative model based on the general GPT-3.5-turbo, a retrieval model based on Sentence-BERT, and a traditional sequence matching model based on BiLSTM. A direct comparison of the performance of the algorithm of this invention with the three traditional algorithms (GPT, Sentence-BERT, and BiLSTM) in the intelligent question-answering task for radiotherapy knowledge was conducted. Accuracy change curves under different training iterations were plotted using Python simulation experiments, as shown below. Figure 7 As shown, the horizontal axis is set to the number of training iterations from 10 to 100 (step size 10), and the vertical axis is the accuracy (range 0.6 to 1.0). Simulation data shows that as the number of training iterations increases, the accuracy of all four algorithms gradually increases. However, the algorithm of this invention (using a circular marker, blue solid line) maintains a significant leading advantage, with its accuracy steadily increasing from 0.72 after 10 training iterations to 0.95 after 100 training iterations. This is attributed to its fusion of the Deep Contrast Learning Semantic Vector Optimization Framework (DeCLUTR) and semantic cue projector technology, which effectively improves the accuracy of cross-modal knowledge representation and semantic matching. The upper limits of accuracy for GPT (using a square marker, orange dashed line), Sentence-BERT (using a triangle marker, green dotted line), and BiLSTM (using a diamond marker, red dotted line) are 81%, 83%, and 78%, respectively. Among them, GPT has a lower accuracy due to the lack of radiotherapy-specific constraints, Sentence-BERT is limited by its ability to generate semantic vectors from a single text and has insufficient cross-modal association, and BiLSTM, as a traditional sequence model, has the defect of limited semantic representation depth. The performance improvement of the three is significantly smaller than that of the algorithm of this invention, which fully verifies the superiority of this invention in the intelligent question answering task of radiotherapy knowledge.

[0168] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0169] It should also be noted that, in the embodiments of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0170] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in the embodiments of this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown in this application, but is to be accorded the widest scope consistent with the principles and novel features disclosed in the embodiments of this application.

Claims

1. A method for intelligent Q&A and guideline updating of radiotherapy knowledge with a feedback optimization mechanism, characterized in that: The method includes: S1: Construct a knowledge base that integrates radiotherapy guideline text with corresponding medical images and generates a structured index through cross-modal semantic feature extraction and semantic prompt embedding steps; The semantic prompt embedding generation step in the construction of the knowledge base is as follows: The cross-modal semantic features are fused with multi-scale visual features extracted from the corresponding medical images through a bidirectional feature pyramid network. The fused features are normalized and subjected to attention processing, and then a basic semantic embedding is generated through a multi-head attention mechanism. The basic semantic embeddings are processed in the following branches: the first branch generates dense cue embeddings containing fine-grained information through adaptive pooling and convolutional layer processing; the second branch generates sparse cue embeddings focusing on core semantic nodes through processing by a multilayer perceptron network; the dense cue embeddings and sparse cue embeddings will serve as structured semantic representations of the corresponding knowledge units in the knowledge base. S2: Receive user queries, match knowledge units related to the query semantics in the knowledge base using a retrieval model optimized by deep contrastive learning, and sort the matching results; generate response content consistent with radiotherapy guidelines by combining a generative model with learnable cross-modal output tokens and a local optimizer. In step S2, the retrieval process includes: A bidirectional long short-term memory network is used to parse user query intent, extract key constraints such as guideline version, cancer type and stage, treatment method and imaging features, and automatically supplement implicit search dimensions for fuzzy queries; A deep contrastive learning-based framework is used to optimize the spatial distribution of semantic vectors in the knowledge base. By using a standardized temperature-scaling cross-entropy loss function, semantically similar knowledge units are clustered in the vector space, while semantically dissimilar units are kept apart. Candidate knowledge units are quickly coarsely screened from the knowledge base based on cosine similarity; a multi-dimensional input feature including semantic matching degree, knowledge timeliness, application frequency and cross-modal relevance is constructed, and the candidate knowledge unit set is finely sorted through a multilayer perceptron to output the most relevant core knowledge unit as the input of the generation module; S3: Monitor the update information of external authoritative radiotherapy guidelines, automatically capture and extract the key update points; filter and sort the updated content through semantic relevance calculation, and after confirmation, integrate the valid updated content into the knowledge base and update the corresponding semantic vector index synchronously. S4: Collect multi-dimensional feedback data from users and experts on the system output, and iteratively optimize the retrieval model and generation model based on the multi-dimensional feedback data to achieve a closed-loop improvement in system performance.

2. The method according to claim 1, characterized in that, In S1, the construction of the knowledge base includes a cross-modal semantic feature extraction step, specifically: The radiotherapy guideline text and corresponding medical images were encoded separately to obtain linguistic and visual features; Feature fusion is achieved through a two-stage progressive attention mechanism: In the first stage, bidirectional multi-head cross-attention calculation is performed on the language features and visual features to obtain preliminary fused text-visual features for text querying vision and visual-text features for visual querying text; wherein, the projection transformation in the attention calculation is implemented through 3×3 depth convolution. In the second stage, the language features and visual features are concatenated and processed by convolution to generate intermediate fusion features. The intermediate fusion features are used as attention queries and cross-attention calculations are performed with the text-visual features and the visual-text features respectively to obtain two deepened fusion features. By fusing the two enhanced fusion features and processing them through a feedforward network, a cross-modal semantic feature that integrates textual semantics and image spatial information is output.

3. The method according to claim 1, characterized in that, In S2, the step of generating response content consistent with radiotherapy guidelines by combining a generative model with learnable cross-modal output tokens and a local optimizer specifically includes: A generative framework is constructed based on a generative pre-trained transformation model with pre-trained weights in the field of fusion radiotherapy, serving as the generative model; The sparse and dense cue embeddings corresponding to the retrieved candidate knowledge units, along with the learnable cross-modal output tokens, are concatenated with the original output token of the generative model and input together into the mask decoder. Simultaneously, the dense cue embeddings are fused with the features of the corresponding medical images as auxiliary inputs to the mask decoder. The mask decoder then generates the original mask. The updated cross-modal output token and the cross-modal semantic features corresponding to the knowledge unit are subjected to nonlinear transformation by a local optimizer, and then a dot product operation is performed to generate an optimized mask. The original mask and the optimized mask are fused to obtain the final generation constraint mask used for the constraint generation process. Based on the generated constraint mask, user identity category, and historical query records, generate and output response content adapted to the current user.

4. The method according to claim 3, characterized in that, In S3, the process of filtering and sorting the updated content through semantic relevance calculation adopts a similarity ranking strategy, which specifically includes the following steps: The system regularly accesses the websites of authoritative radiotherapy guideline publishers via web crawlers and automatically identifies and extracts updated guideline information using natural language processing models. The similarity ranking strategy is as follows: calculate the semantic similarity between the updated content to be integrated and the existing related knowledge in the knowledge base, and prioritize and extract the core points of the updated content based on the similarity score; the valid updated content confirmed by experts is automatically integrated into the knowledge base, and the corresponding semantic index and vector representation are updated.

5. The method according to claim 4, characterized in that, In S4, the multi-dimensional feedback data includes feedback from at least two of the following dimensions: Users directly rate the system's output in terms of accuracy, usability, and comprehensibility, and provide text corrections. The system infers potential satisfaction information by analyzing user interaction data. Professional review comments, ratings, and corrections from radiation oncology experts on the system's output content.

6. The method according to claim 1 or 5, characterized in that, In S4, model iterative optimization includes: Retrieval model optimization: Add the retrieval error samples identified in the multi-dimensional feedback data to the training set to fine-tune the parameters of the semantic vector generation model and the retrieval ranking model; Generative model optimization: The generative model is incrementally trained using the correct answers corrected by experts as positive samples and the incorrect answers from the original system as negative samples, while its internal cross-modal constraint parameters are updated synchronously.

7. The method according to claim 6, characterized in that, In S4, the training of the generative model adopts a combined loss function, which includes: a basic generation loss consisting of a weighted sum of focus loss and dice loss, a refined generation loss based on optimized mask calculation, and an auxiliary loss consisting of mean absolute error loss and cross-entropy loss; the combined loss function is a weighted sum of the basic generation loss, the refined generation loss, and the auxiliary loss.

8. A radiotherapy knowledge intelligent question-and-answer and guideline update system with a feedback optimization mechanism, used to implement the method described in any one of claims 1-7, characterized in that, The system includes: The structured knowledge base module is used to build a knowledge base that integrates radiotherapy guideline text with corresponding medical images and forms a structured index through cross-modal semantic feature extraction and semantic prompt embedding. The retrieval-generation integrated module receives user queries, matches semantically relevant knowledge units in the knowledge base using a retrieval model optimized through deep contrastive learning, and sorts the matching results. It then generates response content consistent with radiotherapy guidelines by combining a learnable cross-modal output token and a local optimizer with a generative model. The dynamic update module is used to monitor the update information of external authoritative radiotherapy guidelines, automatically capture and extract the key update points; filter and sort the updated content through semantic relevance calculation, and after confirmation, integrate the valid updated content into the knowledge base and update the corresponding semantic vector index synchronously. The feedback optimization module is used to collect multi-dimensional feedback data from users and experts on the system output, and to iteratively optimize the retrieval model and the generation model based on the multi-dimensional feedback data, so as to achieve a closed-loop improvement in system performance.

Images