Radiotherapy knowledge intelligent question answering and guide updating method and system with feedback optimization mechanism

Abstract

The invention relates to the crossing field of artificial intelligence and medical technology, and discloses a radiotherapy knowledge intelligent question answering and guide updating method and system with a feedback optimization mechanism, and the method comprises the steps: S1, constructing a knowledge base of structured indexes; s2, user query is received, and semantic-related knowledge units are retrieved and sorted through the retrieval model subjected to deep comparison learning optimization; combining the learnable cross-modal output token with a local optimizer through a generative model to generate response content consistent with the radiotherapy guide basis; s3, updating information of an external authoritative radiotherapy guide is monitored, and updating key points are automatically captured and extracted; the updated contents are screened and sorted, and the effective updated contents are integrated into a knowledge base after confirmation; and S4, collecting feedback data of users and experts, and carrying out iterative optimization on the retrieval model and the generation model to realize closed-loop improvement of system performance. According to the method, radiotherapy knowledge question and answer precision, guide updating dynamic and system performance continuous optimization can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and medical technology, in particular to a radiotherapy knowledge intelligent question answering and guideline updating method and system with feedback optimization mechanism. BACKGROUND

[0002] With the rapid development of radiation oncology, radiotherapy has become one of the core means of tumor treatment, and its knowledge system covers multiple disciplines such as radiation physics, radiation biology, and anatomy, and needs to keep up with the continuous update of clinical guidelines. At present, authoritative organizations such as American Society for Radiation Oncology (ASTRO), European Society for Radiotherapy and Oncology (ESTRO), National Comprehensive Cancer Network (NCCN), and Chinese Society of Clinical Oncology (CSCO) regularly publish or revise radiotherapy clinical guidelines, and promote the iterative upgrade of treatment programs, dose standards, and evidence-based basis.

[0003] The existing radiotherapy knowledge question answering and guideline application system has the following key technical defects: 1. Insufficient accuracy of knowledge representation: traditional systems mostly rely on single text semantic vector to construct knowledge base, which fails to effectively integrate cross-modal information of radiotherapy guideline text and computed tomography (CT) and other medical images, and lacks a standardized prompt embedding generation mechanism, resulting in limited accuracy of knowledge retrieval and generation; 2. Unreasonable distribution of semantic vector space: existing semantic vector generation methods mostly rely on a large amount of labeled data, making it difficult to effectively cluster similar knowledge units and distance apart non-similar units in unsupervised scenarios, resulting in low semantic matching accuracy during retrieval; 3. Poor inhibition effect of AI "hallucination": the generation model lacks effective cross-modal constraint mechanism, and is prone to generate fictional information (such as incorrect dose parameters and non-existent treatment plans) that deviates from the guidelines, which poses a serious safety hazard in high-risk radiotherapy field; 4. Delayed timeliness of guideline update: mostly using manual periodic update mode, which cannot quickly capture the revision dynamics of authoritative guidelines, and lacks precise screening and structured integration mechanism for updated content, resulting in disconnection between system knowledge and clinical practice; 5. No continuous optimization capability of the model: lacking multi-dimensional feedback collection and targeted iteration mechanism, it is difficult to correct retrieval bias and generation errors through user interaction and expert review data, and the system performance cannot adapt to clinical needs for a long time. SUMMARY

[0004] The present application aims at the defects of the prior art, and proposes a radiotherapy knowledge intelligent question answering and guideline updating method and system with a feedback optimization mechanism, which accurately solves the core problems of knowledge time lag, AI "illusion", insufficient information reliability and the like in the prior art through structured knowledge base construction, "retrieval-generation" integrated module design and dynamic updating and feedback optimization mechanism, and realizes accurate radiotherapy knowledge question answering, dynamic guideline updating and continuous optimization of system performance.

[0005] To achieve the above-mentioned purpose, the following technical solutions are implemented: In a first aspect, the present application provides a radiotherapy knowledge intelligent question answering and guideline updating method with a feedback optimization mechanism, which comprises: S1: constructing a knowledge base with structured indexes formed by fusing radiotherapy guideline texts and corresponding medical images and through cross-modal semantic feature extraction and semantic prompt embedding generation steps; S2: receiving a user query, matching knowledge units related to the query semantics in the knowledge base through a retrieval model optimized by deep contrast learning, and sorting the matching results; generating response content consistent with the radiotherapy guidelines through a generation model combined with a learnable cross-modal output token and a local optimizer; S3: monitoring the update information of external authoritative radiotherapy guidelines, automatically capturing and extracting update points; filtering and sorting the update content through semantic correlation calculation, and integrating the effective update content into the knowledge base after confirmation, and synchronously updating the corresponding semantic vector index; S4: collecting multi-dimensional feedback data of users and experts on system output, and iteratively optimizing the retrieval model and the generation model based on the multi-dimensional feedback data to realize closed-loop improvement of system performance.

[0006] Further, in S1, the construction of the knowledge base includes a cross-modal semantic feature extraction step, specifically: The radiotherapy guideline texts and corresponding medical images are encoded respectively to obtain language features and visual features; Feature fusion is performed through a two-stage progressive attention mechanism: In the first stage, bidirectional multi-head cross-attention calculation is performed on the language features and visual features to obtain preliminary fused text-visual features with text query vision and visual-text features with visual query text; wherein the projection transformation in the attention calculation is realized by 3x3 deep convolution; In the second stage, the language features and visual features are spliced and processed by convolution to generate intermediate fusion features; the intermediate fusion features are used as attention queries to perform cross-attention calculation with the text-visual features and the visual-text features respectively to obtain two deepened fusion features; The two deep fusion features are fused and processed by a feedforward network to output a cross-modal semantic feature that fuses text semantics and image spatial information.

[0007] Further, in S1, the construction of the knowledge base further includes a semantic prompt embedding generation step, specifically: The cross-modal semantic feature is fused with multi-scale visual features extracted from the corresponding medical image through a bidirectional feature pyramid network; The fused features are normalized and attention processed, and then a basic semantic embedding is generated through a multi-head attention mechanism; The basic semantic embedding is processed in the following branches: in a first branch, an adaptive pooling operation and a convolutional layer are used to generate a dense prompt embedding containing fine-grained information; in a second branch, a multi-layer perceptron network is used to generate a sparse prompt embedding focusing on core semantic nodes; the dense prompt embedding and the sparse prompt embedding serve as structured semantic representations of corresponding knowledge units in the knowledge base.

[0008] Further, in S2, the retrieval process includes: A bidirectional long short-term memory network is used to analyze the user query intent, extract key constraint conditions such as guide version, cancer stage, treatment method and image features, and automatically supplement the implied retrieval dimensions for fuzzy queries; A deep contrast learning-based framework is used to optimize the spatial distribution of semantic vectors in the knowledge base, and a standardized temperature scaling cross-entropy loss function is used to make semantically similar knowledge units cluster in the vector space and semantically different units repel each other; Based on cosine similarity, a candidate knowledge unit set is quickly and roughly screened out from the knowledge base; multi-dimensional input features including semantic matching degree, knowledge timeliness, application frequency and cross-modal relevance are constructed, and a multi-layer perceptron is used to fine-sort the candidate knowledge unit set, outputting the most relevant core knowledge unit as input for the generation module.

[0009] Further, in S2, the generation model combines a learnable cross-modal output token and a local optimizer to generate a response content consistent with the radiotherapy guidelines, specifically including: A generative pre-training transformation model that fuses pre-training weights in the radiotherapy field is used to construct a generation framework as the generation model; The sparse prompt embedding and the dense prompt embedding corresponding to the candidate knowledge unit obtained by retrieval, as well as the learnable cross-modal output token, are spliced with the original output token of the generation model and input into the mask decoder; at the same time, the dense prompt embedding is fused with the features of the corresponding medical image as auxiliary input for the mask decoder; the mask decoder generates an original mask; The updated cross-modal output token and the cross-modal semantic feature corresponding to the knowledge unit are respectively subjected to non-linear transformation by a local optimizer, and then dot product operation is performed to generate an optimized mask; and the original mask and the optimized mask are fused to obtain a final generation constraint mask used to constrain the generation process. Based on the generated constraint mask, the user identity category, and the historical query record, a response content adapted to the current user is generated and output.

[0010] Further, in S3, the filtering and sorting of the updated content by the semantic correlation degree calculation adopts a similarity sorting strategy, which specifically includes the following process: The network crawler periodically accesses the website of a preset authoritative radiotherapy guideline publishing agency, and automatically identifies and extracts the update information of the guideline by using a natural language processing model; The similarity sorting strategy is: calculating the semantic similarity of the updated content to be integrated and the existing related knowledge in the knowledge base, and prioritizing and extracting the core points of the updated content according to the similarity score; the effective updated content confirmed by the expert review is automatically integrated into the knowledge base, and the corresponding semantic index and vector representation are updated.

[0011] Further, in S4, the multi-dimensional feedback data includes feedback of at least two dimensions: The user's direct score on the accuracy, practicality, and ease of understanding of the system output content and text correction; The potential satisfaction information inferred by the system by analyzing the behavior data of the user and the system interaction; Professional review opinions, ratings, and corrected content of the system output content from a radiation oncology expert.

[0012] Further, in S4, the model iteration optimization includes: Retrieval model optimization: adding the retrieval error samples identified in the multi-dimensional feedback data to the training set to fine-tune the parameters of the semantic vector generation model and the retrieval sorting model; Generation model optimization: using the expert-corrected correct answer as a positive sample and the system's original wrong answer as a negative sample to incrementally train the generation model and update its internal cross-modal constraint parameters.

[0013] Further, in S4, the training of the generation model adopts a combined loss function, which includes: a basic generation loss composed of a focal loss and a dice loss, a refined generation loss calculated based on the optimized mask, and an auxiliary loss composed of a mean absolute error loss and a cross-entropy loss; the combined loss function is a weighted sum of the basic generation loss, the refined generation loss, and the auxiliary loss.

[0014] In a second aspect, the embodiments of the present application also provide a radiotherapy knowledge intelligent question answering and guideline updating system with a feedback optimization mechanism, comprising: a structured knowledge base module for constructing a knowledge base that integrates radiotherapy guideline texts and corresponding medical images, and forming a structured index through cross-modal semantic feature extraction and semantic prompt embedding generation steps; a retrieval-generation integrated module for receiving user queries, matching knowledge units related to query semantics in the knowledge base through a retrieval model optimized by deep contrast learning, and sorting the matching results; generating response content consistent with the radiotherapy guidelines through a generation model combined with a learnable cross-modal output token and a local optimizer; a dynamic updating module for monitoring update information of external authoritative radiotherapy guidelines, automatically capturing and extracting update highlights; filtering and sorting the update content through semantic correlation degree calculation, and integrating the effective update content into the knowledge base after confirmation, and synchronously updating the corresponding semantic vector index; a feedback optimization module for collecting multi-dimensional feedback data of users and experts on system output, iteratively optimizing the retrieval model and the generation model based on the multi-dimensional feedback data, and realizing closed-loop improvement of system performance.

[0015] Compared with the prior art, the present application has the following beneficial effects: 1. Significant improvement in knowledge representation accuracy: The standardized sparse / dense prompt embedding generated by the semantic prompt projector effectively integrates the semantic of radiotherapy texts and the spatial features of images, making the semantic representation of knowledge units more targeted, solving the problem of lack of standardized constraints in traditional knowledge representation, and through the fusion of cross-modal semantic features and multi-scale image features, the sparse prompt embedding focusing on core semantics ( ) and the dense prompt embedding containing fine-grained information ( ) are generated through BiFPN, BN, adaptive pooling, etc., providing accurate semantic constraints for subsequent retrieval and generation, laying a precise foundation, and effectively improving the accuracy of cross-modal knowledge association recognition; 2. High efficiency and accuracy of semantic retrieval: The semantic vector optimization framework (DeCLUTR) based on deep contrast learning optimizes the semantic vector space distribution through the standardized temperature scaling cross-entropy loss function (NT-Xent), solving the problem of low semantic matching accuracy in unsupervised scenarios. The correlation degree of knowledge units is quantified using cosine similarity, and the vector aggregation of similar knowledge units is strengthened through the loss function, improving the semantic matching efficiency during retrieval. Combined with the "coarse retrieval + fine sorting" two-stage strategy, the retrieval accuracy is improved to more than 95%; 3. AI "hallucination" generation effectively suppressed: learning cross-modal output tokens with dual constraints from local optimizers, combined with hard constraints of guideline text and image features, generates content with high consistency with authoritative guidelines, completely avoiding risks such as fictional treatment plans and incorrect dosage parameters; 4. Guideline update timeliness greatly improved: through network crawler monitoring and similarity sorting self-update strategy, fast capture, accurate screening and automatic integration of guideline update points are realized, ensuring knowledge timeliness, shortening guideline update response time, significantly improving efficiency compared with manual update, and accurately extracting core update points to avoid interference from redundant information; 5. System performance continuously self-adapting optimization: multi-dimensional feedback and iterative optimization mechanism enables the model to correct retrieval omissions and generation biases, and with the accumulation of feedback data, the retrieval model and generation model are continuously fine-tuned through explicit, implicit and professional feedback data, realizing closed-loop optimization of system performance, continuously improving retrieval accuracy and generation consistency, and continuously enhancing the ability to adapt to different cancer types and different user scenarios; 6. Wide application scenarios and strong practicality: suitable for professional diagnosis and treatment queries of clinicians, knowledge learning needs of medical students, and popularized consultation of patient families, while supporting historical version tracing and associated knowledge recommendation, balancing professionalism and ease of use, providing efficient support for knowledge dissemination and clinical practice in the field of radiotherapy.

[0016] It should be understood that the content described in the summary section is not intended to limit the key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other features, advantages, and aspects of embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings that illustrate, by way of example, the principles of the application. The drawings serve merely as an example and are not intended to limit the present application. In the drawings, the same or similar reference numerals denote the same or similar elements throughout the several views, in which: Figure 1 is a flowchart of a radiotherapy knowledge intelligent question-answering and guideline updating method with feedback optimization mechanism provided by an embodiment of the present application; Figure 2 is a system overall architecture and data flow diagram of an embodiment of the present application; Figure 3 is a semantic vector optimization framework based on deep contrastive learning of an embodiment of the present application; Figure 4 is a dynamic updating and feedback optimization closed-loop flowchart of an embodiment of the present application; Figure 5This is a schematic diagram of the modules of the intelligent Q&A and guide update system for radiotherapy knowledge with a feedback optimization mechanism provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the system functions according to an embodiment of the present invention; Figure 7 This is a performance comparison chart of the algorithm of this invention with three traditional algorithms: GPT, Sentence-BERT, and BiLSTM. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0020] like Figure 1 and 2 As shown, the intelligent question-and-answer and guideline update method for radiotherapy knowledge with a feedback optimization mechanism includes: S1: Construct a knowledge base that integrates radiotherapy guideline text with corresponding medical images and generates a structured index through cross-modal semantic feature extraction and semantic prompt embedding steps; Step S1 is used for the construction and preprocessing of the structured knowledge base, and includes the following steps: Step S1.1: Integration of authoritative data from multiple sources Based on the clinical guidelines for radiotherapy published by authoritative organizations such as the American Society of Radiation Therapy (ASTRO), the European Society of Radiation Therapy and Oncology (ESTRO), the National Comprehensive Cancer Network (NCCN), the American Society of Clinical Oncology (ASCO), and the Chinese Society of Clinical Oncology (CSCO), and supplemented by core journal articles, treatment guidelines, clinical pathology data standards, and computed tomography (CT) images related to diagnosis and treatment, a comprehensive data source system covering multiple cancer types, stages, and modalities is established.

[0021] Step S1.2: Cross-modal semantic feature extraction Step S1.2 is a cross-modal semantic feature extraction step for constructing the knowledge base, specifically: encoding the radiotherapy guideline text and the corresponding medical image respectively to obtain language features and visual features; feature fusion is performed through a two-stage progressive attention mechanism: in the first stage, bidirectional multi-head cross attention calculation is performed on the language features and visual features to obtain preliminary fusion text-to-vision features and vision-to-text features; wherein the projection transformation in the attention calculation is realized by a 3x3 deep convolution; in the second stage, the original language features and visual features are spliced and processed by convolution to generate intermediate fusion features; the intermediate fusion features are used as attention queries to perform cross attention calculation with the text-to-vision features and the vision-to-text features respectively to obtain two deep fusion features; the two deep fusion features are fused and processed by a feedforward network to output cross-modal semantic features that fuse text semantics and image spatial information. The specific implementation is as follows: A Bidirectional Encoder Representations from Transformers (BERT) model pre-trained in the radiotherapy field is used to automatically extract core entities such as cancer name, treatment technology, dose parameter, and image feature correlation index, and to construct a radiotherapy field-specific entity dictionary.

[0022] A two-stage progressive cross-attention cross-modal semantic interaction mechanism is proposed to associate the guideline text description with the corresponding CT image features. The medical image (wherein represents the depth, height, and width of the image) and the corresponding text description ( is the text length), the text is supplemented with a sentence start marker ([SOS]) and a sentence end marker ([EOS]) and projected into a text embedding sequence , the image and the text are input into corresponding encoders to obtain visual features and language features .

[0023] : Radiotherapy-related medical image data (such as CT images) in the form of a three-dimensional matrix, covering image information of the tumor and surrounding tissues. : The dimension space of the medical image , represents the real number field, corresponding to the depth, height, and width dimensions of the image. : The depth dimension of the medical image, i.e., the number of slices of the image, corresponding to the layer thickness dimension of the CT image in radiotherapy diagnosis and treatment. The height dimension of a medical image refers to the number of pixels in the vertical direction of a single image slice. The width dimension of a medical image refers to the number of pixels in the horizontal direction of a single image slice. The radiotherapy guideline text description data corresponding to medical images is in one-dimensional sequence form, covering core information such as cancer type, treatment plan, and dosage parameters. Text description dimensional space, Indicates the length of the text sequence (i.e., the number of characters or words). Text description The length of the text refers to the total number of characters or words contained in the text. [SOS]: Sentence start marker, used to identify the beginning of the radiotherapy guideline text description, assisting the model in recognizing text boundaries. [EOS]: Sentence end marker, used to identify the end of the radiotherapy guideline text description, assisting the model in fully capturing the text's semantics. The normalized text embedding sequence generated after being supplemented with [SOS] and [EOS] tags is used for subsequent encoder processing. Text embedding sequence dimensional space, For the dimensions of text embedding, Corresponding original text length Add the length of the two markers. The dimension of text embedding, i.e. the dimension of the feature vector obtained after the text is projected, is used to quantify the semantic information of the text. Medical Imaging The visual feature vectors obtained after encoder processing cover spatial feature information such as image texture, contour, and tumor location. Text embedding sequence The language feature vectors obtained after encoder processing cover textual feature information such as semantics and logical relationships in the radiotherapy guidelines.

[0024] The first layer achieves preliminary modality fusion through multi-head cross-attention computation, employing... Depth-convolutional projection replaces direct linear projection to capture local information; the calculation formula is as follows:

[0025] : Multi-head cross-attention function, used to achieve cross-modal fusion of visual and linguistic features. The first parameter is projected as a query vector, and the second parameter is projected as a key vector and a value vector. The output features of the first-level cross-attention operation are obtained through language features. For query, visual features The key and value are obtained by operation, and the language realizes semantic guidance to vision. : The output feature of the first level cross-attention operation, which is obtained by taking the visual feature as the query, the language feature as the key and value, and realizing the feature complement of vision to language. Deep convolution projection: a feature processing method used to replace direct linear projection, which extracts local features through a deep convolution kernel with a size of 1*1, and strengthens the association capture of radiotherapy-related image details and text keywords.

[0026] The second level takes and after splicing, and processes them through a convolution layer to obtain , and takes as the query and , performs cross-attention operation, and the formula is:

[0027] Finally, the sum and the feedforward network (FF) output cross-modal semantic features (CS-Features) , forming a multi-modal knowledge unit of "text + image".

[0028] : Feature splicing function, used to splice the visual feature and the language feature by dimension, and integrate the original feature information of the two modalities. : Convolution operation function, used to perform dimension fusion and redundant information filtering on the spliced features, and output more compact fusion features. : Intermediate output feature of the second level feature processing, which is obtained by performing convolution operation on the splicing result of and , and used as the query vector for subsequent cross-attention operation. : The output feature of the second level cross-attention operation, which is obtained by taking as the query, as the key and value, and deepening the cross-modal semantic fusion. : The output feature of the second level cross-attention operation, which is obtained by taking as the query, as the key and value, and optimizing the cross-modal feature-consistency. : Feedforward network (Feed Forward Network), used to perform nonlinear transformation and dimension adjustment on the features output by the second level cross-attention, and strengthen the feature expression ability. : Cross-modal Semantic Features (CS-Features) that integrate the semantics of the radiotherapy guideline text and the spatial features of the medical image, forming a multi-modal knowledge unit core representation of "text + image".

[0029] In summary, the above two-stage fusion process realizes the transition from preliminary association to deep semantic integration: first, the text and image features are guided and preliminarily aligned through bidirectional cross-attention; then, the original features and the preliminarily aligned features are fused again to generate a more comprehensive intermediate representation, and the preliminary alignment result is calibrated and deepened based on this intermediate representation, finally outputting a unified semantic representation that accurately reflects the correspondence between "text" and "image".

[0030] Step S1.3: Semantic prompt embedding generation Step S1.3 is a semantic prompt embedding generation step for building a knowledge base, specifically: the cross-modal semantic features and the multi-scale visual features extracted from the corresponding medical image are fused through a bidirectional feature pyramid network; the fused features are normalized and attention processed, and then a basic semantic embedding is generated through a multi-head attention mechanism; the basic semantic embedding is processed in the following branches: in the first branch, a dense prompt embedding containing fine-grained information is generated through an adaptive pooling operation and a convolutional layer processing; in the second branch, a sparse prompt embedding focusing on the core semantic nodes is generated through a multi-layer perceptron network; the dense prompt embedding and the sparse prompt embedding will serve as the structured semantic representation of the corresponding knowledge unit in the knowledge base. The specific implementation is as follows: The Semantic Prompt Projector proposed in the embodiments of the present application is a key module for converting original features to standardized prompt embeddings in the present embodiment, which is used to convert cross-modal semantic features and multi-scale image features into standardized prompt embeddings to provide precise semantic constraints for subsequent retrieval and generation. The design purpose is to deeply fuse the text-image fusion features rich in semantics, i.e., cross-modal semantic features and image multi-scale features (MIF) containing different granularity spatial information of the image encoder. Specifically, first, the above multi-scale information is obtained through a bidirectional feature pyramid network (BiFPN) structure . The bidirectional feature pyramid network (BiFPN) is a multi-scale feature fusion structure that propagates through a bidirectional path from top to bottom and from bottom to top, for example, first fuses the high-level semantic features and high-resolution features , and then gradually fuses them with , , Feature aggregation and transmission are performed through a top-down upsampling path and a bottom-up downsampling path, and finally an enhanced multi-modal feature with global semantics and local details is output , realizing deep fusion of four-scale image features and , strengthening the consistency of cross-modal features, and laying a foundation for subsequent generation of distinctive prompt embedding.

[0031] : Cross-modal semantic features, integrating the core representation of radiotherapy guideline text semantics and medical image spatial features, are the basic input for semantic prompt generation. : The four-scale visual feature set output by the image encoder, corresponding to image features at different resolutions, covers multi-level information from global contours to local details (such as tumor edges and tissue texture). : The first scale feature of the image encoder, corresponding to low-resolution global image features, captures the overall position of the tumor and the associated information of the anatomical structure. : The second scale feature of the image encoder, corresponding to medium-low resolution features, strengthens the spatial relationship representation of the tumor and the surrounding tissue. : The third scale feature of the image encoder, corresponding to medium-high resolution features, focuses on the contour and morphological features of the tumor region. : The fourth scale feature of the image encoder, corresponding to high-resolution features, captures fine information such as tumor detail texture and edge sharpness. : The multi-modal feature after BiFPN fusion, integrating visual information of different scales and cross-modal semantic information, provides rich input for subsequent semantic embedding generation.

[0032] After balancing the visual features and through batch normalization (Batch Normalization, BN), semantic embedding is generated through multi-head cross-attention, position embedding is superimposed, and is obtained through convolutional layer processing.

[0033] Batch Normalization (Batch Normalization, BN): Feature standardization operation, used to balance the numerical distribution of visual features and , reduce feature deviation caused by modal difference, and improve model training stability and feature fusion effect. : Semantic embedding vector, obtained by semantic extraction of BN-processed features through multi-head cross-attention mechanism, accurately maps core semantic information related to radiotherapy (such as cancer type, treatment dose, and image feature association). Positional Embedding, used to supplement the spatial position information of features, assist the model in distinguishing knowledge units at different levels and different positions, and improve the position sensitivity of semantic matching. Enhanced semantic embedding after superimposing positional embedding and processing by convolutional layer, further integrating position information and semantic information through convolution operation, and optimizing the expression ability of features.

[0034] Through adaptive pooling (ada-pooling) and convolutional layer, two-layer multilayer perceptron (Two-Layer Multilayer Perceptron, ), dense prompt embedding (Dense Prompt Embedding, ) and sparse prompt embedding (Sparse Prompt Embedding, ) are generated respectively, and the hierarchical structure of "cancer type-staging-treatment plan-adverse reaction-image feature" is labeled. The hierarchical structure is the knowledge labeling rule defined by the present application, which is used for structured labeling of and , so that the prompt embedding is consistent with the organization logic of the knowledge base, and the retrieval matching accuracy is improved.

[0035] Adaptive pooling (ada-pooling): dynamic pooling operation, which can adaptively adjust the pooling window size according to the dimension and semantic importance of the input features, while preserving key semantic information and realizing feature dimension reduction, avoiding loss of radiotherapy-related core information.

[0036] Convolutional layer (Conv): feature extraction and dimension adjustment module, used for local feature enhancement and dimension adaptation of , providing adaptive input for dense / sparse prompt embedding generation.

[0037] Two-layer multilayer perceptron (Two-Layer Multilayer Perceptron, ): neural network module containing two hidden layers, which realizes semantic mapping and dimension conversion of features through nonlinear transformation, and adapts to the generation requirements of prompt embedding.

[0038] Dense prompt embedding (Dense Prompt Embedding, ): high-density semantic prompt representation, containing rich fine-grained semantic information in the field of radiotherapy (such as specific treatment dose parameters, adverse reaction handling details), used to strengthen the accuracy of generated content.

[0039] Sparse prompt embedding (Sparse Prompt Embedding, ): low-density key semantic prompt representation, focusing on the core semantic nodes in the field of radiotherapy (such as cancer name, core treatment plan), providing key semantic constraints for the generation process, and used to improve the efficiency of retrieval and generation.

[0040] Step S1.4: knowledge vector index construction The structured knowledge units are converted into high-dimensional semantic vectors by using a Sentence-BERT model, a mixed structure of "semantic vector index + keyword index" is constructed, and the index weight distribution is optimized by an Extreme Gradient Boosting (XGBoost) algorithm to improve the retrieval efficiency.

[0041] S2: receiving a user query, matching the knowledge units related to the query semantics in the knowledge base through the retrieval model optimized by deep contrast learning, and sorting the matching results; generating a response content consistent with the radiotherapy guideline through a generation model combined with a learnable cross-modal output token and a local optimizer; Step S2 is used to construct a deep learning driven "retrieval-generation" integrated module to generate a response content, including the following steps: Step S2.1: constructing an accurate retrieval mechanism The retrieval mechanism of step S2.1 is as follows: a bidirectional long short-term memory network is used to analyze the user query intention, extract the key constraint conditions of the guideline version, cancer stage, treatment method and image features, and automatically supplement the implicit retrieval dimensions for fuzzy queries; a deep contrast learning based framework is used to optimize the spatial distribution of semantic vectors in the knowledge base, and a standardized temperature scaling cross-entropy loss function is used to make similar semantic units gather in the vector space and dissimilar units move away from each other; based on the cosine similarity, a candidate knowledge unit set is quickly and roughly screened out from the knowledge base; a multi-dimensional input feature including semantic matching degree, knowledge timeliness, application frequency and cross-modal relevance is constructed, and a multi-layer perception machine is used to accurately sort the candidate knowledge unit set, and output the most relevant core knowledge unit as the input of the generation module. The specific implementation is as follows: A bidirectional long short-term memory network (BiLSTM) is used to analyze the user query intention, extract the key constraint conditions of the guideline version, cancer stage, treatment method and image features, and automatically supplement the implicit dimensions (such as the default association with the latest guideline version and the mainstream treatment plan) for fuzzy queries, so as to realize directional retrieval.

[0042] Figure 3 is a schematic diagram of a semantic vector optimization framework based on deep contrast learning according to an embodiment of the present application. As shown in Figure 3As shown, this invention proposes a semantic vector optimization framework based on deep contrastive learning (DeCLUTR), which is an unsupervised semantic optimization module designed in this invention to optimize the semantic vector space distribution of knowledge units in the radiotherapy field and improve the semantic matching accuracy during retrieval. The semantic vector space distribution is optimized by using a standardized temperature-scaled cross-entropy loss function (NT-Xent), which enhances the vector clustering of similar knowledge units and widens the vector distance between dissimilar units, thereby optimizing the semantic space structure. The loss function is defined as:

[0043] : Cosine similarity calculation function, used to quantify the semantic matching degree of any two radiotherapy knowledge semantic vectors. The output value of the NT-Xent loss function measures the degree of optimization of the semantic vector space distribution. The smaller the loss value, the better the vector clustering effect of similar knowledge units, the greater the vector distance of dissimilar units, and the higher the retrieval matching accuracy. The natural logarithm function is used to perform a nonlinear transformation on the loss term, enhancing the sensitivity of the loss value to vector distribution bias and making the model more likely to identify unreasonable distributions in the semantic vector space. The exponential function is used to perform numerical transformation on the results of cosine similarity normalization, amplifying the differences in similarity and enhancing the loss function's ability to perceive the degree of semantic association. The semantic vectors of knowledge units in the radiotherapy field that need to be optimized (such as vector representations of a radiotherapy guideline segment or image-related knowledge) serve as "anchor vectors" in comparative learning and are the core reference benchmarks for similarity calculation. :and Semantic vectors of radiotherapy knowledge units with similar semantics (such as treatment plan knowledge vectors for the same cancer type and stage, and handling rule vectors for the same adverse reaction) serve as "positive sample vectors" in contrastive learning, used to strengthen the vector aggregation of similar knowledge. :and Semantic vectors of radiotherapy domain knowledge units with semantic dissimilarity (such as treatment knowledge vectors for different cancer types, irrelevant diagnostic and treatment guidelines, and rules for handling different adverse reactions), where As a "negative sample vector" in contrastive learning, it is used to widen the vector distance between dissimilar knowledge. Temperature over-parameter, with values ​​of This is used to adjust the weight distribution of cosine similarity, controlling the degree of clustering in the semantic vector space. Smaller... It will amplify the differences in similarity, making the vector space distribution sparser; larger Will reduce the similarity difference, and make the vector space distribution more dense. The present application verifies through experiments that this value can optimally balance the aggregation and discrimination. : anchor point vector Cosine similarity with positive sample vector Quantify the semantic association degree of the two, the larger the value, the more similar the semantic of the corresponding knowledge unit. : needle point vector Cosine similarity with negative sample vector Quantify the semantic association degree of the two, the smaller the value, the more distant the semantic of the corresponding knowledge unit. : The result of the similarity between the anchor point vector and the positive sample vector after exponential conversion and temperature adjustment, as the numerator of the loss function, used to strengthen the association weight between the positive sample and the anchor point vector. : The sum of the similarity between all negative sample vectors and anchor point vectors after exponential conversion, used to construct the denominator of the loss function, reflecting the discrimination between anchor point vectors and positive and negative sample vectors through the ratio of the numerator.

[0044] Adopt the two-stage retrieval strategy of "semantic vector coarse retrieval + multilayer perceptron (MLP) fine sorting": the first stage quickly filters Top50 related knowledge units through cosine similarity calculation; the second stage takes "semantic matching degree between query and knowledge unit, knowledge unit update time, clinical application frequency, cross-modal feature correlation" as input features, and outputs accurate sorting results through MLP model to filter Top10 core knowledge units as input of the generation module.

[0045] Step S2.2: controllable generation strategy The controllable generation strategy of step S2.2: generating a response content consistent with the radiotherapy guideline by combining a generative model with a learnable cross-modal output token and a local optimizer, specifically including: constructing a generative framework based on a generative pre-training transformation model fused with pre-training weights in the radiotherapy field as the generative model; splicing the sparse prompt embedding corresponding to the retrieved candidate knowledge unit, the dense prompt embedding, and the learnable cross-modal output token with the original output token of the generative model, and inputting them into the mask decoder together; at the same time, fusing the dense prompt embedding with the features of the corresponding medical image as the auxiliary input of the mask decoder; the mask decoder generates an original mask based on the above inputs; the cross-modal output token and the cross-modal semantic features corresponding to the knowledge unit are respectively subjected to nonlinear transformation by the local optimizer, and then dot product operation is performed to generate an optimized mask; the original mask and the optimized mask are fused to obtain a generative constraint mask used to constrain the generation process; based on the generative constraint mask, the user identity category and the historical query record, a response content adapted to the current user is generated and output. The specific implementation is as follows: The generative pre-training transformation model (GPT) series model is used as the basis of the generation framework, the BERT model weight pre-trained in the radiotherapy field is fused, the accuracy of the description of medical professional terms and images is strengthened, and the natural language generation capability is strong, which is used to output fluent and grammatically logical radiotherapy knowledge question and answer content.

[0046] Bidirectional Encoder Representations from Transformers (BERT): a language model pre-trained on radiotherapy field data, whose weight is fused into the GPT framework to strengthen the accuracy of the description of radiotherapy professional terms (such as dose parameters and treatment technology names) and image association (such as the correspondence between tumor image features and treatment plans).

[0047] A learnable cross-modal output token (Cross-modal Output Tokens, ) enhancement technology is proposed, which is the generation constraint enhancement module designed in the present application, which captures the cross-modal semantic association through the learnable token to provide accurate multi-modal knowledge constraint for the generation process. Specifically, the sparse prompt embedding , the dense prompt embedding and the original output token of the model , are spliced and input into the mask decoder (Mask Decoder), i.e. , the image embedding and Fused as another input of the decoder, to generate the original mask (

[0048] : A learnable cross-modal output token, used to carry the cross-modal semantic information of the radiotherapy guideline text and medical image, and to optimize the generated content in cooperation with the original output token of the model. : The original output token of the model, the basic output unit of the GPT framework, carrying the basic semantic logic of the generated content. : A multi-vector concatenation function, used to concatenate in dimensions, integrating basic semantics, cross-modal constraints, and core semantic prompts to form the joint input of the decoder. : The input vector of the mask decoder, concatenated by , containing complete semantic constraint information required for generation. : Image embedding vector, obtained by processing medical images (such as CT images) through an encoder, carrying spatial features and anatomical structure information of the image. : Feature fusion operation (element-wise addition in this case), used to fuse image embedding with dense prompt embedding to achieve collaborative constraints of visual features and fine-grained semantic features.Mask Decoder ( ): The core decoding unit of the generation module, receiving and two types of input, and outputting the original mask used to constrain the generated content. : The original mask, generated by the mask decoder, used to preliminarily limit the semantic range of the generated content, avoiding deviating from the knowledge boundary of the radiotherapy field.

[0049] Design a local refiner to process the updated cross-modal output token and respectively through a three-layer multilayer perceptron (Three-Layer Multilayer Perceptron, ), and then perform dot product operation to generate an optimized mask , and finally obtain by summing the logarithm to suppress AI "hallucinations" (such as fictional treatment plans, incorrect dose parameters).

[0050] The local refiner (Local Refiner) is a generated detail optimization module designed by the invention, used to correct local deviations of the original mask and improve the detail accuracy of the generated content. : The updated cross-modal output token, after iteration through the mask decoder, further strengthens the output token's cross-modal semantic association. : Cross-modal semantic features, which integrate the core semantic representations of radiotherapy guideline texts and medical images, provide a basis for local optimization. Three-layer multilayer perceptron, : A neural network module with three hidden layers, used for nonlinear transformation and dimension adaptation of and to enhance the semantic expression ability of features. : Dot product operation, used for element-level multiplication of to focus on the common information of cross-modal semantics and updated tokens, generating accurate local optimization signals. : Optimization mask, obtained through dot product operation, used to correct the local bias of the original mask (such as dose parameter expression errors and image feature association errors). : Log sum operation, used to sum the log results of the original mask and the optimization mask , combining preliminary constraints and local optimization information to form the final generated constraint mask. : The final generated constraint mask used in the constraint generation process, used to strictly constrain the generation process of the GPT model.

[0051] Mean pooling is used to convert the encoder output into a fixed-length embedding. The system automatically adjusts the professional depth and expression of the answer for different user profiles (clinicians, medical students, and patient family members) and recommends related knowledge based on historical query records.

[0052] Mean pooling: a feature aggregation operation used to convert high-dimensional features from the encoder output into a fixed-length semantic embedding, ensuring the consistency of the generated input dimensions while preserving core semantic information.

[0053] User profile: a system-defined three-category core user group (clinicians, medical students, and patient family members) used to trigger adaptive adjustments of generated content (such as professional dose details and evidence-based recommendations for doctors, and simplified treatment process explanations for patients).

[0054] Historical query records: users' past radiotherapy knowledge query data used to mine potential needs and recommend related knowledge (such as after querying "throat cancer radiotherapy plan", automatically recommending "pre-radiotherapy image evaluation" and "post-radiotherapy adverse reaction care").

[0055] S3: Monitor the update information of the external authoritative radiotherapy guidelines, automatically capture and extract the update points; screen and sort the update content through semantic correlation calculation, and after confirmation, integrate the effective update content into the knowledge base, and update the corresponding semantic vector index synchronously; Step S3 is used for constructing a guideline dynamic updating mechanism. Figure 4 is a dynamic updating and feedback optimization closed-loop flowchart of the embodiment of the application. As shown in Figure 4 the guideline dynamic updating: the update content is screened and sorted through semantic correlation calculation, a similarity sorting strategy is adopted: the network crawler regularly accesses the website of the preset authoritative radiotherapy guideline publishing organization, and the natural language processing model is used to automatically identify and extract the update information of the guideline; the similarity sorting strategy is: the semantic similarity of the update content to be integrated and the existing related knowledge in the knowledge base is calculated, and the update content is prioritized and the core points are extracted according to the similarity score; the effective update content confirmed by the expert review is automatically integrated into the knowledge base, and the corresponding semantic index and vector representation are updated. The specific implementation is as follows: A network crawler (Web Crawler) monitoring channel is established, and a natural language processing model is combined to regularly crawl the guideline update announcements of the official websites of authoritative organizations, and automatically identify the new version release or old version revision information. The network crawler (Web Crawler): the information monitoring tool constructed by the application, which is used to regularly access the official websites of the American Society for Radiation Oncology (ASTRO), the European Society for Radiotherapy and Oncology (ESTRO), the National Comprehensive Cancer Network (NCCN), the American Society of Clinical Oncology (ASCO), the Chinese Society of Clinical Oncology (CSCO) and other authoritative organizations, and to crawl the update announcements and new version content of the radiotherapy clinical guidelines. The natural language processing model (Natural Language Processing Model) is used to analyze the information crawled by the network crawler, automatically identify the key update events such as new version release and old version revision of the guideline, and extract core metadata such as update time and update range.

[0056] A similarity sorting self-updating strategy is proposed as an optimization strategy for updating the knowledge base, which realizes the accurate screening and sorting of the update content by calculating the similarity scores between knowledge units, and ensures that high-value update points are preferentially included in the knowledge base.

[0057] The similarity scores of the knowledge units are calculated:

[0058] : the first The similarity score of each radiotherapy knowledge unit is used to quantify the degree of semantic association between the knowledge unit and other knowledge units. The higher the score, the broader the core semantic coverage of the unit, and the higher the update priority. The total number of radiotherapy knowledge units involved in similarity calculation, covering related knowledge units in the same cancer type and the same scenario in the knowledge base (such as a set of knowledge units on treatment plans for different stages of a certain cancer type). The index variable for the knowledge unit, with a value range of 1 to... Used to iterate over all elements except the first one. All other knowledge units outside of this unit. : Constraints to ensure the calculation of the first When scoring the similarity of individual knowledge units, similarity calculations are not performed with themselves to avoid invalid results. : No. Each radiotherapy knowledge unit is a standardized knowledge fragment after structured processing (such as a text + image-related knowledge unit about "radiotherapy dosage for stage III laryngeal cancer" in a certain guideline). : No. A radiotherapy knowledge unit, and Belonging to the same related knowledge set, used with Perform a similarity comparison. Cosine similarity calculation function, used for quantization. and The degree of semantic matching; the larger the value, the closer the semantic relationship between the two. Similarity summation operation, calculating the first similarity... The sum of the cosine similarities between a knowledge unit and all other knowledge units in the set yields the comprehensive similarity score for that unit. .

[0059] After sorting by score, extract key update points such as treatment plan adjustments, dosage standard changes, evidence-based updates, and image feature association rule optimizations, and generate an update summary.

[0060] After review and confirmation by radiation oncology experts, the corresponding knowledge units, semantic vectors, and index structures in the knowledge base are automatically updated. The update timeliness threshold is set to 72 hours to ensure synchronization with the latest guidelines and to retain historical versions for retrospective querying.

[0061] S4: Collect multi-dimensional feedback data from users and experts on the system output, and iteratively optimize the retrieval model and generation model based on the multi-dimensional feedback data to achieve a closed-loop improvement in system performance.

[0062] Step S4 is used to implement feedback optimization. Figure 4 This is a flowchart illustrating the dynamic update and feedback optimization closed-loop process of an embodiment of the present invention. Figure 4As shown, the feedback optimization specifically includes the following steps: S4.1: Multi-dimensional feedback collection and analysis Design multi-dimensional feedback to continuously fine-tune the retrieval model and the generation model through explicit, implicit, and professional feedback data.

[0063] Explicit feedback: Set up "accuracy, practicality, and ease of understanding" three-level scoring buttons and text correction entry to support users to submit specific opinions (such as factual errors, version errors, and image association deviations); Implicit feedback: Capture potential satisfaction by analyzing user interaction behavior (dwell time, click behavior, repeated queries, and knowledge unit usage frequency); Professional feedback: Access a radiology oncology expert review channel to obtain professional ratings, correction opinions, and multi-modal knowledge association optimization suggestions from radiology oncology experts on system output content.

[0064] Step S4.2: Model iterative optimization The model iterative optimization of step S4.2 includes: retrieval model optimization: adding the retrieval error samples identified in the multi-dimensional feedback data to the training set to fine-tune the parameters of the semantic vector generation model and the retrieval ranking model; generation model optimization: using the expert corrected correct answers as positive samples and the system originally generated wrong answers as negative samples to incrementally train the generation model and update its internal cross-modal constraint parameters. Specifically includes the following steps: Step S4.2.1: Retrieval model optimization The system collects multi-dimensional feedback information, including user explicit ratings, text correction opinions, implicit interaction behavior data, and expert professional review opinions. The "retrieval omission" and "association error" samples in the multi-dimensional feedback data are included in the training set to fine-tune the semantic vector generation parameters of the Sentence-BERT model and the contrast learning parameters of the DeCLUTR semantic vector optimization framework based on deep contrast learning, and optimize the weights of the retrieval ranking MLP model.

[0065] “Retrieval omission” samples: samples identified in the feedback data as “user query has relevant knowledge but system did not retrieve” (e.g. user query “Ⅳ stage lung cancer radiotherapy combined regimen”, knowledge base has corresponding guideline content but not retrieved and output). “Association error” samples: samples identified in the feedback data as “system retrieval result does not match user query semantics” (e.g. user query “laryngeal cancer radiotherapy dose”, system retrieves esophageal cancer radiotherapy related content). Training set: sample set used for model optimization, “retrieval omission” and “association error” samples are supplemented to it to strengthen the model’s learning of high error scenarios. Sentence-BERT: model used to generate radiotherapy knowledge unit semantic vectors, by fine-tuning its semantic vector generation parameters (such as attention weights, hidden layer output dimensions), to improve the representation accuracy of semantic vectors. DeCLUTR: framework used to optimize semantic vector space distribution, fine-tune its contrast learning parameters, and further strengthen the vector clustering of similar knowledge units. MLP: model used for retrieval result sorting, by optimizing its weight parameters, adjusting the weight proportion of semantic matching degree, update time, clinical application frequency, etc., to improve the accuracy of the sorting result.

[0066] Step S4.2.2: Model optimization With the expert corrected answer as the positive sample and the original wrong answer as the negative sample, the generation model is incrementally pre-trained, and the cross-modal output token is updated with the local optimizer parameters; the loss function uses a combination of focal loss and dice loss, i.e. where the weight ratio is At the same time, the mean absolute error (MAE) loss is introduced for IoU prediction, and the cross-entropy (CE) loss is used for target detection, and the overall loss is:

[0067] Expert corrected answer: the correct content obtained by the radiation oncology expert after correcting the system’s wrong answer (e.g. correcting the system’s fictional “laryngeal cancer radiotherapy dose” to the standard dose specified in the guideline), which is used as a positive sample for model training.

[0068] Original wrong answer: the system’s generated answer that does not meet the guideline requirements, has “hallucinations” or expression biases, which is used as a negative sample for model training to help the model learn to avoid error generation patterns.

[0069] Incremental pre-training: based on the original pre-trained model, new positive and negative samples are used for additional training, without the need to retrain the entire model, improving optimization efficiency.

[0070] Cross-modal Output Tokens, ): Tokens carrying cross-modal semantic constraints, whose parameters are updated in incremental pre-training to strengthen the capture of information associated with radiotherapy text and images.

[0071] Local Refiner Parameters: Parameters in the local refiner used to correct generated details (such as the weights of , activation function parameters), which are updated to improve the correction ability of generated errors.

[0072] Focal Loss, ): A loss function used to address class imbalance in generated samples, giving higher weights to difficult samples (such as rare cancer treatment schemes) to improve the generation accuracy of complex scenarios.

[0073] Dice Loss, ): A loss function used to measure the overlap between generated content and the true answer (expert corrected answer) to strengthen the factual consistency of generated content.

[0074] : The basic generation loss is obtained by weighted sum of focal loss and dice loss, the formula is , used to constrain the overall accuracy of generated content.

[0075] and weight coefficients, the weight ratio is , balancing the influence degree of two kinds of loss.

[0076] Mean Absolute Error (MAE) Loss ): A loss function used for Intersection over Union (IoU) prediction, measuring the absolute deviation of the IoU prediction value of generated content from the actual value, to improve the boundary accuracy of generated content.

[0077] Intersection over Union (IoU): An index used to quantify the overlap between generated content and the true answer, the larger the value, the higher the overlap, and the better the generation accuracy.

[0078] Cross-Entropy (CE) Loss ): A loss function used for object detection, which is used here to constrain the accuracy of core entities (such as cancer names, dose parameters, and treatment techniques) in generated content, avoiding entity recognition errors.

[0079] : Optimized generation loss, calculated based on the optimization mask output by the local refiner, used to strengthen the detail accuracy of generated content.

[0080] : learnable weight coefficient, used to adjust the loss proportion of the loss of the generation model , which is adaptively optimized by training to balance the overall loss.

[0081] : overall loss function of the generation model, which comprehensively constrains the overall accuracy, detail accuracy and entity correctness of the generated content.

[0082] As shown in Figure 5 and Figure 6 , the radiotherapy knowledge intelligent question answering and guideline updating system with a feedback optimization mechanism comprises: A structured knowledge base module 210 is configured to build a knowledge base that integrates radiotherapy guideline texts and corresponding medical images, and forms a structured index through cross-modal semantic feature extraction and semantic prompt embedding generation steps. A retrieval-generation integrated module 220 is configured to receive user queries, match knowledge units related to the query semantics in the knowledge base through a retrieval model optimized by deep contrast learning, and sort the matching results; through a generation model combined with learnable cross-modal output tokens and a local optimizer, generate response content consistent with the radiotherapy guidelines. A dynamic updating module 230 is configured to monitor the update information of external authoritative radiotherapy guidelines, automatically capture and extract update highlights; filter and sort the update content through semantic correlation degree calculation, and integrate the effective update content into the knowledge base after confirmation, and update the corresponding semantic vector index synchronously. A feedback optimization module 240 is configured to collect multi-dimensional feedback data of users and experts on the system output, and iteratively optimize the retrieval model and the generation model based on the multi-dimensional feedback data, to realize closed-loop improvement of system performance.

[0083] The radiotherapy knowledge intelligent question answering and guideline updating system with a feedback optimization mechanism provided in the embodiments of the present application can perform the radiotherapy knowledge intelligent question answering and guideline updating method with a feedback optimization mechanism provided in any embodiment of the present application, has the corresponding functions and advantages of performing the method, and the detailed process is described in the foregoing method embodiments, which will not be repeated here.

[0084] To verify the effectiveness of the method described in the embodiment of the present application, a comparative experiment was conducted. A test set based on a publicly disclosed tumor image-report data set was constructed, containing 1000 radiotherapy case text-image pairs covering different cancer types and stages. The retrieval accuracy was defined as the proportion of queries that returned at least one standard answer in the top 10 results (Top-10). The consistency of the generated content was evaluated by three experienced radiation oncologists using a double-blind method. The experts gave a consistency percentage score (0-100%) after comprehensive evaluation from the dimensions of factual accuracy, professional expression, and rationality of image association. The average value was taken. The baseline systems for comparison include: a generation model based on general GPT-3.5-turbo, a retrieval model based on Sentence-BERT, and a traditional sequence matching model based on BiLSTM. The performance of the algorithm of the present application and the three traditional algorithms (GPT, Sentence-BERT, and BiLSTM) in the radiotherapy knowledge intelligent question answering task was compared intuitively. The accuracy curve under different training times was drawn through Python simulation experiment, as shown in Figure 7 The simulation data shows that with the increase of training times, the accuracy of the four algorithms shows a gradual upward trend, but the algorithm of the present application (using a circular marker, blue solid line) always maintains a significant leading advantage, with the accuracy increasing steadily from 0.72 at 10 training times to 0.95 at 100 training times. This is due to the deep contrast learning semantic vector optimization framework (DeCLUTR) and the semantic prompt projector technology, which effectively improve the cross-modal knowledge representation and semantic matching accuracy. The upper limit of the accuracy of GPT (using a square marker, orange dotted line), Sentence-BERT (using a triangular marker, green dot-dash line), and BiLSTM (using a diamond marker, red dot line) is 81%, 83%, and 78%, respectively. Among them, GPT has a lower accuracy due to the lack of radiotherapy field-specific constraints, Sentence-BERT is limited by the single text semantic vector generation capability and lacks cross-modal association, and BiLSTM as a traditional sequence model has the limitation of limited semantic representation depth. The performance improvement of the three is significantly less than that of the algorithm of the present application, fully verifying the superiority of the present application in the radiotherapy knowledge intelligent question answering task.

[0085] It should be noted that each embodiment in the present specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0086] It should also be noted that, in the embodiments of the present application, the terms such as first and second, etc. are used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by an "including a" statement does not exclude the existence of additional identical elements in the process, method, article, or apparatus including the element.

[0087] The above description of disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined in the embodiments of the present application can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown in the embodiments of the present application, but will conform to the widest scope consistent with the principles and novel features disclosed in the embodiments of the present application.

Claims

1. A radiotherapy knowledge intelligent question answering and guideline updating method with a feedback optimization mechanism, characterized in that, The method comprises: S1: constructing a fusion radiotherapy guideline text and corresponding medical images, and forming a structured index knowledge base through a cross-modal semantic feature extraction and semantic prompt embedding generation step; S2: receiving a user query, matching the knowledge units related to the query semantics in the knowledge base through a retrieval model optimized by deep contrast learning, and sorting the matching results; through a generation model combined with a learnable cross-modal output token and a local optimizer, a response content consistent with the radiotherapy guideline is generated; S3: Monitor the update information of the external authoritative radiotherapy guideline, automatically extract and extract the update points; filter and sort the update content through semantic correlation calculation, and after confirmation, integrate the effective update content into the knowledge base and update the corresponding semantic vector index synchronously; S4: Collect multi-dimensional feedback data of users and experts on system output, and iteratively optimize the retrieval model and generation model based on the multi-dimensional feedback data to realize closed-loop improvement of system performance.

2. The method of claim 1, wherein, In S1, the construction of the knowledge base includes a cross-modal semantic feature extraction step, specifically: Encode the radiotherapy guideline text and corresponding medical images respectively to obtain language features and visual features; Through a two-stage progressive attention mechanism for feature fusion: The first stage, the language features and visual features are calculated by bidirectional multi-head cross attention, and the text-visual features and visual-text features are obtained; wherein the projection transformation in the attention calculation is realized by 3x3 deep convolution; The second stage, the language features and visual features are spliced and processed by convolution to generate intermediate fusion features; the intermediate fusion features are used as attention queries to perform cross attention calculation with the text-visual features and the visual-text features respectively, and two deep fusion features are obtained; Fuse the two deep fusion features and process them through a feedforward network to output cross-modal semantic features that fuse text semantics and image spatial information.

3. The method of claim 2, wherein, In S1, the construction of the knowledge base also includes a semantic prompt embedding generation step, specifically: Fuse the cross-modal semantic features and the multi-scale visual features extracted from the corresponding medical images through a bidirectional feature pyramid network; Normalize and attention process the fused features, and then generate a basic semantic embedding through a multi-head attention mechanism; The basic semantic embedding is processed through the following branches: the first branch, through adaptive pooling operation and convolution layer processing, generates a dense prompt embedding containing fine-grained information; The second branch, through a multi-layer perceptron network, generates a sparse prompt embedding focusing on the core semantic nodes; The dense prompt embedding and the sparse prompt embedding will serve as the structured semantic representation of the corresponding knowledge unit in the knowledge base.

4. The method of claim 3, wherein, In S2, the retrieval process includes: Use a bidirectional long short-term memory network to analyze the user query intent, extract the key constraint conditions of the guideline version, cancer stage, treatment method and image features, and automatically supplement the implicit retrieval dimensions of the fuzzy query; The framework based on deep contrast learning is used to optimize the spatial distribution of semantic vectors in the knowledge base, and the standard temperature scaling cross-entropy loss function is used to make semantic similar knowledge units gather in the vector space and semantic different units move away from each other. Based on the cosine similarity, a candidate knowledge unit set is quickly screened out from the knowledge base; multi-dimensional input features including semantic matching degree, knowledge timeliness, application frequency and cross-modal correlation are constructed, and the candidate knowledge unit set is sorted by a multi-layer perception machine to output the most relevant core knowledge unit as the input of the generation module.

5. The method of claim 4, wherein, In S2, the generation model is combined with the learnable cross-modal output token and the local optimizer to generate a response content consistent with the radiotherapy guidelines, specifically including: A generative pre-training transformation model based on the fusion of pre-training weights in the radiotherapy field is used to construct a generation framework as the generation model; The sparse prompt embedding corresponding to the candidate knowledge unit retrieved, the dense prompt embedding and the learnable cross-modal output token are spliced with the original output token of the generation model and input into the mask decoder; at the same time, the dense prompt embedding is fused with the features of the corresponding medical image as the auxiliary input of the mask decoder; the mask decoder generates an original mask; The cross-modal semantic features corresponding to the knowledge unit are respectively subjected to nonlinear transformation by the local optimizer, and then dot product operation is performed to generate an optimized mask; the original mask and the optimized mask are fused to obtain a generation constraint mask used to constrain the generation process; Based on the generation constraint mask, the user identity category and the historical query record, a response content suitable for the current user is generated and output.

6. The method of claim 5, wherein, In S3, the similarity sorting strategy is used for screening and sorting the updated content, specifically including the following process: Periodically access the website of the preset authoritative radiotherapy guideline publishing agency through the network crawler, and automatically identify and extract the update information of the guideline by using the natural language processing model; The similarity sorting strategy is: calculating the semantic similarity between the update content to be integrated and the existing related knowledge in the knowledge base, and prioritizing and extracting the core points of the update content according to the similarity score; the effective update content confirmed by the expert audit is automatically integrated into the knowledge base, and the corresponding semantic index and vector representation are updated.

7. The method of claim 6, wherein, In S4, the multi-dimensional feedback data includes feedback of at least two dimensions: User direct scoring of system output content in accuracy, practicality and ease of understanding dimensions and text correction; The potential satisfaction information inferred by the system by analyzing the behavior data of user and system interaction; Professional review opinions, ratings and revised contents of system output content from radiation oncology experts.

8. The method according to claim 1 or 7, characterized in that, In S4, model iteration optimization includes: Retrieval model optimization: add the retrieval error samples identified in the multi-dimensional feedback data to the training set to fine-tune the parameters of the semantic vector generation model and the retrieval sorting model; Model optimization: the correct answers revised by experts are taken as positive samples, and the original wrong answers of the system are taken as negative samples to incrementally train the generation model and update its internal cross-modal constraint parameters.

9. The method of claim 8, wherein, In S4, the training of the generation model adopts a combined loss function, which includes a basic generation loss composed of a focal loss and a dice loss weighted, a refined generation loss calculated based on an optimized mask, and an auxiliary loss composed of a mean absolute error loss and a cross-entropy loss; the combined loss function is a weighted sum of the basic generation loss, the refined generation loss and the auxiliary loss.

10. The intelligent question-answering and guideline updating system for radiotherapy knowledge with feedback optimization mechanism, characterized in that, The system comprises: A structured knowledge base module for constructing a knowledge base that fuses radiotherapy guideline texts and corresponding medical images and forms a structured index through cross-modal semantic feature extraction and semantic prompt embedding generation steps; A retrieval-generation integrated module for receiving user queries, matching knowledge units related to query semantics in the knowledge base through a retrieval model optimized by deep contrast learning, and sorting the matching results; generating response content consistent with the radiotherapy guidelines through a generation model combined with learnable cross-modal output tokens and a local optimizer; A dynamic updating module for monitoring update information of external authoritative radiotherapy guidelines, automatically capturing and extracting update highlights; filtering and sorting the update content through semantic correlation calculation, and integrating the effective update content into the knowledge base and synchronously updating the corresponding semantic vector index after confirmation; A feedback optimization module for collecting multi-dimensional feedback data of the system output by users and experts, iteratively optimizing the retrieval model and the generation model based on the multi-dimensional feedback data, and realizing closed-loop improvement of system performance.

Images