Examination report automatic generation method and system based on cue word

Abstract

The invention relates to the technical field of examination report generation, and particularly discloses an automatic examination report generation method and system based on cue words. The artificial intelligence data analysis and semantic coding technology is adopted to carry out phrasing processing and text convolution coding on the radiographic examination data of the target patient object, and semantic coding is carried out on keyword prompts input by a doctor at the same time; the semantic embedding features based on the key prompt words are used as guidance to carry out self-adaptive significance adjustment on each part of sentence granularity description in the radiation examination sentence granularity semantic coding features so as to intelligently generate an examination report. Therefore, the examination report which is reasonable in structure, detailed in content and prominent in key point can be intelligently generated based on the guidance of the cue word, so that the workload of a doctor is reduced, the doctor is helped to quickly obtain the most important and most relevant information, and the working efficiency and the decision-making quality are improved.

Description

Technical Field

[0001] The present application relates to the technical field of inspection report generation, and more specifically, to a method and system for automatically generating an inspection report based on prompt words. Background Art

[0002] Radiology image reading and report writing are usually completed by professional radiologists. Specifically, radiologists review the images of patients and conduct detailed analysis and diagnosis, and write radiology reports after reading the images. Radiology reports are composed of two parts: imaging findings and diagnostic conclusions. Doctors comprehensively and meticulously record the imaging manifestations of lesions in the imaging findings section and give accurate diagnostic conclusions in the diagnostic opinion section. In the process of writing reports, radiologists face problems such as heavy workload, low template adaptability, and too much content that needs to be added or deleted.

[0003] In recent years, scholars in the industry have proposed solutions to reduce the workload of writing radiology reports mainly from the following aspects: Patent CN111243729B uses CNN to construct a mapping relationship between radiological imaging data and radiological reports, inputs radiological images, and outputs radiological reports. However, current research is limited to chest X-rays. The performance of this model has not been fully verified or optimized for other types of radiological examinations and different diseases that may be found in these examinations. Patent CN108320781B uses voice input to automatically convert the model into report text content. This technology is still a big challenge for inexperienced radiologists. Due to the poor integrity and standardization of the voice input process, clinical acceptance is not as expected. Patent CN117292783 designs a system and method for deriving diagnostic results based on image findings, but there is a lot of content to be entered and modified by doctors in the image findings section, and clinicians still need to enter a large amount of text, which does not significantly reduce the workload.

[0004] Therefore, an optimized automatic inspection report generation solution is desired, which can intelligently generate inspection reports based on prompt words. Summary of the invention

[0005] The present application provides a method and system for automatically generating an examination report based on prompt words, which can intelligently generate an examination report with a reasonable structure, detailed content and highlighted key points based on the guidance of the prompt words, thereby reducing the workload of doctors, helping doctors to quickly obtain the most important and relevant information, and improving work efficiency and decision-making quality.

[0006] In a first aspect, a method for automatically generating an examination report based on prompt words is provided, comprising: obtaining a keyword prompt input by a doctor; obtaining radiological examination data of a target patient object; performing sentence segmentation processing and text convolution coding on the radiological examination data of the target patient object to obtain radiological examination sentence granularity semantic coding aggregation features; performing semantic coding on the keyword prompt input by the doctor to obtain key prompt word semantic embedding features; performing multimodal feature interactive coding on the key prompt word semantic embedding features and the radiological examination sentence granularity semantic coding aggregation features to obtain a sequence of prompt word-guided radiological examination sentence granularity semantic salient coding features, comprising: performing feature decoupling on the radiological examination sentence granularity semantic coding aggregation features to obtain a set of radiological examination sentence granularity local features; performing prompt gate-guided attention interactive fusion on the key prompt word semantic embedding features and the set of radiological examination sentence granularity local features to obtain a sequence of prompt word-guided radiological examination sentence granularity semantic salient coding features; and obtaining an examination report based on the sequence of prompt word-guided radiological examination sentence granularity semantic salient coding features.

[0007] In a second aspect, a prompt word-based automatic generation system for an examination report is provided, comprising: a keyword prompt acquisition module, used to acquire keyword prompts input by a doctor; a radiological examination data acquisition module, used to acquire radiological examination data of a target patient object; a radiological examination sentence granularity semantic encoding module, used to perform sentence segmentation processing and text convolution encoding on the radiological examination data of the target patient object to obtain radiological examination sentence granularity semantic encoding aggregation features; a key prompt word semantic encoding module, used to perform semantic encoding on the keyword prompt input by the doctor to obtain key prompt word semantic embedding features; a prompt word-guided radiological examination encoding module, used to perform multi-modal semantic embedding on the key prompt word semantic embedding features and the radiological examination sentence granularity semantic encoding aggregation features. The state features are interactively encoded to obtain a sequence of semantically significant coding features of the prompt word-guided radiological inspection sentence granularity, wherein the prompt word-guided radiological inspection coding module includes: a radiological inspection feature decoupling unit, which is used to perform feature decoupling on the radiological inspection sentence granularity semantic coding aggregation features to obtain a set of radiological inspection sentence granularity local features; a prompt word-guided radiological inspection interactive fusion unit, which is used to perform a prompt gate-guided attention interactive fusion on the key prompt word semantic embedding features and the set of radiological inspection sentence granularity local features to obtain a sequence of semantically significant coding features of the prompt word-guided radiological inspection sentence granularity; an inspection report generation module, which is used to obtain an inspection report based on the sequence of semantically significant coding features of the prompt word-guided radiological inspection sentence granularity.

[0008] The present application provides a method and system for automatically generating an inspection report based on prompt words, which obtains the keyword prompts input by the doctor, obtains the radiological examination data of the target patient object, and uses artificial intelligence data analysis and semantic coding technology to perform sentence processing and text convolution coding on the radiological examination data of the target patient object, and at the same time performs semantic coding on the keyword prompts input by the doctor, so as to adaptively adjust the significance of each part of the sentence granularity description in the radiological examination sentence granularity semantic coding feature based on the semantic embedding feature of the key prompt words as a guide, so as to intelligently generate an inspection report. In this way, it is possible to intelligently generate an inspection report with a reasonable structure, detailed content and prominent focus based on the guidance of the prompt words, thereby reducing the workload of doctors, helping doctors to quickly obtain the most important and relevant information, and improving work efficiency and decision-making quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0010] Figure 1 This is a schematic flowchart of a method for automatically generating an inspection report based on prompt words according to an embodiment of the present application.

[0011] Figure 2 This is a schematic diagram of the data flow of the image analysis module in the method for automatically generating an inspection report based on prompt words in an embodiment of the present application.

[0012] Figure 3 This is a schematic flowchart of performing sentence segmentation and text convolutional coding on the radiological examination data of the target patient in the automatic generation method of the examination report based on prompt words in an embodiment of the present application to obtain the granular semantic coding aggregation features of the radiological examination sentence.

[0013] Figure 4 In the automatic generation method of inspection reports based on prompt words in an embodiment of the present application, multimodal feature interactive encoding is performed on the semantic embedding features of the key prompt words and the granular semantic encoding aggregation features of the radiological inspection sentence to obtain a schematic flowchart of a sequence of prompt word-guided radiological inspection sentence granular semantic significant encoding features.

[0014] Figure 5 In the automatic generation method of inspection reports based on prompt words in an embodiment of the present application, the set of semantic embedding features of the key prompt words and the set of local features of the radiological inspection sentence granularity are subjected to a prompt gate-guided attention interaction fusion to obtain a schematic flowchart of a sequence of semantically significant encoding features of the radiological inspection sentence granularity guided by the prompt words.

[0015] Figure 6 It is a flow chart of the radiology report auxiliary writing model provided in the embodiment of the present application.

[0016] Figure 7 This is a schematic block diagram of a system for automatically generating inspection reports based on prompt words according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0018] In response to the above technical problems, this application proposes a method for automatically generating an inspection report based on prompt words, which obtains the keyword prompts input by the doctor, obtains the radiological examination data of the target patient object, and uses artificial intelligence data analysis and semantic coding technology to perform sentence processing and text convolution coding on the radiological examination data of the target patient object, and semantically encodes the keyword prompts input by the doctor, so as to adaptively adjust the significance of each part of the sentence granularity description in the radiological examination sentence granularity semantic coding feature based on the semantic embedding feature of the key prompt words as a guide, so as to intelligently generate an inspection report. In this way, based on the guidance of the prompt words, an inspection report with a reasonable structure, detailed content and prominent focus can be intelligently generated, thereby reducing the workload of doctors, helping doctors to quickly obtain the most important and relevant information, and improving work efficiency and decision-making quality.

[0019] like Figure 1 and Figure 2 As shown, the method for automatically generating an examination report based on prompt words includes: S1, obtaining a keyword prompt input by a doctor; S2, obtaining radiological examination data of a target patient object; S3, performing sentence segmentation processing and text convolution encoding on the radiological examination data of the target patient object to obtain radiological examination sentence granularity semantic coding aggregation features; S4, performing semantic encoding on the keyword prompt input by the doctor to obtain key prompt word semantic embedding features; S5, performing multimodal feature interactive encoding on the key prompt word semantic embedding features and the radiological examination sentence granularity semantic coding aggregation features to obtain a sequence of prompt word-guided radiological examination sentence granularity semantic significant coding features; S6, obtaining an examination report based on the sequence of prompt word-guided radiological examination sentence granularity semantic significant coding features.

[0020] Exemplarily, in step S1, a keyword prompt input by a doctor is obtained. It should be understood that the keyword prompt contains key information extracted by the doctor after professional judgment based on the specific imaging data of the target patient, such as the location of the lesion, the nature, number, size, etc. of the lesion. By obtaining the keyword prompts input by the doctor, the significance of the description of each part in the radiological examination report can be adjusted based on these prompt words, so that the semantic weight related to the key prompt words is improved, thereby ensuring that the generated report has a reasonable structure, detailed content and prominent focus.

[0021] Exemplarily, in step S2, radiological examination data of the target patient object is obtained. It should be understood that radiological examination data directly reflects the patient's pathological condition and provides specific information about the lesion, such as location, size, morphology, etc., which is essential for generating accurate and detailed examination reports. By analyzing radiological examination data, the system can capture semantic information such as anatomical structure and lesion characteristics in the image, which is the basic information for doctors to make diagnoses. Therefore, obtaining radiological examination data can ensure that the generated examination report is highly professional and accurate, helping doctors quickly obtain the most important and relevant information, and improving work efficiency and decision-making quality.

[0022] Exemplarily, in step S3, the radiological examination data of the target patient object is sentence-processed and text convolutionally encoded to obtain radiological examination sentence-granular semantic encoding aggregation features. It should be understood that the radiological examination data expresses different aspects of the patient's imaging manifestations, such as examination type, examination site, lesion condition, etc. These different descriptions contain rich semantic information and context associations. By sentence-processing the data, each sentence-level description can be separated, so that the system can analyze and understand the semantic content contained in each specific description in a more fine-grained manner. Through the text convolutional neural network model (Text CNN), short text features, such as the description method of specific lesion features, can be effectively extracted. Text CNN captures phrases or patterned expressions by applying a convolution kernel sliding window to the matrix, thereby generating a highly representative radiological examination sentence-granular semantic encoding aggregation feature map. This method combines Bert's powerful language understanding and context modeling capabilities with the high efficiency of Text CNN in short text feature extraction, and comprehensively reflects the semantic information of the entire radiological examination description.

[0023] In one embodiment, Figure 3As shown, the radiological examination data of the target patient object is subjected to sentence segmentation processing and text convolutional encoding to obtain radiological examination sentence granularity semantic encoding aggregation features, including: S31, the radiological examination data of the target patient object is subjected to sentence segmentation processing to obtain a sequence of radiological examination sentence granularity descriptions; S32, the sequence of radiological examination sentence granularity descriptions is input into a text convolutional neural network model including a Bert layer to obtain a radiological examination sentence granularity semantic encoding aggregation feature map as the radiological examination sentence granularity semantic encoding aggregation feature.

[0024] Exemplarily, in step S31, the radiological examination data of the target patient object is processed by sentence segmentation to obtain a sequence of radiological examination sentence granularity descriptions. It should be understood that, considering that the radiological examination data of the target patient object expresses different aspects of the radiological examination data of the target patient object, such as examination type, examination site, lesion condition, etc. These different examination data descriptions contain different semantic information and associations between contexts. Therefore, in order to better understand the specific meaning of each sentence and accurately capture key information, such as lesion location, size, shape, etc., in the technical solution of the present application, the radiological examination data of the target patient object is processed by sentence segmentation to obtain a sequence of radiological examination sentence granularity descriptions. In this way, the radiological examination data can be segmented into sentence-level descriptions, so that the subsequent model can analyze and understand the semantic content contained in each specific in a more fine-grained manner, laying the foundation for subsequent semantic encoding.

[0025] Exemplarily, in step S32, the sequence of the radiological examination sentence granularity description is input into a text convolutional neural network model including a Bert layer to obtain a radiological examination sentence granularity semantic encoding aggregation feature map as the radiological examination sentence granularity semantic encoding aggregation feature. It should be understood that in order to capture the contextual information and complex semantic relationships between the granular descriptions of each radiological examination sentence, so as to achieve deep semantic understanding and efficient processing of the entire radiological examination data, in the technical solution of the present application, the sequence of the radiological examination sentence granularity description is input into a text convolutional neural network model including a Bert layer to obtain a radiological examination sentence granularity semantic encoding aggregation feature map. It can be understood that Bert is a deep learning model based on the Transformer architecture, which understands the contextual information of the input text through a bidirectional encoder. Unlike the traditional unidirectional model, Bert can consider the context on both sides of a word at the same time, which enables it to better capture the semantic relationship in the granular description of each radiological examination sentence and the semantic association across sentences. The text convolutional neural network model is mainly composed of a one-dimensional convolutional layer, an activation function, a pooling layer, and a fully connected layer. Specifically, each radiology sentence granular description can be represented as a vector (e.g., by word embedding), and the sequence of the entire radiology sentence granular description can be viewed as a matrix composed of a series of such vectors. The text convolutional neural network model captures phrases or patterned expressions, such as the description of specific lesion features, by applying convolution kernel sliding windows to these matrices. Therefore, by combining Bert and the text convolutional neural network model, the advantages of both can be utilized - Bert's powerful language understanding and context modeling capabilities, and the high efficiency of the text convolutional neural network in short text feature extraction, thereby comprehensively reflecting the semantic information of the entire radiology description and generating a highly representative radiology sentence granular semantic encoding aggregate feature map.

[0026] Exemplarily, in step S4, the keyword prompts entered by the doctor are semantically encoded to obtain semantic embedding features of key prompt words. It should be understood that considering that the keyword prompts entered by the doctor contain key information extracted by the doctor after professional judgment based on the specific imaging data of the target patient object, including information such as the location of the lesion, the nature of the lesion, the number, and the size of the lesion. Therefore, in order to be able to deeply understand the meaning behind the keywords entered by the doctor, so as to more accurately identify and process complex medical terms and concepts, in the technical solution of the present application, the keyword prompts entered by the doctor are semantically encoded to obtain a semantic embedding coding vector of the key prompt words. In this way, the professional knowledge of the doctor can be better understood and utilized, thereby generating a more accurate, personalized and valuable examination report. In particular, in a specific embodiment of the present application, the keyword prompts entered by the doctor are firstly segmented to decompose an entire keyword prompt into smaller units to obtain multiple phrases (such as words or subwords), and then each phrase after the segmentation processing is input into a keyword semantic encoder containing a word embedding layer to understand and capture the contextual semantic meaning of each keyword in a fine-grained manner, and obtain a semantic embedding coding vector of the key prompt word.

[0027] In one embodiment, the keyword prompt input by the doctor is semantically encoded to obtain a key prompt word semantic embedding feature, including: performing word segmentation processing on the keyword prompt input by the doctor and then inputting the result into a keyword semantic encoder including a word embedding layer to obtain a key prompt word semantic embedding encoding vector as the key prompt word semantic embedding feature.

[0028] Exemplarily, in step S5, the key prompt word semantic embedding features and the radiology examination sentence granular semantic encoding aggregation features are subjected to multimodal feature interactive encoding to obtain a sequence of prompt word-guided radiology examination sentence granular semantic significant encoding features. It should be understood that the key prompt word semantic embedding features capture the semantic information of the key points and concerns conveyed by the doctor through keywords or phrases. For example, the doctor may specify a specific lesion location, suspected disease type or other important findings. The radiology examination sentence granular semantic encoding aggregation features describe in detail the semantic information such as anatomical structures and lesion characteristics visible in the image. Considering that the descriptive semantics of different parts in the radiology examination sentence granular semantic encoding aggregation features have different degrees of significant information, prompt words as semantic guides can help the system focus more on relevant features when processing radiology examination data. Therefore, in order to adjust the significance of the description of each part in the radiological examination report based on key prompt words so that the semantic weight related to the key prompt words can be improved, in the technical solution of the present application, the key prompt word semantic embedding features and the radiological examination sentence granularity semantic encoding aggregation features are subjected to multimodal feature interactive encoding to obtain a sequence of prompt word-guided radiological examination sentence granularity semantic significance encoding features.

[0029] In one embodiment, Figure 4 As shown, the key prompt word semantic embedding features and the radiographic inspection sentence granularity semantic coding aggregation features are subjected to multimodal feature interactive coding to obtain a sequence of prompt word guided radiographic inspection sentence granularity semantic significant coding features, including: S51, feature decoupling of the radiographic inspection sentence granularity semantic coding aggregation features to obtain a set of radiographic inspection sentence granularity local features; S52, cue gate-guided attention interactive fusion of the key prompt word semantic embedding features and the set of radiographic inspection sentence granularity local features to obtain a sequence of prompt word guided radiographic inspection sentence granularity semantic significant coding features.

[0030] In one embodiment, in step S51, the radiological inspection sentence granular semantic coding aggregate features are feature decoupled to obtain a set of radiological inspection sentence granular local features, including: expanding each feature matrix along the channel dimension of the radiological inspection sentence granular semantic coding aggregate feature map into a feature vector to obtain a set of radiological inspection sentence granular semantic local fine-grained feature vectors as the set of radiological inspection sentence granular local features. Specifically, the process can be expressed as: ;in, is the granular semantic encoding aggregate feature map of the radiological examination sentence, For Perform feature decoupling operations. , and They are the first and the second in the set of radiological examination sentence granular semantic local fine-grained feature vectors. and A radiometric inspection sentence-level semantic local fine-grained feature vector.

[0031] That is, the feature matrices along the channel dimension of the radiological examination sentence granular semantic encoding aggregation feature map are expanded into feature vectors to obtain a set of radiological examination sentence granular semantic local fine-grained feature vectors as a set of radiological examination sentence granular local features. This process is to simplify complex multidimensional data into a series of one-dimensional data that is easier to process. In this conversion process, different semantic information originally represented in two-dimensional or higher-dimensional space is decomposed, and each feature matrix represents a specific aspect of semantic information or contextual association in the image or text description. When these feature matrices are expanded into feature vectors along the channel dimension, they are actually converted into a series of elements that can be analyzed independently while maintaining the integrity of the original semantic information. After doing so, a set of multiple fine-grained feature vectors is obtained, each of which captures a local area or a specific type of semantic information in the original feature map. This not only helps to capture subtle changes and local features in image data more finely, which is especially important for medical images, because even small changes may mean important pathological information; it also reduces the redundancy and correlation between different features, making subsequent processing more efficient and targeted. In addition, this feature expression method also supports cross-modal interaction, that is, it can emphasize the relevant local features based on the keyword prompts provided by the doctor, so as to highlight the key information. For example, when generating a report, the importance of describing this part can be adjusted based on the keyword prompts (such as "right upper lobe nodule").

[0032] In one embodiment, Figure 5 As shown, in step S52, the key prompt word semantic embedding features and the set of radiological examination sentence granularity local features are subjected to cue gate guided attention interaction fusion to obtain a sequence of the prompt word guided radiological examination sentence granularity semantic salient coding features, including: S521, based on the key prompt word semantic embedding coding vector, calculating the weighted attention weights of each radiological examination sentence granularity semantic local fine-grained feature vector in the set of the radiological examination sentence granularity semantic local fine-grained feature vectors to obtain a set of key prompt word-radiological examination sentence granularity cross-modal semantic interaction attention weights; S522, based on the set of key prompt word-radiological examination sentence granularity cross-modal semantic interaction attention weights, cross-modal interaction optimization is performed on the key prompt word semantic embedding coding vector and the set of radiological examination sentence granularity local features to obtain a sequence of prompt word guided radiological examination sentence granularity semantic salient coding vectors as the sequence of prompt word guided radiological examination sentence granularity semantic salient coding features.

[0033] In one embodiment, in step S521, based on the key prompt word semantic embedding coding vector, the weighted attention weights of each radiological inspection sentence granularity semantic local fine-grained feature vector in the set of the radiological inspection sentence granularity semantic local fine-grained feature vectors are calculated to obtain a set of key prompt word-radiological inspection sentence granularity cross-modal semantic interaction attention weights, including: linearly transforming the key prompt word semantic embedding coding vector to obtain a key prompt word semantic query coding vector and a key prompt word semantic value coding vector; performing converter-based cross-modal prompt coding on each radiological inspection sentence granularity semantic local fine-grained feature vector in the set of the key prompt word semantic query coding vector, the key prompt word semantic value coding vector and the radiological inspection sentence granularity semantic local fine-grained feature vector to obtain a set of key prompt word-radiological inspection sentence granularity semantic cross-modal prompt semantic coding vectors; performing decoding gate modulation on the set of the key prompt word-radiological inspection sentence granularity semantic cross-modal prompt semantic coding vectors to obtain the set of the key prompt word-radiological inspection sentence granularity cross-modal semantic interaction attention weights. Specifically, the process can be expressed by the formula: ;in, is the first in the set of local fine-grained feature vectors of the radiological inspection sentence granularity semantics A radiometric sentence-level semantic local fine-grained feature vector, is the semantic embedding encoding vector of the key prompt word, is the query embedding matrix, is matrix multiplication, is the semantic query encoding vector of the key word, is the value embedding matrix, is the semantic value encoding vector of the key prompt word, yes The transposed vector of yes Length, is the normalized exponential function, yes and The key cue words between - radiological examination sentence granularity semantic cross-modal cue semantic encoding vector, is the decoding weight matrix, It is a decoder. yes Corresponding key cue word-radiology check sentence granularity cross-modal semantic interaction attention weights.

[0034] That is, first, by linearly transforming the key cue word semantic embedding encoding vector, the system converts it into a form suitable for subsequent processing - namely, the key cue word semantic query encoding vector and the key cue word semantic value encoding vector. This transformation introduces trainable parameters to adjust the spatial distribution of features, making them more suitable for cross-modal comparison and fusion. The query encoding vector is used to locate the correlation with other modal features, while the value encoding vector preserves the important information of the original features. This step is crucial for the subsequent cross-modal interaction because it ensures that information from different modalities can be compared and fused in a unified way. Next, the key cue word semantic query encoding vector, the key cue word semantic value encoding vector, and each feature vector in the set of radiology sentence granular semantic local fine-grained feature vectors are respectively input into the cross-modal prompt encoder based on the transformer structure. In this process, the transformer structure allows the model to effectively capture the complex interaction patterns between different modal features. Specifically, the query encoding vector helps the system find which local details in the radiology data are most relevant to the keyword prompts provided by the doctor, while the value encoding vector preserves the core information of the keyword prompts, ensuring that important diagnostic clues are not lost. The local fine-grained feature vector of radiology examination represents the specific description of different parts in the imaging data. By combining with the query and value encoding vectors, the corresponding pathological features can be matched more accurately. The set of semantic encoding vectors of cross-modal hints of key cue words-radiology examination sentence granularity semantics generated in this way not only retains the information of the original modality features, but also integrates the complementary knowledge from other modalities, improving the richness and accuracy of feature representation. Finally, the set of semantic encoding vectors of cross-modal hints of key cue words-radiology examination sentence granularity semantics is decoded and gated to obtain the set of cross-modal semantic interaction attention weights of key cue words-radiology examination sentence granularity semantics. This process is similar to the information screening mechanism of the human brain, which ensures the effectiveness and rationality of the final fusion result. The information gating unit dynamically adjusts the importance of information from different sources according to task requirements to avoid over-reliance on a certain modality information and ignoring other clues that may be equally important. By optimizing the attention weights, the model can more accurately capture the complementarity and synergy between different modalities.

[0035] In one embodiment, in step S522, based on the set of attention weights of cross-modal semantic interaction at the key prompt word-radioscopic inspection sentence granularity, the key prompt word semantic embedding coding vector and the set of local features at the radioscopic inspection sentence granularity are cross-modally optimized to obtain a sequence of semantically significant coding vectors guided by the prompt word at the radioscopic inspection sentence granularity, including: cross-modally interacting the key prompt word semantic embedding coding vector with the set of local features at the radioscopic inspection sentence granularity to obtain a set of cross-modal interaction local feature vectors at the key prompt word-radioscopic inspection sentence granularity; based on the set of attention weights of cross-modal semantic interaction at the key prompt word-radioscopic inspection sentence granularity, optimizing each key prompt word-radioscopic inspection sentence granularity cross-modal interaction local feature vector in the set of cross-modal interaction local feature vectors at the key prompt word-radioscopic inspection sentence granularity to obtain a sequence of semantically significant coding vectors guided by the prompt word at the radioscopic inspection sentence granularity. Specifically, the process can be expressed by the formula: ;in, , and They are the first and the second in the set of radiological examination sentence granular semantic local fine-grained feature vectors. and A radiometric sentence-level semantic local fine-grained feature vector, They are and The corresponding key word-radiology inspection sentence granularity cross-modal semantic interaction attention weight, is the semantic embedding encoding vector of the key prompt word, It is the point multiplication by position. It is a sequence of semantically significant encoding vectors of the sentence granularity guided by the cue words.

[0036] That is, first, these keywords are converted into semantic embedding encoding vectors of key prompt words. Then, the system will cross-modally interact these encoding vectors with the set of local feature vectors extracted from the radiological examination data. Through this interaction, the system can find the correlation between keyword prompts and image features, such as paying special attention to the features of the right upper lobe region in the image data, and compare and match them with the keyword prompts. This interaction not only enables the model to focus more carefully on local details while maintaining global understanding, but also improves the recognition accuracy of specific pathological features. Next, the system calculates the attention weights of cross-modal semantic interactions at the granularity of key prompt words-radiological examination sentences to determine which information is more important or relevant. In the medical image analysis scenario, this means that the system can adjust the importance of different image regions based on the keyword prompts provided by the doctor. For example, the prompt of "right upper lobe nodule" will cause the system to give a higher weight to this region, while relatively reducing the attention to other non-critical regions. Direct interaction of features between modalities is achieved by using the method of positional dot multiplication, effectively capturing the complementary information of the two modalities at the same spatial position and enhancing the spatial consistency of feature representation. On this basis, each local feature vector is further optimized in combination with the attention weights to emphasize those parts that are most important for diagnosis. This not only improves the quality of feature representation, but also ensures that the final generated examination report can accurately reflect the patient's pathological condition and highlight the key information that the doctor is concerned about. After the above optimization, a sequence of granular semantically significant encoding vectors of radiological examination sentences guided by prompt words is obtained. This sequence has a reasonable structure because the guidance information from the keyword prompt is introduced. The generated report is more in line with clinical needs in structure and the logic is clear and easy to understand.

[0037] Exemplarily, in step S6, an inspection report is obtained based on the sequence of semantically significant coding features of the radiological examination sentence granularity guided by the prompt words. In one embodiment, an inspection report is obtained based on the sequence of semantically significant coding features of the radiological examination sentence granularity guided by the prompt words, including: inputting the sequence of semantically significant coding vectors of the radiological examination sentence granularity guided by the prompt words into a trained large language model to obtain the inspection report. That is, the sequence of semantically significant coding features of the radiological examination sentence granularity guided by the prompt words obtained by cross-modal interaction with the semantic embedding coding vector of the key prompt words and the granular semantic coding aggregation feature map of the radiological examination sentence granularity is generated and processed, so as to intelligently generate an inspection report. In this way, based on the guidance of the prompt words, an inspection report with a reasonable structure, detailed content and prominent emphasis can be intelligently generated, thereby reducing the workload of doctors, helping doctors to quickly obtain the most important and relevant information, and improving work efficiency and decision-making quality.

[0038] Here, when the key prompt word semantic embedding coding vector and the radiology inspection sentence granular semantic coding aggregate feature map represent the embedded semantic features of the keyword prompt and the convolutional text semantic features of the radiology inspection data respectively, after performing cross-modal interactive optimization encoding based on the cross-modal prompt gate, the sequence of radiology inspection sentence granular semantic saliency coding vectors guided by the prompt word will also have significant interactive optimization distribution space structure differences due to the differences in cross-modal semantic promptability, affecting the convergence consistency of the generator based on the large language model, and thus affecting the semantic accuracy of the inspection report.

[0039] Therefore, in a preferred embodiment, the sequence of the cue-word guided radiological inspection sentence granular semantically significant encoding vectors is optimized, and the optimization process includes: calculating the absolute value sum of all eigenvalues ​​of the sequence of the cue-word guided radiological inspection sentence granular semantically significant encoding vectors and the square root of the sum of squares to obtain the first cue-word guided radiological inspection sentence granular semantically significant spatial structure value and the second cue-word guided radiological inspection sentence granular semantically significant spatial structure value, that is: .

[0040] The second prompt word guided radiological examination sentence granular semantic significant spatial structure value is divided by the first prompt word guided radiological examination sentence granular semantic significant spatial structure value to obtain the prompt word guided radiological examination sentence granular semantic significant encoding probability value: .

[0041] The feature mean of all feature values ​​of the sequence of the granular semantic significant coding vector of the prompt word guided radiological examination sentence is multiplied by the probability value of the granular semantic significant coding of the prompt word guided radiological examination sentence to obtain the statistical field value of the granular semantic significant coding of the prompt word guided radiological examination sentence .

[0042] Subtract one from the statistical field value of the granular semantic significant coding of the prompt word leading the radiological examination sentence and divide it by the statistical field value of the granular semantic significant coding of the prompt word leading the radiological examination sentence to obtain the partial probability value of the granular semantic significant coding of the prompt word leading the radiological examination sentence .

[0043] Calculate the concatenated vector of the sequence of semantically significant encoding vectors of the sentence granularity guided by the prompt word The probability value of the partial distribution of the semantic significant encoding of the granularity of the radiological examination sentence guided by the prompt word is an exponential power function , and multiplied by the partial probability value of the granular semantic significant encoding of the prompt word guided radiological inspection sentence to obtain the micro-representation vector of the granular semantic significant encoding of the prompt word guided radiological inspection sentence .

[0044] The concatenated vector of the sequence of semantically significant encoding vectors of the sentence granularity guided by the prompt word is After point multiplication with the partial probability value of the granular semantic significant encoding of the prompt word guided radiological examination sentence, an exponential function with a natural constant as the base is calculated to obtain the macroscopic mapping vector of the granular semantic significant encoding of the prompt word guided radiological examination sentence .

[0045] After calculating the base 2 logarithm of the micro-representation vector of the granular semantic significant encoding of the prompt word guided radiological examination sentence, the vector is weighted and summed with the macro-mapping vector of the granular semantic significant encoding of the prompt word guided radiological examination sentence to obtain the sequence of optimized granular semantic significant encoding vectors of the prompt word guided radiological examination sentence. .

[0046] That is, the partial low-order derivatives of the statistical distribution field corresponding to the sequence of the granular semantically significant coding vectors of the prompt word-guided radiological inspection sentences are used as non-overlapping macro-feature representation behavior patches of the sequence of the granular semantically significant coding vectors of the prompt word-guided radiological inspection sentences, so as to strengthen the dynamic sensitivity of the long-range micro-complex information distribution of the sequence of the granular semantically significant coding vectors of the prompt word-guided radiological inspection sentences to the macro-representation behavior of the class probability based on the different macro-behavior patch organization space under the non-isotropic backbone structure of the sequence of the granular semantically significant coding vectors of the prompt word-guided radiological inspection sentences, thereby promoting the iterative dynamic consistency of the generation target between the generation target and the extracted features in the feature space-generation probability mapping, so as to improve the semantic accuracy of the inspection report obtained by the large language model trained by the sequence input of the granular semantically significant coding vectors of the prompt word-guided radiological inspection sentences.

[0047] In another preferred embodiment, the trained large language model is a radiology report auxiliary writing model, and its construction and training methods are as follows: Figure 6 As shown, the method includes: S71, obtaining a report of the radiological examination type and performing data cleaning on the report content; S72, constructing a pre-training data set, building a radiological report auxiliary writing model and a pre-training module, and performing pre-training; S73, constructing an instruction data set InD based on keyword prompts, and building a fine-tuning module to fine-tune the radiological report auxiliary writing model; S74, deploying a radiological report auxiliary writing system to generate a target radiological report according to the keyword prompts input by the doctor.

[0048] Among them, step S71 includes: obtaining a report of the type of radiological examination, and performing data cleaning on the content of the report. This embodiment collects the report content according to the type of radiological examination. The type of radiological examination is different from ultrasonic examination, pathological examination, nuclear medicine examination, etc. The types of radiological examinations involved in this embodiment include the following: magnetic resonance imaging (MR), electronic computed tomography (CT), computer X-ray photography (CR), digital X-ray photography (DR), mammography (MG), etc. The radiological examinations covered in this embodiment include: multiple parts such as the head, facial features, chest, abdomen, pelvis, bones, and breast.

[0049] The radiology report of this embodiment includes the image findings and the diagnosis conclusion part. The radiology examination report is a report text written by the doctor after a detailed analysis and diagnosis of the radiology images of the patient.

[0050] In some embodiments, the imaging findings include the doctor's detailed record of the structural divisions, lesion size, number, boundaries, morphology and other imaging features of one or more examined organs, which can also be referred to as an imaging description. For example, imaging findings: "The chests on both sides are symmetrical, the textures of the two lungs are clear and run naturally, a ground glass nodule (thin layer Im140) is found in the middle lobe of the right lung, with a long diameter of about 4 mm, and no obvious consolidation shadows are found in the other two lungs, and the hilum of the two lungs is not large. The mediastinum window shows that the mediastinum is centered, and no obvious enlarged lymph nodes are found in the mediastinum. No obvious signs of pleural effusion are found." The doctor gives a conclusive diagnosis in the diagnostic opinion section, which briefly includes the type and degree of the lesion. The diagnostic opinion is also called the diagnostic conclusion. For example, the diagnostic opinion corresponding to the above example image is as follows: "Micronodules in the right middle lobe of the lung, follow-up examination is recommended." It is not difficult to see that the diagnostic opinion corresponds to the medical representation described in the image.

[0051] Furthermore, a preprocessing module PM is constructed to clean the acquired radiology reports. The cleaned report text should meet the following characteristics: no sensitive privacy information, no irrelevant radiology report information and noise, no errors and non-standard data, and no duplicate data.

[0052] In step S72, in order to make the radiology report assisted writing model familiar with the radiology report corpus, a pre-training module is constructed to pre-train the model. First, the pre-training data set PD is constructed in combination with the radiology report corpus cleaned by the preprocessing module PM. No manual annotation and secondary processing are performed during the construction of PD. Each text sample contained in the data set PD consists of two parts: image findings and diagnostic conclusions. The PD data set is in a computer-processable form, such as JSON format, CSV format, etc. Then, a radiology report assisted writing model and a pre-training module are built. The radiology report assisted writing model uses a Transformer structure for decoding and encoding, in which each Transformer layer is a multi-headed self-attention mechanism. After the original text is converted into tokens, it is input into the pre-training module of the model, and then the output probability distribution is obtained through the fully connected layer.

[0053] In this embodiment, the purpose of building a pre-training module is to make the radiology report auxiliary writing model familiar with the radiology report corpus. The pre-training module includes two tasks: random word masking and clause sorting. The model jointly constructs a loss function based on the two subtasks.

[0054] Furthermore, the random word masking task randomly masks the tokens in the input sequence and replaces the original sequence tokens with replacement words, and then the model outputs the predicted value of the corresponding masked position. In short, the model not only needs to learn the text before masking, but also needs to make predictions based on the text content after masking.

[0055] Assume that the input text sequence is ,in, Indicates In the random word masking task, a portion of words are randomly selected for masking, and the masked input sequence is obtained. The language model is based on the masked sequence Predict the probability distribution of masked words. Its loss function is the cross entropy loss function: ;in, is the indicator function, when It is 1 if it is equal to the jth word, otherwise it is 0.

[0056] Furthermore, the goal of the clause ordering task is to predict the order in a sentence. The approach is to divide a single sample text into words and sentences, randomly select connected clauses, with positive samples being AB and negative samples being BA. The model predicts the order of sentences by learning the coherence of the clause content. Specifically, part of the training data is sentence pairs in the original order (positive samples), and the other part is sentence pairs with the order swapped (negative samples). The model receives these sentences as input and makes predictions.

[0057] For two sequences: Represent the vector representation of the two sentences respectively, record The sentence order is The probability of The sentence order is The probability of , its loss function is: ,in represents the indicator function; Furthermore, the pre-training module combines the loss function of the above subtasks with the joint loss function. The joint loss function can be expressed as: , among which represents the weight of the subtask loss function. It is worth mentioning that in this embodiment Take 0.7, The best effect is achieved when the value is 0.3.

[0058] The pre-training module will receive the training data of the two sub-tasks at the same time, and update the parameters through back propagation and Adam optimization algorithm to complete the pre-training task. After the pre-training module, the model is more fully trained, and more accurately understands the relevance and logic of the term description and text clauses of the radiology report, improves the model's ability to understand and generate report texts, and enhances the applicability and reliability of the radiology report auxiliary writing model.

[0059] Step S73, in combination with the actual scenario of writing radiological reports, construct a data set InD of radiological report text instructions with keyword prompts and their matching ones, and build a fine-tuning model for the fine-tuning module. First, construct a data set InD of radiological report text instructions with keyword prompts and their matching ones. The keywords directly affect the model effect, so the selection of keywords is crucial. The keyword for the radiological report text without lesions contained in the data set InD is: 'no abnormality', which will not be elaborated here. The keywords corresponding to the radiological report text with confirmed lesions or abnormalities in the data InD include: lesion information such as lesion site, lesion nature, number, size, morphology, etc.; when the radiological examination contains uncertain lesion nature, the keywords include further suggestions in addition to lesion site, number, size, morphology, etc.; if it is an MR report text, it should also include signal intensity. The lesion site and lesion nature are required keywords, and whether the number, size, and morphology are required keywords depends on the lesion type. For example, if lung nodules are found during a chest CT scan, the report keywords must include the location and nature of the lung nodules, as well as the number, size, and morphology of the lung nodules. If rib fractures are found during a chest CT scan, the report keywords only need to specify the location and type of fracture.

[0060] The following table gives the keyword content, examples, necessity and other instructions.

[0061]

[0062] Typically, keywords are concluded by radiologists after understanding the patient's basic medical history and browsing the radiological images. However, the cost of manually refining keywords is too high. This embodiment uses a semi-supervised learning method to collect keywords corresponding to the radiological report text, and then constructs an InD dataset.

[0063] The following table is an example of the radiology report text corresponding to different keywords.

[0064]

[0065] First, the doctor restores the radiological diagnosis scenario and annotates M high-quality keyword-radiological report text data pairs, where the keywords correspond accurately to the radiological reports. The higher the quality of the keywords, the more fully they can describe the lesions, and the stronger their inspiration and guidance capabilities for the model. Then, using the few-shot method, M samples are fed into the model as examples, so that the model can learn to extract keywords based on radiological reports, and further generate N InD data sets, which contain a large number of high-quality keyword-radiological report text data pairs.

[0066] Construct the InD dataset, that is, construct the following sample json data: { instruction: ['Generate imaging findings and diagnostic conclusions based on the following keywords'] Input: ['The middle phalanx of the left ring finger is fractured.'] Output: ['Image findings: No obvious dislocation fracture was found in the middle finger of the left hand, and no obvious swelling was found in the surrounding soft tissue. Avulsion fracture at the base of the middle phalanx of the left ring finger. The bone structure of the remaining left hand components was regular, with intact bone structure. No obvious dislocation fracture or dislocation was found. The joint space was clear and the joint surface was smooth. Diagnosis conclusion: Avulsion fracture at the base of the middle phalanx of the left ring finger. Reexamination is recommended. '] },Furthermore, combined with the constructed InD dataset, a fine-tuning module was built to fine-tune the radiology report auxiliary writing model to improve the model's ability to generate complete radiology report texts based on keywords.

[0067] The fine-tuning module adds a trainable low-rank transformation matrix to each Transformer layer of the model, and only trains the low-rank transformation matrix during fine-tuning. The low-quality transformation matrix includes a dimension reduction matrix L and a dimension increase matrix I with a dimension of r.

[0068] During training, the parameters of the model are fixed, and only the dimension reduction matrix L and the dimension increase matrix I are trained, while the input and output dimensions of the model remain unchanged. When outputting, the L and I matrices are superimposed with the original parameters of the model. Specifically, L is initialized with a random Gaussian distribution and I is initialized with a zero matrix. In this way, only a small number of parameters need to be updated to obtain excellent fine-tuning effects.

[0069] In step S74, the deployment of the radiology report assisted writing model is realized. After the above pre-training process and fine-tuning process, a model specially adapted for radiology report assisted writing is obtained. Finally, it can be published as a service through FastAPI to achieve the connection with the front-end page. Finally, the doctor enters the prompt keyword, and the radiology report generation unit generates the radiology report.

[0070] In summary, the automatic generation method of the inspection report based on the prompt words according to the embodiment of the present application is explained, which obtains the keyword prompts input by the doctor, obtains the radiological examination data of the target patient object, and uses artificial intelligence data analysis and semantic coding technology to perform sentence processing and text convolution coding on the radiological examination data of the target patient object, and at the same time, semantic coding is performed on the keyword prompts input by the doctor, so as to adaptively adjust the significance of each part of the sentence granularity description in the radiological examination sentence granularity semantic coding feature based on the semantic embedding feature of the key prompt words as a guide, so as to intelligently generate the inspection report. In this way, it is possible to intelligently generate an inspection report with a reasonable structure, detailed content and prominent focus based on the guidance of the prompt words, thereby reducing the workload of doctors, helping doctors to quickly obtain the most important and relevant information, and improving work efficiency and decision-making quality.

[0071] Figure 7 FIG. 1 is a schematic flow chart of a system for automatically generating an inspection report based on prompt words according to an embodiment of the present application. Figure 7As shown, the automatic generation system 100 of examination reports based on prompt words includes: a keyword prompt acquisition module 110, which is used to acquire keyword prompts input by doctors; a radiological examination data acquisition module 120, which is used to acquire radiological examination data of target patient objects; a radiological examination sentence granularity semantic encoding module 130, which is used to perform sentence segmentation and text convolution encoding on the radiological examination data of the target patient objects to obtain radiological examination sentence granularity semantic encoding aggregation features; a key prompt word semantic encoding module 140, which is used to perform semantic encoding on the keyword prompts input by doctors to obtain key prompt word semantic embedding features; a prompt word guided radiological examination encoding module 150, which is used to perform multimodal feature interactive encoding on the key prompt word semantic embedding features and the radiological examination sentence granularity semantic encoding aggregation features to obtain a sequence of prompt word guided radiological examination sentence granularity semantic significant encoding features; and an examination report generation module 160, which is used to obtain an examination report based on the sequence of prompt word guided radiological examination sentence granularity semantic significant encoding features.

[0072] In one embodiment, the prompt word guided radiological inspection encoding module is used to perform multimodal feature interactive encoding on the key prompt word semantic embedding features and the radiological inspection sentence granularity semantic encoding aggregation features to obtain a sequence of prompt word guided radiological inspection sentence granularity semantic significant encoding features, including: a radiological inspection feature decoupling unit, used to perform feature decoupling on the radiological inspection sentence granularity semantic encoding aggregation features to obtain a set of radiological inspection sentence granularity local features; a prompt word guided radiological inspection interactive fusion unit, used to perform prompt gate guided attention interactive fusion on the key prompt word semantic embedding features and the set of radiological inspection sentence granularity local features to obtain a sequence of prompt word guided radiological inspection sentence granularity semantic significant encoding features.

[0073] In one embodiment, the radiology examination sentence granularity semantic encoding module is used to: perform sentence processing on the radiology examination data of the target patient object to obtain a sequence of radiology examination sentence granularity descriptions; input the sequence of radiology examination sentence granularity descriptions into a text convolutional neural network model including a Bert layer to obtain a radiology examination sentence granularity semantic encoding aggregation feature map as the radiology examination sentence granularity semantic encoding aggregation feature.

[0074] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned inspection report automatic generation system based on prompt words have been referred to above. Figures 1 to 6 The description of the automatic generation method of inspection reports based on prompt words has been introduced in detail, and therefore, its repeated description will be omitted.

Claims

1. A method for automatically generating an inspection report based on prompt words, characterized in that: include: S1, obtaining keyword prompts input by doctors; S2, obtaining radiological examination data of the target patient object; S3, performing sentence segmentation processing and text convolution encoding on the radiological examination data of the target patient object to obtain radiological examination sentence granularity semantic encoding aggregation features; S4, performing semantic encoding on the keyword prompts input by the doctor to obtain semantic embedding features of key prompt words; S5, performing multimodal feature interactive encoding on the key prompt word semantic embedding features and the radiological examination sentence granular semantic encoding aggregation features to obtain a sequence of prompt word-guided radiological examination sentence granular semantic significant encoding features; S6, obtaining an inspection report based on the sequence of semantically significant coding features of the radiological inspection sentence granularity guided by the prompt word; wherein, the S5, includes: S51, performing feature decoupling on the aggregated features of the semantic coding granularity of the radiological inspection sentence to obtain a set of local features of the radiological inspection sentence granularity; S52, performing a prompt gate-guided attention interaction fusion on the semantic embedding features of the key prompt word and the set of local features of the radiological inspection sentence granularity to obtain a sequence of semantically significant coding features of the radiological inspection sentence granularity guided by the prompt word.

2. The method for automatically generating an inspection report based on prompt words according to claim 1, characterized in that: The S3 comprises: performing sentence processing on the radiology examination data of the target patient object to obtain a sequence of radiology examination sentence granularity descriptions; inputting the sequence of radiology examination sentence granularity descriptions into a text convolutional neural network model including a Bert layer to obtain a radiology examination sentence granularity semantic encoding aggregation feature map as the radiology examination sentence granularity semantic encoding aggregation feature.

3. The method for automatically generating an inspection report based on prompt words according to claim 2, characterized in that: The S4 comprises: performing word segmentation processing on the keyword prompt input by the doctor and inputting the result into a keyword semantic encoder including a word embedding layer to obtain a key prompt word semantic embedding coding vector as the key prompt word semantic embedding feature.

4. The method for automatically generating an inspection report based on prompt words according to claim 3, characterized in that: The S51 includes: expanding each feature matrix along the channel dimension of the radiological inspection sentence granular semantic encoding aggregation feature map into a feature vector to obtain a set of radiological inspection sentence granular semantic local fine-grained feature vectors as the set of radiological inspection sentence granular local features.

5. The method for automatically generating an inspection report based on prompt words according to claim 4, characterized in that: The S52 includes: S521, based on the key prompt word semantic embedding coding vector, calculating the weighted attention weights of each radiological inspection sentence granularity semantic local fine-grained feature vector in the set of the radiological inspection sentence granularity semantic local fine-grained feature vectors to obtain a set of key prompt word-radiological inspection sentence granularity cross-modal semantic interaction attention weights; S522, based on the set of key prompt word-radiological inspection sentence granularity cross-modal semantic interaction attention weights, cross-modal interaction optimization of the key prompt word semantic embedding coding vector and the set of radiological inspection sentence granularity local features to obtain a sequence of prompt word-guided radiological inspection sentence granularity semantic significant coding vectors as a sequence of prompt word-guided radiological inspection sentence granularity semantic significant coding features.

6. The method for automatically generating an inspection report based on prompt words according to claim 5, characterized in that: The S521 includes: performing a linear transformation on the key prompt word semantic embedding coding vector to obtain a key prompt word semantic query coding vector and a key prompt word semantic value coding vector; performing a converter-based cross-modal prompt coding on each radiological inspection sentence granularity semantic local fine-grained feature vector in the set of the key prompt word semantic query coding vector, the key prompt word semantic value coding vector and the radiological inspection sentence granularity semantic local fine-grained feature vector to obtain a set of key prompt word-radiological inspection sentence granularity semantic cross-modal prompt semantic coding vectors; performing decoding gated modulation on the set of key prompt word-radiological inspection sentence granularity semantic cross-modal prompt semantic coding vectors to obtain a set of key prompt word-radiological inspection sentence granularity cross-modal semantic interaction attention weights.

7. The method for automatically generating an inspection report based on prompt words according to claim 6, characterized in that: The S522 includes: performing cross-modal interaction on the key prompt word semantic embedding coding vector and the set of radiographic inspection sentence granularity local features to obtain a set of key prompt word-radiographic inspection sentence granularity cross-modal interaction local feature vectors; based on the set of key prompt word-radiographic inspection sentence granularity cross-modal semantic interaction attention weights, optimizing each key prompt word-radiographic inspection sentence granularity cross-modal interaction local feature vector in the set of key prompt word-radiographic inspection sentence granularity cross-modal interaction local feature vectors to obtain a sequence of the prompt word-guided radiographic inspection sentence granularity semantic saliency coding vectors.

8. The method for automatically generating an inspection report based on prompt words according to claim 7, characterized in that: The S6 comprises: inputting the sequence of granular semantically significant encoding vectors of the prompt words guiding the radiological examination sentence into the trained large language model to obtain the examination report.

9. A system for automatically generating inspection reports based on prompt words, characterized in that: include: A keyword prompt acquisition module, used to acquire keyword prompts input by doctors; A radiological examination data acquisition module, used to acquire radiological examination data of a target patient object; A radiology examination sentence granularity semantic coding module is used to perform sentence segmentation processing and text convolution coding on the radiology examination data of the target patient object to obtain radiology examination sentence granularity semantic coding aggregation features; A key prompt word semantic encoding module, used for semantically encoding the key word prompt input by the doctor to obtain a key prompt word semantic embedding feature; A prompt word guided radiological examination encoding module is used to perform multimodal feature interactive encoding on the key prompt word semantic embedding features and the radiological examination sentence granular semantic encoding aggregation features to obtain a sequence of prompt word guided radiological examination sentence granular semantic significant encoding features; An inspection report generation module is used to obtain an inspection report based on the sequence of semantically significant coding features of radiological inspection sentences guided by the prompt words; wherein the prompt word-guided radiological inspection coding module includes: a radiological inspection feature decoupling unit, used to perform feature decoupling on the radiological inspection sentence granularity semantic coding aggregation features to obtain a set of radiological inspection sentence granularity local features; a prompt word-guided radiological inspection interactive fusion unit, used to perform a prompt gate-guided attention interactive fusion on the key prompt word semantic embedding features and the set of radiological inspection sentence granularity local features to obtain a sequence of semantically significant coding features of radiological inspection sentences guided by the prompt word.

10. The automatic generation system of inspection report based on prompt words according to claim 9, characterized in that: The radiology examination sentence granularity semantic encoding module is used to: perform sentence segmentation processing on the radiology examination data of the target patient object to obtain a sequence of radiology examination sentence granularity descriptions; input the sequence of radiology examination sentence granularity descriptions into a text convolutional neural network model including a Bert layer to obtain a radiology examination sentence granularity semantic encoding aggregation feature map as the radiology examination sentence granularity semantic encoding aggregation feature.

Images