A diagnostic report generation system and method based on medical image concept

By introducing image feature extraction and conceptual text alignment modules into the medical image subtitle model, a clear diagnostic report is generated, which solves the problem of poor interpretation of diagnostic reports in the prior art and improves the efficiency of medical image-assisted diagnosis.

CN118098481BActive Publication Date: 2024-11-22SHIXIANG MEDICAL TECH (DALIAN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410273866.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-11
Publication Date
2024-11-22
Estimated Expiration
2044-03-11

AI Technical Summary

Technical Problem

The existing medical image subtitle model lacks interpretability and cannot clearly reflect the correspondence between the diagnostic report text and the medical image, resulting in the inability to effectively assist medical professionals in diagnosis.

Method used

A diagnostic report generation system based on medical images is designed, including image feature extraction module, concept extraction module, concept text alignment module, decoder module and output module. By extracting image features, the corresponding relationship between lesion features and dictionary word features is calculated. , generate clear diagnostic reports.

Benefits of technology

It realizes clear expression of the correlation between images and diagnostic reports, improves the interpretability and auxiliary diagnostic efficiency of diagnostic reports, and gives full play to the automation and efficient role of medical image diagnostic reports.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118098481B_ABST
    Figure CN118098481B_ABST
Patent Text Reader

Abstract

The present invention discloses a diagnostic report generation system and method based on medical image concepts, including a concept extraction module, a concept text alignment module and a word feature extraction module; the concept extraction module is used to extract all lesion features in the image feature map; the concept text alignment module is used to obtain the most similar dictionary word feature corresponding to each lesion feature in a number of dictionary word features; the word feature extraction module is used to obtain the correspondence between the text in the plain text diagnostic report and the several most similar dictionary word features; and finally outputs a final diagnostic report corresponding to the text in the plain text diagnostic report and the lesion features. The final diagnostic report generated by the system can clearly give the correlation between the medical image and the plain text diagnostic report, so as to give full play to the role of efficiently assisting doctors in diagnosis, improve diagnostic efficiency and thus serve more patients.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a diagnosis report generation system and method based on medical image concepts. Background Art

[0002] Recent developments in the healthcare field have increasingly emphasized the potential of medical artificial intelligence in assisting in the generation of cost-effective and safe diagnostic reports, and the automated generation of medical image diagnostic reports plays an important role in improving the efficiency of the medical system. Image captioning is an important technology to achieve this goal, but the existing image captioning models lack interpretability, that is, the correspondence between medical images and the corresponding generated diagnostic report text is unclear, and it cannot reflect which lesion areas on the medical image the diagnostic conclusions in the diagnostic report text are based on. This makes the existing diagnostic reports unable to effectively assist medical professionals in their diagnosis work in the actual diagnosis process, and does not give full play to the efficient role of the automation of medical image diagnostic reports. Therefore, a truly reliable image captioning model should be able to generate high-quality reports and provide understandable explanations for the generated content. Summary of the invention

[0003] The present invention provides a diagnostic report generation system and method based on the concept of medical images, so as to overcome the technical problem that the diagnostic report generated in the prior art cannot reflect which lesion areas on the medical image the diagnostic conclusions in the diagnostic report text are drawn based on, so that in the actual diagnosis process, the existing diagnostic report cannot effectively assist medical professionals in diagnosis work and fails to give full play to the automated and efficient role of the medical image diagnostic report.

[0004] In order to achieve the above object, the technical solution of the present invention is:

[0005] A diagnosis report generation system based on medical image concepts comprises an image feature extraction module, a concept extraction module, a concept text alignment module, a decoder module, a word feature extraction module and an output module; the image feature extraction module is used to extract image features of medical images to obtain an image feature map, wherein the image features include texture features and shape features; the concept extraction module is used to extract all lesion features in the image feature map; the concept text alignment module is used to obtain the most similar dictionary word feature corresponding to each lesion feature among a number of dictionary word features; the decoder module is used to decode each most similar dictionary word feature to obtain a plain text diagnosis report; the word feature extraction module is used to obtain the correspondence between the text in the plain text diagnosis report and a number of the most similar dictionary word features; the output module is used to obtain a final diagnosis report corresponding to the text in the plain text diagnosis report and the lesion features based on the correspondence between the text in the plain text diagnosis report and a number of the most similar dictionary word features and the most similar dictionary word feature corresponding to each lesion feature among a number of dictionary word features among all the lesion features.

[0006] Furthermore, the process of extracting all lesion features in the image feature map by the concept extraction module is as follows:

[0007] 1) Inputting the image feature map F into the concept extraction module;

[0008] First, the image feature map F is position-encoded to obtain the image feature map F′ after adding the position coding, which is expressed as:

[0009] F′=F+P

[0010] Where P is the position code;

[0011] The position code P is calculated based on the sine and cosine functions for each spatial position and feature dimension, which is specifically expressed as:

[0012]

[0013]

[0014] Where P (pos,2t) and P (pos,2t+1) are the position codes of the spatial position pos in dimensions 2t and 2t+1 respectively; t is the index of the current dimension d;

[0015] The image feature map F′ after adding the position code is stretched to obtain a stretched image feature map F″, wherein the dimension of the image feature map F″ is d×l, where l=hw, and h and w are the height and width of the image feature map F″ respectively;

[0016] 2) Input the image feature map F′ after adding the position encoding into the learnable visual concept prototype VCPs, where VCPs is a learnable matrix W = {c1, c2, …c k},c1,c2,…c k are learnable vectors of dimension d×1 corresponding to each VCP in VCPs, k is the number of concepts, c1, c2, … c k Focus on a specific area of ​​a lesion in F′, through c1, c2, …c k Calculate the lesion features of k VCP focus areas and obtain the set of all lesion features F * , where the calculation process of the lesion characteristics of each VCP focus area is;

[0017] Based on the slot-attention algorithm, the attention representation between each VCP and F″ is calculated to obtain the region of interest of each VCP. The calculation process includes:

[0018] Each VCP and the stretched image feature map F″ are input into the first multilayer perceptron Q and the second multilayer perceptron K respectively for information enhancement to obtain a′ i , the calculation formula is:

[0019] a′ i =Q(c i ) T K(F″)

[0020] Among them, c i ∈W, i = 1, 2, ... k; Q and K are both structures with three sequentially connected "fully connected layers-ReLU nonlinear activation function layers"; a′ i ∈A′={a′1,a′2,…a′ k};

[0021] a′ i Input the activation function layer and finally get the region of interest a i , the calculation formula is:

[0022]

[0023] In the formula, the region of interest a i is a one-dimensional vector with a length of l; is the sigmoid activation function, a i ∈A={a1,a2,…a k};

[0024] By region of interest a i Visualize the image position corresponding to the VCP focus, so as to extract the lesion features v in the VCP focus area i, the calculation formula is:

[0025]

[0026] In the formula, v i ∈F * , F1 is the feature vector with dimension d×l obtained by stretching the spatial dimension of the image feature map F, and the lesion feature v i is a one-dimensional vector with length d.

[0027] Furthermore, the plurality of dictionary word features are obtained by the diagnosis report generation system through training in the training phase, and each dictionary word feature is a vector with a dimension of d;

[0028] The process of the concept text alignment module obtaining the most similar dictionary word feature corresponding to each lesion feature among a plurality of dictionary word features is as follows:

[0029] The optimal allocation is calculated using the Hungarian matrix, i.e., the sum of the allocation losses is minimized, where the allocation loss is the Euclidean distance between each lesion feature and each dictionary word feature, and the following operations are performed:

[0030] Subtract the minimum value: Subtract the minimum value from each row of the Hungarian matrix, and then subtract the minimum value from each column so that each row and column has at least one zero;

[0031] Strikethrough: Cover all zeros with the minimum number of horizontal or vertical lines;

[0032] Check the number of lines: If the number of lines is equal to the order of the Hungarian matrix, that is, the number of rows or columns, the optimal match is found. At this time, the sum of the distribution losses is the smallest, indicating that the similarity between the lesion feature and the dictionary word feature is the highest, that is, the most similar dictionary word feature corresponding to each lesion feature is obtained. Otherwise, the matrix adjustment operation is performed;

[0033] Adjust the matrix: find the minimum value not covered by the line, then subtract this minimum value from each element not covered by the line, and add this minimum value to each element covered by two lines, and the elements covered by one line remain unchanged; then repeat the steps of drawing lines, checking the number of lines and adjusting the matrix until the optimal match is found, then end the operation.

[0034] Furthermore, the process of the word feature extraction module obtaining the correspondence between the text in the plain text diagnosis report and a plurality of most similar dictionary word features is as follows:

[0035] The word feature extraction module is a language model, which obtains a number of word meta-tokens by segmenting the plain text diagnosis report;

[0036] Map several word units to the dictionary space to obtain several most similar dictionary word features corresponding to the text in the plain text diagnosis report. The formula is:

[0037]

[0038] Where s is the number of tokens after mapping; d is the length of dictionary word features in the encoding dictionary; T is the plain text diagnosis report; T′ is the dictionary space; LM is the language model.

[0039] Furthermore, during the training process of the concept extraction module, the loss function adopts the intra-individual consistency loss function and the inter-individual distinctiveness loss function;

[0040] The calculation formula of the intra-individual consistent loss function is:

[0041]

[0042] In the formula, l con represents the intra-concept consistency loss function; H i is the number of samples of the i-th concept in a training batch; k is the number of concepts; v′ i and v i ″ indicates H i The lesion features of two different samples; sim(·) is the operation of calculating the cosine similarity of two different lesion features;

[0043] The calculation formula of the inter-individual discrimination loss function is:

[0044]

[0045] In the formula, l dis represents the loss function for distinguishing concepts; and are the means of all lesion features in the i′th VCP and the i″th VCP in a training batch, respectively, i′,i″∈k.

[0046] Furthermore, the image feature extraction module is a convolutional neural network CNNs model or a visual Transformer model.

[0047] A diagnostic report generation method of a diagnostic report generation system based on medical image concepts, the specific steps comprising:

[0048] S1: Acquire a medical image and extract an image feature map of the medical image;

[0049] S2: extracting all lesion features in the image feature map;

[0050] S3: Calculating the optimal allocation based on the Hungarian matrix to obtain the most similar dictionary word feature corresponding to each lesion feature in all the lesion features among a plurality of dictionary word features;

[0051] S4: Decode each most similar dictionary word feature to obtain a plain text diagnosis report;

[0052] S5: Obtaining the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features through a language model;

[0053] S6: Based on the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features and the most similar dictionary word feature corresponding to each lesion feature in all the lesion features among several dictionary word features, a final diagnosis report corresponding to the text in the plain text diagnosis report and the lesion features is obtained.

[0054] Beneficial effect: The present invention extracts all lesion features from medical images by setting a concept extraction module, and the most similar dictionary word features corresponding to all lesion features can be determined by the concept text alignment module, so as to decode and obtain a plain text diagnosis report. After obtaining the plain text diagnosis report, the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features is obtained by the word feature extraction module, and finally based on the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features and the most similar dictionary word features corresponding to each lesion feature in all lesion features in several dictionary word features, a final diagnosis report corresponding to the text in the plain text diagnosis report and the lesion features is obtained. The final diagnosis report generated by the present invention can clearly give the correlation between the medical image and the plain text diagnosis report, so as to give full play to the role of efficiently assisting doctors in diagnosis, improve the diagnosis efficiency and serve more patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0056] Figure 1 It is a structural schematic diagram of a diagnosis report generation system based on medical image concept in the present invention;

[0057] Figure 2 is a workflow diagram of a concept extraction module in an embodiment of the present invention;

[0058] Figure 3 A flowchart of a method for generating a diagnosis report based on medical image concepts in the present invention;

[0059] Figure 4 This is a sample diagram of a final diagnosis report in which the text in a plain text diagnosis report obtained after inputting a medical image corresponds to the lesion features in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] This embodiment provides a diagnosis report generation system based on medical image concepts, such as Figure 1 As shown, it includes an image feature extraction module, a concept extraction module, a concept text alignment module, a decoder module, a word feature extraction module and an output module;

[0062] The image feature extraction module is used to extract image features of the medical image to obtain an image feature map, wherein the image features include texture features and shape features; specifically, the image feature extraction module is a convolutional neural network CNNs model or a visual Transformer model; for example, the medical image is input into the convolutional neural network CNNs model to obtain an image feature map F, which is expressed as:

[0063]

[0064] Where x is the input medical image, is a real number, d is the feature map dimension, h', w' are the height and width of the feature map respectively.

[0065] The concept extraction module is used to extract all lesion features in the image feature map;

[0066] In a specific embodiment, the process of extracting all lesion features in the image feature map by the concept extraction module is as follows: Figure 2 As shown:

[0067] 1) Inputting the image feature map F into the concept extraction module;

[0068] First, the image feature map F is position-encoded to obtain the image feature map F′ after adding the position coding, which is expressed as:

[0069] F′=F+P

[0070] Where P is the position code;

[0071] The position code P is calculated based on the sine and cosine functions for each spatial position and feature dimension, which is specifically expressed as:

[0072]

[0073]

[0074] Where P (pos,2t) and P (pos,2t+1) are the position codes of the spatial position pos in dimensions 2t and 2t+1 respectively; t is the index of the current dimension d;

[0075] The image feature map F′ after adding the position code is stretched to obtain a stretched image feature map F″, wherein the dimension of the image feature map F″ is d×l, where l=hw, and h and w are the height and width of the image feature map F″ respectively;

[0076] 2) Referring to the paper “Learning Bottleneck Concepts in Image Classification” from Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition 2023, the image feature map F′ with position encoding is input into the learnable visual concept prototype VCPs, where VCPs is a learnable matrix W = {c1, c2, …c k},c1,c2,…c k are learnable vectors of dimension d×1 corresponding to each VCP in VCPs, k is the number of concepts, c1, c2, … c k Focus on a specific area of ​​a lesion in F′, through c1, c2, …c k Calculate the lesion features of k VCP focus areas and obtain the set of all lesion features F * , where the calculation process of the lesion characteristics of each VCP focus area is;

[0077] Referring to the slot-attention algorithm mentioned in the NeurlPS paper “Object-Centric Learning with Slot Attention”, the attention representation between each VCP and F″ is calculated to obtain the region of interest of each VCP. The calculation process includes:

[0078] Each VCP and the stretched image feature map F″ are input into the first multilayer perceptron Q and the second multilayer perceptron K respectively for information enhancement to obtain a′ i , the calculation formula is:

[0079] a′ i =Q(c i ) T K(F″)

[0080] Among them, c i ∈W, i = 1, 2, ... k; Q and K are both structures with three sequentially connected "fully connected layers-ReLU nonlinear activation function layers"; a′ i ∈A′={a′1,a′2,…a′ k};

[0081] a′ i Input the activation function layer and finally get the region of interest a i , the calculation formula is:

[0082]

[0083] In the formula, the region of interest a i is a one-dimensional vector with a length of l; is the sigmoid activation function, a i ∈A={a1,a2,…a k};

[0084] By region of interest a i Visualize the image position corresponding to the VCP focus, so as to extract the lesion features v in the VCP focus area i , the calculation formula is:

[0085]

[0086] In the formula, v i ∈F * , F1 is the feature vector with dimension d×l obtained by stretching the spatial dimension of the image feature map F, and the lesion feature v i is a one-dimensional vector with length d.

[0087] In a specific embodiment, during the training process of the concept extraction module, in order to enhance the learning ability of each VCP, the loss function adopts the intra-individual consistency loss function so that each VCP can consistently capture a lesion feature, and adopts the inter-individual distinctiveness loss function so that all VCPs can capture different lesion features as much as possible; the calculation formula of the intra-individual consistency loss function is:

[0088]

[0089] In the formula, l con represents the intra-concept consistency loss function; H i is the number of samples of the i-th concept in a training batch; k is the number of concepts; v′ i and v i ″ indicates H i The lesion features of two different samples; sim(·) is the operation of calculating the cosine similarity of two different lesion features;

[0090] The calculation formula of the inter-individual discrimination loss function is:

[0091]

[0092] In the formula, l dis represents the loss function for distinguishing concepts; and are the means of all lesion features in the i′th VCP and the i″th VCP in a training batch, respectively, i′,i″∈k.

[0093] The concept text alignment module is used to obtain the most similar dictionary word feature corresponding to each lesion feature among a plurality of dictionary word features;

[0094] Specifically, the process of the concept text alignment module obtaining the most similar dictionary word feature corresponding to each lesion feature among a plurality of dictionary word features is as follows:

[0095] The optimal allocation is calculated using the Hungarian matrix, i.e., the sum of the allocation losses is minimized, where the allocation loss is the Euclidean distance between each lesion feature and each dictionary word feature, and the following operations are performed:

[0096] Subtract the minimum value: Subtract the minimum value from each row of the Hungarian matrix, and then subtract the minimum value from each column so that each row and column has at least one zero;

[0097] Strikethrough: Cover all zeros with the minimum number of horizontal or vertical lines;

[0098] Check the number of lines: If the number of lines is equal to the order of the Hungarian matrix, that is, the number of rows or columns, the optimal match is found. At this time, the sum of the distribution losses is the smallest, indicating that the similarity between the lesion feature and the dictionary word feature is the highest, that is, the most similar dictionary word feature corresponding to each lesion feature is obtained. Otherwise, the matrix adjustment operation is performed;

[0099] Adjust the matrix: find the minimum value not covered by the line, then subtract this minimum value from each element not covered by the line, and add this minimum value to each element covered by two lines, and the elements covered by one line remain unchanged; then repeat the steps of drawing lines, checking the number of lines and adjusting the matrix until the optimal match is found, then end the operation.

[0100] The decoder module is used to decode each most similar dictionary word feature to obtain a plain text diagnosis report; specifically, the decoder module is used to decode the most similar dictionary word feature to obtain the diagnosis report text. In this embodiment, the decoder is a model component for generating text. In deep learning, especially in the context of natural language processing (NLP), a decoder generally refers to a network structure that can generate a coherent text sequence. This embodiment uses a model based on the Transformer architecture as a decoder.

[0101] In a specific embodiment, the several dictionary word features are obtained by training the diagnosis report generation system in a training phase. In the training phase of the diagnosis report generation system, training medical images and corresponding training diagnosis report texts are obtained; and the language model in the diagnosis report generation system is trained based on the training diagnosis report texts, thereby obtaining several dictionary word features, each of which is a vector with a dimension of d; this embodiment uses an encoding dictionary composed of several dictionary word features to perform structured encoding on text content, thereby promoting the matching between text and visual concepts, so as to ensure that the diagnosis report generation system can capture the richness and complexity of language. This method refers to the ACL paper "Evidence-Aware Inferential Text Generation with Vector QuantisedVariational AutoEncoder".

[0102] The word feature extraction module is used to obtain the correspondence between the text in the plain text diagnosis report and a plurality of most similar dictionary word features;

[0103] In a specific embodiment, the word feature extraction module is a language model, and the process of obtaining the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features by the word feature extraction module is:

[0104] By segmenting the plain text diagnostic report, a number of word-unit tokens are obtained;

[0105] Map several word units to the dictionary space to obtain several most similar dictionary word features corresponding to the text in the plain text diagnosis report. The formula is:

[0106]

[0107] Wherein, s is the number of word units after mapping; d is the length of the dictionary word features in the encoding dictionary; T is the plain text diagnosis report; T′ is the dictionary space; LM is the language model. In this embodiment, the language model is the T5 language model.

[0108] The output module is used to obtain a final diagnosis report corresponding to the text in the plain text diagnosis report and the lesion features based on the correspondence between the text in the plain text diagnosis report and the most similar dictionary word features and the most similar dictionary word features corresponding to each lesion feature in the total lesion features, such as Figure 4 shown.

[0109] This embodiment also provides a method for generating a diagnosis report using a diagnosis report generating system based on medical image concepts, such as Figure 3 As shown, the specific steps include:

[0110] S1: Acquire a medical image and extract an image feature map of the medical image;

[0111] S2: extracting all lesion features in the image feature map;

[0112] S3: Calculating the optimal allocation based on the Hungarian matrix to obtain the most similar dictionary word feature corresponding to each lesion feature in all the lesion features among a plurality of dictionary word features;

[0113] S4: Decode each most similar dictionary word feature to obtain a plain text diagnosis report;

[0114] S5: Obtaining the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features through a language model;

[0115] Specifically, since it is unknown which most similar dictionary word feature the text in the plain text diagnosis report specifically corresponds to, the plain text diagnosis report is input into a language model to obtain a correspondence between the text in the plain text diagnosis report and several most similar dictionary word features;

[0116] S6: Based on the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features and the most similar dictionary word feature corresponding to each lesion feature in all the lesion features among several dictionary word features, a final diagnosis report corresponding to the text in the plain text diagnosis report and the lesion features is obtained.

[0117] Specifically, the most similar dictionary word features corresponding to the words in the known plain text diagnosis report are obtained through S5, and the most similar dictionary word features corresponding to each lesion feature in all lesion features among several dictionary word features are also determined by calculating the optimal allocation through the Hungarian matrix. Therefore, the final diagnosis report corresponding to the words in the plain text diagnosis report and the lesion features can be obtained.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A diagnostic report generation system based on medical image concepts, characterized in that: It includes an image feature extraction module, a concept extraction module, a concept text alignment module, a decoder module, a word feature extraction module and an output module; The image feature extraction module is used to extract image features of medical images to obtain image feature maps, wherein the image features include texture features and shape features; The concept extraction module is used to extract all lesion features in the image feature map; The process of extracting all lesion features in the image feature map by the concept extraction module is as follows: 1) Inputting the image feature map F into the concept extraction module; First, the image feature map F is position-encoded to obtain the image feature map F′ after adding the position coding, which is expressed as: F′=F+P Where P is the position code; The position code P is calculated based on the sine and cosine functions for each spatial position and feature dimension, which is specifically expressed as: Where P (pos,2t) and P (pos,2t+1) They are the position encodings of the spatial position pos in dimensions 2t and 2t+1 respectively; t is the index of the current dimension d; The image feature map F′ after adding the position code is stretched to obtain a stretched image feature map F″, wherein the dimension of the image feature map F″ is d×l, where l=hw, and h and w are the height and width of the image feature map F″ respectively; 2) Input the image feature map F′ after adding the position encoding into the learnable visual concept prototype VCPs, where VCPs is a learnable matrix W = {c1, c2, ...c k }, c1, c2, ... c k are learnable vectors of dimension d×1 corresponding to each VCP in VCPs, k is the number of concepts, c1, c2, ...c k Focus on a specific area of ​​a lesion in F', through c1, c2, ...c k Calculate the lesion features of k VCP focus areas and obtain the set F* of all lesion features, where the calculation process of the lesion features of each VCP focus area is: Based on the slot-attention algorithm, the attention representation between each VCP and F″ is calculated to obtain the region of interest of each VCP. The calculation process includes: Each VCP and the stretched image feature map F″ are input into the first multilayer perceptron Q and the second multilayer perceptron K respectively for information enhancement to obtain a′ i , the calculation formula is: a′ i =Q(c i ) T K(F″) Among them, c i ∈W, i = 1, 2, ... k; Q and K are both structures with three sequentially connected "fully connected layers-ReLU nonlinear activation function layers"; a′ i ∈A′={a′1, a′2,...a′ k }; a′ i Input the activation function layer and finally get the region of interest a i , the calculation formula is: In the formula, the region of interest a i is a one-dimensional vector with a length of l; is the sigmoid activation function, a i ∈A={a1,a2,...a k }; By region of interest a i Visualize the image position corresponding to the VCP focus, so as to extract the lesion features v in the VCP focus area i , the calculation formula is: In the formula, v i ∈F * , F1 is the feature vector with dimension d×l obtained by stretching the spatial dimension of the image feature map F, and the lesion feature v i is a one-dimensional vector with a length of d; The concept text alignment module is used to obtain the most similar dictionary word feature corresponding to each lesion feature among a plurality of dictionary word features; The decoder module is used to decode each most similar dictionary word feature to obtain a plain text diagnosis report; The word feature extraction module is used to obtain the correspondence between the text in the plain text diagnosis report and a plurality of most similar dictionary word features; The output module is used to obtain a final diagnosis report corresponding to the text in the plain text diagnosis report and the lesion features based on the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features and the most similar dictionary word feature corresponding to each lesion feature in all the lesion features among several dictionary word features.

2. The diagnostic report generation system based on medical image concept according to claim 1, characterized in that: The plurality of dictionary word features are obtained by the diagnosis report generation system through training in the training phase, and each dictionary word feature is a vector with a dimension of d; The process of the concept text alignment module obtaining the most similar dictionary word feature corresponding to each lesion feature among a plurality of dictionary word features is as follows: The optimal allocation is calculated using the Hungarian matrix, i.e., the sum of the allocation losses is minimized, where the allocation loss is the Euclidean distance between each lesion feature and each dictionary word feature, and the following operations are performed: Subtract the minimum value: Subtract the minimum value from each row of the Hungarian matrix, and then subtract the minimum value from each column so that each row and column has at least one zero; Strikethrough: Cover all zeros with the minimum number of horizontal or vertical lines; Check the number of lines: If the number of lines is equal to the order of the Hungarian matrix, that is, the number of rows or columns, the optimal match is found. At this time, the sum of the distribution losses is the smallest, indicating that the similarity between the lesion feature and the dictionary word feature is the highest, that is, the most similar dictionary word feature corresponding to each lesion feature is obtained. Otherwise, the matrix adjustment operation is performed; Adjust the matrix: find the minimum value not covered by the line, then subtract this minimum value from each element not covered by the line, and add this minimum value to each element covered by two lines, and the elements covered by one line remain unchanged; then repeat the steps of drawing lines, checking the number of lines and adjusting the matrix until the optimal match is found, then end the operation.

3. The diagnostic report generation system based on medical image concept according to claim 2, characterized in that: The process of the word feature extraction module obtaining the correspondence between the text in the plain text diagnosis report and a plurality of most similar dictionary word features is as follows: The word feature extraction module is a language model, which obtains a number of word meta-tokens by segmenting the plain text diagnosis report; Map several word units to the dictionary space to obtain several most similar dictionary word features corresponding to the text in the plain text diagnosis report. The formula is: Where s is the number of tokens after mapping; d is the length of the dictionary word features in the encoding dictionary; T is a plain text diagnosis report; T′ is a dictionary space; LM is a language model.

4. The diagnostic report generation system based on medical image concept according to claim 3, characterized in that: During the training process of the concept extraction module, the loss function adopts the intra-individual consistency loss function and the inter-individual distinctiveness loss function; The calculation formula of the intra-individual consistent loss function is: In the formula, l con represents the intra-individual consistent loss function; H i is the number of samples of the i-th concept in a training batch; k is the number of concepts; v i ′ and v i ″ indicates H i The lesion characteristics of two different samples are shown inside; sim(·) is an operation to calculate the cosine similarity of two different lesion features; The calculation formula of the inter-individual discrimination loss function is: In the formula, l dis represents the loss function for distinguishing between individuals; and are the means of all lesion features in the i′th VCP and i″th VCP in a training batch, respectively. ′ ,i ′′ ∈k.

5. The diagnostic report generation system based on medical image concept according to claim 1, characterized in that: The image feature extraction module is a convolutional neural network (CNNs) model or a visual Transformer model.

6. A method for generating a diagnostic report using the diagnostic report generating system based on medical image concepts as claimed in any one of claims 1 to 5, characterized in that: The specific steps include: S1: Acquire a medical image and extract an image feature map of the medical image; S2: extracting all lesion features in the image feature map; S3: Calculating the optimal allocation based on the Hungarian matrix to obtain the most similar dictionary word feature corresponding to each lesion feature in all the lesion features among a plurality of dictionary word features; S4: Decode each most similar dictionary word feature to obtain a plain text diagnosis report; S5: Obtaining the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features through a language model; S6: Based on the correspondence between the text in the plain text diagnosis report and several most similar dictionary word features and the most similar dictionary word feature corresponding to each lesion feature in all the lesion features among several dictionary word features, a final diagnosis report corresponding to the text in the plain text diagnosis report and the lesion features is obtained.