Image diagnosis report generation method and device and storage medium

Abstract

The invention discloses an image diagnosis report generation method and device and a storage medium. The method comprises the following steps: determining the matching degree on three different levels, namely, the spatial matching degree between a diagnosis text and a candidate template on an anatomical part level, the semantic matching degree between the diagnosis text and the candidate template and related to pathological information, and the clinical association degree between the diagnosis text and the candidate template; according to the method, the target template matched with the diagnosis text is searched based on the diagnosis text input by the user, and the search is further optimized through the time decay factor, the clinical priority and the conflict degree among the regions of interest, so that the accuracy of searching the required image diagnosis template for the user is improved, and the user experience is improved. Therefore, the technical problems that in the prior art, a traditional keyword matching algorithm is poor in keyword matching effect in a medical scene, and the accuracy of searching out a correct template is reduced are solved.

Description

Technical Field

[0001] The present application relates to the field of medical information technology, and in particular to a method, device and storage medium for generating an imaging diagnostic report. Background Art

[0002] Currently, imaging diagnostic reports are usually generated by doctors manually entering keywords or sentences to search in the template library of imaging diagnostic reports based on imaging results, thereby obtaining the required imaging diagnostic report template. The doctor then modifies the imaging diagnostic report template to obtain a complete imaging diagnostic report.

[0003] In existing technologies, imaging report generation systems typically perform simple semantic matching between keywords (or sentences) and information related to imaging report templates. For example, pre-trained language models (such as the BERT model and its variants) are used to calculate the semantic similarity between sentences or words for matching, thereby searching for the corresponding imaging report template and providing it to the doctor. However, existing technologies have the following problems: Traditional semantic matching algorithms perform poorly in medical scenarios. For example, "miliary nodules" and "micronodule clusters" have the same meaning in medical contexts, but traditional semantic matching algorithms often fail to match these two terms. Consequently, this approach reduces the accuracy of finding the correct template.

[0004] With respect to the technical problem of poor accuracy in searching for imaging diagnostic report templates in the above-mentioned prior art, no effective solution has been proposed yet. Summary of the Invention

[0005] The embodiments of the present disclosure provide a method, apparatus, and storage medium for generating an imaging diagnostic report, so as to at least solve the technical problem of poor accuracy in searching for imaging diagnostic report templates in the prior art.

[0006] According to one aspect of an embodiment of the present disclosure, a method for generating an imaging diagnostic report is provided, comprising: receiving diagnostic text corresponding to a medical image input by a user from a terminal device of the user; determining the degree of spatial matching between the diagnostic text and a candidate template at the anatomical site level, the degree of semantic matching between the diagnostic text and the candidate template related to pathological information, and the degree of clinical association between the diagnostic text and the candidate template; determining a first degree of matching between the candidate template and the diagnostic text based on the degree of spatial matching, the degree of semantic matching, and the degree of clinical association; determining a target template from the candidate template based on the first degree of matching; and sending the target template to the terminal device, and receiving an imaging diagnostic report corresponding to the target template from the terminal device.

[0007] According to another aspect of the embodiments of the present disclosure, a storage medium is further provided. The storage medium includes a stored program, wherein the above method is executed by a processor when the program is running.

[0008] According to another aspect of an embodiment of the present disclosure, a device for generating an imaging diagnostic report is also provided, including: a receiving module for receiving a diagnostic text corresponding to a medical image input by a user from a terminal device of the user; a three-level matching module for determining the degree of spatial matching between the diagnostic text and the candidate template at the anatomical site level, the degree of semantic matching between the diagnostic text and the candidate template related to pathological information, and the degree of clinical correlation between the diagnostic text and the candidate template; a matching degree determination module for determining a first matching degree between the candidate template and the diagnostic text based on the spatial matching degree, the semantic matching degree and the clinical correlation degree; a selection module for determining a target template from the candidate template based on the first matching degree; and a sending module for sending the target template to the terminal device and receiving an imaging diagnostic report corresponding to the target template from the terminal device.

[0009] According to another aspect of an embodiment of the present disclosure, a device for generating an imaging diagnostic report is also provided, comprising: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: receiving diagnostic text corresponding to a medical image input by a user from a terminal device of the user; determining the degree of spatial matching between the diagnostic text and the candidate template at the anatomical site level, the degree of semantic matching between the diagnostic text and the candidate template related to pathological information, and the degree of clinical correlation between the diagnostic text and the candidate template; determining a first degree of matching between the candidate template and the diagnostic text based on the degree of spatial matching, the degree of semantic matching, and the degree of clinical correlation; determining a target template from the candidate template based on the first degree of matching; and sending the target template to the terminal device, and receiving an imaging diagnostic report corresponding to the target template from the terminal device.

[0010] In the disclosed embodiment, according to the technical solution of this embodiment, the matching degree at three different levels, i.e., the spatial matching degree between the diagnostic text and the candidate template at the anatomical site level, the semantic matching degree between the diagnostic text and the candidate template related to pathological information, and the clinical correlation degree between the diagnostic text and the candidate template, can be used to search for a target template that matches the diagnostic text based on the diagnostic text input by the user, thereby improving the accuracy of searching for the required imaging diagnostic template for the user, thereby solving the technical problem in the prior art that the traditional keyword matching algorithm has poor keyword matching effect in medical scenarios and reduces the accuracy of searching for the correct template. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings: Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to embodiment 1 of the present disclosure; Figure 2 is a schematic diagram of the imaging diagnostic report generation system according to Example 1 of the present disclosure; Figure 3 is a flowchart of the method for generating an imaging diagnostic report according to the first aspect of embodiment 1 of the present disclosure; Figure 4 is a schematic structural diagram for training a linear transformation matrix according to the first aspect of embodiment 1 of the present disclosure; Figure 5 A schematic diagram of a process for performing feature extraction by using a first feature extraction model combined with a self-attention mechanism, provided in Example 1 of this specification; Figure 6 is a schematic diagram of a path according to the first aspect of embodiment 1 of the present disclosure; Figure 7 is a schematic diagram of the imaging diagnostic report generating device according to embodiment 2 of the present disclosure; and Figure 8 This is a schematic diagram of the imaging diagnostic report generating device according to Example 3 of the present disclosure. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0013] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0014] Example 1 According to this embodiment, a method embodiment of a method for generating an imaging diagnostic report is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0015] The method embodiment provided in this embodiment can be executed in a computer terminal, a server or a similar computing device. Figure 1 The figure shows a hardware structure block diagram of a computing device for implementing a method for generating an image diagnosis report. Figure 1 As shown, a computing device may include one or more processors (the processor may include, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA) or other processing device), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. Those skilled in the art will understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0016] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computing device. As discussed in the embodiments of the present disclosure, the data processing circuitry functions as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0017] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the imaging diagnostic report generation method in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the imaging diagnostic report generation method of the above-mentioned application. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0018] The transmission device is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of the computing device. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0019] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computing device.

[0020] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing device described above.

[0021] Figure 2Schematic diagram of the imaging diagnostic report generation system according to this embodiment. Figure 2 As shown, the system includes a terminal device 100 and a server 200. The terminal device 100 is connected to the server 200 via a network. A physician in the imaging department can enter diagnostic text corresponding to a medical image on the terminal device 100 (for searching for a template for a medical imaging diagnostic report). The terminal device 100 then sends the diagnostic text to the server 200. The server 200 then determines a target template based on the diagnostic text and sends the target template to the terminal device 100. The physician then completes the imaging diagnostic report based on the target template via the terminal device 100.

[0022] It should be noted that the terminal device 100 and the server 200 in the system can both be applicable to the hardware structure described above.

[0023] Under the above operating environment, according to the first aspect of this embodiment, a method for generating an imaging diagnostic report is provided. Figure 2 The server 200 shown in FIG. Figure 4 A schematic diagram showing the process of the method is shown in FIG. Figure 3 As shown, the method includes: S302: receiving a diagnosis text corresponding to a medical image input by a user from a terminal device of the user; S304: Determining the degree of spatial matching between the diagnosis text and the candidate template at the anatomical site level, the degree of semantic matching between the diagnosis text and the candidate template related to pathological information, and the degree of clinical relevance between the diagnosis text and the candidate template; S306: Determining a first matching degree between the candidate template and the diagnosis text based on the spatial matching degree, the semantic matching degree, and the clinical relevance degree; and S308: Determine a target template from the candidate templates based on the first matching degree; and S310: Send the target template to the terminal device, and receive an image diagnosis report corresponding to the target template from the terminal device.

[0024] Specifically, in this embodiment, a user (physician) can enter diagnostic text corresponding to a medical image through terminal device 100. This diagnostic text is used by the user to search for imaging diagnostic report templates. This diagnostic text can be a short text describing the diagnosis of the medical image. Terminal device 100 can send this diagnostic text to server 200. Server 200 receives the diagnostic text corresponding to the medical image input from the user's terminal device 100 (S302). It should be noted that this diagnostic text can be input by voice.

[0025] Next, server 200 needs to retrieve the imaging diagnostic report template required by the user based on the diagnostic text. To do this, server 200 determines the degree of spatial matching between the diagnostic text and the candidate template at the anatomical level, the degree of semantic matching between the diagnostic text and the candidate template related to pathological information, and the degree of clinical relevance between the diagnostic text and the candidate template (S304). Based on the spatial matching, semantic matching, and clinical relevance, server 200 determines a first matching degree between the candidate template and the diagnostic text (S306). The candidate templates mentioned here can be the imaging diagnostic report templates in the template library.

[0026] The spatial match degree focuses on the anatomical level, that is, the correlation between the anatomical parts mentioned in the diagnostic text and the anatomical parts mentioned in the candidate template. The semantic match degree focuses on the semantic correlation related to pathological information between the diagnostic text and the candidate template at the text level. The clinical relevance degree can be determined using a medical knowledge graph (details will be explained later), which represents the clinical relevance between the diagnostic text and the candidate template. The specific methods for determining the spatial match degree, semantic match degree, and clinical relevance degree will be explained in detail later.

[0027] That is to say, by determining the matching degrees at three different levels (spatial matching degree, semantic matching degree and clinical correlation degree), the above-mentioned first matching degree integrates multiple matching degrees between the diagnostic text and the candidate templates. The required imaging diagnostic report template can be selected for the user through the integrated first matching level.

[0028] Therefore, the server 200 can then determine the target template from the candidate templates based on the first matching degree (S308), and send the target template to the terminal device 100. The user edits the target template through the terminal device 100 to complete the complete imaging diagnosis report. The terminal device 100 sends the imaging diagnosis report to the server 200, so that the server 200 receives the imaging diagnosis report corresponding to the target template from the terminal device 100 (S310).

[0029] As described in the background, existing imaging report generation systems typically perform a simple semantic match between keywords and information related to imaging report templates, thereby searching for the corresponding imaging report templates and providing them to doctors. However, existing technologies suffer from the following issues: Traditional keyword matching algorithms (such as TF-IDF) are poorly effective in matching keywords in medical contexts. For example, "miliary nodules" and "micronodule clusters" have the same meaning in medical contexts, but traditional keyword matching algorithms are generally unable to match these two terms together. Consequently, this approach reduces the accuracy of searching for the correct template.

[0030] In view of this, according to the technical solution of this embodiment, the matching degree at three different levels, namely, the spatial matching degree between the diagnostic text and the candidate template at the anatomical site level, the semantic matching degree between the diagnostic text and the candidate template related to pathological information, and the clinical correlation degree between the diagnostic text and the candidate template, can be used to search for a target template that matches the diagnostic text based on the diagnostic text input by the user, thereby improving the accuracy of searching for the required imaging diagnostic template for the user, thereby solving the technical problem in the prior art that the traditional keyword matching algorithm has poor keyword matching effect in medical scenarios, thereby reducing the accuracy of searching for the correct template.

[0031] Optionally, the operation of determining the degree of spatial matching between the diagnostic text and the candidate template at the anatomical part level includes: determining the first anatomical part information in the candidate template, and determining the second anatomical part information in the diagnostic text; determining the part similarity based on the first anatomical part information and the second anatomical part information; determining the spatial position adjacency based on the proximity relationship between the anatomical parts corresponding to the first anatomical part information and the second anatomical part information in the medical image; and determining the degree of spatial matching based on the part similarity and the spatial position adjacency.

[0032] The first anatomical part information represents the anatomical part mentioned in the candidate template, and the second anatomical part information represents the anatomical part mentioned in the diagnostic text. The above-mentioned part similarity can indicate the degree of overlap between the anatomical parts mentioned in the candidate template and the anatomical parts mentioned in the diagnostic text. Specifically, this can be calculated using the BM25 algorithm, which will be explained in detail below. The above-mentioned spatial position proximity is used to add a new dimension based on the part similarity to determine the degree of spatial matching between the diagnostic text and the candidate template.

[0033] Optionally, based on the proximity relationship between the anatomical parts corresponding to the first anatomical part information and the second anatomical part information in the medical image, an operation for determining spatial position proximity includes: determining first coordinate information of the anatomical part corresponding to the first anatomical part information in the medical image, and determining second coordinate information of the anatomical part corresponding to the second anatomical part information in the medical image; determining a first coordinate feature corresponding to the first coordinate information and a second coordinate feature corresponding to the second coordinate information; and determining spatial position proximity based on the first coordinate feature and the second coordinate feature.

[0034] In other words, spatial proximity represents the proximity of the anatomical parts mentioned in the candidate template and the anatomical parts mentioned in the diagnostic text in the medical image. The position of each anatomical part is represented by its three-dimensional coordinates (x, y, z) in the medical image. This method can be used to obtain the first and second coordinate information mentioned above. The first and second coordinate information can be standardized and mapped to a high-dimensional vector space. The expressive power of the position information can be enhanced through linear transformation, thereby obtaining the corresponding first and second coordinate features.

[0035] Specifically, the first coordinate information and the second coordinate information can be standardized by using a Z-score method, so as to obtain a first standardized coordinate corresponding to the first coordinate information and a second standardized coordinate corresponding to the second coordinate information. For example, the first coordinate information is , the second coordinate information is You can use Z-score to normalize each dimension of the x-axis, y-axis, and z-axis. Taking the x-axis as an example, the formula for normalization using Z-score is as follows:

[0036] Thus, we can get the first coordinate information The corresponding first standardized coordinate And the second coordinate information The corresponding second standardized coordinate

[0037] Then through the linear transformation matrix ∈R d×3 , mapping the three-dimensional coordinates to a high-dimensional vector space, the formula is as follows:

[0038]

[0039] in, is the first normalized coordinate, is the second normalized coordinate, and That is the high-dimensional vector after mapping (assuming d=128, then and are all 128-dimensional vectors), is the first coordinate feature, is the second coordinate feature. The linear transformation matrix can be obtained through training.

[0040] Specifically, in order to train the linear transformation matrix, a training sample set consisting of standardized coordinate p'-anatomical part training samples can be constructed. The sample set is specifically as follows: Table 1

[0041] Then, the architecture used to train the linear transformation matrix is ​​as follows Figure 4 As shown in Figure 1, it includes a linear transformation matrix W, a neural network, and a softmax classifier. The linear transformation matrix W is used to map the normalized coordinates into high-dimensional vectors; the neural network is used to extract features from the high-dimensional vectors; and the softmax classifier has a dimension corresponding to the number of anatomical parts. It outputs the probability value corresponding to each anatomical part for the input normalized coordinates.

[0042] Then, using the above sample set Figure 4 The neural network and the transformation matrix W shown are trained to obtain the above-mentioned linear transformation matrix W. Thus, the above-mentioned high-dimensional and .

[0043] Then, it can be determined and The cosine similarity between them, as the spatial proximity, can be expressed as follows:

[0044] in, is the determined spatial position proximity.

[0045] When determining the above-mentioned spatial position proximity, it is possible to determine whether the anatomical parts mentioned in the candidate template and the anatomical parts mentioned in the diagnosis text are anatomically adjacent (or overlap) based on the first coordinate feature and the second coordinate feature. If so, the spatial position proximity can be determined as (where a and b represent the corresponding anatomical parts in the candidate template and the diagnosis text, respectively). If not, the spatial position adjacency can be set to 0.

[0046] If the proximity between the first coordinate feature and the second coordinate feature is greater than a preset reference proximity, it can be determined that the anatomical part mentioned in the candidate template and the anatomical part mentioned in the diagnosis text are anatomically adjacent (or overlap).

[0047] The formula for determining the degree of spatial matching is as follows:

[0048] Among them, A represents the diagnosis text, B represents the candidate template, Indicates the degree of spatial matching, The similarity between the diagnostic text and the candidate template can be determined using the BM25 algorithm. β is the weight corresponding to the spatial proximity and can be manually set in advance, specifically to 0.15. is a proximity function that indicates the spatial proximity between the anatomical parts mentioned in the candidate template and the corresponding anatomical parts mentioned in the diagnosis text. j and b j are the corresponding anatomical parts in the candidate template and diagnosis text respectively.

[0049] The method of determining the part similarity by the BM25 algorithm can be specifically shown in the following formula:

[0050] Where D represents the first anatomical part information in the candidate template, Q represents the second anatomical part information in the diagnosis text, It can represent the i-th anatomical part mentioned in the second anatomical part information. Represents The corresponding inverse document frequency, represents the average number of anatomical parts included in the first anatomical part information and the second anatomical part information, represents the number of anatomical parts included in the first anatomical part information, express The frequency of occurrence in the first anatomical part information, and It can be a preset parameter.

[0051] For example, the diagnostic text reads, "A 2 cm solid nodule with irregular margins, lobulation, and adjacent pleural extension was found in the patient's right upper lobe. Imaging findings strongly suggest malignancy, and a puncture biopsy is recommended." The anatomical site information (second anatomical site information) extracted from this diagnostic text is "right upper lobe." Candidate template A mentions the right upper lobe, candidate template B mentions the right lower lobe, candidate template C mentions both lungs, and candidate template D mentions the right upper lobe and pleura. Table 2 shows examples of the site similarity, spatial proximity, and spatial match between the diagnostic text and each candidate template.

[0052] Table 2

[0053] Candidate template D mentions "right upper lobe" and "pleura," and the location similarity between candidate template D and the diagnosis text is 0.90. Candidate template D mentions two anatomical parts, "right upper lobe" and "pleura," and both need to be considered when calculating spatial proximity. Since both anatomical parts mentioned in candidate template D are adjacent to (or overlap with) the anatomical parts mentioned in the diagnosis text, the determined spatial proximity is 0.30 (0.15 × 2).

[0054] Optionally, the operation of determining the degree of semantic matching between the candidate template and the diagnostic text related to pathological information includes: based on a first feature extraction model, performing feature extraction on the descriptive information of the candidate template, and performing feature extraction on the diagnostic text, to obtain a first semantic feature corresponding to the descriptive information, and a second semantic feature corresponding to the diagnostic text, wherein the first feature extraction model is obtained by fine-tuning a preset semantic feature extraction model based on training samples related to medical scenarios; based on the first semantic feature and the second semantic feature, determining the degree of semantic matching between the candidate template and the diagnostic text related to pathological information.

[0055] In other words, the semantic feature extraction model mentioned above is a semantic model used in common scenarios. For example, this semantic feature extraction model can be a BERT model trained with a large number of text samples. The first feature extraction model mentioned above is obtained by fine-tuning the semantic feature extraction model based on training samples relevant to medical scenarios. Specifically, fine-tuning can be performed using the MIMIC-III dataset. The semantic feature extraction model can be fine-tuned by constructing training samples using medical records, diagnostic reports, and related radiological descriptions. This makes the fine-tuned semantic feature extraction model (i.e., the first feature extraction model) more suitable for specific medical scenarios.

[0056] In addition, preferably, the semantic feature extraction model proposed in this application can further adopt a weighted attention mechanism that is different from the self-attention mechanism and is based on the entity relationship between words for feature extraction based on the BERT model based on the self-attention mechanism. That is, the first feature extraction model can include a fine-tuned semantic feature extraction model, a named entity recognition module, a relationship extraction module, and a weighted attention model. The following is an example of diagnostic text. Those skilled in the art should know that the same method can also be used to extract features from the description information of the candidate template. Figure 5 The process of feature extraction by combining the first feature extraction model with the self-attention mechanism is shown.

[0057] Figure 5 A schematic diagram of a process for performing feature extraction by combining a first feature extraction model with a self-attention mechanism is provided in Example 1 of this specification.

[0058] According to the technical solution of this application, entity types corresponding to medical terms and diagnosis conclusions are predefined. Table 3 below shows some examples of entity types: Table 3

[0059] Therefore, after the server 200 performs pre-processing such as text cleaning, word segmentation, and word embedding on the diagnosis text, it determines the word vector corresponding to each word through the BERT model, and determines the entity type corresponding to each word in the diagnosis text through the named entity recognition module (specifically, the named entity recognition model). For example: The entity type of "right lung" is "anatomical location entity"; the entity type of "finding" is "qualitative description entity"; "2" is "numerical entity"; "cm" is "unit entity"; "solid nodule" is "sign description entity"; and so on.

[0060] In addition, according to the technical solution of this application, relationship types for describing relationships between entities are also predefined. Table 4 below shows some examples of relationship types: Table 4

[0061] Server 200 can then further determine the relationships between entities using a relationship extraction module (specifically, a relationship extraction model for extracting relationships between entities). For example, the relationship between "right lung" and "upper lobe" is a positional relationship; the relationship between "right lung" and "solid nodule" is a positive sign relationship; the relationship between "edge" and "irregular" is a positive sign relationship; and so on.

[0062] In addition, further referring to Table 3, in the technical solution of the present application, weight coefficients corresponding to different relationship types are defined according to different relationship types, that is, for two words, the entities corresponding to the two words can be determined, and then the relationship between the corresponding entities can be determined, and the weight coefficient corresponding to the relationship type of the relationship can be used as the weight coefficient between the two words. The weight coefficient can be determined based on experience and manually adjusted during the actual training process. With reference to Table 3, it can be seen that the weight coefficient can reflect the relationship between words related to medical diagnosis. For example, when the relationship between words is a positive sign relationship or a negative sign relationship, the weight coefficient between the words is relatively large, which is 2.0. For another example, when the relationship between words is a state expression relationship, the weight coefficient between the words is relatively small, which is 1.3. For another example, when there is no entity relationship between words, the weight coefficient is 1.0.

[0063] The server 200 can then use the weighted attention model to perform feature extraction based on the weight coefficients on the word vectors output by the BERT model, thereby determining the semantic features corresponding to each word (i.e., the second semantic features, which are the first semantic features when the first feature extraction model extracts features from the description information of the candidate template). In other words, the weight coefficients corresponding to different relationship types are integrated into the self-attention mechanism to achieve weighted attention that is different from the ordinary self-attention mechanism. The internal calculation process of the weighted attention model can be as follows: Determine the vectors x1~x n The corresponding key vector k1~k n , query vector q1~q n , and the value vector v1~v n And the semantic features f1~f corresponding to each word are determined according to the following formula n (1) (2) (3) Among them, f i is the semantic feature corresponding to each word in the diagnostic text (i.e., the second semantic feature, which is the first semantic feature when semantic feature extraction is performed on the description information of the candidate template). The weight coefficients between the various words are determined based on Table 3, that is, the weight coefficients between the various words are determined by the weight coefficients corresponding to different relationship types.

[0064] The above description is based on the example of the diagnosis text. The semantic features (ie, the first semantic features) can also be extracted for the candidate templates based on the above method.

[0065] In this way, based on the weighted attention mechanism, attention can be weighted based on the entity relationships between words related to medical diagnosis. The semantic features determined in this way can strengthen the information that is more important for matching the diagnostic text with the candidate template, thereby improving the matching accuracy.

[0066] Among them, when determining the degree of semantic matching related to pathological information between the candidate template and the diagnostic text based on the first semantic feature and the second semantic feature, the degree of semantic matching can be determined specifically by determining the similarity between the first semantic feature and the second semantic feature, such as Euclidean distance or cosine similarity.

[0067] Furthermore, based on the first semantic feature and the second semantic feature, the degree of semantic matching between the candidate template and the diagnostic text related to the pathological information is determined, including: determining a first maximum pooling result and a first average pooling result corresponding to the first semantic feature, and weightedly fusing the first maximum pooling result and the first average pooling result to obtain a first fusion feature corresponding to the description information; determining a second maximum pooling result and a second average pooling result corresponding to the second semantic feature, and weightedly fusing the second maximum pooling result and the second average pooling result to obtain a second fusion feature corresponding to the diagnostic text; and based on the first fusion feature and the second fusion feature, determining the degree of semantic matching between the candidate template and the diagnostic text related to the pathological information.

[0068] That is to say, the above process is to better extract the global information and key information of the description information of the candidate template and the diagnostic text after performing semantic feature extraction on the description information and diagnostic text of the candidate template. Specifically, after the server 200 performs feature extraction through the first feature extraction model, the first semantic features corresponding to the description information of the candidate template and the second semantic features corresponding to the diagnostic text are respectively subjected to maximum pooling and average pooling to obtain the first maximum pooling result and the first average pooling result corresponding to the description information of the candidate template, and the second maximum pooling result and the second average pooling result corresponding to the diagnostic text.

[0069] After determining the corresponding maximum pooling results and average pooling results for the descriptive information and the diagnostic text respectively, weighted fusion can be performed on the corresponding maximum pooling results and average pooling results. The specific formula is shown below.

[0070]

[0071] Where E is the word embedding matrix (i.e., the first semantic feature or the second semantic feature), and α can be a dynamic weight determined by the attention mechanism or hyperparameter adjustment. When E is the first semantic feature, HybridVec is the first fused feature; when E is the second semantic feature, HybridVec is the second fused feature.

[0072] Then, the server 200 can directly determine the degree of semantic matching between the candidate template and the diagnostic text based on the first fusion feature and the second fusion feature, thereby more accurately determining the degree of semantic matching between the candidate template and the diagnostic text based on the first semantic feature and the second semantic feature.

[0073] Continuing with the above example, the diagnosis text and each candidate template remain unchanged, and examples of the semantic relevance between the description information of each candidate template and the diagnosis text are shown in Table 5 below.

[0074] Table 5

[0075] Optionally, the operation of determining the degree of clinical correlation between the candidate template and the diagnostic text includes: determining a first keyword in the candidate template and a second keyword in the diagnostic text; based on a preset medical knowledge graph, querying a node that matches the first keyword as a first starting node, and querying a node that matches the second keyword as a second starting node, the medical knowledge graph being used to represent the association between radiological features, pathological types, and anatomical parts; according to the medical knowledge graph, querying a first path with the first starting node as the starting point, and querying a second path with the second starting node as the starting point; and determining the degree of clinical correlation between the candidate template and the diagnostic text based on the first path and the second path.

[0076] The aforementioned medical knowledge graph may include radiological feature nodes (representing radiological features such as nodules and masses), pathology type nodes (representing pathology types such as cancer and inflammation), and anatomical site nodes (representing anatomical sites such as lungs and liver). This medical knowledge graph can be pre-built with a large number of nodes. The edges between nodes in this medical knowledge graph represent the associations between them, and edge weights represent the degree of association between the nodes connected by the corresponding edge, such as the association between "pulmonary nodules" and "adenocarcinoma." Edge weights can be determined through statistical analysis and expert annotation.

[0077] The degree of clinical relevance needs to be determined based on the aforementioned medical knowledge graph. First, server 200 can determine the first keyword in the candidate template and the second keyword in the diagnostic text. Keywords representing radiological features, pathological types, and anatomical sites can be extracted from the candidate template and the diagnostic text, thereby obtaining the first keyword corresponding to the candidate template and the second keyword corresponding to the diagnostic text. Both the first keyword and the second keyword can contain more than one word.

[0078] Server 200 can then search the medical knowledge graph for the node corresponding to the first keyword (referred to as the first starting node above) and the node corresponding to the second keyword (referred to as the second starting node above). Either the first starting node or the second starting node can be used as a starting point. By initiating a search in the medical knowledge graph from the first starting node (or the second starting node) as a starting point, a path starting from the first starting node (or the second starting node) is obtained, thereby obtaining the first path or the second path. The length of the path can be a fixed, preset length, which can be manually set in advance.

[0079] The corresponding path can be determined through multi-hop reasoning. For example, the first starting node (or second starting node) can be used as the starting point, and a target neighbor node (hereinafter referred to as the first neighbor node) can be selected from the neighbor nodes of the starting point as the second node in the path. Then, using the first neighbor node as a reference, a target neighbor node (hereinafter referred to as the second neighbor node) can be selected from the neighbor nodes of the first neighbor node as the third node in the path. This process continues in this manner until the path reaches a predetermined length.

[0080] The method for selecting the target neighbor node may be as follows: for a node, when selecting the target neighbor node of the node, the node type to which the node belongs is determined (for example, a radiological feature node, a pathological type node, or an anatomical part node), and based on the node type, the target node type associated with it is determined (for example, the target node type associated with the radiological feature node type may be a pathological type node type, and the target node type associated with the pathological type node may be an anatomical part node type, and at least one target node type associated with a certain node type may be pre-set). Then, from the neighbor nodes of the target node type, neighbor nodes whose edge weights exceed a preset threshold are selected, and nodes whose correlations with the nodes corresponding to the first keyword (or the second keyword) exceed a preset correlation are selected as target neighbor nodes.

[0081] The degree of association between nodes can be calculated by first determining the node features of each node through a graph neural network (such as Graphsage). The degree of association between the node features is then used as the degree of association between the nodes. Furthermore, the preset threshold can be different for different types of edge weights. For example, the edge weight between a radiological feature node and a pathology type node, and the edge weight between a pathology type node and an anatomical site node are different types of edge weights, and different preset thresholds can be set accordingly.

[0082] Continuing with the previous example, we can use the diagnosis text in the above example as an example to query the nodes related to the keywords "right upper lobe of the lung", "solid nodule" and "malignant" in the medical knowledge graph based on the diagnosis text. For example, starting from the "nodule" node corresponding to "solid nodule", we can query the path starting from the "nodule" node as "nodule → 1 hop: adenocarcinoma (weight 0.9) → 2 hops: pleural invasion (weight 0.85)". Figure 6 shown.

[0083] Figure 6 This is a path diagram according to the first aspect of Example 1 of the present disclosure.

[0084] Figure 6The diagram shows a path example described above. Starting from a nodule node, the pathology type node "adenocarcinoma" is selected from its neighboring nodes. After reaching the "adenocarcinoma" node, the pathology type node "pleural invasion" is selected from its neighboring nodes, thus forming a path. Table 6 shows examples of the first paths corresponding to each candidate template and the clinical correlation levels determined based on the first paths corresponding to the candidate templates and the second paths corresponding to the diagnostic text.

[0085] Table 6

[0086] Specifically, the similarity between the first path corresponding to the candidate template and the second path corresponding to the diagnosis text may be used as the degree of clinical association between the candidate template and the diagnosis text.

[0087] Then, the server 200 can determine the first degree of match between the candidate template and the diagnosis text based on the spatial matching degree, the semantic matching degree and the clinical correlation degree. Specifically, corresponding weights can be preset for the spatial matching degree, the semantic matching degree and the clinical correlation degree. For example, the weight of the spatial matching degree can be set to 40%, the weight of the semantic matching degree can be set to 35% and the weight of the clinical correlation degree can be set to 25%. Then, according to the weights corresponding to the spatial matching degree, the semantic matching degree and the clinical correlation degree respectively, the spatial matching degree, the semantic matching degree and the clinical correlation degree are weighted and summed to obtain the first degree of match.

[0088] Continuing with the above example, Table 7 shows an example of the first matching degree of each candidate template. It should be noted that the values ​​shown in the tables in this specification are for illustrating the process of the method and are only examples, and do not represent actual calculated values.

[0089] Table 7

[0090] Optionally, the operation of determining the target template from the candidate templates based on the first matching degree includes: correcting the first matching degree based on the creation time of the candidate template, the lesion type corresponding to the candidate template, and the degree of conflict between the candidate template and the diagnostic text in the region of interest, and determining a corrected second matching degree; and determining the target template from the candidate template based on the second matching degree.

[0091] After determining the first degree of match, templates meeting preset conditions can be selected from the candidate templates based on the first degree of match as candidate templates. For example, templates with a first degree of match not less than a first preset threshold can be selected as candidate templates. Alternatively, the top k templates (k can be a preset value) can be selected by sorting the templates from highest to lowest according to the first degree of match. The first degree of match of the candidate template can then be modified as described above to obtain a second degree of match. This second degree of match is then used to select the target template that will ultimately be recommended to the user.

[0092] How to determine the first matching degree by creation time, lesion type, and conflict degree will be described in detail below.

[0093] Optionally, the operation of correcting the first matching degree based on the creation time of the candidate template, the lesion type corresponding to the candidate template, and the degree of conflict between the candidate template and the diagnostic text in the region of interest includes: determining the time attenuation factor corresponding to the candidate template based on the creation time; determining the clinical priority corresponding to the candidate template based on the lesion type; and correcting the first matching degree based on the time attenuation factor, clinical priority and degree of conflict to obtain a second matching degree.

[0094] Specifically, the time decay factor can be determined by the time interval between the creation time and the current time. The larger the time interval, the smaller the time decay factor (the lower the priority of the template). Specifically, the time decay factor can be set to: , for example, γ can be set to 0.9, It is the time interval (in months) between the creation time and the current time. You can also set corresponding clinical priorities for different types of lesions. For example, the clinical priority corresponding to malignant lesions can be set to 1.2, the clinical priority corresponding to acute lesions can be set to 1.1, and the clinical priority corresponding to chronic lesions can be set to 0.9. The degree of conflict between the candidate template and the diagnostic text in the region of interest mentioned above represents the degree of difference between the part of interest (anatomical part) mentioned in the candidate template and the part of interest (anatomical part) mentioned in the diagnostic text. Specifically, the intersection-union ratio between the part of interest (anatomical part) mentioned in the candidate template and the part of interest (anatomical part) mentioned in the diagnostic text can be determined, and then the degree of conflict can be obtained through 1-intersection-union ratio.

[0095] The following is an example of correcting the first matching degree to obtain the second matching degree, as shown in Tables 8 and 9. In Tables 8 and 9, it is assumed that candidate template A and candidate template D are selected as pending templates based on the first matching degree. These two templates need to be determined according to the clinical priority, time attenuation factor and conflict degree to determine the second matching degree corresponding to the two templates.

[0096] Table 8

[0097] Table 9

[0098] As can be seen from Table 6, the second matching degree can be determined by the following formula.

[0099]

[0100] Server 200 can then select a target template based on the second degree of match. For example, it can select a template whose second degree of match is no less than a first preset threshold as the target template. Alternatively, it can sort the templates from highest to lowest according to the second degree of match and select the top-a template (a can be a preset value) as the target template. By sending this target template to terminal 100, the user can complete the corresponding imaging diagnostic report based on the target template.

[0101] In addition, reference Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0102] Therefore, according to the technical solution of this embodiment, the spatial matching degree between the candidate template and the diagnostic text, the semantic matching degree between the candidate template and the diagnostic text related to pathological information, and the clinical correlation degree between the candidate template and the diagnostic text determined in combination with the medical knowledge graph can be combined to determine the first matching degree that integrates multiple information. Moreover, not only is the target template determined based on the first matching degree, but the first matching degree is further corrected by combining the time attenuation factor, clinical priority and the degree of conflict between the regions of interest. The target template is determined by the second matching degree obtained by correcting the first matching degree, thereby more accurately searching for the template required by the user for completing the imaging diagnostic report, thereby improving the efficiency of the user in completing the imaging diagnostic report.

[0103] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0104] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0105] Example 2 Figure 7 FIG2 shows an imaging diagnostic report generating apparatus 700 according to this embodiment, which corresponds to the method according to the first aspect of embodiment 1. Figure 7 As shown, the device 700 includes: a receiving module 710, which is used to receive a diagnostic text corresponding to a medical image input by a user from a terminal device of the user; a three-level matching module 720, which is used to determine the spatial matching degree between the diagnostic text and the candidate template at the anatomical site level, the semantic matching degree between the diagnostic text and the candidate template related to pathological information, and the clinical correlation degree between the diagnostic text and the candidate template; a matching degree determination module 730, which is used to determine a first matching degree between the candidate template and the diagnostic text based on the spatial matching degree, the semantic matching degree and the clinical correlation degree; a selection module 740, which is used to determine a target template from the candidate templates based on the first matching degree; and a sending module 750, which is used to send the target template to the terminal device and receive an image diagnosis report corresponding to the target template from the terminal device.

[0106] Optionally, the three-level matching module 720 is specifically used to determine the first anatomical part information in the candidate template and the second anatomical part information in the diagnostic text; determine the part similarity based on the first anatomical part information and the second anatomical part information; determine the spatial position adjacency based on the proximity relationship between the anatomical parts corresponding to the first anatomical part information and the second anatomical part information in the medical image; and determine the degree of spatial matching based on the part similarity and the spatial position adjacency.

[0107] Optionally, the three-level matching module 720 is specifically used to determine the first coordinate information of the anatomical part corresponding to the first anatomical part information in the medical image, and to determine the second coordinate information of the anatomical part corresponding to the second anatomical part information in the medical image; perform feature embedding on the first coordinate information and the second coordinate information to obtain a first coordinate feature corresponding to the first coordinate information and a second coordinate feature corresponding to the second coordinate information; and determine the spatial position proximity based on the first coordinate feature and the second coordinate feature.

[0108] Optionally, the three-level matching module 720 is specifically used to perform feature extraction on the descriptive information of the candidate template and the diagnostic text based on the first feature extraction model to obtain a first semantic feature corresponding to the descriptive information and a second semantic feature corresponding to the diagnostic text, wherein the first feature extraction model is obtained by fine-tuning a preset semantic feature extraction model based on training samples related to the medical scenario; and based on the first semantic feature and the second semantic feature, determine the degree of semantic matching between the candidate template and the diagnostic text related to the pathological information.

[0109] Optionally, the three-level matching module 720 is specifically used to determine the first maximum pooling result and the first average pooling result corresponding to the first semantic feature, and perform weighted fusion on the first maximum pooling result and the first average pooling result to obtain a first fusion feature corresponding to the description information; determine the second maximum pooling result and the second average pooling result corresponding to the second semantic feature, and perform weighted fusion on the second maximum pooling result and the second average pooling result to obtain a second fusion feature corresponding to the diagnostic text; and determine the degree of semantic matching related to pathological information between the candidate template and the diagnostic text based on the first fusion feature and the second fusion feature.

[0110] Optionally, the three-level matching module 720 is specifically used to determine the first keyword in the candidate template and the second keyword in the diagnostic text; based on a preset medical knowledge graph, query the node that matches the first keyword as the first starting node, and query the node that matches the second keyword as the second starting node, the medical knowledge graph is used to represent the association between radiological features, pathological types and anatomical parts; according to the medical knowledge graph, query the first path with the first starting node as the starting point, and query the second path with the second starting node as the starting point; and determine the degree of clinical correlation between the candidate template and the diagnostic text based on the first path and the second path.

[0111] Optionally, the selection module 740 is specifically used to correct the first matching degree based on the creation time of the candidate template, the lesion type corresponding to the candidate template, and the degree of conflict between the candidate template and the diagnostic text in the area of ​​interest, and determine a corrected second matching degree; and determine the target template from the candidate template based on the second matching degree.

[0112] Optionally, the selection module 740 is specifically used to determine the time attenuation factor corresponding to the candidate template based on the creation time; determine the clinical priority corresponding to the candidate template based on the lesion type; and correct the first matching degree based on the time attenuation factor, clinical priority and conflict degree to obtain a second matching degree.

[0113] Therefore, according to the technical solution of this embodiment, the spatial matching degree between the candidate template and the diagnostic text, the semantic matching degree between the candidate template and the diagnostic text related to pathological information, and the clinical correlation degree between the candidate template and the diagnostic text determined in combination with the medical knowledge graph can be combined to determine the first matching degree that integrates multiple information. Moreover, not only is the target template determined based on the first matching degree, but the first matching degree is further corrected by combining the time attenuation factor, clinical priority and the degree of conflict between the regions of interest. The target template is determined by the second matching degree obtained by correcting the first matching degree, thereby more accurately searching for the template required by the user for completing the imaging diagnostic report, thereby also improving the efficiency of the user in completing the imaging diagnostic report. Example 3 Figure 8 FIG2 shows an imaging diagnostic report generating apparatus 800 according to this embodiment, which corresponds to the method according to the first aspect of embodiment 1. Figure 8 As shown, the device 800 includes: a processor 810; and a memory 820, which is connected to the processor 810 and is used to provide the processor 810 with instructions for processing the following processing steps: receiving a diagnostic text corresponding to a medical image input by a user from a user's terminal device; determining the degree of spatial matching between the diagnostic text and the candidate template at the anatomical site level, the degree of semantic matching between the diagnostic text and the candidate template related to pathological information, and the degree of clinical association between the diagnostic text and the candidate template; determining a first degree of matching between the candidate template and the diagnostic text based on the spatial matching degree, the semantic matching degree and the clinical association degree; determining a target template from the candidate template based on the first matching degree; and sending the target template to the terminal device, and receiving an image diagnostic report corresponding to the target template from the terminal device.

[0114] Optionally, the operation of determining the degree of spatial matching between the diagnostic text and the candidate template at the anatomical part level includes: determining the first anatomical part information in the candidate template, and determining the second anatomical part information in the diagnostic text; determining the part similarity based on the first anatomical part information and the second anatomical part information; determining the spatial position adjacency based on the proximity relationship between the anatomical parts corresponding to the first anatomical part information and the second anatomical part information in the medical image; and determining the degree of spatial matching based on the part similarity and the spatial position adjacency.

[0115] Optionally, based on the proximity relationship between the anatomical parts corresponding to the first anatomical part information and the second anatomical part information in the medical image, an operation for determining spatial position proximity includes: determining first coordinate information of the anatomical part corresponding to the first anatomical part information in the medical image, and determining second coordinate information of the anatomical part corresponding to the second anatomical part information in the medical image; performing feature embedding on the first coordinate information and the second coordinate information to obtain a first coordinate feature corresponding to the first coordinate information and a second coordinate feature corresponding to the second coordinate information; and determining spatial position proximity based on the first coordinate feature and the second coordinate feature.

[0116] Optionally, the operation of determining the degree of semantic matching between the candidate template and the diagnostic text related to pathological information includes: based on a first feature extraction model, performing feature extraction on the descriptive information of the candidate template, and performing feature extraction on the diagnostic text, to obtain a first semantic feature corresponding to the descriptive information, and a second semantic feature corresponding to the diagnostic text, wherein the first feature extraction model is obtained by fine-tuning a preset semantic feature extraction model based on training samples related to medical scenarios; based on the first semantic feature and the second semantic feature, determining the degree of semantic matching between the candidate template and the diagnostic text related to pathological information.

[0117] Optionally, based on the first semantic feature and the second semantic feature, the degree of semantic matching between the candidate template and the diagnostic text related to the pathological information is determined, including: determining the first maximum pooling result and the first average pooling result corresponding to the first semantic feature, and weightedly fusing the first maximum pooling result and the first average pooling result to obtain a first fusion feature corresponding to the description information; determining the second maximum pooling result and the second average pooling result corresponding to the second semantic feature, and weightedly fusing the second maximum pooling result and the second average pooling result to obtain a second fusion feature corresponding to the diagnostic text; and based on the first fusion feature and the second fusion feature, determining the degree of semantic matching between the candidate template and the diagnostic text related to the pathological information.

[0118] Optionally, the operation of determining the degree of clinical correlation between the candidate template and the diagnostic text includes: determining a first keyword in the candidate template and a second keyword in the diagnostic text; based on a preset medical knowledge graph, querying a node that matches the first keyword as a first starting node, and querying a node that matches the second keyword as a second starting node, the medical knowledge graph being used to represent the association between radiological features, pathological types, and anatomical parts; according to the medical knowledge graph, querying a first path with the first starting node as the starting point, and querying a second path with the second starting node as the starting point; and determining the degree of clinical correlation between the candidate template and the diagnostic text based on the first path and the second path.

[0119] Optionally, the operation of determining the target template from the candidate templates based on the first matching degree includes: correcting the first matching degree based on the creation time of the candidate template, the lesion type corresponding to the candidate template, and the degree of conflict between the candidate template and the diagnostic text in the region of interest, and determining a corrected second matching degree; and determining the target template from the candidate template based on the second matching degree.

[0120] Optionally, the operation of correcting the first matching degree based on the creation time of the candidate template, the lesion type corresponding to the candidate template, and the degree of conflict between the candidate template and the diagnostic text in the region of interest includes: determining the time attenuation factor corresponding to the candidate template based on the creation time; determining the clinical priority corresponding to the candidate template based on the lesion type; and correcting the first matching degree based on the time attenuation factor, clinical priority and degree of conflict to obtain a second matching degree.

[0121] Therefore, according to the technical solution of this embodiment, the spatial matching degree between the candidate template and the diagnostic text, the semantic matching degree between the candidate template and the diagnostic text related to pathological information, and the clinical correlation degree between the candidate template and the diagnostic text determined in combination with the medical knowledge graph can be combined to determine the first matching degree that integrates multiple information. Moreover, not only is the target template determined based on the first matching degree, but the first matching degree is further corrected by combining the time attenuation factor, clinical priority and the degree of conflict between the regions of interest. The target template is determined by the second matching degree obtained by correcting the first matching degree, thereby more accurately searching for the template required by the user for completing the imaging diagnostic report, thereby improving the efficiency of the user in completing the imaging diagnostic report.

[0122] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0123] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0125] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0126] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0127] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0128] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for generating an imaging diagnostic report, characterized in that: include: receiving, from a terminal device of the user, a diagnosis text corresponding to the medical image input by the user; Determining a degree of spatial matching between the diagnostic text and the candidate template at the anatomical site level, a degree of semantic matching between the diagnostic text and the candidate template with respect to pathological information, and a degree of clinical relevance between the diagnostic text and the candidate template; Determining a first matching degree between the candidate template and the diagnosis text according to the spatial matching degree, the semantic matching degree, and the clinical correlation degree; determining a target template from the candidate templates according to the first matching degree; as well as The target template is sent to the terminal device, and an image diagnosis report corresponding to the target template is received from the terminal device.

2. The method according to claim 1, characterized in that The operation of determining the degree of spatial matching between the diagnosis text and the candidate template at the anatomical site level includes: Determining first anatomical part information in the candidate template and determining second anatomical part information in the diagnosis text; determining part similarity based on the first anatomical part information and the second anatomical part information; determining a spatial position proximity based on a proximity relationship between the anatomical parts corresponding to the first anatomical part information and the second anatomical part information in the medical image; The spatial matching degree is determined according to the part similarity and the spatial position proximity.

3. The method according to claim 2, characterized in that The operation of determining the spatial proximity of the anatomical parts corresponding to the first anatomical part information and the second anatomical part information in the medical image, includes: Determining first coordinate information of the anatomical part corresponding to the first anatomical part information in the medical image, and determining second coordinate information of the anatomical part corresponding to the second anatomical part information in the medical image; Performing feature embedding on the first coordinate information and the second coordinate information to obtain a first coordinate feature corresponding to the first coordinate information and a second coordinate feature corresponding to the second coordinate information; The spatial position proximity is determined according to the first coordinate feature and the second coordinate feature.

4. The method according to claim 1, wherein The operation of determining the degree of semantic matching between the candidate template and the diagnosis text related to pathological information includes: Based on a first feature extraction model, feature extraction is performed on the description information of the candidate template, and feature extraction is performed on the diagnosis text to obtain a first semantic feature corresponding to the description information, and a second semantic feature corresponding to the diagnosis text, wherein the first feature extraction model is obtained by fine-tuning a preset semantic feature extraction model based on training samples related to medical scenarios; The degree of semantic matching between the candidate template and the diagnosis text related to pathological information is determined based on the first semantic feature and the second semantic feature.

5. The method according to claim 4, characterized in that The operation of determining the degree of semantic matching related to pathological information between the candidate template and the diagnosis text according to the first semantic feature and the second semantic feature includes: Determining a first maximum pooling result and a first average pooling result corresponding to the first semantic feature, and performing weighted fusion on the first maximum pooling result and the first average pooling result to obtain a first fused feature corresponding to the description information; Determining a second maximum pooling result and a second average pooling result corresponding to the second semantic feature, and performing weighted fusion on the second maximum pooling result and the second average pooling result to obtain a second fused feature corresponding to the diagnostic text; and The degree of semantic matching related to pathological information between the candidate template and the diagnosis text is determined based on the first fusion feature and the second fusion feature.

6. The method according to claim 1, characterized in that The operation of determining the degree of clinical relevance between the candidate template and the diagnosis text comprises: determining a first keyword in the candidate template and a second keyword in the diagnosis text; Based on a preset medical knowledge graph, searching for a node matching the first keyword as a first starting node, and searching for a node matching the second keyword as a second starting node, the medical knowledge graph being used to represent associations between radiological features, pathological types, and anatomical sites; According to the medical knowledge graph, querying a first path with the first starting node as a starting point, and querying a second path with the second starting node as a starting point; and The degree of clinical association between the candidate template and the diagnosis text is determined based on the first path and the second path.

7. The method according to claim 1, characterized in that The operation of determining a target template from the candidate templates according to the first matching degree includes: Correcting the first matching degree based on the creation time of the candidate template, the lesion type corresponding to the candidate template, and the degree of conflict between the candidate template and the diagnosis text in the region of interest, and determining a corrected second matching degree; and According to the second matching degree, the target template is determined from the candidate templates, and wherein The operation of correcting the first matching degree based on the creation time of the candidate template, the lesion type corresponding to the candidate template, and the conflict degree between the candidate template and the diagnosis text in the region of interest, and determining a corrected second matching degree includes: Determining a time attenuation factor corresponding to the candidate template according to the creation time; Determining a clinical priority corresponding to the candidate template according to the lesion type; and The first matching degree is modified according to the time decay factor, the clinical priority, and the conflict degree to obtain the second matching degree.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the processor executes the method according to any one of claims 1 to 7.

9. An imaging diagnostic report generating device, characterized in that: include: A receiving module, configured to receive a diagnosis text corresponding to a medical image input by a user from a terminal device of the user; a tertiary matching module for determining the degree of spatial matching between the diagnostic text and the candidate template at the anatomical site level, the degree of semantic matching between the diagnostic text and the candidate template with respect to pathological information, and the degree of clinical association between the diagnostic text and the candidate template; a matching degree determination module, configured to determine a first matching degree between the candidate template and the diagnosis text based on the spatial matching degree, the semantic matching degree, and the clinical correlation degree; a selection module, configured to determine a target template from the candidate templates based on the first matching degree; as well as The sending module is used to send the target template to the terminal device and receive an imaging diagnosis report corresponding to the target template from the terminal device.

10. An imaging diagnostic report generating device, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: receiving, from a terminal device of the user, a diagnosis text corresponding to the medical image input by the user; Determining a degree of spatial matching between the diagnostic text and the candidate template at the anatomical site level, a degree of semantic matching between the diagnostic text and the candidate template with respect to pathological information, and a degree of clinical relevance between the diagnostic text and the candidate template; Determining a first matching degree between the candidate template and the diagnosis text according to the spatial matching degree, the semantic matching degree, and the clinical correlation degree; determining a target template from the candidate templates according to the first matching degree; as well as The target template is sent to the terminal device, and an image diagnosis report corresponding to the target template is received from the terminal device.

Images