A method and system for automatically generating inspection reports

Abstract

The present application discloses a method and system for automatically generating inspection reports, which constructs a radiological prior knowledge base, obtains the radiological inspection information of the target object, and uses a data information processing and encoding method based on deep learning to convert the radiological inspection information of the target object into a text description, and then performs sentence processing and semantic encoding. Then, the embedded features of the radiological prior knowledge document fragment are matched for each radiological inspection information sentence granularity description feature from the radiological prior knowledge base, and the embedded features and the corresponding sentence granularity description features are fused, so as to intelligently generate an inspection report based on the semantically enhanced expression of the embedded features of the fused prior enhanced radiological inspection information sentence granularity description. In this way, it is possible to capture the specific inspection semantic features in each sentence more finely based on the constructed radiological prior knowledge base and through a deep learning model and an intelligent matching mechanism, so as to automatically generate a high-quality and professional inspection report.

Description

Technical Field

[0001] The present application relates to the field of intelligent generation, and more specifically, to a method and system for automatically generating inspection reports. Background Art

[0002] Radiology plays a pivotal role in medical diagnosis and is an indispensable examination method in clinical practice and medical research. The traditional process of writing radiology examination reports consumes a lot of time, especially for less experienced radiologists, the timeliness, professionalism and accuracy of report writing are more difficult to guarantee. In view of this, with the continuous development of large language model technology, the demand for automatic generation of radiology reports is becoming increasingly urgent. This technology aims to reduce the workload of radiologists while improving the speed and accuracy of writing radiology reports.

[0003] While large language models have developed rapidly in general fields, scholars have gradually applied them to proprietary fields. For example, patent CN117037994A uses large language model technology to guide students to update diagnostic reports based on standard diagnostic information. This application provides assistance for students to write diagnostic reports but does not fully automate the generation of reports. In addition, the application does not embed additional prior knowledge to enhance the professionalism of medical terminology. Patent CN117828050A provides a Chinese medicine question-and-answer method based on long document retrieval and enhanced generation. Its document segmentation is only based on the document length threshold and the overlap between the previous and next documents, but does not involve the sorting or integration of summary information of each document block.

[0004] Therefore, an optimized inspection report generation solution is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method and system for automatically generating inspection reports, which constructs a radiological prior knowledge base and obtains the radiological inspection information of the target object, and uses a data information processing and encoding method based on deep learning to convert the radiological inspection information of the target object into a text description and then performs sentence processing and semantic encoding. Then, the embedded features of the radiological prior knowledge document fragments are matched for each radiological inspection information sentence granularity description feature from the radiological prior knowledge base, and the embedded features and the corresponding sentence granularity description features are fused, so as to intelligently generate an inspection report based on the semantically enhanced expression of the embedded features of the fused priori enhanced radiological inspection information sentence granularity description. In this way, it is possible to capture the specific inspection semantic features in each sentence more finely based on the constructed radiological prior knowledge base and through a deep learning model and an intelligent matching mechanism, thereby automatically generating a high-quality and professional inspection report.

[0006] According to one aspect of the present application, a method for automatically generating an inspection report is provided, which includes: constructing a radiological prior knowledge base, the radiological prior knowledge base including embedded coding vectors of multiple radiological prior knowledge document fragments; obtaining radiological inspection information of a target object, and performing text conversion on the radiological inspection information of the target object to obtain a text description of the radiological inspection information of the target object; performing sentence processing and semantic coding on the text description of the radiological inspection information of the target object to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors; embedding the radiological inspection information sentence granularity description embedded coding vectors from the radiological prior knowledge base for each radiological inspection information sentence granularity description in the sequence of radiological inspection information sentence granularity description embedded coding vectors; The method comprises the following steps: matching the embedded coding vectors of at least one radiological prior knowledge document fragment respectively; semantically fusing each radiological inspection information sentence granularity description embedded coding vector in the sequence of radiological inspection information sentence granularity description embedded coding vectors with the embedded coding vector of at least one radiological prior knowledge document fragment to obtain a sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors; performing sequence-dependent conditional feature semantic enhancement on the sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors; and obtaining an inspection report based on the sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors.

[0007] According to another aspect of the present application, a system for automatically generating an inspection report is provided, comprising:

[0008] A radiological prior knowledge base construction module is used to construct a radiological prior knowledge base, wherein the radiological prior knowledge base includes embedded coding vectors of multiple radiological prior knowledge document fragments; a radiological inspection information acquisition text conversion module is used to obtain the radiological inspection information of the target object, and perform text conversion on the radiological inspection information of the target object to obtain a text description of the radiological inspection information of the target object; a semantic encoding module is used to perform sentence processing and semantic encoding on the text description of the radiological inspection information of the target object to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors; a radiological prior knowledge document fragment matching module is used to match at least one radiological inspection information sentence granularity description embedded coding vector in the sequence of the radiological inspection information sentence granularity description embedded coding vectors from the radiological prior knowledge base. a sequence of embedded coding vectors of radiological prior knowledge document fragments; a semantic fusion module, used to semantically fuse each radiological inspection information sentence granularity description embedded coding vector in the sequence of embedded coding vectors of the radiological inspection information sentence granularity description with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of embedded coding vectors of prior-enhanced radiological inspection information sentence granularity description; a conditional feature semantic enhancement module, used to perform sequence-dependent conditional feature semantic enhancement on the sequence of embedded coding vectors of prior-enhanced radiological inspection information sentence granularity description to obtain a sequence of embedded coding vectors of semantic expression-enhanced radiological inspection information sentence granularity description; an inspection report generation module, used to obtain an inspection report based on the sequence of embedded coding vectors of the semantic expression-enhanced radiological inspection information sentence granularity description.

[0009] Compared with the prior art, the present application provides an automatic generation method and system for inspection reports, which constructs a radiological prior knowledge base, obtains the radiological inspection information of the target object, and uses a data information processing and encoding method based on deep learning to convert the radiological inspection information of the target object into a text description, and then performs sentence processing and semantic encoding. Then, the embedded features of the radiological prior knowledge document fragments are matched for each radiological inspection information sentence granularity description feature from the radiological prior knowledge base, and the embedded features and the corresponding sentence granularity description features are fused, so as to intelligently generate an inspection report based on the semantically enhanced expression of the fused embedded features of the priori enhanced radiological inspection information sentence granularity description. In this way, it is possible to capture the specific inspection semantic features in each sentence more finely based on the constructed radiological prior knowledge base and through a deep learning model and an intelligent matching mechanism, thereby automatically generating a high-quality and professional inspection report. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 The present invention is a flowchart of a method for automatically generating an inspection report according to an embodiment of the present application.

[0012] Figure 2 Schematic diagram of data flow of the method for automatically generating inspection reports according to an embodiment of the present application.

[0013] Figure 3 This is a flowchart of sub-step S6 of the method for automatically generating an inspection report according to an embodiment of the present application.

[0014] Figure 4 It is a block diagram of a system for automatically generating inspection reports according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0016] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0018] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0020] In the technical solution of the present application, a method for automatically generating an inspection report is proposed. Figure 1 The present invention is a flowchart of a method for automatically generating an inspection report according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the method for automatically generating an inspection report according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the method for automatically generating an inspection report includes the following steps: S1, constructing a radiological prior knowledge base, wherein the radiological prior knowledge base includes embedded coding vectors of multiple radiological prior knowledge document fragments; S2, acquiring the radiological inspection information of the target object, and performing text conversion on the radiological inspection information of the target object to obtain a text description of the radiological inspection information of the target object; S3, performing sentence processing and semantic coding on the text description of the radiological inspection information of the target object to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors; S4, matching the radiological inspection information sentence granularity description embedded coding vectors in the sequence of the radiological inspection information sentence granularity description embedded coding vectors from the radiological prior knowledge base, respectively. S5, semantically fuse each radiological inspection information sentence granularity description embedded coding vector in the sequence of radiological inspection information sentence granularity description embedded coding vectors with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors; S6, perform sequence-dependent conditional feature semantic enhancement on the sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors; S7, obtain an inspection report based on the sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors.

[0021] In particular, the S1 constructs a radiological prior knowledge base, which includes embedded coding vectors of multiple radiological prior knowledge document fragments. It should be understood that the radiological prior knowledge base is a database or information system designed specifically for the field of radiology, which stores a large amount of medical knowledge, case studies, diagnostic criteria, expert opinions and other information. After encoding and structuring, this information can be used to assist radiologists in interpreting imaging data, generating inspection reports, and supporting clinical decision-making. Therefore, in order to make the generated inspection reports more professional and reliable, in the technical solution of the present application, a radiological prior knowledge base is constructed, which includes embedded coding vectors of multiple radiological prior knowledge document fragments, that is, by integrating a large amount of abnormal diagnosis and treatment knowledge, radiological technology, expert opinions and other medical knowledge, the radiological prior knowledge base can help the system identify image features more accurately and assist doctors in making more accurate diagnoses. In addition, the rich medical terms and descriptions contained in the radiological prior knowledge base can significantly speed up the generation of inspection reports, reduce the time for doctors to manually write, and enable them to focus more on direct care of patients and other complex medical decisions.

[0022] In one example, a radiological prior knowledge base is constructed. Prior knowledge in the field of radiology is acquired, and a sliding window method is introduced to optimize the document segmentation method, thereby constructing a radiological prior knowledge base; constructing a radiological prior knowledge base specifically involves the collection of data, cleaning of radiological knowledge and document segmentation, persistent storage of knowledge, and updating of the knowledge base. The prior knowledge in the field of radiology is derived from radiological-related knowledge in various formats such as e-books, PDF reports, academic papers, case data, etc., specifically including the anatomical structure of each organ system, radiological report templates, abnormal diagnosis and treatment knowledge, radiological technology-related knowledge, etc. The prior knowledge data in the field of radiology are collected and processed and cleaned. Specifically, non-textual radiological prior data are converted into plain text data with the help of data processing technology. Furthermore, the extracted text data is integrated and preprocessed such as removing punctuation marks and stop words to obtain a summarized radiological prior knowledge document. Preferably, an optimized document segmentation and separation method is adopted, that is, the radiological prior knowledge document is segmented based on a sliding window method, thereby completing the construction of a radiological prior knowledge base. The specific technical details are as follows: the original document Split into A block of text, denoted as . Define variables Represents a window size, that is The number of blocks to consider on either side of the text block, the method window size It is dynamically adjusted based on indicators to determine the optimal context range of the text block.

[0023] For each block of text , generate a summary , the abstract contains The information of the text block. Then calculate each block Additive contextual summary of , Include Text Block and context The correlation between . is the attention mechanism function, which is used to The information in is weighted averaged.

[0024] Finally, each block With its additive contextual summary Combine to form the final input block : , is a function that combines the original block and its additive context summary. ;in, is the weight matrix, is the bias term, is the activation function.

[0025] The sliding window document segmentation method dynamically adjusts the window size and context summary function , which can optimize the performance of the splitting method , specifically expressed as: ,in is a performance evaluation function, based on the final block set Evaluate system performance.

[0026] Preferably, the document is processed in combination with the sliding window slicing method and persistently stored and updated irregularly, thereby completing the construction of a radiological prior knowledge base.

[0027] In particular, the S2 obtains the radiological examination information of the target object, and performs text conversion on the radiological examination information of the target object to obtain a text description of the radiological examination information of the target object. First, the radiological examination information of the target object is obtained. In one example, in order to ensure that the information obtained is comprehensive and accurate, a multi-channel data collection method is designed. The most direct data source is to directly extract the patient's radiological examination images and related metadata from the hospital's internal information system (HIS or RIS). In addition, images and related information can also be obtained from local storage devices, cloud servers or picture archiving and communication systems (PACS) through the standard format of digital imaging and communication in the medical field (DICOM files). For radiological examination information in some special cases, users may also need to manually enter or upload relevant information.

[0028] In addition to the image itself, a series of auxiliary information needs to be collected to help understand and interpret the image data. This includes but is not limited to the patient's basic information, such as name, gender, age and other non-sensitive personal information; examination details, such as examination date, examination site, equipment model used and parameter settings; actual X-rays, CT scans, MRI images and other imaging data; and sometimes some notes or preliminary impressions added by radiologists during the examination process, which are equally important and can provide additional professional opinions for the final report.

[0029] Once the raw data is obtained, the next step is to perform necessary preprocessing to ensure data quality and consistency. Data from different sources are formatted uniformly, such as converting all timestamps to a unified time format, standardizing and calibrating images generated by different devices, and ensuring the consistency of all input data. In order to protect patient privacy, any personally identifiable information, such as name, ID number, etc., is removed or encrypted without affecting the diagnosis results. At the same time, check whether the image is clear and complete, exclude poor quality images caused by equipment failure, etc., and verify the accuracy of other text information, such as confirming that the examination date and time are correct.

[0030] The preprocessed information needs to be further integrated to form a structured data set to facilitate subsequent sentence processing and semantic encoding. Organize the above information according to certain logical relationships, such as sorting by examination date or classifying by examination site, to ensure that each image has clear description information to match. Establishing the association between images and corresponding metadata to ensure that each image can find its corresponding examination record and other relevant information is crucial to improving diagnostic accuracy.

[0031] Next, the radiological examination information of the target object is converted into text to obtain a text description of the radiological examination information of the target object. It should be understood that since the radiological examination information of the target object contains images of the internal structure of the human body generated using technologies such as X-rays, CT (computed tomography), MRI (magnetic resonance imaging), ultrasound or nuclear medicine, in order to be able to convert the radiological examination information of the target object into structured, easy-to-understand and process text information, thereby supporting more efficient medical diagnosis and subsequent processing, in the technical solution of the present application, the radiological examination information of the target object is converted into text. In particular, in a specific embodiment of the present application, the radiological examination information of the target object can be processed by using an encoder-decoder architecture. Specifically: the CNN in the encoder-decoder architecture processes images as an encoder, which automatically learns the key feature representations at different scales in the image through a series of convolutional layers and pooling layers, and then receives the initial state of the image features from the encoder through RNN, LSTM or GRU as a decoder and gradually generates descriptive text according to the context. In this way, the radiological examination images in the radiological examination information can be converted into text descriptions, and the text information such as patient information, examination type and location can be used as the text description of the target object radiological examination information to provide richer data support for the subsequent generation of examination reports.

[0032] In particular, the S3 performs sentence processing and semantic encoding on the text description of the target object radiological examination information to obtain a sequence of radiological examination information sentence granularity description embedded coding vectors. That is, in a specific example of the present application, first, considering that the description sentences of different examination information in the text description of the target object radiological examination information contain different semantic content information, such as clinical symptoms, diagnostic conclusions, etc., the semantic information expressed in the meanings of these different sentences, therefore, in order to be able to more carefully analyze and understand the semantic information in each examination information, in the technical solution of the present application, the text description of the target object radiological examination information is sentence processed to divide the longer text into smaller, more manageable sentences or phrases, and obtain a sequence of radiological examination information sentence granularity descriptions, so as to better understand the text information conveyed by each sentence, especially when this information involves complex medical terms or diagnostic conclusions. Then, each radiological examination information sentence granularity description in the sequence of the radiological examination information sentence granularity description is semantically encoded to obtain a sequence of the radiological examination information sentence granularity description embedded coding vectors. That is, in order to convert unstructured text information into a numerical representation that can be understood and processed by a computer, and at the same time to more accurately understand the semantic association between the semantic information and the context in each radiological inspection information, in the technical solution of the present application, each radiological inspection information sentence granularity description in the sequence of the radiological inspection information sentence granularity description is semantically encoded to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors, so as to help the model understand the logical relationship and contextual dependency between sentences more clearly, which is crucial for generating a coherent and accurate inspection report, and to provide more accurate matching results from the radiological prior knowledge base in the subsequent process. In particular, in a specific embodiment of the present application, a radiological information semantic encoder including a sentence embedding layer is used to semantically encode each radiological inspection information sentence granularity description in the sequence of the radiological inspection information sentence granularity description to capture and mine the deep inspection information semantic features of each sentence, and obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors.

[0033] In particular, the S4 matches at least one embedded coding vector of a radiological prior knowledge document fragment from the radiological prior knowledge base for each radiological examination information sentence granularity description embedded coding vector in the sequence of the radiological examination information sentence granularity description embedded coding vector. In order to be able to use existing medical knowledge to enhance the understanding and interpretation of new data, provide additional background information and reference cases for the semantic content of the inspection information at each sentence granularity, thereby improving the accuracy and reliability of the inspection report generation, in the technical solution of the present application, at least one embedded coding vector of a radiological prior knowledge document fragment is matched from the radiological prior knowledge base for each radiological examination information sentence granularity description embedded coding vector in the sequence of the radiological examination information sentence granularity description embedded coding vector.

[0034] In one example, a comprehensive search vectorization module is constructed. A comprehensive search vectorization module is developed, and a re-ranking mechanism is introduced to improve the recall performance of knowledge base documents. Preferably, the comprehensive search combines semantic similarity search and full-text search technology. Semantic similarity search is based on segmented text vector indexing, and the similarity between the input text and the knowledge base document fragment is calculated by vectorization. The vectorization module of the method further combines the keyword matching technology of full-text search, and achieves accurate comprehensive search by weighted averaging the vectorization similarity and keyword matching results. The specific definitions are as follows:

[0035] The query text vector is: , the i-th document block vector is: , then the vector retrieval similarity between the query text vector and the i-th document block is expressed as: ;in, represents the dot product of the text vector and the document block vector, Represents the Euclidean distance between two vectors. From this, we can get the full-text retrieval score of the two vectors: ;in, Representing terms The inverse document frequency is calculated as follows: ;in, is the total number of documents, is a term containing The number of documents in the above formula Representing terms In the documentation The probability of occurrence in represents the adjustment parameter, Represents the adjustment parameters to control the degree of normalization of the document. Indicates the length of the document block, represents the average length of the documents in the entire document set; and the final weighted average comprehensive retrieval score is: ;in, ; By calculating the weighted average similarity score as above, the search results will be re-ranked according to the comprehensive similarity score. The re-ranking mechanism in this application enables the document fragments most relevant to the user query to be returned first, thereby improving the recall performance.

[0036] In particular, the S5 semantically fuses each radiological inspection information sentence granularity description embedded coding vector in the sequence of radiological inspection information sentence granularity description embedded coding vectors with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of prior-enhanced radiological inspection information sentence granularity description embedded coding vectors. That is, in the technical solution of the present application, each radiological inspection information sentence granularity description embedded coding vector in the sequence of radiological inspection information sentence granularity description embedded coding vectors is cascaded with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of prior-enhanced radiological inspection information sentence granularity description embedded coding vectors. Here, considering that each radiological examination information sentence granularity description embedded coding vector matches the radiological prior knowledge document information related to it, therefore, in order to be able to enhance and enrich the semantic expression of the original examination information, so as to provide a more comprehensive and standard representation, to improve the task of subsequent report generation, in the technical solution of the present application, each radiological examination information sentence granularity description embedded coding vector in the sequence of the radiological examination information sentence granularity description embedded coding vector is cascaded with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of prior enhanced radiological examination information sentence granularity description embedded coding vectors. In other words, through the cascade operation, the original inspection information of each sentence granularity can be combined with the relevant prior knowledge, so that each sentence not only contains its own inspection semantic features, but also incorporates rich medical background information and expert opinions, so as to obtain a more detailed, professional and medically standard-compliant representation, and ensure the standardization and completeness of the content of the subsequent inspection report.

[0037] In particular, the S6 performs sequence-dependent conditional feature semantic enhancement on the sequence of the prior-enhanced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of semantic expression-enhanced radiological inspection information sentence granularity description embedded coding vectors. Considering that each prior-enhanced radiological inspection information sentence granularity description embedded coding vector contains key and deep-level semantic information expressions, and each sentence granularity information has different influence and importance in the essential core attributes of the entire inspection information. Therefore, in order to further optimize and enhance the semantic expression capabilities of these embedded coding vectors, so as to more finely capture the specific key inspection features contained in each sentence, thereby ensuring that the semantic expression is more accurate and detailed, in the technical solution of the present application, the sequence of the prior-enhanced radiological inspection information sentence granularity description embedded coding vectors is subjected to sequence-dependent conditional feature semantic enhancement to obtain a sequence of semantic expression-enhanced radiological inspection information sentence granularity description embedded coding vectors. In this way, it can be ensured that they can better capture and convey the key features in the radiological inspection information, so as to improve the quality and accuracy of subsequent tasks (report generation). Specifically, in a specific example of the present application, such as Figure 3 As shown, the S6 includes: S61, performing field mapping on each prior enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vector to obtain a sequence of admission prior enhanced radiological inspection information sentence granularity description embedded coding vectors; S62, calculating the prior enhanced radiological inspection information field essential feature vector of the sequence of admission prior enhanced radiological inspection information sentence granularity description embedded coding vectors; S63, using the prior enhanced radiological inspection information field essential feature vector as a conditional feature vector, respectively performing distinguishable enhancement on each prior enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vector to obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors.

[0038] Specifically, the S61 performs field mapping on each priori enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of the priori enhanced radiological inspection information sentence granularity description embedded coding vector to obtain a sequence of entry priori enhanced radiological inspection information sentence granularity description embedded coding vectors. That is, a field mapper is used to perform field mapping on each priori enhanced radiological inspection information sentence granularity description embedded coding vector to obtain a sequence of entry feature vectors. That is, a field mapping operation is performed on each priori enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of the priori enhanced radiological inspection information sentence granularity description embedded coding vector, with the purpose of converting these features into a form that is more suitable for the current model or task requirements, that is, obtaining a sequence of entry priori enhanced radiological inspection information sentence granularity description embedded coding vectors. This stage is essentially a transformation of the feature space, involving nonlinear transformation or other forms of preprocessing technology, so that the features can better reflect the intrinsic relationship between samples or are more easily captured by the algorithm. Key information. More specifically, the following field mapping formula is used to perform field mapping on each priori enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of priori enhanced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of admission priori enhanced radiological inspection information sentence granularity description embedded coding vectors; wherein the field mapping formula is: ;in, The granularity description of the a priori enhanced radiological examination information sentence is embedded in the sequence of the encoding vector. A priori enhanced radiological examination information sentence granularity description embedding encoding vector, and are the first weight matrix and the second weight matrix respectively, yes The corresponding admission prior enhanced radiological examination information sentence granularity description embedding encoding vector.

[0039] Specifically, the S62 calculates the field essential characteristic vector of the a priori enhanced radiological inspection information of the sequence of the granular description of the entry a priori enhanced radiological inspection information sentence embedded coding vectors. That is, in the embodiment of the present application, the field depth factors of each entry a priori enhanced radiological inspection information sentence granular description embedded coding vector in the sequence of the granular description of the entry a priori enhanced radiological inspection information sentence embedded coding vectors are first calculated to obtain a sequence of field depth factors of the a priori enhanced radiological inspection information; such factors are obtained one by one through all the entry a priori enhanced radiological inspection information sentence granular description embedded coding vectors in the sequence of the granular description of the entry a priori enhanced radiological inspection information sentence embedded coding vectors to obtain a sequence of field depth factors of the a priori enhanced radiological inspection information. Furthermore, based on the sequence of field depth factors of the a priori enhanced radiological inspection information, the field essential characteristic vector of the entry a priori enhanced radiological inspection information sentence granular description embedded coding vector is calculated. That is, using the sequence of the depth factors of the prior enhanced radiological inspection information field obtained above as a basis, the field effect normalization mechanism is used to extract a feature distribution that comprehensively reflects the core attributes of the embedding coding vector of the granular description of the whole batch of admission prior enhanced radiological inspection information sentences, that is, the prior enhanced radiological inspection information field essential feature vector. More specifically, the following essential feature calculation formula is used to calculate the prior enhanced radiological inspection information field essential feature vector of the sequence of the embedding coding vector of the granular description of the admission prior enhanced radiological inspection information sentence; wherein, the essential feature calculation formula is: ;in, To calculate the square of the length of a vector, represents the logarithmic function value with base 2, For calculation The depth of field factor, for The corresponding prior enhanced radiological inspection information field depth factor, It means to calculate the exponential function value with the natural constant e as the base. for The corresponding standardized a priori enhanced radiological inspection information field depth factor, It is the essential characteristic vector of the prior enhanced radiological inspection information field.

[0040] Specifically, the S63 uses the prior enhanced radiological inspection information field essential feature vector as a conditional feature vector, and performs distinguishable enhancement on each prior enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of semantically expressed enhanced radiological inspection information sentence granularity description embedded coding vectors. That is, in an embodiment of the present application, a sigmoid function is first used to perform fully connected encoding semantic association on the conditional feature vector and each prior enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vector to obtain a sequence of prior enhanced radiological inspection information sentence granularity description fully connected embedded coding vectors; then, the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vectors is point-wise multiplied with each group of corresponding prior enhanced radiological inspection information sentence granularity description embedded coding vectors and the prior enhanced radiological inspection information sentence granularity description fully connected embedded coding vector in the sequence of the prior enhanced radiological inspection information sentence granularity description fully connected embedded coding vector to obtain the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors. That is, using the a priori enhanced radiological inspection information field essential feature vector as additional conditional information, the sequence of the a priori enhanced radiological inspection information sentence granularity description embedded coding vectors is subjected to element-by-element distinguishable enhancement processing, so as to specifically amplify important features in certain aspects according to specific rules or weight adjustment schemes, while suppressing other less important parts, so as to achieve the purpose of optimizing feature expression, and obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors. More specifically, using the a priori enhanced radiological inspection information field essential feature vector as the conditional feature vector, the following distinguishable enhancement formula is used to distinguishably enhance each a priori enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of the a priori enhanced radiological inspection information sentence granularity description embedded coding vector to obtain the sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors; wherein, the distinguishable enhancement formula is: ;in, is the priori enhanced radiological examination information essential weight matrix, is the local prior enhanced radiological examination information weight matrix, is the bias vector, yes function, It is the point multiplication by position. Yes The semantic expression after semantic enhancement is used to embed the encoding vector of the granular description of the radiological examination information sentence.

[0041] In particular, the S7 obtains the inspection report based on the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors. That is, in the technical solution of the present application, the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors is input into the trained inspection report generator based on the large language model to obtain the inspection report. That is, the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded features obtained by semantic expression enhancement using the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vectors is generated and processed, so as to intelligently generate the inspection report.

[0042] In one example, a reinforcement feedback learning approach is used to fine-tune a large language model.

[0043] Specifically, a high-quality fine-tuning instruction dataset is constructed through manual evaluation and scoring by models such as GPT-4. The constructed data contains user instructions, positive outputs, and negative outputs. The specific format is as follows:

[0044] {

[0045] "conversation":[

[0046] {

[0047] “from”: “human”,

[0048] “value”: “Please generate a radiology report based on the input information. The output is required to be generated in JSON format: “tissue + abnormal information””

[0049] }

[0050] ],

[0051] "chosen":{

[0052] “from”: “LLM”,

[0053] “value”: “{“Image findings”: “ XXXX” , “Diagnosis conclusion”: “XXXX”}”

[0054] },

[0055] "rejected":{

[0056] “from”: “LLM”,

[0057] “value”: “Based on the information you entered, I can generate the following radiology report: Imaging findings: XXXX; Diagnosis conclusion: XXXX.”

[0058] },

[0059] }

[0060] According to the above template, the specific input information, imaging findings, and diagnostic conclusions are extracted from the radiology report text after desensitization and cleaning, thereby completing the construction of a radiology report text generation data set of N. Furthermore, the ORPO method, a reinforcement feedback learning method, is selected to fine-tune the large language model.

[0061] Through ORPO, LLM can learn to answer instructions and satisfy human preferences at the same time. The loss function is calculated as follows during fine-tuning: ;in, is a contrastive loss function, The model parameters are , the input is Correct category Odds, The model parameters are , the input is Error category Odds, is the sigmoid function, Strengthens the model's ability to distinguish between correct and incorrect categories. is the standard cross entropy loss, and the hyperparameter For adjustment and Previous weight.

[0062] After fine-tuning, the large language model can learn to answer commands and give data formats that satisfy human preferences.

[0063] Next, the extracted radiology prior knowledge is used to generate accurate radiology reports using a fine-tuned large language model. Specifically, the system uses the extracted radiology prior knowledge as contextual supplementary information for the fine-tuned large language model, and the model generates a complete radiology report containing imaging findings and diagnostic conclusions strictly based on the current contextual knowledge.

[0064] Here, in the case where each prior reinforced radiological inspection information sentence granularity description embedded coding vector in the sequence of the prior reinforced radiological inspection information sentence granularity description embedded coding vector represents the embedded coding semantic features of the radiological inspection information combined with the prior knowledge fragment, after performing feature distinguishable enhancement based on the sequence conditional description, the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors will also have significant distinguishable enhancement distribution space long-range structural consistency differences due to the local semantic serialization conditional descriptive differences, thereby affecting the semantic accuracy of the inspection report obtained by the inspection report generator based on the large language model.

[0065] Therefore, in a preferred example, the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors is optimized, and the optimization process includes: arranging the respective feature values ​​of the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors in ascending order to obtain a semantic expression enhanced radiological inspection information sentence granularity description embedded coding sequence set; The eigenvalue and The absolute value of the difference between the eigenvalues ​​is less than or equal to the distance difference hyperparameter , calculate the The eigenvalues ​​are similar to the The weighted sum between the eigenvalues ​​is the optimized Eigenvalue ;in, and They represent the first Eigenvalues ​​and Eigenvalues, and represents different weighting hyperparameters.

[0066] In response to the semantic expression, the radiological inspection information sentence granularity description is embedded in the encoding sequence set. The eigenvalue and The absolute value of the difference between the eigenvalues ​​is greater than the distance difference hyperparameter .

[0067] Calculate the square root of the sum of squares of all feature values ​​of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding sequence set, multiply the square root by 2 and then divide it by the square of the size of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding sequence set to obtain the semantic expression enhanced radiological inspection information sentence granularity description embedded coding space primitive value .

[0068] The semantic expression enhanced radiological inspection information sentence granularity description embedding coding space primitive value is multiplied by the first After the eigenvalue, calculate the product with the first The weighted reduction between the eigenvalues ​​is the optimized Eigenvalue , and Denotes different weighting hyperparameters.

[0069] Combining the semantic expression to strengthen the radiological inspection information sentence granularity description embedding coding sequence set optimization The feature value is used to obtain a sequence of embedding coding vectors of the semantic expression-enhanced radiological inspection information sentence granularity description, wherein the first feature value of the embedding coding sequence set of the semantic expression-enhanced radiological inspection information sentence granularity description remains unchanged. Finally, the sequence of embedding coding vectors of the semantic expression-enhanced radiological inspection information sentence granularity description is input into the trained inspection report generator based on the large language model to obtain the inspection report.

[0070] In this way, in order to solve the problem that the feature set of the sequence of embedded coding vectors described at the granularity of the semantic expression enhanced radiological inspection information sentence is insufficient in the convergence representation ability of the global feature space structure caused by the long distance exceeding the predetermined local distribution interval threshold under the predetermined feature value sequential distribution, the high-dimensional feature space primitive representation based on self-inner product fusion of the sequence of embedded coding vectors described at the granularity of the semantic expression enhanced radiological inspection information sentence is used to capture the complex structure of the global network interaction of its feature values, so as to reconstruct the spatial structure aggregation regression relationship between the feature values ​​of the sequence of embedded coding vectors described at the granularity of the semantic expression enhanced radiological inspection information sentence by simulating the scale-based high-dimensional feature space potential primitives, so as to realize the encoding reconstruction of the real sequence distribution behavior of the sequence of embedded coding vectors described at the granularity of the semantic expression enhanced radiological inspection information sentence under long distance, improve the consistent expression effect of the sequence of embedded coding vectors described at the granularity of the semantic expression enhanced radiological inspection information sentence, and improve the semantic accuracy of the inspection report generator based on the large language model whose input training is completed.

[0071] In summary, the automatic generation method of the inspection report according to the embodiment of the present application is explained, which constructs a radiological prior knowledge base, obtains the radiological inspection information of the target object, and uses a data information processing and encoding method based on deep learning to convert the radiological inspection information of the target object into a text description and then performs sentence processing and semantic encoding. Then, the embedded features of the radiological prior knowledge document fragments are matched for each radiological inspection information sentence granularity description feature from the radiological prior knowledge base, and the embedded features and the corresponding sentence granularity description features are fused, so as to intelligently generate an inspection report based on the semantically enhanced expression of the embedded features of the fused prior enhanced radiological inspection information sentence granularity description. In this way, it is possible to capture the specific inspection semantic features in each sentence more finely based on the constructed radiological prior knowledge base and through a deep learning model and an intelligent matching mechanism, thereby automatically generating a high-quality and professional inspection report.

[0072] Furthermore, a system for automatically generating an inspection report is also provided.

[0073] Figure 4 FIG. 1 is a block diagram of a system for automatically generating an inspection report according to an embodiment of the present application. Figure 4As shown, according to the embodiment of the present application, the automatic generation system 300 of the inspection report includes: a radiological prior knowledge base construction module 310, which is used to construct a radiological prior knowledge base, and the radiological prior knowledge base includes multiple embedded coding vectors of radiological prior knowledge document fragments; a radiological inspection information acquisition text conversion module 320, which is used to obtain the radiological inspection information of the target object, and perform text conversion on the radiological inspection information of the target object to obtain a text description of the radiological inspection information of the target object; a semantic encoding module 330, which is used to perform sentence processing and semantic encoding on the text description of the radiological inspection information of the target object to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors; a radiological prior knowledge document fragment matching module 340, which is used to match each radiological inspection information sentence granularity description in the sequence of embedded coding vectors from the radiological prior knowledge base. The description embedded coding vectors are matched to the embedded coding vectors of at least one radiological prior knowledge document fragment respectively; a semantic fusion module 350 is used to semantically fuse each radiological inspection information sentence granularity description embedded coding vector in the sequence of the radiological inspection information sentence granularity description embedded coding vectors with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors; a conditional feature semantic enhancement module 360 ​​is used to perform sequence-dependent conditional feature semantic enhancement on the sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors; an inspection report generation module 370 is used to enhance the sequence of radiological inspection information sentence granularity description embedded coding vectors based on the semantic expression to obtain an inspection report.

[0074] As described above, the inspection report automatic generation system 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an inspection report automatic generation algorithm. In a possible implementation, the inspection report automatic generation system 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the inspection report automatic generation system 300 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the inspection report automatic generation system 300 can also be one of the many hardware modules of the wireless terminal.

[0075] Alternatively, in another example, the inspection report automatic generation system 300 and the wireless terminal may also be separate devices, and the inspection report automatic generation system 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0076] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for automatically generating an inspection report, characterized in that: include: Constructing a radiological prior knowledge base, the radiological prior knowledge base comprising embedded coding vectors of a plurality of radiological prior knowledge document fragments; acquiring radiological examination information of a target object, and performing text conversion on the radiological examination information of the target object to obtain a text description of the radiological examination information of the target object; performing sentence processing and semantic coding on the text description of the radiological examination information of the target object to obtain a sequence of embedded coding vectors of sentence granularity descriptions of the radiological examination information; Matching the embedded coding vector of at least one radiological prior knowledge document fragment to each radiological inspection information sentence granularity description embedded coding vector in the sequence of radiological inspection information sentence granularity description embedded coding vectors from the radiological prior knowledge base; semantically fusing each radiological inspection information sentence granularity description embedded coding vector in the sequence of radiological inspection information sentence granularity description embedded coding vectors with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of prior reinforced radiological inspection information sentence granularity description embedded coding vectors; Performing sequence-dependent conditional feature semantic enhancement on the sequence of the priori enhanced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors; Based on the semantic expression, the radiological inspection information sentence granularity description is enhanced to embed a sequence of coding vectors to obtain an inspection report.

2. The method for automatically generating an inspection report according to claim 1, characterized in that: The text description of the target object's radiological inspection information is subjected to sentence processing and semantic encoding to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors, including: sentence processing of the target object's radiological inspection information text description to obtain a sequence of radiological inspection information sentence granularity descriptions; semantic encoding of each radiological inspection information sentence granularity description in the sequence of radiological inspection information sentence granularity descriptions to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors.

3. The method for automatically generating an inspection report according to claim 2, characterized in that: Semantically encoding each radiological inspection information sentence granularity description in the sequence of radiological inspection information sentence granularity descriptions to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors, including: using a radiological information semantic encoder including a sentence embedding layer to semantically encode each radiological inspection information sentence granularity description in the sequence of radiological inspection information sentence granularity descriptions to obtain a sequence of radiological inspection information sentence granularity description embedded coding vectors.

4. The method for automatically generating an inspection report according to claim 3, characterized in that: Semantically fusing each radiological inspection information sentence granularity description embedded coding vector in the sequence of radiological inspection information sentence granularity description embedded coding vectors with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors, including: cascading each radiological inspection information sentence granularity description embedded coding vector in the sequence of radiological inspection information sentence granularity description embedded coding vectors with the embedded coding vector of the at least one radiological prior knowledge document fragment to obtain the sequence of prior enhanced radiological inspection information sentence granularity description embedded coding vectors.

5. The method for automatically generating an inspection report according to claim 4, characterized in that: The sequence of the prior reinforced radiological inspection information sentence granularity description embedded coding vectors is subjected to sequence-dependent conditional feature semantic enhancement to obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors, including: performing field mapping on each prior reinforced radiological inspection information sentence granularity description embedded coding vector in the sequence of the prior reinforced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of admission prior reinforced radiological inspection information sentence granularity description embedded coding vectors; calculating the prior reinforced radiological inspection information field essential feature vector of the sequence of admission prior reinforced radiological inspection information sentence granularity description embedded coding vectors; using the prior reinforced radiological inspection information field essential feature vector as the conditional feature vector, respectively performing distinguishable enhancement on each prior reinforced radiological inspection information sentence granularity description embedded coding vector in the sequence of the prior reinforced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors.

6. The method for automatically generating an inspection report according to claim 5, characterized in that: Calculating the prior enhanced radiological inspection information field essential characteristic vector of the sequence of the granular description of the entry prior enhanced radiological inspection information sentence embedded in the coding vector, including: calculating the field depth factor of each entry prior enhanced radiological inspection information sentence granular description embedded in the coding vector in the sequence of the granular description of the entry prior enhanced radiological inspection information sentence embedded in the coding vector to obtain a sequence of the prior enhanced radiological inspection information field depth factors; based on the sequence of the prior enhanced radiological inspection information field depth factors, calculating the prior enhanced radiological inspection information field essential characteristic vector of the sequence of the granular description of the entry prior enhanced radiological inspection information sentence embedded in the coding vector.

7. The method for automatically generating an inspection report according to claim 6, characterized in that: Taking the prior enhanced radiological inspection information field essential feature vector as the conditional feature vector, each prior enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vector is distinguishably enhanced to obtain the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vector, including: using a sigmoid function to perform full connection encoding semantic association on the conditional feature vector and each prior enhanced radiological inspection information sentence granularity description embedded coding vector in the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vector to obtain a sequence of prior enhanced radiological inspection information sentence granularity description fully connected embedded coding vectors; performing position point multiplication on the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vectors and each group of corresponding prior enhanced radiological inspection information sentence granularity description embedded coding vectors and prior enhanced radiological inspection information sentence granularity description fully connected embedded coding vectors in the sequence of the prior enhanced radiological inspection information sentence granularity description fully connected embedded coding vectors to obtain the sequence of the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors.

8. The method for automatically generating an inspection report according to claim 7, characterized in that: Based on the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vector sequence, an inspection report is obtained, including: inputting the semantic expression enhanced radiological inspection information sentence granularity description embedded coding vector sequence into a trained inspection report generator based on a large language model to obtain the inspection report.

9. An automatic inspection report generation system, characterized in that: include: A radiological prior knowledge base construction module is used to construct a radiological prior knowledge base, wherein the radiological prior knowledge base includes embedded coding vectors of multiple radiological prior knowledge document fragments; a radiological inspection information acquisition text conversion module is used to obtain radiological inspection information of a target object, and perform text conversion on the radiological inspection information of the target object to obtain a text description of the radiological inspection information of the target object; a semantic encoding module is used to perform sentence processing and semantic encoding on the text description of the radiological inspection information of the target object to obtain a sequence of embedded coding vectors of radiological inspection information sentence granularity descriptions; a radiological prior knowledge document fragment matching module is used to match the embedded coding vector of at least one radiological prior knowledge document fragment from the radiological prior knowledge base for each radiological inspection information sentence granularity description embedded coding vector in the sequence of embedded coding vectors of the radiological inspection information sentence granularity descriptions; A semantic fusion module, used for semantically fusing each radiological inspection information sentence granularity description embedding coding vector in the sequence of radiological inspection information sentence granularity description embedding coding vectors with the embedding coding vector of the at least one radiological prior knowledge document fragment to obtain a sequence of prior reinforced radiological inspection information sentence granularity description embedding coding vectors; A conditional feature semantic enhancement module is used to perform sequence-dependent conditional feature semantic enhancement on the sequence of the prior enhanced radiological inspection information sentence granularity description embedded coding vectors to obtain a sequence of semantic expression enhanced radiological inspection information sentence granularity description embedded coding vectors; The inspection report generation module is used to enhance the sequence of radiological inspection information sentence granularity description embedded coding vectors based on the semantic expression to obtain an inspection report.

Images