Image analysis method and device, computing equipment and storage medium

Abstract

The invention provides an image analysis method and device, computing equipment and a storage medium. The method may include: obtaining flow chart data generated by a large language model based on a diagnostic thought description text, the flow chart data representing a flow chart including at least one node, the diagnostic thought description text being a natural language text describing a diagnostic thought for performing diagnosis according to a medical image; and based on the flow chart data, according to one or more medical images associated with the target human body, obtaining an analysis result about the target human body.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to an image analysis method, apparatus, computing device and storage medium. Background Art

[0002] With the rapid development of medical imaging technology, the demand for human image analysis in the medical field is increasing. How to analyze human images efficiently and accurately has become an important direction of current technological development.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] According to one aspect of the present disclosure, a computer-implemented image analysis method is provided, comprising: obtaining flowchart data generated by a large language model based on a diagnostic idea description text, the flowchart data being used to represent a flowchart including at least one node, the diagnostic idea description text being a natural language text describing a diagnostic idea for diagnosis based on a medical image; and obtaining an analysis result about the target human body based on the flowchart data and according to one or more medical images associated with the target human body.

[0005] According to another aspect of the present disclosure, an image analysis device is provided, comprising: a flowchart obtaining unit, for obtaining flowchart data generated by a large language model based on a diagnostic idea description text, wherein the flowchart data is used to represent a flowchart including at least one node, and the diagnostic idea description text is a natural language text that describes a diagnostic idea for diagnosis based on a medical image; and an analysis unit, for obtaining an analysis result about the target human body based on the flowchart data and one or more medical images associated with the target human body.

[0006] According to another aspect of the present disclosure, a computing device is provided, comprising: a memory, a processor, and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement an image analysis method according to one or more embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, the image analysis method according to one or more embodiments of the present disclosure is implemented.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the image analysis method according to one or more embodiments of the present disclosure when executed by a processor.

[0009] These and other aspects of the disclosure will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0011] Figure 1 is a schematic diagram illustrating an example system in which the various methods described herein may be implemented according to an exemplary embodiment;

[0012] Figure 2 is a flow chart illustrating an image analysis method according to an exemplary embodiment;

[0013] Figure 3A is a schematic diagram of an image application scenario according to an exemplary embodiment;

[0014] Figure 3B is a schematic diagram illustrating flow chart data obtained by an image analysis method according to an exemplary embodiment;

[0015] Figure 4 is a schematic block diagram illustrating an image analysis apparatus according to an exemplary embodiment;

[0016] Figure 5 is a block diagram illustrating an exemplary computer device that can be applied to the exemplary embodiments. DETAILED DESCRIPTION

[0017] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0018] The terms used in the description of various examples described in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. As used herein, the term "plurality" means two or more, and the term "based on" should be interpreted as "based at least in part on". In addition, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations.

[0019] Exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0020] Figure 1 is a schematic diagram illustrating an example system 100 in which the various methods described herein may be implemented, according to an example embodiment.

[0021] refer to Figure 1 The system 100 includes a client device 110 , a server 120 , and a network 130 that communicatively couples the client device 110 and the server 120 .

[0022] The client device 110 includes a display 114 and a client application (APP) 112 that can be displayed via the display 114. The client application 112 can be an application that needs to be downloaded and installed before running or a small program (liteapp) as a lightweight application. In the case where the client application 112 is an application that needs to be downloaded and installed before running, the client application 112 can be pre-installed on the client device 110 and activated. In the case where the client application 112 is a small program, the user 102 can directly run the client application 112 on the client device 110 by searching for the client application 112 in the host application (for example, by the name of the client application 112, etc.) or scanning the graphic code of the client application 112 (for example, a bar code, a QR code, etc.), without installing the client application 112. In some embodiments, the client device 110 can be any type of mobile computer device, including a mobile computer, a mobile phone, a wearable computer device (for example, a smart watch, a head-mounted device, including smart glasses, etc.) or other types of mobile devices. In some embodiments, the client device 110 may alternatively be a stationary computer device, such as a desktop computer, a server computer or other types of stationary computer devices. In some optional embodiments, the client device 110 may also be or include a medical image printing device.

[0023] The server 120 is typically a server deployed by an Internet Service Provider (ISP) or an Internet Content Provider (ICP). The server 120 may represent a single server, a cluster of multiple servers, a distributed system, or a cloud server that provides basic cloud services (such as cloud databases, cloud computing, cloud storage, and cloud communications). It will be understood that although Figure 1 The server 120 is shown in FIG. 1 in communicating with only one client device 110 , but the server 120 may provide background services for multiple client devices simultaneously.

[0024] Examples of network 130 include a local area network (LAN), a wide area network (WAN), a personal area network (PAN), and / or a combination of communication networks such as the Internet. Network 130 can be a wired or wireless network. In some embodiments, the data exchanged through network 130 is processed using technologies and / or formats including hypertext markup language (HTML), extensible markup language (XML), etc. In addition, encryption technologies such as secure socket layer (SSL), transport layer security (TLS), virtual private network (VPN), Internet protocol security (IPsec) can also be used to encrypt all or some links. In some embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0025] The system 100 may also include an image acquisition device 140. In some embodiments, Figure 1The image acquisition device 140 shown can be a medical scanning device, including but not limited to scanning or imaging devices used in positron emission tomography (PET), positron emission tomography with computerized tomography (PET / CT), single photon emission computed tomography with computerized tomography (SPECT / CT), computerized tomography (CT), medical ultrasonography, nuclear magnetic resonance imaging (NMRI), magnetic resonance imaging (MRI), cardiovascular angiography (CA), digital radiography (DR), etc. For example, the image acquisition device 140 may include a digital subtraction angiography scanner, a magnetic resonance angiography scanner, a tomography angiography scanner, a positron emission tomography scanner, a positron emission computed tomography scanner, a single photon emission computed tomography scanner, a computed tomography scanner, a medical ultrasound examination device, a nuclear magnetic resonance imaging scanner, a magnetic resonance imaging scanner, a digital radiography scanner, etc. The image acquisition device 140 may communicate with a server (e.g., Figure 1 The image processing system is connected to the server 120 in the figure or a separate server of the imaging system (not shown in the figure) to realize image data processing, including but not limited to converting the scan data (for example, converting it into a medical image sequence), compressing it, correcting its pixels, and reconstructing it in three dimensions.

[0026] The image acquisition device 140 may be connected to the client device 110 , for example, via the network 130 , or directly connected to the client device in other ways to communicate with the client device.

[0027] Optionally, the system may further include an intelligent computing device or computing card 150. The image acquisition device 140 may include or be connected (e.g., removably connected) to such a computing card 150, etc. As an example, the computing card 150 may implement image data processing, including but not limited to conversion, compression, pixel correction, reconstruction, etc. As another example, the computing card 150 may implement an image analysis method according to an embodiment of the present disclosure.

[0028] The system may also include other parts not shown, such as a data storage unit. The data storage unit may be a database, a data repository, or one or more devices for data storage in other forms, and may be a conventional database, or may include a cloud database, a distributed database, etc. For example, direct image data formed by the image acquisition device 140 or a medical image sequence or three-dimensional image data obtained through image processing may be stored in the data storage unit for subsequent retrieval by the server 120 and the client device 110 from the data storage unit. In addition, the above-mentioned image acquisition device 140 may also directly provide the image data or the medical image sequence or three-dimensional image data obtained through image processing to the server 120 or the client device 110, etc.

[0029] The user can use the client device 110 to control the acquisition of images or videos, view the acquired images or videos (including preliminary image data or images that have been analyzed and processed, etc.), view the analysis results, interact with the acquired images or analysis results, input acquisition instructions, configure data, etc. The client device 110 can send configuration data, instructions or other information to the image acquisition device 140 to control the acquisition of the image acquisition device, process data, etc.

[0030] For the purpose of the embodiments of this disclosure, Figure 1In the example of , the client application 112 may be an image sequence management application, which may provide various functions, such as storage management, indexing, sorting, and classification of the acquired image sequences. Correspondingly, the server 120 may be a server used together with the image sequence management application. The server 120 may provide image sequence management services to the client application 112 running in the client device 110 based on user requests or instructions generated according to the embodiments of the present disclosure, such as managing the image sequence storage in the cloud, storing and classifying the image sequence according to a specified index (including, for example, but not limited to, sequence type, patient identification, body part, acquisition target, acquisition stage, acquisition machine, whether lesions are detected, severity, etc.), and retrieving and providing the image sequence to the client device according to the specified index, etc. Alternatively, the server 120 may also provide or allocate such service capabilities or storage space to the client device 110, and the client application 112 running in the client device 110 may provide corresponding image sequence management services according to user requests or instructions generated according to the embodiments of the present disclosure, etc. It is to be understood that the above is only an example, and the present disclosure is not limited thereto.

[0031] Figure 2 2 is a flowchart illustrating an image analysis method 200 according to an exemplary embodiment. The method 200 may be performed on a client device (eg, Figure 1 The execution of each step of the method 200 may be performed at the client device 110 shown in FIG. Figure 1 In some embodiments, the method 200 may be performed on a server (e.g., Figure 1 In some embodiments, method 200 may be performed by a client device (eg, client device 110) and a server (eg, server 120) in combination.

[0032] In the following, each step of the method 200 is described in detail.

[0033] refer to Figure 2 At step 210, flowchart data generated by a large language model based on a diagnostic idea description text is obtained, wherein the flowchart data is used to represent a flowchart including at least one node, and the diagnostic idea description text is a natural language text that describes a diagnostic idea for diagnosing a medical image.

[0034] At step 220 , based on the flowchart data and according to one or more medical images associated with the target human body, an analysis result on the target human body is obtained.

[0035] Through the above method, the image can be analyzed and processed based on the diagnostic idea to obtain the analysis result. Therefore, the generated analysis result can be made more interpretable and a more accurate processing result can be obtained.

[0036] For example, the diagnosis idea description text may be a natural language text describing a diagnosis based on at least one medical purpose, and the analysis result may be an analysis result of an analysis performed with respect to the at least one medical purpose.

[0037] Exemplarily, the method may include obtaining one or more medical images about the target human body, such as acquiring, controlling acquisition, or retrieving, reading, or otherwise obtaining one or more medical images about the target human body from an already acquired image library. Exemplarily, the step of obtaining one or more medical images about the target human body may be before, after, or simultaneously with step 210. As a specific non-limiting example, at least a portion of the one or more medical images may be acquired according to the flowchart data, for example, the flowchart data may indicate which images are read or acquired.

[0038] Exemplarily, the flowchart data may correspond to one or more flowcharts. Exemplarily, the diagnosis idea description text may be a diagnosis idea description text from a user.

[0039] Exemplarily, the at least one node may include at least one image acquisition node. In such an example, at least one medical image among the one or more medical images is acquired based on the at least one image acquisition node.

[0040] Exemplarily, the at least one image acquisition node may correspond to a description of a medical image required to achieve the at least one medical purpose in the diagnostic idea description text. Exemplarily, the method further comprises acquiring at least a portion of the one or more medical images based on the at least one image acquisition node in the flowchart data.

[0041] Exemplarily, the at least one image acquisition node comprises a node for acquiring a medical image of the target human body through at least one image acquisition device. In such an example, the corresponding image acquisition node may be referred to as an image acquisition node.

[0042] Exemplarily, the at least one image acquisition node (e.g., image acquisition node) may indicate at least one of the following parameters: acquisition device type, acquisition parameters, acquisition position. As a specific non-limiting example, based on natural language such as "take a lung CT of a patient", it may be determined that the acquisition device is a CT scanning system, the acquisition position is the lung region of the target human body (and the corresponding scanning coordinate axis or scanning movement range may be further generated), and other parameters such as scanning accuracy, intensity, etc. may be determined.

[0043] Exemplarily, the at least one image acquisition node includes a node for acquiring a medical image related to the target human body from a medical image library. In such an example, the corresponding image acquisition node may be referred to as an image retrieval node.

[0044] Exemplarily, the at least one image acquisition node (e.g., image retrieval node) may indicate at least one of the following parameters: medical image acquisition time, medical image type, medical image parameters, medical image acquisition location. Exemplarily, the medical image acquisition time may correspond to the image acquisition time, for example, the last few days, the last year, every three months for image comparison follow-up, and so on. The medical image type may correspond to the acquisition device, whether to perform contrast imaging, whether to enhance, and so on. The medical image acquisition location may correspond to the lungs, the brain, or other contents that can be understood by those skilled in the art. The medical image parameters may correspond to various medical image parameters that can be understood by those skilled in the art, and as non-limiting examples, may include image clarity, acquisition device version, 2D or 3D, and so on.

[0045] As a further non-limiting example, the flowchart may include first determining whether there are existing images that meet the conditions, and searching if the conditions are met, and controlling the image acquisition device to acquire images if the conditions are not met, etc. For example, corresponding to the natural language of "see if the patient's lung CT has xxx", the existing image library, patient information or other medical databases may be first retrieved or otherwise queried to determine whether there are images that meet the conditions, which may correspond to a conditional judgment node or a retrieval node (or both), and if there are images, they are read, and if not or the conditions are not met (for example, they are not recent, the acquisition accuracy does not meet the requirements, or there are other problems that do not meet the image processing requirements, etc.), the image acquisition step is performed.

[0046] Exemplarily, the at least one node may include at least one image processing node, and the at least one image processing node may be used to process at least one medical image among the one or more medical images to obtain at least one intermediate processing result.

[0047] Exemplarily, the at least one image processing node may correspond to a description of the processing of the medical image and / or one or more variables required to achieve the at least one medical purpose in the diagnosis idea description text.

[0048] Exemplarily, the at least one image processing node may include a node for identifying at least one target in at least one of the one or more medical images. In such an embodiment, the at least one intermediate processing result may include a recognition result for the at least one target. Exemplarily, such a node may be referred to as a recognition node, a target recognition node, etc., and the present disclosure is not limited thereto.

[0049] For example, such an image processing node may correspond to an explicit description in the diagnostic idea description text such as "determine whether there is a plaque", or may correspond to an implicit description that includes "plaque in the blood vessel" but does not explicitly indicate that it is to be identified or judged. For example, the diagnostic idea description text may not explicitly specify that a specific target needs to be identified, but may only mention a specific target in the image, or mention other processing such as comparison and measurement of a specific target in the image. For example, the diagnostic idea description text may correspond to first performing an identification step on the specific target (corresponding to an identification node), and then performing other optional corresponding processing (for example, there may be one or more optional other processing nodes after the identification node).

[0050] As a specific non-limiting example, the diagnostic idea description text may include "measure the stenosis rate based on the plaque in the blood vessel", which does not explicitly indicate the identification of the plaque in the blood vessel. The flowchart generated thereby may first include a vascular plaque identification node, optionally may include an identification node and a segmentation node, and then include a measurement node, and so on. Exemplarily, the at least one image processing node may include a node for measuring at least one scale in at least one of the one or more medical images, and wherein the at least one intermediate processing result includes a measurement result for the at least one scale.

[0051] Continuing with the example of stenosis rate, there may be stenosis rate = [1 - (stenotic artery diameter ÷ normal artery diameter)] x 100%. More specifically, for carotid artery stenosis rate, there may be a NASCET stenosis rate calculation method and an ECST stenosis rate calculation method.

[0052] For the NASCET stenosis rate calculation method, stenosis rate = (normal diameter distal to the stenosis - the narrowest diameter of the stenosis segment) / normal diameter distal to the stenosis. Figure 3A , you can have

[0053]

[0054] The calculation method for ECST stenosis rate is: stenosis rate = (estimated initial diameter at the narrowest diameter of the stenosis segment - the narrowest diameter of the stenosis segment) / estimated initial diameter at the narrowest diameter of the stenosis segment. Figure 3B , you can have

[0055]

[0056] Therefore, for the NASCET stenosis rate calculation method, the values ​​a and b can be measured. For the ECST stenosis rate, c and a can be measured. These measurement values ​​corresponding to specific medical purposes or medical analysis purposes or the corresponding measurement positions can be pre-stored or read through an interface with a predetermined medical resource library. Alternatively, these measurement values ​​can be obtained by asking the user.

[0057] Exemplarily, the at least one node may include a node for calculating at least a portion of the at least one intermediate result according to a rule corresponding to the diagnostic idea description text to obtain a calculated intermediate result. Exemplarily, these image processing nodes may be referred to as calculation nodes. Exemplarily, the rule corresponding to the diagnostic idea description text may be a rule defined in the diagnostic idea description text, such as an explicitly or implicitly defined rule, or may correspond to the diagnostic idea in the diagnostic idea description text in other ways.

[0058] Continuing with the example of stenosis rate, the calculation node may determine the corresponding intermediate result based on a calculation formula, calculation rule, etc. that is pre-stored, read from a medical resource library through a corresponding interface, or otherwise obtained. Exemplarily, the flowchart may include a node that outputs the intermediate result, such as a node that controls a display system to output. Alternatively, the intermediate result may not be output to the user but may only be stored or archived, provided to the user only when the user actively requests it, or output to the user together with the final result or final analysis report, and the present disclosure is not limited thereto.

[0059] Exemplarily, generating an image processing node including a computing node may include generating a node that itself includes computing capabilities. Alternatively, the node itself may not have computing capabilities, but may call other computing resources that have computing capabilities. Exemplarily, the measurement and calculation functions may be combined into a single node, for example, a single node has the ability to measure and calculate stenosis, or it may also be a separate node, and the present disclosure is not limited thereto.

[0060] Exemplarily, the at least one node may include at least one analysis node, and the at least one analysis node is used to obtain the analysis result for the target human body based on the output of at least one other node in the flowchart. For example, the analysis node may correspond to a statement in the diagnostic idea description text that explicitly indicates the determination of the analysis result, such as "determine the lesion grade", or may correspond to a statement in the diagnostic idea description text that implicitly mentions the analysis result, such as "based on the lesion grade, perform xx treatment" or "if the grade result is XX, then there is a possibility of XX", etc.

[0061] Exemplarily, the analysis result may include a classification result for at least one identified target in the one or more medical images.

[0062] Result grading can utilize the understanding ability of the large language model. For example, result grading can include fuzzy grading, or the diagnostic idea description text can include fuzzy instructions, such as "when xxxx, it is likely to be malignant". The diagnostic idea description text can contain multiple judgment conditions, such as multiple fuzzy judgment conditions, for example, "the lesion length is greater than xx cm or there is xx tissue around it, which may indicate that the lesion is malignant" or "if the stenosis rate is too high and the surrounding xx tissue presents xx morphology, then there is a high probability that xx exists". The large language model can understand these judgment conditions separately, judge them separately, and give the final judgment result. Exemplarily, the large language model can give the judgment result and judgment probability corresponding to each judgment condition, and integrate based on each judgment result to give the final analysis result. Exemplarily, the analysis result may include the probability of at least one disease or condition related to the target human body.

[0063] Continuing with the previous example, for example, the "too high" in "the stenosis rate is too high" can be a fuzzy judgment, and "xx presents xx morphology" can also have a certain morphological approximation range, and illustratively, the large language model can output "the stenosis rate is 60%, which is relatively high, the morphology of xx tissue is approximately xx shape, and the difference from the standard xx morphology is xx%. Based on the above judgment conditions, the probability of the existence of xx risk is xx", and so on.

[0064] As described above in conjunction with some embodiments, the flowchart may include a process corresponding to the judgment step. For example, if it is determined that the current image is insufficient to obtain the desired target analysis result, the user may be prompted to obtain an additional image. Alternatively, the judgment step may correspond to a judgment branch in the analysis logic, such as "when conditions A and B are met, symptom C may exist; when A is met but B is not met, further determine whether condition D is met, and if so, symptom E may exist", and so on.

[0065] Exemplarily, the at least one node may include at least a first node and a second node, and the flowchart data includes a first edge connecting the first node and the second node, wherein the first edge is configured with a first determination condition.

[0066] In other embodiments, instead of being set on the edge, the determination condition can be set as a separate node. Exemplarily, the determination condition can be generated based on the diagnostic idea description text. For example, the determination condition can correspond to "see if the stenosis rate reaches xx value, if so, see if there is xx problem in the adjacent tissue" in the diagnostic idea description text.

[0067] According to the embodiments of the present disclosure, it is possible to obtain an analysis process based on defined diagnostic ideas based on the diagnostic experience and ideas from the user, with the help of the language understanding ability of the large model, and utilize the image processing capabilities of the software to obtain interpretable results that can gain the trust of the user.

[0068] For example, the diagnostic idea description text may be a diagnostic idea description text from a user. The user here may be a doctor, an expert, a professor, etc. Exemplarily, the intermediate variable may include a segmentation result, a recognition result (e.g., a recognition result that is not directly related to the disease to be recognized, but a recognition result of other related signs), etc. Exemplarily, the processing operation may include segmenting, enhancing, converting, detecting, recognizing, etc. the image.

[0069] According to some embodiments, the diagnostic idea description text may come from a first user, and the analysis result may be used to present to the first user. According to some embodiments, the diagnostic idea description text may come from a first user, and the analysis result may be used to present to a second user different from the first user. For example, the first user and the second user may be different doctors, doctors from different hospitals, professors and students or interns, and so on. Thus, a tool is provided that can effectively draw on the experience and wisdom of doctors, for example, in places where doctors' resources, knowledge, and qualifications are relatively limited, interpretable image analysis can also be achieved.

[0070] According to some embodiments, the at least one diagnostic idea description text may come from a user in a predetermined user pool. For example, a qualified doctor resource pool may be maintained to achieve the effect of sharing a doctor knowledge base, so that even in scenarios where medical resources are limited, doctor knowledge is limited, or the current doctor wants to view or refer to the advice and diagnostic ideas of other peers, an interpretable analysis result can be obtained.

[0071] According to some embodiments, the method may further include obtaining a processing target for the one or more medical images, wherein the processing target includes a name of a disease to be detected or a symptom related to the target human body. In such an embodiment, the analysis and processing of the one or more medical images may also be based on the processing target.

[0072] According to some embodiments, the diagnostic idea description text may include a processing target description text. In such an example, the flowchart data may be generated by a large language model based on at least the diagnostic idea description text and the processing target description text. For example, a doctor may orally say "I want to see if it is xx disease", and input it into the large language model together with the diagnostic idea, thereby obtaining the corresponding flowchart data.

[0073] Since imaging diagnosis still requires a lot of medical knowledge and reasoning, doctors in many hospitals have their own diagnostic experience and ideas when diagnosing diseases. Therefore, if the imaging diagnosis software provided allows doctors to define diagnostic ideas, it will greatly help doctors in diagnosis.

[0074] By generating a flowchart based on the text describing the diagnostic ideas, we can use the diagnostic ideas provided by doctors, and use AI to provide anatomical cognition, analysis and execution of the doctor's diagnostic process, and typical data collection. The process can be customized to adapt to the needs of different doctors and can be executed automatically. Specific data can also be collected for scientific research needs. Such medical imaging AI-assisted tools will make diagnosis more explainable. The problem of medical explainability in AI analysis, which was once questioned by doctors, can also be solved by allowing doctors to define explainability themselves.

[0075] According to one or more embodiments, the method may include obtaining voice information associated with the collected image, the voice information including the diagnostic idea. Then, the voice information may be recognized to obtain a diagnostic idea description text. The diagnostic idea description text may be input into a large language model to obtain corresponding flowchart data, and the image may be processed based on the flowchart data. The method may include outputting one or more intermediate results and a final analysis result.

[0076] It is to be understood that throughout this article, any large language model known to those skilled in the art or appearing after the filing date of this application can be adopted, and those skilled in the art will understand its training method and interface calling mode, and will not be repeated here. In addition, it is to be understood that throughout this article, any reference to "large language model" can refer to the same large language model, or to different large language models, including different iterative versions of the same large language model, or multiple large language models trained independently. As a specific non-limiting example, different large language models can be used for diagnostic thinking texts from different users, for example, different large language models are selected according to region, language, user preference, network resources, resource usage of each large language model, etc. In another embodiment, the same model can be used to extract texts from different users and generate flow chart data. It is to be understood that the present disclosure is not limited to this.

[0077] For example, the pre-written big model interface can be used to input the diagnostic ideas of doctors or other users into the big model. As a specific non-limiting example, the prompt text input to the model can be in the form of "The following is the diagnostic idea text from the doctor. Please generate flowchart data based on the doctor's diagnostic idea for appropriate processing of the image to comply with the instructions of the diagnostic idea text", but it can be understood that the above is only an example and the present disclosure is not limited to this. The big language model can be integrated with the image processing module or separated from it, and the present disclosure is not limited to this.

[0078] It is understood that although the various operations are depicted in the drawings as being in a particular order, this should not be understood as requiring that these operations must be performed in the particular order shown or in a sequential order, nor should it be understood as requiring that all the operations shown must be performed to obtain the desired results. For example, two steps described in order herein may be performed in reverse order, or may be performed concurrently. For another example, one or more steps in the various embodiments of the present disclosure may be omitted.

[0079] In addition, it is understood that the method for predicting or determining data involved in one or more embodiments of the present disclosure is not a method for a doctor to directly determine the diagnosis result, but involves data processing or information processing in the medical process, and the data processing result can be used for reference by the doctor, thereby assisting the doctor's medical operation. It is understood that the information processing method, data prediction method, determination method, decision-making method, etc. involved in one or more embodiments of the present disclosure are executed by a computer or a device including a computer.

[0080] It is to be understood that throughout the present disclosure, an image or an image sequence may be or may include two-dimensional image data, or may be or include three-dimensional image data. An image or an image sequence may be image data that is directly acquired and stored or otherwise sent to a terminal device for use by a user. An image or an image sequence may also be processed image data after various image processing. An image or an image sequence may undergo other analysis processes (e.g., an analysis process for whether there are pathological features or lesions) and include analysis results (e.g., circling of an area of ​​interest, segmentation results of a tissue, etc.). It is to be understood that the present disclosure is not limited thereto.

[0081] According to some embodiments, obtaining one or more medical images of the target human body may include: selecting images from a medical imaging device. The medical imaging device may include an X-ray device, a CT scanning device, an MRI scanning device, an ultrasound device, or a digital pathology slice scanning device.

[0082] According to some embodiments, the flowchart data may further include a node for generating an image analysis report, the image analysis report including, for example, image processing results, diagnosis conclusions, and recommended treatment plans. For example, the analysis report may be generated according to an analysis report requirement or analysis report template input by a user, predetermined, or otherwise obtained.

[0083] Figure 3B is a flowchart corresponding to the flowchart data according to some non-limiting exemplary embodiments of the present disclosure. Figure 3B As shown, the flowchart may start with, for example, an image acquisition process, such as determining whether to retrieve an image or collect an image or both. If an image needs to be retrieved, the image library is searched through a corresponding interface. Afterwards, it may be determined whether the retrieved image meets the conditions, and if not, the process of collecting an image may be triggered.

[0084] It will be understood that the above are merely examples, and in other examples, such as the diagnostic idea description text corresponding to "perform xx processing on these images" that explicitly or implicitly does not require additional images, the steps of retrieving, collecting or otherwise acquiring images may be omitted.

[0085] Return to reference Figure 3B For the image that has been obtained, the flowchart may include nodes for processing the image, for example, the image may be enhanced, segmented, etc. It is understandable that in the case where there are multiple processing steps, the multiple processing steps may correspond to a single node, for example, a single node has multiple computing processing capabilities or has an interface for calling multiple computing processing capabilities; or, the multiple processing steps may correspond to corresponding nodes respectively.

[0086] Continue to refer Figure 3B, the flowchart data may include nodes for target identification, such as identifying a first type of object, such as a blood vessel, tissue, lesion, plaque, etc. The flowchart data may include nodes for measuring based on the identification results. The flowchart data may also include nodes for determining intermediate results based on the measurement, such as nodes having computing capabilities or capable of calling computing capabilities. Exemplarily, the flowchart data may also include nodes for comparing the first intermediate result with the first predetermined criterion, as described above, where the comparison may be a fuzzy comparison, such as corresponding to the fuzzy instructions "larger", "smaller", "having xx shape", "suspected existence of xx", etc. in the diagnostic idea description text.

[0087] As an optional exemplary process, the flowchart may also include additional judgment nodes for judging whether additional processing steps are required. If additional processing steps are required, such as if the judgment conditions in the diagnostic idea description text are not met, the predetermined accuracy or credibility is not reached, etc., additional image processing, additional target recognition (such as identifying the second type of object), additional measurement and calculation (not shown), determination of the second intermediate result, comparison of the second intermediate result with the second predetermined criterion, etc. may be performed. If no additional processing steps are required, or the additional processing steps have been completed, the node for outputting the analysis result may be proceeded to. The analysis result may also include, for example, analysis logic, analysis process, result probability, etc. As a specific non-limiting example, the flowchart node for judging whether additional processing steps are required may correspond to, for example, the following diagnostic idea description text: If the stenosis rate is greater than 90%, it is considered that xxxx exists; if the stenosis rate is between 60%-90%, and there is xx around, it is considered that xxxx... It is understandable that there may be more than one branch judgment of the additional processing step, and the generated flowchart may include more or less additional processing step judgment nodes, additional image processing nodes, additional target recognition nodes, etc., and the present disclosure is not limited thereto. With the help of the fuzzy language understanding ability of the large model, the doctor's experience judgment and complex logic can be effectively used to generate a corresponding flowchart, so that the medical image can be processed and analyzed in a logical and interpretable process to obtain a reliable analysis result.

[0088] Figure 44 is a schematic block diagram illustrating an image analysis device 400 according to an exemplary embodiment. The image analysis device 400 may include a flowchart obtaining unit 410 and an analyzing unit 420. The flowchart obtaining unit 410 may be used to obtain flowchart data generated by a large language model based on a diagnostic idea description text, wherein the flowchart data is used to represent a flowchart including at least one node, and the diagnostic idea description text is a natural language text describing a diagnostic idea for diagnosis based on a medical image. The analyzing unit 420 may be used to obtain an analysis result about the target human body based on the flowchart data and one or more medical images associated with the target human body.

[0089] According to some embodiments, the apparatus may further include an image acquisition unit for acquiring a desired medical image, such as retrieving a medical image, or controlling one or more acquisition devices to acquire a medical image, etc. The image acquisition unit may correspond to the image acquisition step in various embodiments of the present disclosure, and will not be described in detail herein.

[0090] According to some embodiments, the device may further include a user interface module, which is used to receive a diagnosis idea description text and / or a processing target, etc. input by a user in a voice form, a text form or other forms.

[0091] It should be understood that Figure 4 The various modules of the apparatus 400 shown in FIG. 4 can be used in conjunction with the reference Figure 2 The steps in the method 200 described above correspond to each other. Therefore, the operations, features and advantages described above for the method 200 and its variants are also applicable to the device 400 and the modules included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0092] According to an embodiment of the present disclosure, a computing device is also disclosed, including a memory, a processor, and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement the steps of the image analysis method according to the embodiment of the present disclosure and its variant examples.

[0093] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium is also disclosed, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the image analysis method according to the embodiment of the present disclosure and its variant examples are implemented.

[0094] According to an embodiment of the present disclosure, a computer program product is also disclosed, including a computer program, wherein when the computer program is executed by a processor, the steps of the image analysis method according to the embodiment of the present disclosure and its variant examples are implemented.

[0095] Although specific functions are discussed above with reference to specific modules, it should be noted that the functions of the various modules discussed herein may be divided into multiple modules, and / or at least some functions of multiple modules may be combined into a single module. The specific module discussed herein performs an action including the specific module itself performing the action, or alternatively the specific module calls or otherwise accesses another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, the specific module that performs the action may include the specific module itself that performs the action and / or another module that the specific module calls or otherwise accesses to perform the action. For example, the various modules or units described in accordance with one or more embodiments of the present disclosure may be combined into a single module or unit in some embodiments. For another example, two or more modules or units may be described in parallel in one or more embodiments of the present disclosure, while in some other embodiments, there may be one or more inclusion relationships between these modules and units. As used herein, the phrase "entity A initiates action B" or "entity A causes action B to be performed" may refer to entity A issuing an instruction to perform action B, but entity A itself does not necessarily perform the action B. For example, the phrase “the display module causes display ..." may mean that the display module instructs a display (not shown) or other possible display device to display, while the display module itself does not need to perform the action of “displaying”.

[0096] It should also be understood that various technologies can be described herein in the general context of software hardware elements or program modules. The various modules described above with respect to FIG. 3 can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program codes / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuits. For example, in some embodiments, one or more of the modules or units described according to one or more embodiments of the present disclosure can be implemented together in a system on chip (System on Chip, SoC). SoC can include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and can optionally execute the received program code and / or include embedded firmware to perform functions.

[0097] According to one aspect of the present disclosure, a computing device is provided, which includes a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any method embodiment described above.

[0098] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method embodiment described above are implemented.

[0099] According to one aspect of the present disclosure, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the steps of any method embodiment described above are implemented.

[0100] In the following, combined Figure 5 Illustrative examples of such a computer device, non-transitory computer-readable storage medium, and computer program product are described.

[0101] Figure 5 An example configuration of a computer device 500 that can be used to implement the methods described herein is shown. For example, Figure 1 The server 120 and / or the client device 110 shown in FIG. 5 may include an architecture similar to the computer device 500. The above-mentioned image analysis device / apparatus may also be implemented in whole or at least in part by the computer device 500 or a similar device or system.

[0102] Computer device 500 can be a variety of different types of devices, such as a server of a service provider, a device associated with a client (e.g., a client device), a system on a chip, and / or any other suitable computer device or computing system. Examples of computer device 500 include, but are not limited to, a desktop computer, a server computer, a laptop or netbook computer, a mobile device (e.g., a tablet computer, a cellular or other wireless phone (e.g., a smart phone), a notepad computer, a mobile station), a wearable device (e.g., glasses, a watch), an entertainment device (e.g., an entertainment appliance, a set-top box communicatively coupled to a display device, a game console), a television or other display device, a car computer, and the like. Therefore, computer device 500 can range from a full-resource device with a large amount of memory and processor resources (e.g., a personal computer, a game console) to a low-resource device with limited memory and / or processing resources (e.g., a traditional set-top box, a handheld game console).

[0103] Computer device 500 may include at least one processor 502, memory 504, communication interface(s) 506, a display device 508, other input / output (I / O) devices 510, and one or more mass storage devices 512, all capable of communicating with one another, such as via a system bus 514 or other appropriate connections.

[0104] The processor 502 may be a single processing unit or multiple processing units, all of which may include a single or multiple computing units or multiple cores. The processor 502 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, the processor 502 may be configured to obtain and execute computer-readable instructions stored in the memory 504, mass storage device 512, or other computer-readable media, such as program code of an operating system 516, program code of an application program 518, program code of other programs 520, and the like.

[0105] The memory 504 and the mass storage device 512 are examples of computer-readable storage media for storing instructions that are executed by the processor 502 to implement the various functions described above. For example, the memory 504 may generally include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.). In addition, the mass storage device 512 may generally include a hard drive, a solid-state drive, a removable medium, including external and removable drives, a memory card, a flash memory, a floppy disk, an optical disk (e.g., a CD, a DVD), a storage array, a network attached storage, a storage area network, etc. The memory 504 and the mass storage device 512 may all be collectively referred to herein as memory or computer-readable storage media, and may be a non-transitory medium capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by the processor 502 as a specific machine configured to implement the operations and functions described in the examples herein.

[0106] A number of program modules may be stored on mass storage device 512. These programs include operating system 516, one or more application programs 518, other programs 520, and program data 522, and they may be loaded into memory 504 for execution. Examples of such applications or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing components / functionality including method 200 (including any suitable steps of method 200) and / or other embodiments described herein.

[0107] Although in Figure 5504 of the computer device 500, but modules 516, 518, 520, and 522, or portions thereof, may be implemented using any form of computer-readable media accessible by the computer device 500. As used herein, "computer-readable media" includes at least two types of computer-readable media, namely, computer storage media and communication media.

[0108] Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented by any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data. Computer storage media includes but is not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by a computer device.

[0109] In contrast, communication media may embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism. Computer storage media as defined herein does not include communication media.

[0110] The computer device 500 may also include one or more communication interfaces 506 for exchanging data with other devices, such as through a network, direct connection, etc., as discussed above. Such communication interfaces may be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), a wired or wireless (such as IEEE802.11 wireless LAN (WLAN)) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a wireless network interface. TM The communication interface 506 may facilitate communication within a variety of network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. The communication interface 506 may also provide for communication with external storage devices (not shown) such as in a storage array, network attached storage, storage area network, etc.

[0111] In some examples, a display device 508 such as a monitor may be included for displaying information and images to the user. Other I / O devices 510 may be devices that receive various inputs from the user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and the like.

[0112] Although the present disclosure has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative and exemplary rather than restrictive; the present disclosure is not limited to the disclosed embodiments. Variations to the disclosed embodiments will be understood and implemented by those skilled in the art in practicing the claimed subject matter by studying the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps that are not listed, and the word "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A computer-implemented image analysis method comprising: Obtaining flowchart data generated by a large language model based on a diagnostic idea description text, wherein the flowchart data is used to represent a flowchart including at least one node, and the diagnostic idea description text is a natural language text describing a diagnostic idea for diagnosing a medical image; as well as Based on the flowchart data, an analysis result on the target human body is obtained according to one or more medical images associated with the target human body.

2. The method according to claim 1, wherein: The at least one node includes at least one image obtaining node, and wherein at least one medical image of the one or more medical images is obtained based on the at least one image obtaining node.

3. The method according to claim 2, wherein: The at least one image acquisition node includes a node for acquiring a medical image of the target human body through at least one image acquisition device.

4. The method according to claim 3, wherein: The at least one image acquisition node indicates at least one of the following parameters: acquisition device type, acquisition parameters, and acquisition location.

5. The method according to any one of claims 2 to 4, wherein: The at least one image acquisition node includes a node for acquiring a medical image related to the target human body from a medical image library.

6. The method according to claim 5, wherein: The at least one image acquisition node indicates at least one of the following parameters: medical image acquisition time, medical image type, medical image parameters, and medical image acquisition location.

7. The method according to any one of claims 1 to 6, wherein: The at least one node includes at least one image processing node, and the at least one image processing node is used to process at least one medical image among the one or more medical images to obtain at least one intermediate processing result.

8. The method according to claim 7, wherein: The at least one image processing node comprises a node for identifying at least one target in at least one of the one or more medical images, and wherein the at least one intermediate processing result comprises a recognition result for the at least one target.

9. The method according to claim 7 or 8, wherein: The at least one image processing node comprises a node for measuring at least one scale in at least one of the one or more medical images, and wherein the at least one intermediate processing result comprises a measurement result for the at least one scale.

10. The method according to any one of claims 7 to 9, wherein: The at least one node includes a node for calculating at least a part of the at least one intermediate result according to a rule corresponding to the diagnosis idea description text to obtain a calculated intermediate result.

11. The method according to any one of claims 1 to 10, wherein: The at least one node includes at least one analysis node, and the at least one analysis node is used to obtain an analysis result for the target human body based on an output of at least one other node in the flow chart.

12. The method according to claim 11, wherein: The analysis result includes a classification result for at least one identified target in the one or more medical images.

13. The method according to claim 11 or 12, wherein: The analysis result includes a probability of at least one disease or condition associated with the target human body.

14. The method according to any one of claims 1 to 13, wherein: The at least one node includes at least a first node and a second node, and the flowchart data includes a first edge connecting the first node and the second node, wherein the first edge is configured with a first determination condition.

15. An image analysis device, comprising: A flowchart obtaining unit, used to obtain flowchart data generated by a large language model based on a diagnostic idea description text, wherein the flowchart data is used to represent a flowchart including at least one node, and the diagnostic idea description text is a natural language text describing a diagnostic idea for diagnosing according to a medical image; as well as An analysis unit is used to obtain an analysis result on the target human body based on the flowchart data and according to one or more medical images associated with the target human body.

16. A computing device comprising: a memory, a processor and a computer program stored on the memory, The processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 14.

17. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.

18. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.

Images