Spinal arthritis analysis method and device based on learning model

Abstract

The invention relates to a spinal arthritis analysis method and device based on a learning model. The method belongs to the field of image recognition in a scene of applying Agent based on a large language model to medical image analysis, and comprises the following steps: performing primary matching on a description text vector corresponding to a target part in a magnetic resonance image sequence to be recognized and a description text vector in each piece of preset part information, and determining similar part information; a description text vector is generated through an Agent of a large language model and can be adaptively integrated in an existing axSpA auxiliary diagnosis agent based on an expert model and the large language model so as to improve the accuracy of a medical image recognition result. Therefore, the preset part information with high similarity with the target part can be quickly screened out. And the local magnetic resonance image is used for image similarity calculation, so that the recognition accuracy of the generated target part is improved.

Description

Background Art

[0002] Magnetic Resonance Imaging (MRI) is a non-invasive medical imaging technique that uses strong magnetic fields and radio waves to generate detailed images of the internal structures of the human body. Magnetic Resonance Imaging (MRI) is also a tomographic imaging technique. MRI provides a detailed view of the internal structures of the human body by generating multiple two-dimensional tomographic images. Compared with CT images, MRI images have an important position in medical imaging due to their higher soft tissue contrast, radiation-free risk, and multi-planar imaging capabilities. These images can display structures such as bones, organs, soft tissues, and blood vessels, and play an important role in the discovery and monitoring of specific tissue structure sites.

[0003] Generally, the specific number of tomographs in a set of magnetic resonance images can vary according to different factors, including the scanned site, the required image resolution, clinical needs, and the type of magnetic resonance machine used, etc. In recent years, deep learning and neural networks have made remarkable progress in medical image analysis, especially in the automatic detection and localization of lesions in magnetic resonance images. For example, in magnetic resonance image analysis, CNN can be used to detect and classify various structural sites, etc.

[0004] At the same time, due to the fact that large language models (LLMs) have hundreds of millions or even more parameters, they are able to capture complex language structures and patterns and demonstrate powerful capabilities in various tasks, such as text generation, question answering, translation, etc. An Agent combined with a large language model can then utilize its profound understanding of language to perform more complex and user-friendly interactions and automatically complete tasks. Agents based on large language models are gradually being used in the scenario of medical image analysis.

[0005] Although convolutional neural networks (CNNs) have made remarkable progress in medical image analysis, due to the defects of CNNs themselves, such as the small number of training samples for certain lesions, the presence of noise in the training data labels, insufficient or excessive model training, etc., deep learning and neural networks may still be inaccurate when detecting and classifying various lesions. Furthermore, in magnetic resonance images, the image regions corresponding to the target sites identified by deep learning and neural networks have errors, which can easily mislead users. Summary of the Invention

[0006] In view of the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] According to one aspect of the present invention, there is provided a method for analyzing spondyloarthritis based on a learning model, the method comprising the following steps:

[0008] Cluster the multiple images to be recognized in the magnetic resonance image sequence to be recognized to generate at least one target part cluster; the images to be recognized are local magnetic resonance images corresponding to the target part ROI region; the ROI region is obtained through a deep learning model;

[0009] Use the description text vector corresponding to the target part in the magnetic resonance image sequence to be recognized to perform a primary match with the description text vector in each preset part information to determine at least one similar part information; each preset part information includes the description text vector corresponding to the part and the local magnetic resonance image sequence of the ROI region corresponding to the part;

[0010] Use the images to be recognized in each target part cluster to perform a secondary match with the local magnetic resonance image sequences in each similar part information to generate the recognition accuracy information of the target part in each target part cluster;

[0011] The secondary match includes:

[0012] According to the tomographic interval H of the first part magnetic resonance image sequence 1 and the tomographic interval H of the second part magnetic resonance image sequence 2 , determine the magnetic resonance image to be matched corresponding to each magnetic resonance image in the first part magnetic resonance image sequence in the second part magnetic resonance image sequence; among the local magnetic resonance image sequences in the target part cluster and the local magnetic resonance image sequences in the similar part information, the one with the smaller number of images is the first part magnetic resonance image sequence, and the one with the larger number of images is the second part magnetic resonance image sequence; wherein, the serial number interval of the magnetic resonance image to be matched corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i , A 1 i , A 2 i respectively satisfy the following conditions:

[0013]

[0014] wherein, A 2 max is the largest image serial number in the second part magnetic resonance image sequence;

[0015] According to the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched, generate the accuracy P of the target part in the target part cluster; P satisfies the following conditions:

[0016]

[0017] wherein, and the similarities between the i-th magnetic resonance image in the magnetic resonance image sequence of the first part and the corresponding first and last magnetic resonance images to be matched respectively; n is the total number of magnetic resonance images in the magnetic resonance image sequence of the first part;

[0018] Generate the recognition accuracy information of the target parts in the target part cluster according to the accuracy of the target parts in the target part cluster and the accuracy threshold.

[0019] Further, generating the recognition accuracy information of the target parts in the target part cluster according to the accuracy of the target parts in the target part cluster and the accuracy threshold includes:

[0020] If P > Y1, mark all the images to be recognized in the target part cluster with the first color;

[0021] If Y1 > P > Y2, mark all the images to be recognized in the target part cluster with the second color;

[0022] If P < Y2, mark all the images to be recognized in the target part cluster with the third color; Y1 and Y2 are the first accuracy threshold and the second accuracy threshold respectively, and Y1 > Y2.

[0023] Further, performing clustering processing on multiple images to be recognized in the magnetic resonance image sequence to be recognized to generate at least one target part cluster, including:

[0024] If the overlap degree of the two rectangular bounding boxes corresponding to the target parts in two adjacent frames of the images to be recognized is greater than the overlap degree threshold, determine that the two images to be recognized belong to the same target part cluster.

[0025] Further, the description text vector is obtained according to the following method:

[0026] Input the description text related to the target part into a preset deep learning network to generate the description text vector corresponding to the target part.

[0027] Further, the obtaining of the similarity between each magnetic resonance image in the magnetic resonance image sequence of the first part and the corresponding magnetic resonance image to be matched includes the following steps:

[0028] If the size specifications of two magnetic resonance images are the same, use the structural similarity algorithm to obtain the similarity between the two magnetic resonance images.

[0029] Further, the obtaining of the similarity between each magnetic resonance image in the magnetic resonance image sequence of the first part and the corresponding magnetic resonance image to be matched further includes the following steps:

[0030] If the size specifications of two magnetic resonance images are different, the size specifications of the two magnetic resonance images need to be adjusted to be the same before calculating the similarity.

[0031] Further, the size specifications of the ROI region of the image to be recognized are the same as those of the ROI region in the preset part information.

[0032] Further, Y1 = 0.9 and Y2 = 0.8.

[0033] According to the second aspect of the present invention, a spondyloarthritis analysis device based on a learning model, the device includes:

[0034] An identification clustering model for clustering a plurality of images to be recognized in a magnetic resonance image sequence to be recognized, generating at least one target part cluster; the image to be recognized is a local magnetic resonance image corresponding to the ROI region of the target part; the ROI region is obtained through a deep learning model;

[0035] A vector matching model for performing primary matching between the description text vector corresponding to the target part in the magnetic resonance image sequence to be recognized and the description text vector in each preset part information to determine at least one similar part information; each preset part information includes the description text vector of the corresponding part and the local magnetic resonance image sequence of the ROI region of the corresponding part;

[0036] An image matching model for performing secondary matching between the images to be recognized in each target part cluster and the local magnetic resonance image sequence in each similar part information, generating the recognition accuracy information of the target part in each target part cluster;

[0037] The secondary matching includes:

[0038] According to the slice interval H of the first part magnetic resonance image sequence 1 and the slice interval H of the second part magnetic resonance image sequence 2 , determining the magnetic resonance image to be matched corresponding to each magnetic resonance image in the first part magnetic resonance image sequence in the second part magnetic resonance image sequence; in the local magnetic resonance image sequence in the target part cluster and the local magnetic resonance image sequence in the similar part information, the one with fewer images is the first part magnetic resonance image sequence, and the one with more images is the second part magnetic resonance image sequence; wherein, the serial number interval of the magnetic resonance image to be matched corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i , A 1 i , A 2 i respectively satisfy the following conditions:

[0039]

[0040] Among them, A 2 max is the largest image serial number in the second - part magnetic resonance image sequence;

[0041] According to the similarity between each magnetic resonance image in the first - part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched, generate the accuracy P of the target part in the target part cluster; P satisfies the following conditions:

[0042]

[0043] Among them, and are respectively the similarities between the i - th magnetic resonance image in the first - part magnetic resonance image sequence and the corresponding first and last magnetic resonance images to be matched; n is the total number of magnetic resonance images in the first - part magnetic resonance image sequence;

[0044] According to the accuracy of the target part in the target part cluster and the accuracy threshold, generate the recognition accuracy information of the target part in the target part cluster.

[0045] Furthermore, it further includes a human - machine interaction module, and the human - machine interaction module is used to obtain operation instructions.

[0046] The present invention has at least the following beneficial effects:

[0047] In the present invention, on the basis of the images identified by deep learning and neural network, re - recognition verification can be carried out to improve the recognition accuracy of the target part in the magnetic resonance image. For example, this solution can be applied to the scenario of confirming the recognition accuracy of the target part in the magnetic resonance image of axial spondyloarthritis (axSpA). Specifically, in this solution, first, the description text vector corresponding to the target part is used to perform vector matching with the preset part information in the preset matching library to quickly screen out the preset part information with high similarity to the target part. Then, the local magnetic resonance image sequence corresponding to each preset part information is used to calculate the image similarity with multiple images to be recognized corresponding to the target part, and finally, the recognition accuracy of the target part is generated. Thus, through the double verification of the description text vector and the magnetic resonance image, it can be further determined the recognition accuracy of the target part in the magnetic resonance image after being identified by deep learning and neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0049] Figure 1 It is a flowchart of a spondyloarthritis analysis method based on a learning model provided by an embodiment of the present invention;

[0050] Figure 2 It is a flowchart for obtaining a description text vector provided by an embodiment of the present invention;

[0051] Figure 3 It is a schematic structural diagram of an axSpA auxiliary diagnosis intelligent agent based on an expert model and a large language model provided by an embodiment of the present invention. Detailed implementation manners

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0053] As a possible embodiment of the present invention, as Figure 1 shown, a spondyloarthritis analysis method based on a learning model is provided, and the method includes the following steps:

[0054] S100: Cluster a plurality of images to be recognized in the magnetic resonance image sequence to be recognized to generate at least one target part cluster. The images to be recognized are local magnetic resonance images corresponding to the target part ROI (Region of Interest). The ROI region is obtained through a deep learning model.

[0055] The magnetic resonance image sequence to be recognized in this embodiment can be an image sequence that has been recognized by existing deep learning and neural networks. Among them, each frame of magnetic resonance image indicates that the ROI region corresponding to the target site has been marked. Of course, it can also be that the image region corresponding to the target site is marked using an existing segmentation network. Since the solution in this embodiment can be used to further identify and confirm the lesion regions (such as axSpA and subchondral bone marrow edema, etc.) in magnetic resonance images to improve the accuracy of the recognition results, and can also be used to further identify and confirm the images corresponding to certain organ tissues (such as ligaments, muscles, heart, and blood vessels, etc.), so as to conduct intensive training for beginners in medical image analysis to recognize the corresponding parts. Therefore, the target site in this embodiment can be either some lesion sites or some organ tissue sites.

[0056] Specifically, S100 includes:

[0057] S101: If the overlap degree of the two rectangular bounding boxes corresponding to the target site in two adjacent frames of the images to be recognized is greater than the overlap degree threshold, it is determined that the two images to be recognized belong to the same target site cluster.

[0058] Generally, whether it is a lesion site or some normal organs, they are all three-dimensional structures, which have a certain correlation in space, and the structures before and after are also slowly and continuously changing, without sudden changes. At the same time, when MRI performs tomographic scanning, in order to more clearly reflect the corresponding part structure, the tomographic interval will be set correspondingly. Therefore, usually for an organizational structure, there will be a large overlap degree in two adjacent frames of magnetic resonance images, while there is basically no overlap degree between different organizational structures.

[0059] Therefore, in this embodiment, each image to be recognized can be placed in the same coordinate system, and then the overlap degree of the two rectangular bounding boxes corresponding to the target site in two adjacent frames of images is calculated in turn. Whether they are the same target site is judged by the overlap degree, and then all the images to be recognized corresponding to the same target site are placed in the same target site cluster.

[0060] S200: Use the description text vector corresponding to the target site in the magnetic resonance image sequence to be recognized to perform a primary match with the description text vector in each preset site information, and determine at least one similar site information. Each preset site information includes the description text vector of the corresponding site and the local magnetic resonance image sequence of the ROI region of the corresponding site.

[0061] The description text vector is obtained according to the following method:

[0062] S201: Input the description text related to the target site into a preset deep learning network to generate the description text vector corresponding to the target site.

[0063] In this step, an existing deep learning network can be used to extract semantic features in the description text to generate a description text vector. Specifically, corresponding methods in the field of natural language processing can be used for semantic extraction, such as using the BERT (Bidirectional Encoder Representations from Transformers) model or the Doc2Vec model.

[0064] Such as Figure 2 As shown, in the present invention, the description text corresponding to each part can be either descriptive text information about the shape attributes of the corresponding part in the MRI image. If the part is a lesion, it can also be the corresponding clinical symptoms caused by the lesion, or the text information of its corresponding treatment plan. Specifically, both the clinical symptoms and the treatment plan can be obtained through information such as medical records and chief complaints. Or the text information in its corresponding examination materials, such as blood routine, erythrocyte sedimentation rate, imaging data, and CRP (C-Reactive Protein) and other examination information. These examination information can be input into an existing large language model, and through the information processing ability of the large language model, they can be sorted into a text information in a preset standard format. In order to improve the accuracy of the above text sorting, relevant corpora in this field can also be used to retrain the general large language model to form a corresponding dedicated model to replace the general large language model.

[0065] This part can be composed of a Prompt constructor and a large language model. The large language model (either the API interface of LLM, such as GPT-4, or an open-source large language model can be deployed locally, such as ChatGLM) is responsible for analyzing and sorting the above information. The Prompt constructor is responsible for preprocessing the input of the large language model to better stimulate the ability of the large language model.

[0066] Such as Figure 2 As shown, in addition, due to the differences in modalities in the above materials (such as the existence of images, tables, and texts), when sorting the materials, in order to obtain information of different modalities more accurately, an expert model can be configured. The expert model includes small dedicated models corresponding to the information configuration of each modality. The small dedicated model can be an existing analysis model for relevant modality information. The present invention can obtain the corresponding information through API calls.

[0067] Taking the information sorting and extraction of axSpA as an example, the main sources of its descriptive information are clinical manifestations, imaging data such as MRI, and biochemical test results. Therefore, it is necessary to call existing MRI edema scoring models, structural scoring models, and biochemical result extraction models. The MRI edema model includes three modules and steps: MRI image preprocessing, ROI segmentation of the sacroiliac joint, and ROI regional edema scoring (edema recognition, edema depth recognition, and edema brightness recognition). The structural scoring includes three modules and steps: image preprocessing, ROI segmentation of the sacroiliac joint, and ROI regional SSS scoring (including four aspects: erosion, backfill, fatty metaplasia, and ankylosis). The biochemical result extraction model uses traditional machine learning algorithms to analyze and extract biochemical test indicators.

[0068] Taking the lesion area of axSpA as the target site as an example for illustration:

[0069] As Figure 2 shown, the descriptive text related to the target site can include the patient's chief complaint, past medical history, family medical history, and the presence of symptoms such as inflammatory low back pain, peripheral arthritis, and enthesitis. Sacroiliac joint MRI images and laboratory results reports such as Human leukocyte antigen (HLA)-B27, erythrocyte sedimentation rate, and C-reactive protein. After integrating and sorting all the above materials through a large language model, a descriptive text in a standard format will be generated, and then the semantic vector will be extracted through the corresponding natural language processing model. Of course, deep learning methods such as convolutional neural network (CNN) can also be used in advance for image processing of sacroiliac joint MRI to extract key features from the images, such as the size, shape, and location of the lesion area, to generate corresponding information convenient for the large language model to process. Image processing includes noise removal, image enhancement, etc.

[0070] Similarly, when constructing the preset site information, the above descriptive text of the confirmed axSpA can also be collected in the same way, and the magnetic resonance image sequence of the corresponding lesion site can be collected as a reference for later comparison. For the acquisition of preset site information, relevant data can be collected from various medical devices and systems, including electronic medical record systems (EMR / EHR): recorded medical history, medical records, treatment plans, etc.; imaging devices such as CT and MRI, providing medical image data; laboratory testing devices: providing test results of samples such as blood. Connect to different medical devices and systems through standard interfaces to obtain data. Establish a real-time data stream channel to ensure that the data obtained from the devices can be transmitted back in a timely manner. Establish reliable interfaces and communication mechanisms to support interaction with other systems and modules. Ensure the real-time and accuracy of the data.

[0071] S300: Use the images to be recognized in each target part cluster to perform secondary matching with the local magnetic resonance image sequences in each similar part information, and generate the recognition accuracy information of the target parts in each target part cluster.

[0072] The secondary matching includes:

[0073] S301: According to the slice interval H of the first part magnetic resonance image sequence 1 and the slice interval H of the second part magnetic resonance image sequence 2 , determine the magnetic resonance image to be matched corresponding to each magnetic resonance image in the first part magnetic resonance image sequence in the second part magnetic resonance image sequence. Among the local magnetic resonance image sequences in the target part cluster and the local magnetic resonance image sequences in the similar part information, the one with fewer images is the first part magnetic resonance image sequence, and the one with more images is the second part magnetic resonance image sequence. Among them, the serial number interval of the magnetic resonance image to be matched corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i , A 1 i , A 2 i respectively satisfy the following conditions:

[0074]

[0075] Among them, A 2 max is the largest image serial number in the second part magnetic resonance image sequence.

[0076] Generally, the slice intervals that can be scanned by different nuclear magnetic resonance devices may vary, and during two nuclear magnetic resonance scans, the slice intervals of the two magnetic resonance image sequences may also vary due to different settings. As a result, there will be a certain front-back error in the scanned parts corresponding to the images with the same serial number in the two image sequences. Therefore, if subsequent image similarity comparison is to be performed, it is necessary to first determine which two small images are to be used for similarity calculation according to the relationship between the slice intervals. In this embodiment, since different slice intervals will cause different scanned positions in the two image sequences, in order to ensure the accuracy of subsequent similarity calculation, each magnetic resonance image in the first part magnetic resonance image sequence will be used to calculate the similarity with the magnetic resonance images in a region. That is, multiple magnetic resonance images to be matched will be configured for each magnetic resonance image in the first part magnetic resonance image sequence.

[0077] S302: Generate the accuracy P of the target part in the target part cluster according to the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched. P satisfies the following conditions:

[0078]

[0079] Wherein, and are the similarities between the i-th magnetic resonance image in the first part magnetic resonance image sequence and the first and the last corresponding magnetic resonance images to be matched respectively. n is the total number of magnetic resonance images in the first part magnetic resonance image sequence. In this step means calculating the similarities between each magnetic resonance image in the first part magnetic resonance image sequence and multiple corresponding magnetic resonance images to be matched respectively, and taking the maximum similarity as the final similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the magnetic resonance image to be matched.

[0080] More preferably, corresponding weight coefficients can be configured for each image similarity to adjust the proportion of the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the magnetic resonance image to be matched in the accuracy. Specifically, the weight coefficients corresponding to the first and the last magnetic resonance images in the first part magnetic resonance image sequence are smaller than those of the middle magnetic resonance images.

[0081] Specifically, the acquisition of the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched includes the following steps:

[0082] S312: If the size specifications of two magnetic resonance images are the same, use the structural similarity algorithm to obtain the similarity between the two magnetic resonance images.

[0083] Calculating the similarity between two images is a common task in image processing and computer vision, and there are many different methods for this purpose. Such as pixel-level similarity, Structural Similarity Index (SSIM), and histogram comparison can all be achieved.

[0084] To ensure that the size specifications of two magnetic resonance images are the same. The size specifications of the ROI area of the image to be recognized can be the same as those of the ROI area in the preset part information. Specifically, the size of the ROI can be set to ensure that the size specifications of two magnetic resonance images are the same.

[0085] S322: If the size specifications of two magnetic resonance images are different, before calculating the similarity, the size specifications of the two magnetic resonance images need to be adjusted to be the same.

[0086] The most common method is to scale the images to the same size. This can be achieved through interpolation methods such as bilinear interpolation or nearest neighbor interpolation. Of course, a fully convolutional network (FCN) can also be used to process input images of any size to extract feature vectors of the images, and then calculate the similarity between the vectors to generate the corresponding image similarity.

[0087] S303: Generate the recognition accuracy information of the target parts in the target part cluster according to the accuracy of the target parts in the target part cluster and the accuracy threshold.

[0088] S303 includes:

[0089] S313: If P > Y1, mark all the images to be recognized in the target part cluster with the first color.

[0090] S323: If Y1 > P > Y2, mark all the images to be recognized in the target part cluster with the second color.

[0091] S333: If P < Y2, mark all the images to be recognized in the target part cluster with the third color. Y1 and Y2 are the first accuracy threshold and the second accuracy threshold respectively, and Y1 > Y2. For example, Y1 = 0.9 and Y2 = 0.8.

[0092] By comparing the accuracy with the corresponding threshold, the accuracy of the image regions corresponding to the target parts recognized by the prior art (such as deep learning and neural networks) in the images to be recognized can be determined, and different colors are used for marking, so as to facilitate the subsequent use by relevant personnel.

[0093] As another possible embodiment of the present invention, there is also provided a spondyloarthritis analysis device based on a learning model, and the device includes:

[0094] An identification clustering model, which is used to perform clustering processing on multiple images to be recognized in the magnetic resonance image sequence to be recognized, and generate at least one target part cluster; the images to be recognized are local magnetic resonance images corresponding to the target part ROI region; the ROI region is obtained through a deep learning model;

[0095] A vector matching model, which is used to perform primary matching on the description text vectors corresponding to the target parts in the magnetic resonance image sequence to be recognized and the description text vectors in each preset part information to determine at least one similar part information; each preset part information includes the description text vector of the corresponding part and the local magnetic resonance image sequence of the corresponding part ROI region;

[0096] An image matching model is used to perform secondary matching between the images to be recognized in each target part cluster and the local magnetic resonance image sequences in each similar part information, and generate the recognition accuracy information of the target parts in each target part cluster;

[0097] The secondary matching includes:

[0098] According to the section interval H of the first part magnetic resonance image sequence 1 and the section interval H of the second part magnetic resonance image sequence 2 , determine the magnetic resonance image to be matched corresponding to each magnetic resonance image in the first part magnetic resonance image sequence in the second part magnetic resonance image sequence; among the local magnetic resonance image sequences in the target part cluster and the local magnetic resonance image sequences in the similar part information, the one with fewer images is the first part magnetic resonance image sequence, and the one with more images is the second part magnetic resonance image sequence; where, the serial number interval of the magnetic resonance image to be matched corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i , A 1 i , A 2 i respectively satisfy the following conditions:

[0099]

[0100] where, A 2 max is the largest image serial number in the second part magnetic resonance image sequence;

[0101] According to the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched, generate the accuracy P of the target part in the target part cluster; P satisfies the following conditions:

[0102]

[0103] where, and are respectively the similarities between the i-th magnetic resonance image in the first part magnetic resonance image sequence and the corresponding first and last magnetic resonance images to be matched; n is the total number of magnetic resonance images in the first part magnetic resonance image sequence;

[0104] According to the accuracy of the target part in the target part cluster and the accuracy threshold, generate the recognition accuracy information of the target part in the target part cluster.

[0105] A spinal arthritis analysis device based on a learning model provided in the present invention can mainly be used to further identify and confirm the recognition results of target parts identified by deep learning and neural networks in the MRI images corresponding to spinal arthritis, so as to ensure the accuracy of the recognition results. Therefore, the device in this embodiment can be effectively integrated after the existing module for analyzing and identifying target regions in MRI images to further determine the recognition results.

[0106] Specifically, the spinal arthritis analysis device based on a learning model in this embodiment can be adaptively integrated into the existing axSpA auxiliary diagnosis intelligent agent based on an expert model and a large language model to improve the accuracy of medical image recognition results. For example, it can be specifically integrated into the recognition module of the target inflammation site of the sacroiliac joint MRI image in the medical decision-making module (i.e., the medical decision-making device) to improve the recognition accuracy.

[0107] Specifically, as Figure 3 shown, the axSpA auxiliary diagnosis intelligent agent based on an expert model and a large language model aims to deal with a large number of patients with low back pain in the clinical work environment, realize primary screening and triage, and assist rheumatologists in one-stop diagnosis of axSpA, thereby improving the accuracy and efficiency of diagnosis. This intelligent agent includes a human-computer interaction module, a task planning module, a medical decision-making module, a data acquisition module, an expert model module, and a knowledge base module.

[0108] Human-computer interaction module: This module is responsible for the interaction between the user (doctor or patient) and the intelligent agent. The user inputs data such as clinical manifestations, laboratory tests, and imaging examinations into the intelligent agent through a web application (or mobile application), a robot, etc. The intelligent agent outputs a detailed imaging report, diagnosis, and corresponding guidance by calling imaging interpretation, diagnostic tools based on an expert model, or diagnostic tools and a large language model.

[0109] The human-computer interaction interface corresponding to the human-computer interaction module is designed friendly, which is convenient for doctors and patients to use. The user only needs to input a question, and the intelligent agent will automatically determine the patient information according to the question and obtain the patient data. The user views the reply of the intelligent agent through the human-computer interaction interface, which improves the usability and user experience of the system.

[0110] Task Planning Module: This module consists of a task decomposer, a task executor, and a prompt builder. The main function of the task decomposer is to analyze and refine the problems or requirements input by the user into executable subtasks. Parse the user's intention, identify keywords and context, extract the required inflammatory low back pain in the diagnosis, complete blood count, erythrocyte sedimentation rate, and CRP in laboratory tests, understand the user input, identify the dependencies between subtasks, and ensure a reasonable task order. The task executor assigns subtasks to the corresponding modules or algorithms according to the task type and required resources. For example, for the imaging data provided by the user, it calls the image processing module. This module ensures the efficient collaborative work of each functional module.

[0111] Medical Decision-making Module: This is the center of the system, responsible for processing and analyzing the input information to generate diagnostic and medical decision-making suggestions. This module uses the deep learning algorithm of the large model to comprehensively analyze the patient's medical history, clinical symptoms, imaging data, etc., so as to provide accurate diagnostic results and personalized treatment plans. The medical decision-making module first needs to integrate data from different sources for comprehensive analysis. These data usually include: current medical history, past medical history, family medical history, and the presence of symptoms such as inflammatory low back pain, peripheral arthritis, enthesitis, etc., sacroiliac joint MRI images, and laboratory results such as HLA-B27, erythrocyte sedimentation rate, and C-reactive protein reports. The medical decision-making module uses a deep learning model for data analysis and decision generation. After deeply analyzing the patient data, the module will generate a preliminary diagnostic result. Match symptoms, analyze images, and comprehensively judge in combination with the medical history to generate a comprehensive diagnostic opinion.

[0112] Data Acquisition Module: Responsible for collecting patient data from various medical devices and systems, including electronic medical record systems (EMR / EHR): recording the patient's medical history, medical records, treatment plans, etc.; imaging devices such as CT and MRI, providing medical imaging data; laboratory testing devices: providing test results of samples such as blood. Connect to different medical devices and systems through standard interfaces to obtain data. Establish a real-time data stream channel to ensure that the data obtained from the devices can be transmitted to the data acquisition module in a timely manner. Establish a reliable interface and communication mechanism to support interaction with other systems and modules. This module ensures the real-time and accuracy of data and provides comprehensive information support for the medical decision-making module.

[0113] Expert Model Module: This module integrates expert models of several modalities. It can simulate the diagnostic thinking of experts, provide support for the medical decision-making module, and enhance the reliability and accuracy of the system. The expert model module plays a crucial role in the medical system. By integrating multiple expert models, simulating the diagnostic thinking of experts, and providing support for the medical decision-making module, it improves the reliability and accuracy of the system. The expert model module usually consists of the following main components: Imaging Model: Focuses on medical image analysis to help identify and diagnose abnormalities in images. Diagnostic Model: Provides comprehensive diagnostic suggestions based on the patient's medical records and clinical data. The imaging model is specifically used to process and analyze sacroiliac joint MRI. Deep learning methods such as convolutional neural networks (CNNs) are used for image processing, including noise removal, image enhancement, etc. Key features are extracted from the images, such as the size, shape, and location of the lesion area. Abnormalities in the images are identified. The diagnostic model comprehensively considers the patient's medical records, symptoms, and laboratory test results to provide diagnostic suggestions. Natural language processing techniques are used to analyze the patient's clinical symptoms and chief complaints. The patient's symptoms and examination results are matched with known disease patterns to generate preliminary diagnostic results. Combining the patient's medical records, imaging data, and laboratory results, comprehensive diagnostic and treatment suggestions are provided. The expert model module is an effective tool. This part integrates multiple models for analyzing examination results, such as the edema scoring model for MRI, the structural damage scoring model, the biochemical examination result analysis model, etc. The expert model is used to comprehensively analyze the patient's examination data to improve the final diagnostic accuracy.

[0114] Knowledge Base Module: Contains the latest research papers, clinical trial data, and medical advancements to help the intelligent agent understand the current state of the art in the field. Assembles various authoritative clinical guidelines and standard operating procedures to provide evidence-based medicine recommendations. Knowledge accumulated by the intelligent agent during the actual diagnostic process, including insights and lessons learned from historical cases. The content of the knowledge base is updated regularly to ensure it includes the latest treatment options and research findings. Dynamically expands the information in the knowledge base based on the usage and learning progress of the intelligent agent, adding new cases and discoveries. Introduces medical experts to review and validate the content of the knowledge base to ensure its authority and scientificity. This module also provides an efficient search engine that allows users to quickly retrieve knowledge and information related to axSpA, improving the response speed and efficiency of the system.

[0115] On the one hand, the knowledge base module of this intelligent agent can be connected to the network to regularly search for and update the latest axSpA-related materials. On the other hand, during the diagnostic process of the intelligent agent, each diagnostic process and result will be organized into the local knowledge base for future diagnostic reference. The intelligent agent realizes self-update and continuous learning through the knowledge base module.

[0116] The data acquisition module of the agent can obtain various examination results of patients. For each patient's specific situation, combined with the analysis results of the examinations and relevant knowledge in the local knowledge base, it assists doctors in formulating the best treatment plan.

[0117] Through the collaborative work of each module, this agent system aims to improve the diagnostic accuracy of axSpA and provide a powerful support tool for doctors and patients. The system can not only provide accurate diagnoses, but also put forward personalized treatment suggestions based on the latest research and clinical data, helping doctors make the best medical decisions and improving the treatment effect and satisfaction of patients at the same time.

[0118] The medical decision-making module of this agent system utilizes the capabilities of the large model to comprehensively analyze the patient's medical history, chief complaints, analysis results of the expert model, etc., conforms to the axSpA gestalt diagnostic model, and provides more accurate and consistent axSpA diagnostic results. For primary medical institutions or non-specialist doctors, when facing patients with low back pain, they can directly apply the intelligent interpretation results to guide the clinic, providing efficient and accurate primary screening and triage for diagnosis; through automated diagnosis and decision support, it assists rheumatologists in diagnosing axSpA, improving the diagnostic accuracy and efficiency, and compensating for the imbalance in medical levels to a certain extent.

[0119] In addition, although the steps of the methods in this disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0120] From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described here can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on the network, including several instructions to enable a computing device (which can be a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0121] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above method is also provided.

[0122] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuits", "modules", or "systems" here.

[0123] An electronic device according to this embodiment of the present invention. The electronic device is only an example and should not impose any restrictions on the functions and usage scopes of the embodiments of the present invention.

[0124] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one of the above-mentioned processors, at least one of the above-mentioned memories, and a bus connecting different system components (including the memory and the processor).

[0125] Among them, the memory stores program codes, and the program codes can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present invention described in the "Exemplary Method" section of this specification.

[0126] The memory may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory, and may further include a read-only memory (ROM).

[0127] The memory may further include a program / utility having a set (at least one) of program modules. Such program modules include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0128] The bus may represent one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure in a variety of bus structures.

[0129] The electronic device can also communicate with one or more external devices (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device, and / or communicate with any device that enables the electronic device to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface. Moreover, the electronic device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in combination with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0130] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or can be implemented by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0131] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium, on which a program product capable of implementing the above method of this specification is stored. In some possible implementation manners, various aspects of the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of this specification.

[0132] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0133] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0134] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0135] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0136] In addition, the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, rather than for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the time sequence of these processes. Additionally, it is also easy to understand that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0137] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-mentioned modules or units may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided and embodied by multiple modules or units.

[0138] The above are only the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for analyzing spondyloarthritis based on a learning model, characterized in that: The method includes the following steps: Performing clustering processing on multiple images to be recognized in the magnetic resonance image sequence to be recognized, generating at least one target part cluster; the images to be recognized are local magnetic resonance images corresponding to the ROI region of the target part; the ROI region is obtained through a deep learning model; Performing primary matching on the description text vectors corresponding to the target parts in the magnetic resonance image sequence to be recognized and the description text vectors in each preset part information to determine at least one similar part information; Each preset part information includes a description text vector corresponding to the corresponding part and a local magnetic resonance image sequence of the ROI region of the corresponding part; the description text vector is generated by a large language model; Performing secondary matching on the images to be recognized in each target part cluster and the local magnetic resonance image sequences in each similar part information to generate the recognition accuracy information of the target parts in each target part cluster.

2. The method according to claim 1, characterized in that The secondary matching includes: According to the slice interval H1 of the first part magnetic resonance image sequence and the slice interval H2 of the second part magnetic resonance image sequence, the corresponding magnetic resonance image to be matched in the second part magnetic resonance image sequence for each magnetic resonance image in the first part magnetic resonance image sequence is determined; in the local magnetic resonance image sequence in the target part cluster and the local magnetic resonance image sequence in the similar part information, the one with fewer images is the first part magnetic resonance image sequence, and the one with more images is the second part magnetic resonance image sequence; wherein the sequence number interval of the magnetic resonance image to be matched corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i ], A 1 i , A 2 i The following conditions are met respectively: Among them, A2 max is the largest image number in the magnetic resonance image sequence of the second part; Generating the accuracy P of the target part in the target part cluster according to the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched; P satisfies the following conditions: in, and are the similarities between the i-th magnetic resonance image in the magnetic resonance image sequence of the first part and the corresponding first and last magnetic resonance images to be matched respectively; n is the total number of magnetic resonance images in the magnetic resonance image sequence of the first part; Generating the recognition accuracy information of the target part in the target part cluster according to the accuracy of the target part in the target part cluster and the accuracy threshold; and The generating the recognition accuracy information of the target part in the target part cluster according to the accuracy of the target part in the target part cluster and the accuracy threshold includes: If P > Y1, then marking all the images to be recognized in the target part cluster as the first color; If Y1 > P > Y2, then marking all the images to be recognized in the target part cluster as the second color; If P < Y2, then marking all the images to be recognized in the target part cluster as the third color; Y1 and Y2 are the first accuracy threshold and the second accuracy threshold respectively, and Y1 > Y2.

3. The method according to claim 2, characterized in that Performing clustering processing on multiple images to be recognized in the magnetic resonance image sequence to be recognized, generating at least one target part cluster, including: If the overlap degree of two rectangular bounding boxes corresponding to the target parts in two adjacent frames of the images to be recognized is greater than the overlap degree threshold, then determining that the two images to be recognized belong to the same target part cluster.

4. The method according to claim 2, characterized in that: The description text vector is obtained according to the following method: Inputting the description text related to the target part into a preset deep learning network to generate the description text vector corresponding to the target part.

5. The method according to claim 2, characterized in that: The obtaining of the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched includes the following steps: If the size specifications of two magnetic resonance images are the same, then using the structural similarity algorithm to obtain the similarity between the two magnetic resonance images.

6. The method according to claim 5, characterized in that The obtaining of the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched further includes the following steps: If the size specifications of two magnetic resonance images are different, then before calculating the similarity, it is necessary to adjust the size specifications of the two magnetic resonance images to be the same.

7. The method according to claim 2, characterized in that The size specifications of the ROI region of the image to be recognized are the same as those of the ROI region in the preset part information.

8. The method according to claim 2, characterized in that: Y1 = 0.9, Y2 = 0.

8.

9. A spinal arthritis analysis device based on a learning model, characterized in that: The device includes: An identification clustering model is used to perform clustering processing on a plurality of images to be identified in a sequence of magnetic resonance images to be identified, and generate at least one target part cluster; the image to be identified is a local magnetic resonance image corresponding to a target part ROI region; the ROI region is obtained through a deep learning model; A vector matching model is used to perform primary matching between the description text vector corresponding to the target part in the magnetic resonance image sequence to be identified and the description text vector in each preset part information to determine at least one similar part information; each preset part information includes the description text vector of the corresponding part and the local magnetic resonance image sequence of the corresponding part ROI area; The image matching model is used to perform secondary matching using the image to be identified in each target part cluster and the local magnetic resonance image sequence in each similar part information to generate the recognition accuracy information of the target part in each target part cluster.

10. The device according to claim 9, characterized in that It also includes a human-computer interaction module, which is used to obtain operation instructions.

Images