A learning model based method and apparatus for analyzing spondyloarthritis

Abstract

The application relates to a learning model-based ankylosing spondylitis analysis method and device. Belonging to the field of image recognition in the scene of applying an agent based on a large language model to medical image analysis, the method comprises the following steps: performing primary matching on a description text vector corresponding to a target part in a to-be-recognized magnetic resonance image sequence and a description text vector in each preset part information to determine similar part information; the description text vector is generated by an agent of a large language model and can be adaptively integrated in an existing axSpA auxiliary diagnosis intelligent agent based on an expert model and a large language model to improve the accuracy of a medical image recognition result; in the scheme, the description text vector is used for vector matching to quickly screen out preset part information with high similarity to the target part; and then local magnetic resonance images are used for image similarity calculation to improve the recognition accuracy of the target part.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition processing, in particular to a spine arthritis analysis method and device based on a learning model. BACKGROUND

[0002] Magnetic resonance imaging (MRI) is a non-invasive medical imaging technique that uses strong magnetic fields and radio waves to generate detailed images of the internal structures of the human body. MRI is also a tomographic imaging technique, which provides a detailed view of the internal structures of the human body by generating multiple two-dimensional tomographic images. MRI images have an important position in medical imaging because of their higher soft tissue contrast, no radiation risk, and multi-planar imaging capabilities compared to CT images. These images can show structures such as bones, organs, soft tissues, and blood vessels, and play an important role in the discovery and monitoring of specific tissue structure sites.

[0003] The specific number of slices in a set of magnetic resonance images can vary depending on different factors, including the scanned site, the required image resolution, clinical needs, and the type of magnetic resonance machine used. In recent years, deep learning and neural networks have made significant progress in medical image analysis, especially in automatically detecting and locating lesions in magnetic resonance images. For example, in magnetic resonance image analysis, CNNs can be used to detect and classify various structure sites, etc.

[0004] At the same time, due to the fact that large language models (LLM) have hundreds of millions or even more parameters, they are able to capture complex language structures and patterns and perform well on various tasks such as text generation, question answering, translation, etc. Agents combined with large language models can use their deep understanding of language to interact more complexly and humanly, and automatically complete tasks. Agents based on large language models are gradually being used in the context of medical image analysis.

[0005] Although convolutional neural networks (CNN) have made significant progress in medical image analysis, due to the limitations of CNN itself, such as the small amount of training samples for some lesions, the presence of label noise in training data, insufficient or excessive model training, etc., deep learning and neural networks may still be inaccurate when detecting and classifying various lesions. This further causes the image region corresponding to the target site identified by deep learning and neural networks in magnetic resonance images to have errors, which can easily mislead users. SUMMARY

[0006] To solve the above technical problems, the technical solution adopted by the present application is:

[0007] According to one aspect of the present application, there is provided a learning model-based spondyloarthritis analysis method, the method comprising the following steps:

[0008] performing clustering processing on a plurality of to-be-identified images in a to-be-identified magnetic resonance image sequence to generate at least one target part cluster; the to-be-identified images are local magnetic resonance images corresponding to a region of interest (ROI) of a target part; the ROI is obtained through a deep learning model;

[0009] performing primary matching between a description text vector corresponding to the target part in the to-be-identified magnetic resonance image sequence and a description text vector in each preset part information to determine at least one similar part information; each preset part information comprises a description text vector of a corresponding part and a local magnetic resonance image sequence of a corresponding ROI of the part;

[0010] performing secondary matching between the to-be-identified images in each target part cluster and the local magnetic resonance image sequence in each similar part information to generate identification accuracy information of the target part in each target part cluster;

[0011] The secondary matching comprises:

[0012] determining, according to a slice interval H1 of a first part magnetic resonance image sequence and a slice interval H2 of a second part magnetic resonance image sequence, a to-be-matched magnetic resonance image corresponding to each magnetic resonance image in the first part magnetic resonance image sequence in the second part magnetic resonance image sequence; in the local magnetic resonance image sequence in the target part cluster and the local magnetic resonance image sequence in the similar part information, the one with a smaller number of images is the first part magnetic resonance image sequence, and the one with a larger number of images is the second part magnetic resonance image sequence; wherein, the serial number interval of the to-be-matched magnetic resonance image corresponding to the i th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i ], A 1 i , A 2 i respectively satisfy the following conditions:

[0013]

[0014] wherein, A2 max is the largest image serial number in the second part magnetic resonance image sequence;

[0015] generating the accuracy P of the target part in the target part cluster according to the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding to-be-matched magnetic resonance image; P satisfies the following condition:

[0016]

[0017] wherein, and respectively are the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding first and last magnetic resonance image to be matched respectively; n is the total number of magnetic resonance images in the first part magnetic resonance image sequence;

[0018] According to the accuracy of the target part in the target part cluster and the accuracy threshold, the identification accuracy information of the target part in the target part cluster is generated.

[0019] Further, according to the accuracy of the target part in the target part cluster and the accuracy threshold, the identification accuracy information of the target part in the target part cluster is generated, comprising:

[0020] If P>Y1, all the images to be identified in the target part cluster are marked as the first color;

[0021] If Y1>P>Y2, all the images to be identified in the target part cluster are marked as the second color;

[0022] If P<Y2, all the images to be identified in the target part cluster are marked as the third color; Y1 and Y2 are respectively the first accuracy threshold and the second accuracy threshold, Y1>Y2.

[0023] Further, the plurality of images to be identified in the sequence of magnetic resonance images to be identified are clustered to generate at least one target part cluster, comprising:

[0024] If the coincidence degree of the two rectangular bounding boxes corresponding to the target part in the two adjacent frames of images to be identified is greater than the coincidence degree threshold, it is determined that the two images to be identified belong to the same target part cluster.

[0025] Further, the description text vector is obtained by the following method:

[0026] The description text related to the target part is input into a preset deep learning network to generate a description text vector corresponding to the target part.

[0027] Further, the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched includes the following steps:

[0028] If the size specifications of the two magnetic resonance images are the same, the structural similarity algorithm is used to obtain the similarity between the two magnetic resonance images.

[0029] Further, the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding magnetic resonance image to be matched includes the following steps:

[0030] If the size specifications of the two magnetic resonance images are different, the size specifications of the two magnetic resonance images need to be adjusted to be the same before the similarity is calculated.

[0031] Further, the size specifications of the ROI region of the to-be-identified image and the ROI region in the preset part information are the same.

[0032] Further, Y1=0.9 and Y2=0.8.

[0033] According to a second aspect of the present application, a learning model-based spondyloarthritis analysis device, the device comprises:

[0034] The recognition clustering model is configured to perform clustering processing on a plurality of to-be-identified images in the to-be-identified magnetic resonance image sequence, and generate at least one target part cluster; the to-be-identified image is a local magnetic resonance image corresponding to a target part ROI region; the ROI region is obtained by a deep learning model;

[0035] The vector matching model is configured to perform primary matching between a description text vector corresponding to a target part in the to-be-identified magnetic resonance image sequence and a description text vector in each preset part information, and determine at least one similar part information; each preset part information comprises a description text vector of a corresponding part and a local magnetic resonance image sequence of a corresponding part ROI region;

[0036] The image matching model is configured to perform secondary matching between a to-be-identified image in each target part cluster and a local magnetic resonance image sequence in each similar part information, and generate identification accuracy information of the target part in each target part cluster;

[0037] The secondary matching comprises:

[0038] According to the interval H1 of the first part magnetic resonance image sequence and the interval H2 of the second part magnetic resonance image sequence, the to-be-matched magnetic resonance image corresponding to each magnetic resonance image in the first part magnetic resonance image sequence is determined; in the local magnetic resonance image sequence in the target part cluster and the local magnetic resonance image sequence in the similar part information, the first part magnetic resonance image sequence has fewer images, and the second part magnetic resonance image sequence has more images; wherein the serial number interval of the to-be-matched magnetic resonance image corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i ], A 1 i , A 2 i respectively satisfy the following conditions:

[0039]

[0040] wherein A2 max is the maximum image sequence number in the second part magnetic resonance image sequence;

[0041] According to the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding to-be-matched magnetic resonance image, the accuracy P of the target part in the target part cluster is generated; P satisfies the following conditions:

[0042]

[0043] wherein, and are the similarities between the i-th magnetic resonance image in the first part magnetic resonance image sequence and the corresponding first and last to-be-matched magnetic resonance images respectively; n is the total number of magnetic resonance images in the first part magnetic resonance image sequence;

[0044] According to the accuracy of the target part in the target part cluster and the accuracy threshold, the recognition accuracy information of the target part in the target part cluster is generated.

[0045] Further, a human-computer interaction module is further included, and the human-computer interaction module is used to acquire an operation instruction.

[0046] The present application has at least the following beneficial effects:

[0047] In the present application, after the image is recognized by deep learning and neural network, the image can be re-identified and verified to improve the recognition accuracy of the target part in the magnetic resonance image. For example, the present application can be applied to the recognition accuracy confirmation scene of the target part in the magnetic resonance image of axial spondyloarthritis (axSpA). Specifically, in the present application, first, the description text vector corresponding to the target part is matched with the preset part information in the preset matching library to quickly screen out the preset part information with high similarity to the target part. Then, the local magnetic resonance image sequence corresponding to each preset part information is used to calculate the image similarity with the multiple to-be-identified images corresponding to the target part, and finally the recognition accuracy of the target part is generated. Therefore, through the double verification of the description text vector and the magnetic resonance image, it can be further determined that the recognition accuracy of the target part in the magnetic resonance image after deep learning and neural network recognition. BRIEF DESCRIPTION OF DRAWINGS

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a learning model-based method for analyzing spondyloarthritis, provided as an embodiment of the present invention;

[0050] Figure 2 A flowchart illustrating the process of obtaining descriptive text vectors provided in this embodiment of the invention;

[0051] Figure 3 This is a schematic diagram of the structure of the axSpA-assisted diagnostic agent based on expert models and large language models provided in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] As one possible embodiment of the present invention, such as Figure 1 As shown, a learning model-based method for analyzing spondyloarthritis is provided, which includes the following steps:

[0054] S100: Clustering is performed on multiple images in the magnetic resonance image sequence to generate at least one target region cluster. The images to be identified are local magnetic resonance images corresponding to the ROI (Region of Interest) regions of the target regions. The ROI regions are obtained through a deep learning model.

[0055] The sequence of magnetic resonance images to be identified in the embodiment can be the sequence of images after being identified by the existing deep learning and neural network. Each frame of the magnetic resonance image represents the ROI region corresponding to the eye part which has been labeled, and of course the image region representing the eye part can also be labeled using the existing segmentation network. Since the scheme in the embodiment can be used for further identification and confirmation of the lesion region (such as axSpA and subchondral bone marrow edema) in the magnetic resonance image to improve the accuracy of the identification result, and can also be used for further identification and confirmation of the image corresponding to some organ tissue (such as ligament, muscle, heart and blood vessel), so as to strengthen the training of the corresponding part recognition for beginners of medical image analysis. Therefore, the target part in the embodiment can be some lesion part or some organ tissue part.

[0056] Specifically, S100 includes:

[0057] S101: If the coincidence degree of the two rectangular bounding boxes corresponding to the target part in the two adjacent images to be identified is greater than the coincidence degree threshold, it is determined that the two images to be identified belong to the same target part cluster.

[0058] Generally, whether it is a lesion part or some normal organ, it is a three-dimensional structure, which has certain relevance in space, and the structures before and after are also slowly and continuously changing, and will not mutate. At the same time, since the MRI is tomographically scanned, the tomographic interval is set correspondingly to more clearly reflect the structure of the corresponding part. Therefore, generally, a tissue structure has a large coincidence degree in adjacent two frames of magnetic resonance images, and different tissue structures have little coincidence degree.

[0059] Therefore, in the embodiment, each image to be identified can be placed in the same coordinate system, and then the coincidence degree of the two rectangular bounding boxes corresponding to the target part in adjacent two images is calculated in sequence, whether it is the same target part is determined by the coincidence degree, and then all the images to be identified corresponding to the same target part are placed in the same target part cluster.

[0060] S200: Perform primary matching between the description text vector corresponding to the target part in the sequence of magnetic resonance images to be identified and the description text vector in each preset part information to determine at least one similar part information. Each preset part information includes a description text vector of the corresponding part and a local magnetic resonance image sequence of the ROI region of the corresponding part.

[0061] The description text vector is obtained by the following method:

[0062] S201: Input the description text related to the target part into the preset deep learning network to generate the description text vector corresponding to the target part.

[0063] In this step, an existing deep learning network can be used to extract semantic features in the description text to generate a description text vector. Specifically, semantic extraction can be performed using corresponding methods in the field of natural language processing, such as using a BERT (Bidirectional Encoder Representations from Transformers) model or a Doc2Vec model.

[0064] As shown in Figure 2 In the present application, the description text corresponding to each part can be descriptive text information of the shape attribute of the corresponding part in the MRI image. If the part is a lesion, it can also be text information of the corresponding clinical symptoms or treatment plan caused by the lesion. Specifically, the clinical symptoms and treatment plan can be obtained from medical records and complaints. Or the corresponding text information in the examination data, such as blood routine, blood sedimentation, imaging data, and CRP (C-Reactive Protein) examination information. These examination information can be input into an existing large language model, and through the information processing capability of the large language model, it can be sorted into a preset standard format text information. In order to improve the accuracy of the above text sorting, a related corpus in the field can also be used to retrain the general large language model to form a corresponding special model to replace the general large language model.

[0065] This part can be constructed by a Prompt constructor and a large language model. The large language model (which can use the API interface of LLM, such as GPT-4, or deploy an open source large language model locally, such as ChatGLM) is responsible for analyzing and sorting the above information. The Prompt constructor is responsible for pre-processing the input of the large language model, so as to better stimulate the ability of the large language model.

[0066] As shown in Figure 2 In addition, due to the difference in the above-mentioned modalities (such as images, tables, and text), when the material is combed, in order to more accurately obtain information of different modalities, a specialist model can be configured, which includes a small special model corresponding to the information configuration of each modality. The small special model can be an existing related modality information analysis model, and the present application can obtain the corresponding information through API calling.

[0067] As an example of information extraction from axSpA, the main sources of description information are clinical manifestations, MRI image data, and biochemical test results. Therefore, existing MRI edema scoring models, structural scoring models, and biochemical result extraction models need to be called. The MRI edema model includes MRI image preprocessing, sacroiliac joint ROI segmentation, and ROI region edema scoring (edema recognition, edema depth recognition, and edema brightness recognition). The structural score includes image preprocessing, sacroiliac joint ROI segmentation, and ROI region SSS score (including erosion, backfill, fat degeneration, and rigidity). The biochemical result extraction model uses traditional machine learning algorithms to analyze and extract biochemical test indicators.

[0068] As an example of the target site being the lesion area of axSpA:

[0069] As shown in Figure 2 , the target site related description text can include the patient's chief complaint, medical history, family history, and whether there are symptoms such as inflammatory low back pain, peripheral arthritis, and enthesitis, sacroiliac joint MRI images, and laboratory results such as Human leukocyte antigen (HLA)-B27, erythrocyte sedimentation rate, and C-reactive protein. After integrating and combing all the above materials through a large language model, a standard format description text is generated, and then a corresponding natural language processing model is used to extract semantic vectors. Of course, a convolutional neural network (CNN) or other deep learning method can be used to process sacroiliac joint MRI images in advance to extract key features such as the size, shape, and location of the lesion area to generate corresponding information that is easy for the large language model to process. Image processing includes noise removal and image enhancement.

[0070] Similarly, when constructing the preset site information, the above description text of axSpA diagnosed patients can also be collected in the same way, and the corresponding magnetic resonance image sequence of the lesion site can also be collected as a reference for later comparison. For the acquisition of preset site information, relevant data can be collected from various medical devices and systems, including electronic medical record system (EMR / EHR): recorded medical history, medical record, treatment plan, etc.; imaging devices such as CT and MRI, providing medical image data; laboratory testing equipment: providing test results of blood samples. Through standard interfaces, connect with different medical devices and systems to obtain data. Establish real-time data flow channels to ensure that data obtained from devices can be returned in time. Establish a reliable interface and communication mechanism to support interaction with other systems and modules. Ensure the real-time and accuracy of the data.

[0071] S300: Perform secondary matching between the to-be-identified images in each target site cluster and the local magnetic resonance image sequences in each similar site information to generate identification accuracy information of the target site in each target site cluster.

[0072] The secondary matching includes:

[0073] S301: Determine a to-be-matched magnetic resonance image corresponding to each magnetic resonance image in the first site magnetic resonance image sequence in the second site magnetic resonance image sequence according to the interval H1 of the first site magnetic resonance image sequence and the interval H2 of the second site magnetic resonance image sequence. In the local magnetic resonance image sequences in the target site cluster and the local magnetic resonance image sequences in the similar site information, the one with fewer images is the first site magnetic resonance image sequence, and the one with more images is the second site magnetic resonance image sequence. Wherein, the sequence number interval of the to-be-matched magnetic resonance image corresponding to the i-th magnetic resonance image in the first site magnetic resonance image sequence is [A 1 i , A 2 i ], A 1 i , A 2 i respectively satisfy the following conditions:

[0074]

[0075] Wherein, A2 max is the largest image sequence number in the second site magnetic resonance image sequence.

[0076] Generally, the interval of the layers that can be scanned by different nuclear magnetic resonance devices may be different, and the interval of the layers of the two magnetic resonance image sequences may also be different due to different settings during the two nuclear magnetic resonance scans. Therefore, the images with the same sequence number in the two image sequences have a certain front-back error in the corresponding scanning positions. Therefore, if subsequent image similarity comparison is to be performed, it is necessary to determine which two small images are to be used for similarity calculation according to the relationship between the intervals of the layers. In this embodiment, since different intervals of the layers will result in different scanning positions in the two image sequences, in order to ensure the accuracy of the subsequent similarity calculation, each magnetic resonance image in the first site magnetic resonance image sequence will be used for similarity calculation with the magnetic resonance images in a region. That is, each magnetic resonance image in the first site magnetic resonance image sequence will be configured with multiple to-be-matched magnetic resonance images.

[0077] S302: Generate the accuracy P of the target site in the target site cluster according to the similarity between each magnetic resonance image in the first site magnetic resonance image sequence and the corresponding to-be-matched magnetic resonance image. P satisfies the following condition:

[0078]

[0079] wherein, and are the similarity between the i-th magnetic resonance image in the first part magnetic resonance image sequence and the corresponding first and last to-be-matched magnetic resonance image, respectively. n is the total number of magnetic resonance images in the first part magnetic resonance image sequence. In this step indicates that the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and a plurality of corresponding to-be-matched magnetic resonance images is calculated respectively, and the maximum similarity is taken as the final similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the to-be-matched magnetic resonance image.

[0080] More preferably, a corresponding weight coefficient can be configured for each image similarity to adjust the proportion of the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the to-be-matched magnetic resonance image in the accuracy. Specifically, the weight coefficients corresponding to the first magnetic resonance image and the last magnetic resonance image in the first part magnetic resonance image sequence are smaller than the weight coefficients of the intermediate magnetic resonance images.

[0081] Specifically, the acquisition of the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding to-be-matched magnetic resonance image includes the following steps:

[0082] S312: If the size specifications of the two magnetic resonance images are the same, the structural similarity algorithm is used to obtain the similarity between the two magnetic resonance images.

[0083] The calculation of the similarity between two images is a common task in image processing and computer vision, and there are many different methods that can be used for this purpose. For example, pixel-level similarity, structural similarity index (SSIM), and histogram comparison can all be implemented.

[0084] In order to ensure that the size specifications of the two magnetic resonance images are the same, the ROI region of the to-be-identified image and the ROI region in the preset part information can have the same size specifications. Specifically, the size of the ROI can be set to ensure that the size specifications of the two magnetic resonance images are the same.

[0085] S322: If the size specifications of the two magnetic resonance images are not the same, the size specifications of the two magnetic resonance images need to be adjusted to the same before calculating the similarity.

[0086] The most common method is to scale the images to the same size. This can be achieved by interpolation methods such as bilinear interpolation or nearest neighbor interpolation. Of course, a fully convolutional network (FCN) can also be used to process input images of arbitrary size to extract feature vectors of the images, and then generate corresponding image similarity through similarity calculation between vectors.

[0087] S303: generating identification accuracy information of the target part in the target part cluster according to the accuracy of the target part in the target part cluster and the accuracy threshold.

[0088] S303 includes:

[0089] S313: if P>Y1, marking all the to-be-identified images in the target part cluster as the first color.

[0090] S323: if Y1>P>Y2, marking all the to-be-identified images in the target part cluster as the second color.

[0091] S333: if P<Y2, marking all the to-be-identified images in the target part cluster as the third color. Y1 and Y2 are the first accuracy threshold and the second accuracy threshold respectively, and Y1>Y2. For example, Y1=0.9 and Y2=0.8.

[0092] By comparing the accuracy with the corresponding threshold, the accuracy of the image region corresponding to the target part identified by the existing technology (such as deep learning and neural network) in the to-be-identified image can be determined, and different colors are used for marking, so as to facilitate subsequent related personnel to use.

[0093] As another possible embodiment of the present application, a learning model-based analysis device for ankylosing spondylitis is also provided, which comprises:

[0094] The identification clustering model is used for clustering processing of a plurality of to-be-identified images in the to-be-identified magnetic resonance image sequence, and generating at least one target part cluster; the to-be-identified image is a local magnetic resonance image corresponding to a target part ROI region; the ROI region is obtained by a deep learning model;

[0095] The vector matching model is used for primary matching of the description text vector corresponding to the target part in the to-be-identified magnetic resonance image sequence with the description text vector in each preset part information, and determining at least one similar part information; each preset part information includes a description text vector of a corresponding part and a local magnetic resonance image sequence of a corresponding part ROI region;

[0096] The image matching model is configured to perform secondary matching between the to-be-identified image in each target part cluster and the local magnetic resonance image sequence in each similar part information, and generate the identification accuracy information of the target part in each target part cluster.

[0097] The secondary matching comprises:

[0098] According to the interval H1 of the first part magnetic resonance image sequence and the interval H2 of the second part magnetic resonance image sequence, the to-be-matched magnetic resonance image corresponding to each magnetic resonance image in the first part magnetic resonance image sequence is determined; in the local magnetic resonance image sequence in the target part cluster and the local magnetic resonance image sequence in the similar part information, the first part magnetic resonance image sequence has less images, and the second part magnetic resonance image sequence has more images; wherein the serial number interval of the to-be-matched magnetic resonance image corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i ], A 1 i , A 2 i respectively satisfy the following conditions:

[0099]

[0100] wherein A2 max is the largest image serial number in the second part magnetic resonance image sequence;

[0101] According to the similarity between each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding to-be-matched magnetic resonance image, the accuracy P of the target part in the target part cluster is generated; P satisfies the following conditions:

[0102]

[0103] wherein, and are the similarities between the i-th magnetic resonance image in the first part magnetic resonance image sequence and the corresponding first and last to-be-matched magnetic resonance images respectively; n is the total number of magnetic resonance images in the first part magnetic resonance image sequence.

[0104] According to the accuracy of the target part in the target part cluster and the accuracy threshold, the identification accuracy information of the target part in the target part cluster is generated.

[0105] The learning model-based spondyloarthritis analysis device provided in the present application can be mainly used for further identification confirmation of the identification result of the target part in the MRI image corresponding to spondyloarthritis identified through deep learning and neural network, so as to ensure the accuracy of the identification result. Therefore, the device in the present embodiment can be effectively integrated after the existing module for analyzing and identifying the target region of the MRI image, to further determine the identification result.

[0106] Specifically, the learning model-based spondyloarthritis analysis device in the present embodiment can be adaptively integrated in the existing axSpA auxiliary diagnosis intelligent agent based on an expert model and a large language model, to improve the accuracy of medical image identification result. For example, it can be integrated in the identification module of the target inflammatory site of the sacroiliac joint MRI image in the medical decision module (i.e. medical decision maker), to improve the identification accuracy.

[0107] Specifically, as shown in Figure 3 The axSpA auxiliary diagnosis intelligent agent based on an expert model and a large language model aims to deal with a large number of patients with low back pain in the clinical work environment, to realize preliminary screening and shunting and assist rheumatologists in one-stop diagnosis of axSpA, so as to improve the accuracy and efficiency of diagnosis. The intelligent agent includes a human-computer interaction module, a task planning module, a medical decision module, a data acquisition module, an expert model module, and a knowledge base module.

[0108] Human-computer interaction module: this module is responsible for the interaction between the user (doctor or patient) and the intelligent agent. The user inputs the clinical manifestations, laboratory tests, image examinations, etc. through the web (or mobile) application, robot, etc. into the intelligent agent. The intelligent agent outputs detailed image reports, diagnoses and corresponding guidance by calling the image interpretation and diagnosis tools based on the expert model or the diagnosis tools and the large language model.

[0109] The human-computer interaction interface corresponding to the human-computer interaction module is designed friendly, which is convenient for doctors and patients to use. The user only needs to input the question, and the intelligent agent will automatically determine the patient information and obtain the patient data according to the question. The user views the reply of the intelligent agent through the human-computer interaction interface, which improves the ease of use and user experience of the system.

[0110] Task planning module: This module consists of task decomposer, task executor, and prompt builder. The main function of the task decomposer is to analyze and refine the user's input question or requirement into executable subtasks. Analyze user intent, identify keywords and context, and extract the need for inflammatory low back pain, blood routine, and erythrocyte sedimentation rate in laboratory examination. Understand user input, identify dependencies between subtasks, and ensure reasonable task order. Task executor, according to the task type and required resources, assigns subtasks to corresponding modules or algorithms, such as user-provided image data, calls image processing module. This module ensures efficient collaboration of various functional modules.

[0111] Medical decision module: This is the core of the system, responsible for processing and analyzing input information, generating diagnosis and medical decision suggestions. This module uses deep learning algorithms of large models to comprehensively analyze patient's medical history, clinical symptoms, image data, etc., to provide accurate diagnosis results and personalized treatment plans. The medical decision module first needs to integrate data from different sources for comprehensive analysis. These data usually include: current medical history, past medical history, family medical history, and whether there are symptoms such as inflammatory low back pain, peripheral arthritis, and enthesis inflammation, sacroiliac joint MRI image, and laboratory results HLA-B27, erythrocyte sedimentation rate, and C-reactive protein report. The medical decision module uses deep learning models for data analysis and decision generation. After deep analysis of patient data, the module generates preliminary diagnosis results. Match symptoms, analyze images, and combine medical history to generate comprehensive diagnosis opinions.

[0112] Data acquisition module: responsible for collecting patient data from various medical devices and systems, including electronic medical record system (EMR / EHR): records patient's medical history, medical record, treatment plan, etc.; imaging devices CT, MRI, etc., provide medical image data; laboratory testing equipment: provides test results of blood samples. Connect with different medical devices and systems through standard interfaces to obtain data. Establish real-time data flow channels to ensure that data obtained from devices can be transmitted to the data acquisition module in a timely manner. Establish reliable interfaces and communication mechanisms to support interaction with other systems and modules. This module ensures the real-time and accuracy of data, providing comprehensive information support for the medical decision module.

[0113] Expert Model Module: This module integrates several modal expert models. This module can simulate the diagnostic thinking of experts and provide support for the medical decision-making module, improving the reliability and accuracy of the system. The expert model module plays a key role in the medical system, by integrating multiple expert models, simulating the diagnostic thinking of experts, providing support for the medical decision-making module, and thus improving the reliability and accuracy of the system. The expert model module usually contains the following main components: Image Model: Focuses on medical image analysis, helping to identify and diagnose abnormalities in images. Diagnosis Model: Provides comprehensive diagnostic recommendations based on patient medical records and clinical data. The image model is specifically used to process and analyze sacroiliac joint MRI, using deep learning methods such as convolutional neural networks (CNN) for image processing, including noise removal, image enhancement, etc. Extract key features from images, such as lesion area size, shape, and location. Identify abnormalities in images. The diagnosis model considers patient medical records, symptoms, and laboratory test results to provide diagnostic recommendations. Use natural language processing techniques to analyze patient clinical symptoms and complaints. Match patient symptoms and test results with known disease patterns to generate preliminary diagnostic results. Combine patient medical records, image data, and laboratory results to provide comprehensive diagnostic and treatment recommendations. The expert model module is an effective tool that integrates multiple models for analyzing test results, such as MRI edema score models, structural damage score models, and biochemical test result analysis models. Use expert models to comprehensively analyze patient test data to improve the accuracy of the final diagnosis.

[0114] Knowledge Base Module: Contains the latest research papers, clinical trial data, and medical advances, helping the agent understand the latest developments in the field. Collect various authoritative clinical guidelines and standard operating procedures to provide evidence-based medical recommendations. The agent accumulates knowledge during actual diagnosis, including insights and lessons learned from historical cases. The content of the knowledge base is regularly updated to ensure that it contains the latest treatment options and research findings. Based on the agent's usage and learning progress, dynamically expand the information in the knowledge base, adding new cases and discoveries. Introduce medical experts to review and verify the content of the knowledge base to ensure its authority and scientific nature. This module also provides an efficient search engine that allows users to quickly search for axSpA-related knowledge and information, improving the system's response speed and efficiency.

[0115] On the one hand, the knowledge base module of the agent can be connected to the network to search for the latest axSpA-related information and update it regularly, and on the other hand, during the diagnosis process of the agent, the diagnosis process and results of each time are sorted into the local knowledge base for future diagnosis reference. The agent realizes self-updating and continuous learning through the knowledge base module.

[0116] The data acquisition module of the intelligent agent can obtain various examination results of the patient, and based on the specific conditions of each patient, the analysis results of the examination and the relevant knowledge in the local knowledge base, assist the doctor in formulating the best treatment plan.

[0117] Through the cooperative work of each module, the intelligent agent system aims to improve the accuracy of axSpA diagnosis and provide strong support tools for doctors and patients. The system not only provides accurate diagnosis, but also provides personalized treatment recommendations based on the latest research and clinical data, helping doctors make the best medical decisions and improving the treatment effect and satisfaction of patients.

[0118] The medical decision-making module of the intelligent agent system utilizes the capabilities of large models to comprehensively analyze the patient's medical history, chief complaints, and expert model analysis results, conforming to the axSpA Gestalt diagnosis pattern, and providing more accurate and consistent axSpA diagnosis results. For primary medical institutions or non-specialist doctors, when facing patients with low back pain, the intelligent interpretation results can be directly applied to guide clinical practice, providing efficient and accurate initial screening and diversion; through automated diagnosis and decision support, the system assists rheumatologists in diagnosing axSpA, improving diagnostic accuracy and efficiency, and to some extent, supplementing the imbalance of medical levels.

[0119] In addition, although the steps of the methods in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all of the steps shown must be performed to achieve the desired results. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps, etc.

[0120] From the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or network, and includes a number of instructions to make a computing device (which can be a personal computer, server, mobile terminal, or network device, etc.) execute the method according to the embodiments of the present disclosure.

[0121] In the exemplary embodiments of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0122] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be embodied in a form of entirely hardware, entirely software (including firmware, microcode, etc.), or a combination of hardware and software, which can be collectively referred to as "circuitry", "module" or "system".

[0123] The electronic device according to this embodiment of the present application. The electronic device is merely an example and should not bring any limitation to the function and use range of the embodiments of the present application.

[0124] The electronic device is in the form of a general computing device. The components of the electronic device can include, but are not limited to, the at least one processor described above, the at least one memory described above, and a bus connecting different system components, including the memory and the processor.

[0125] The memory stores program codes which can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present application described in the "Exemplary Method" section of the present specification.

[0126] The memory can include a readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory, and can further include a read-only memory (ROM).

[0127] The memory can further include programs / utilities with a set of (at least one) program modules, such as an operating system, one or more application programs, other program modules, and program data, each of which or some combination of which can include the implementation of a network environment.

[0128] The bus can be one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures.

[0129] The electronic device can also communicate with one or more external devices such as a keyboard or a pointing device, a Bluetooth device, or a device for enabling

[0130] Those skilled in the art will readily understand that the example embodiments described herein can be implemented by software and / or by hardware combined with software essential for the software. Accordingly, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a number of instructions for causing a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0131] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which stores a program product capable of implementing the above-mentioned method of the present disclosure. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform the steps according to various example embodiments of the present disclosure described in the above-mentioned “example method” section of the present disclosure when the program product is run on the terminal device.

[0132] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or a flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0133] Computer readable signal media can include a propagated data signal with instructions embodied in data signals. Such propagated signal can take a wide variety of forms, including but not limited to electro-magnetic signals, optical signals, and so forth. Computer readable signal media can be any medium that can be involved in providing instructions to a machine for execution.

[0134] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0135] Program code, used by or in connection with the routines described herein, can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider. The application program code can be embodied on a computer readable medium, which include a magnetic or optical storage device such as diskette, DVD, or compact disk, a memory such as a semiconductor memory device or a wireless channel.

[0136] Furthermore, the above-described diagrams merely illustrate a schematic of the processes included in the method according to the exemplary embodiments of the present application, and are not intended for limiting purposes. It is readily understood that the processes shown in the above-described diagrams do not indicate or limit the time sequence of these processes. In addition, it is readily understood that these processes can be executed, for example, synchronously or asynchronously in a plurality of modules.

[0137] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. Indeed, according to an embodiment of the present disclosure, the features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functionalities of one module or unit described above can be further divided into embodied by a plurality of modules or units.

[0138] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A learning model-based spondyloarthritis analysis method, characterized by, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: Each preset part information comprises a description text vector of a corresponding part and a local magnetic resonance image sequence of a corresponding part ROI region; the description text vector is generated by a large language model. According to the slice interval H1 of the first part magnetic resonance image sequence and the slice interval H2 of the second part magnetic resonance image sequence, determine the corresponding to-be-matched magnetic resonance image of each magnetic resonance image in the first part magnetic resonance image sequence in the second part magnetic resonance image sequence; in the local magnetic resonance image sequence in the target part cluster and the local magnetic resonance image sequence in the similar part information, the one with less number of images is the first part magnetic resonance image sequence, and the one with more number of images is the second part magnetic resonance image sequence; wherein the serial number interval of the to-be-matched magnetic resonance image corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i ] and A 1 i , A 2 i respectively satisfy the following conditions: ； ； wherein A2 max is the largest image number in the second site magnetic resonance image sequence; The method comprises the following steps: ； wherein, and respectively are the similarity between the i-th magnetic resonance image in the first sequence of magnetic resonance images and the corresponding first and last magnetic resonance image to be matched; n is the total number of magnetic resonance images in the first sequence of magnetic resonance images; According to the accuracy of the target part in the target part cluster and the accuracy threshold, the recognition accuracy information of the target part in the target part cluster is generated; and The method comprises the following steps: If P>Y1, all the to-be-identified images in the target part cluster are marked as a first color; If Y1>P>Y2, all the to-be-identified images in the target part cluster are marked as a second color; If P<Y2, all the to-be-identified images in the target part cluster are marked as a third color; Y1 and Y2 are a first accuracy threshold and a second accuracy threshold respectively, and Y1>Y2.

2. The method of claim 1, wherein, The method comprises the following steps: If the coincidence degree of the two rectangular bounding boxes corresponding to the target part in the to-be-identified images of two adjacent frames is greater than a coincidence degree threshold, it is determined that the two to-be-identified images belong to the same target part cluster.

3. The method of claim 1, wherein, The method comprises the following steps: The method comprises the following steps:

4. The method of claim 1, wherein, The method comprises the following steps: If the size specifications of the two magnetic resonance images are the same, the structural similarity algorithm is used to obtain the similarity between the two magnetic resonance images.

5. The method of claim 4, wherein, The method comprises the following steps: If the size specifications of the two magnetic resonance images are not the same, the size specifications of the two magnetic resonance images need to be adjusted to be the same before calculating the similarity.

6. The method of claim 1, wherein, The ROI region of the to-be-identified image and the ROI region in the preset part information have the same size specifications.

7. The method of claim 1, wherein, Y1=0.9, Y2=0.

8.

8. A learning model-based spondyloarthritis analysis apparatus characterized by comprising: The device comprises: The recognition clustering model is used for clustering processing of a plurality of to-be-recognized images in a to-be-recognized magnetic resonance image sequence, to generate at least one target part cluster; the to-be-recognized image is a local magnetic resonance image corresponding to a target part ROI region; the ROI region is obtained through a deep learning model; The vector matching model is used for primary matching of a description text vector corresponding to a target part in the to-be-recognized magnetic resonance image sequence with a description text vector in each preset part information, to determine at least one similar part information; each preset part information includes a description text vector of a corresponding part and a local magnetic resonance image sequence of a corresponding part ROI region; The image matching model is used for secondary matching of a to-be-recognized image in each target part cluster with a local magnetic resonance image sequence in each similar part information, to generate recognition accuracy information of a target part in each target part cluster; According to the slice interval H1 of the first part magnetic resonance image sequence and the slice interval H2 of the second part magnetic resonance image sequence, determine the corresponding to-be-matched magnetic resonance image of each magnetic resonance image in the first part magnetic resonance image sequence in the second part magnetic resonance image sequence; in the local magnetic resonance image sequence in the target part cluster and the local magnetic resonance image sequence in the similar part information, the first part magnetic resonance image sequence is the one with less number of images, and the second part magnetic resonance image sequence is the one with more number of images; wherein the serial number interval of the to-be-matched magnetic resonance image corresponding to the i-th magnetic resonance image in the first part magnetic resonance image sequence is [A 1 i , A 2 i ] and A 1 i , A 2 i respectively satisfy the following conditions: ； ； wherein A2 max is the largest image number in the second site magnetic resonance image sequence; According to the similarity of each magnetic resonance image in the first part magnetic resonance image sequence and the corresponding to-be-matched magnetic resonance image, the accuracy P of the target part in the target part cluster is generated; P satisfies the following conditions: ； wherein, and respectively are the similarity between the i-th magnetic resonance image in the first sequence of magnetic resonance images and the corresponding first and last magnetic resonance image to be matched; n is the total number of magnetic resonance images in the first sequence of magnetic resonance images; According to the accuracy of the target part in the target part cluster and the accuracy threshold, the recognition accuracy information of the target part in the target part cluster is generated; and The generation of the recognition accuracy information of the target part in the target part cluster according to the accuracy of the target part in the target part cluster and the accuracy threshold includes: If P>Y1, all to-be-recognized images in the target part cluster are marked as a first color; If Y1>P>Y2, all to-be-recognized images in the target part cluster are marked as a second color; If P<Y2, all to-be-recognized images in the target part cluster are marked as a third color; Y1 and Y2 are a first accuracy threshold and a second accuracy threshold respectively, and Y1>Y2.

9. The apparatus of claim 8, wherein, Further comprising a human-computer interaction module, the human-computer interaction module is used for obtaining an operation instruction.

Images