Joint image automatic analysis and diagnosis system based on artificial intelligence

Abstract

The invention discloses a joint image automatic analysis and diagnosis system based on artificial intelligence, and relates to the technical field of image processing and artificial intelligence. The method comprises the following steps: carrying out spatial standardization alignment and cutting on X-ray and MRI images through key point detection; respectively extracting skeleton form and soft tissue texture features, and calculating correlation between the skeleton form and the soft tissue texture features in space and channel dimensions to realize deep feature complementation and fusion; constructing a graph convolutional network based on a joint anatomy topological structure, mapping fusion features to graph nodes, aggregating adjacent node information through a graph message passing mechanism, simulating medical logic to perform joint reasoning, and outputting a diagnosis result; generating a visual lesion thermodynamic diagram by using a gradient weighting algorithm to position a lesion, and generating a structured diagnosis report containing lesion positions, properties and degrees according to the pathological description knowledge base; according to the method, the problems of single-modal information limitation and poor model interpretability are effectively solved, and high-precision, interpretable and high-efficiency joint image intelligent analysis is realized.

Description

Technical Field

[0001] This invention belongs to the field of image processing and artificial intelligence technology, and specifically relates to an automatic joint image analysis and diagnosis system based on artificial intelligence. Background Technology

[0002] Joint degenerative diseases have become a global public health challenge, especially knee osteoarthritis, whose prevalence continues to rise among the elderly. With the popularization of medical imaging technology and the development of artificial intelligence, using computer vision technology to achieve automated and accurate diagnosis of joint diseases has become an important research direction for improving diagnostic and treatment efficiency and alleviating the pressure on medical resources.

[0003] Currently, automated image reading tools based on convolutional neural networks have been applied to joint image analysis. These methods typically perform end-to-end feature learning and classification directly on single-modality two-dimensional images, achieving preliminary lesion detection and grading. However, their design paradigm has fundamental limitations: the model is highly dependent on single-modality data and lacks explicit modeling of the correlation between anatomical structure and pathology, resulting in limited analytical dimensions.

[0004] This reveals the following problems with existing technologies: 1. Data uniformity: They often rely on X-rays from a single perspective or MRI from a single modality, making it difficult to capture the complex three-dimensional structure and soft tissue lesions of joints (such as the knee and hip joints); 2. Poor interpretability: Traditional deep learning models only output diagnostic results (such as "stage III arthritis"), but doctors cannot know which part of the image the model is based on, leading to low clinical trust; 3. Missed detection of small lesions: For early-stage small narrowing of the joint space or slight cartilage wear, general convolutional neural networks are prone to feature loss due to downsampling operations. Summary of the Invention

[0005] (a) Technical problems to be solved In view of the problems in related technologies, the present invention provides an artificial intelligence-based automatic joint image analysis and diagnosis system to overcome the above-mentioned technical problems existing in the existing related technologies.

[0006] (II) Technical Solution To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: This invention provides an artificial intelligence-based automatic joint image analysis and diagnosis system, comprising: Data acquisition and standardization module: It is configured to acquire X-ray and MRI images of the target joint area of ​​the patient to be diagnosed, and perform spatial standardization correction and cropping to obtain processed X-ray and MRI images; Dual-stream feature extraction and fusion module: configured to construct a dual-stream feature extraction network containing structural flow and texture flow branches; input the processed X-ray images and MRI images into the structural flow and texture flow branches respectively to obtain bone morphology features and soft tissue texture features; calculate the correlation matrix between structural flow features and texture flow features in spatial and channel dimensions to generate multimodal fusion feature vectors; Anatomical Atlas Reasoning Module: It is configured to map multimodal fused feature vectors to joint anatomical topology knowledge graph, use the final graph convolutional neural network to mine the spatial adjacency edges and functional association edges between nodes in the anatomical topology knowledge graph, and calculate the contribution of each node to the target diagnostic grading to obtain the diagnostic results of joint lesions and the set of key lesion factors. Visualization and interpretation generation module: It is configured to generate visual lesion heatmaps for different pathological types based on the diagnosis results of joint lesions and to calculate the gradient information of the backpropagation of the dual-stream feature extraction network in S2 using the gradient weighted class activation mapping algorithm. Intelligent diagnostic report generation module: It is configured to locate the anatomical region based on the visualized lesion heat map, retrieve the set of key lesion factors from the preset pathological description knowledge base, and convert them into natural language descriptions containing the location, nature and severity of the lesion. The descriptions are then overlaid on the original images to obtain a structured diagnostic report. Preferably, the implementation steps of the data acquisition and standardization module include: S11. Use the DICOM protocol interface to read the X-ray and MRI images of the target joint of the patient to be diagnosed from the hospital PACS system; S12. Use ResNet-50 as the backbone network to build the initial keypoint detection model; collect historical joint image data and the coordinates of key anatomical landmarks in historical joint image data to obtain historical joint training data; use the historical joint training data to train the initial keypoint detection model to obtain the final keypoint detection model. The X-ray and MRI images of the target joint of the patient to be diagnosed are input into the final key point detection model to obtain a set of real-time key anatomical landmark coordinates. S13. Calculate the joint midline tilt angle based on the real-time key anatomical landmark coordinate set, and use the affine transformation matrix to perform rotation correction on the X-ray and MRI images of the target joint of the patient to be diagnosed, so that the joint midline coincides with the vertical axis of the image, and obtain the corrected X-ray and MRI images. S14. Using the joint center point of the corrected X-ray and MRI images as a reference, the region of interest is cropped according to the preset physical size ratio; the cropped image is scaled to a uniform resolution and histogram equalization is performed to enhance it, thus obtaining the processed X-ray and MRI images. Preferably, the implementation steps of the dual-stream feature extraction and fusion module include: S21. Construct a dual-flow feature extraction network that includes a structure flow branch and a texture flow branch; input the processed X-ray image and MRI image into the structure flow branch and the texture flow branch respectively, and obtain the bone morphology feature map output by the structure flow branch and the soft tissue texture feature map output by the texture flow branch. S22. Compress the skeletal morphology feature map and the soft tissue texture feature map into single-channel feature maps respectively, and generate the spatial weight matrix of the skeletal morphology feature map and the spatial weight matrix of the soft tissue texture feature map through the Sigmoid function. S23. Calculate the spatial overlap between the skeletal morphology feature map and the soft tissue texture feature map under the anatomical view, and construct a cross-modal attention fusion unit; calculate the correlation matrix of different modal feature maps in the channel dimension and spatial dimension through the cross-modal attention fusion unit; S24. Based on the spatial weight matrix of the skeletal morphology feature map, the spatial weight matrix of the soft tissue texture feature map, the skeletal morphology feature matrix, the soft tissue texture feature matrix, and the attention matrix, a multimodal fusion feature vector is obtained through the feature fusion formula. Preferably, the implementation steps of S23 include: S231. Reshape the skeletal morphology feature map and the soft tissue texture feature map into skeletal morphology feature matrices and soft tissue texture feature matrices, respectively. S232. Use the skeletal morphology feature matrix as the query vector and the soft tissue texture feature matrix as the key vector; calculate the product of the skeletal morphology feature matrix and the soft tissue texture feature matrix and normalize it to obtain the attention matrix; Preferably, the implementation steps of the anatomical atlas reasoning module include: S31. Define a set of atlas node types; the set of atlas node types includes anatomical structure nodes, pathological manifestation nodes, and clinical sign nodes; S32. Based on the set of graph node types, construct the physical structure of the anatomical topology knowledge graph to obtain the joint anatomical topology knowledge graph; S33. Map the multimodal fusion feature vectors to the joint anatomy topology knowledge graph, and map the feature vectors to specific nodes in the graph according to the anatomical location; establish spatial adjacency edges based on biomechanical transmission relationships, and establish functional association edges based on left-right symmetry structure to obtain the instantiated anatomy topology knowledge graph. S34. Construct an initial graph convolutional neural network based on the joint anatomical topology; collect historical X-ray and MRI images, as well as anatomical structure annotations and graph-level labels, and train the initial graph convolutional neural network. During the training process, combine optimization algorithms to find the weight parameters of the initial graph convolutional neural network and obtain the optimal solution; use the optimal solution as the weight parameters of the initial graph convolutional neural network to obtain the final graph convolutional neural network. S35. Based on the instantiated anatomical topology knowledge graph, the final graph convolutional neural network is used to take the attribute features of the nodes as the initial embedding vectors. After multiple layers of graph convolution, the neighbor node information is aggregated through the graph message passing mechanism, the node representation vector is updated, and after passing through a fully connected classification layer, the diagnosis result of joint lesions is obtained. S36. For any node, obtain the influence weight of the node feature vector on the final diagnostic grading probability after passing through the fully connected classification layer; if the influence weight exceeds the preset threshold, the node is determined to be a key lesion node, and its corresponding feature vector and adjacency relationship are extracted to obtain the key lesion factor set. Preferably, the step in S34 where the optimization algorithm is used to find the weight parameters of the initial graph convolutional neural network during training to obtain the optimal solution includes: S341. Construct a chromosome population and set the size of the chromosome population; use each chromosome in the chromosome population as the weight parameters of the initial graph convolutional neural network; set the maximum number of training iterations; S342. After each round of training, the predicted classification data is obtained; the similarity between the predicted classification data and the anatomical structure annotation and the graph-level label is calculated to obtain the classification accuracy; the classification accuracy and single prediction classification time of the initial graph convolutional neural network are set; the fitness of each chromosome in the chromosome population is set according to the classification accuracy and single prediction classification time to obtain the fitness formula. S343. Begin iteration. In each iteration, calculate the fitness of each chromosome in the chromosome population according to the fitness formula to obtain the chromosome fitness set. Select chromosomes in the chromosome population according to the fitness set to obtain the selected chromosome population. Perform crossover and mutation operations on the chromosomes in the selected chromosome population to obtain the operated chromosome population. S344. Repeat S343. When the maximum number of training iterations is reached, stop the iteration and obtain the optimal solution. Preferably, the implementation steps in S35, which involve aggregating neighbor node information through multi-layer graph convolution and updating the node representation vector using a graph message passing mechanism, include: S351. For a target node, identify its set of first-order neighbor nodes; the set of first-order neighbor nodes includes anatomical structure nodes directly connected by spatial adjacency edges. S352. Calculate the feature similarity between the first-order neighbor node and the target node to obtain the edge weight coefficient; S353. The feature vectors of first-order neighbor nodes are weighted and summed based on the edge weight coefficients, and then concatenated with the feature vector of the target node itself. After linear transformation and nonlinear activation, the updated node vector is obtained. Preferably, the implementation steps of the visualization explanation generation module include: S41. Based on the target category score in the diagnosis results of joint lesions, calculate the gradient of the target category score relative to the feature map of the last convolutional layer of the dual-stream feature extraction network; S42. Perform global average pooling on the gradient values ​​to obtain channel importance weights; then perform linear weighted superposition of the channel importance weights and the corresponding feature maps, and process them through the ReLU activation function to obtain a coarse-grained localization map. S43. Upsample the coarse-grained localization map to the original image resolution, and use a color mapping table to map the activation values ​​to RGB color values ​​to generate a visual lesion heat map superimposed on the original image. Preferably, the implementation steps of the intelligent diagnostic report generation module include: S51. Traverse the high-response areas in the visualized lesion heatmap generated in S4, determine the corresponding anatomical structure nodes, and extract the set of key lesion factors calculated in S3 for that node. S52. Using a pre-set pathological description knowledge base, match the corresponding medical term templates according to the feature types of key lesion factors; the feature types include osteophyte formation, joint space narrowing, cartilage wear and bone marrow edema; fill the extracted feature parameters into the template slots to generate natural language descriptions; S53. For the anatomical structure nodes determined in S51, extract local features from the multimodal fusion feature vector obtained in S2, compare them with historical local features, calculate the similarity, and obtain the pathological feature type; render the original image according to the pathological feature type to obtain intelligent diagnostic image. S54. Based on the framework of the standard diagnostic report template, fill the structured text description generated in S52 into the "Image View" column, embed the intelligent diagnostic image rendered in S53 into the "Key Image" column, complete the structured assembly of the text and image content, and finally render and export the complete intelligent analysis report. Preferably, the implementation steps for constructing the pathological description knowledge base in S52 include: S521. Collect historical joint imaging diagnostic reports and extract the image description paragraphs and diagnostic conclusions from them; S522. Utilize natural language processing technology to perform entity recognition and relation extraction on the descriptive paragraphs, and construct a set of triplets of <anatomical location, pathological features, degree modifiers>. S523. Conduct frequency statistics and expert review on the triplet set, select high-frequency and standardized medical terminology combinations, establish a standardized key-value pair mapping table, and form a pathological description knowledge base.

[0007] (III) Beneficial Effects The present invention has the following beneficial effects: This invention significantly improves the accuracy and comprehensiveness of joint disease diagnosis through multimodal data fusion and anatomical atlas reasoning. The system comprehensively utilizes bone structure information from X-rays and soft tissue texture information from MRI to overcome the limitations of single-modal vision. Furthermore, by explicitly modeling the biomechanical relationships between anatomical sites through graph convolutional networks, it achieves more sensitive detection and joint reasoning for early minor lesions (such as minor cartilage wear and bone marrow edema) and complex concurrent signs, effectively reducing the missed diagnosis rate.

[0008] This invention significantly enhances the clinical interpretability and practicality of diagnostic results through a visual explanation and intelligent report generation mechanism. Gradient-weighted heatmaps intuitively identify the lesion areas upon which decisions are based, and structured reports automatically generate standardized descriptive text, making the AI's judgment process transparent and credible to doctors. This not only serves as an efficient auxiliary review tool, shortening report writing time, but also helps doctors quickly understand the AI's reasoning logic, build clinical trust, and promote the implementation of human-machine collaborative diagnostic models.

[0009] This invention provides a modular and scalable intelligent joint analysis framework with broad application prospects and industry value. Its core design, such as attention-based feature fusion units, configurable anatomical atlases, and standardized pathological knowledge bases, possesses excellent generalization capabilities and can be easily adapted to image analysis of other sites such as the hip and ankle joints. This solution advances medical image analysis from a single-modality, black-box model to a multimodal, interpretable, and knowledge-driven artificial intelligence system.

[0010] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the artificial intelligence-based automatic joint image analysis and diagnosis method of the present invention. Figure 2This is a schematic diagram of the modules of the AI-based automatic joint image analysis and diagnosis system of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the invention, and not all embodiments. Based on the embodiments of the invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the invention.

[0014] To resolve the above issues, please refer to [link / reference]. Figure 1 This invention discloses an artificial intelligence-based automatic joint image analysis and diagnosis system, comprising: Data acquisition and standardization module: It is configured to acquire X-ray and MRI images of the target joint area of ​​the patient to be diagnosed, and perform spatial standardization correction and cropping to obtain processed X-ray and MRI images; Dual-stream feature extraction and fusion module: configured to construct a dual-stream feature extraction network containing structural flow and texture flow branches; input the processed X-ray images and MRI images into the structural flow and texture flow branches respectively to obtain bone morphology features and soft tissue texture features; calculate the correlation matrix between structural flow features and texture flow features in spatial and channel dimensions to generate multimodal fusion feature vectors; Anatomical Atlas Reasoning Module: It is configured to map multimodal fused feature vectors to joint anatomical topology knowledge graph, use the final graph convolutional neural network to mine the spatial adjacency edges and functional association edges between nodes in the anatomical topology knowledge graph, and calculate the contribution of each node to the target diagnostic grading to obtain the diagnostic results of joint lesions and the set of key lesion factors. Visualization and interpretation generation module: It is configured to generate visual lesion heatmaps for different pathological types based on the diagnosis results of joint lesions and to calculate the gradient information of the backpropagation of the dual-stream feature extraction network in S2 using the gradient weighted class activation mapping algorithm. Intelligent diagnostic report generation module: It is configured to locate the anatomical region based on the visualized lesion heat map, retrieve the set of key lesion factors from the preset pathological description knowledge base, and convert them into natural language descriptions containing the location, nature and severity of the lesion. The descriptions are then overlaid on the original images to obtain a structured diagnostic report. The above embodiments construct an anatomical topology knowledge graph by aligning the depth of heterogeneous images to simulate the anatomical logical reasoning of doctors, and output diagnostic reports using visualized lesion heat maps and automated text generation technology. This overcomes the three major pain points of traditional solutions: lack of interpretability, easy to miss small lesions, and low report writing efficiency. It realizes full-chain intelligence from image acquisition and lesion localization to report generation, and can automatically generate auxiliary diagnostic reports with clear lesions, rigorous logic, and visualized evidence, significantly improving the accuracy and efficiency of clinical diagnosis.

[0015] Data acquisition and standardization module: configured to acquire X-ray and MRI images of the target joint of the patient to be diagnosed, and perform spatial standardization correction and cropping to obtain processed X-ray and MRI images; this embodiment aims to solve the problem of insufficient information in single-modal images and difficulty in comprehensively assessing joint lesions; The implementation steps of the data acquisition and standardization module in the above embodiments include: S11. Use the DICOM protocol interface to read the X-ray and MRI images of the target joint of the patient to be diagnosed from the hospital PACS system; In specific implementation, the above embodiment S11 is as follows: The embodiments of the present invention all take the knee joint as an example. Through the DICOM interface of the hospital PACS system, the knee joint image data of the patient ID "P2025001" is obtained, including: bilateral knee weight-bearing anteroposterior X-ray (structured data, reflecting interosseous spaces and osteophytes) and knee joint MRI sequence (T2 fat-suppressed sequence, reflecting cartilage, meniscus and bone marrow edema); the X-ray is a 2D projection image, and the MRI is a 3D slice sequence; S12. Use ResNet-50 as the backbone network to build the initial keypoint detection model; collect historical joint image data and the coordinates of key anatomical landmarks in historical joint image data to obtain historical joint training data; use the historical joint training data to train the initial keypoint detection model to obtain the final keypoint detection model. The X-ray and MRI images of the target joint of the patient to be diagnosed are input into the final key point detection model to obtain a set of real-time key anatomical landmark coordinates. In specific implementation, the above embodiment S12 specifically includes: the coordinates of the key anatomical landmarks include the femoral center point (x f , y f ), Tibial center point (x) t , y t ), patellar center point (x p , y p ) and the vertex of the fibular head (x fib , y fib ); The keypoint detection model uses ResNet-50 as its backbone network. After removing the original classification head, a prediction head consisting of three deconvolutional layers and one 1x1 convolutional layer is added. The model input is a standardized 512x512 single-channel joint image, and the output is a coordinate heatmap and confidence score of four preset key anatomical landmarks (such as the femoral center, tibial center, patellar center, and fibular head apex) for the target joint (such as the knee joint). During training, a dataset containing historical joint images and their precisely labeled coordinates is used. End-to-end training is performed using weighted mean squared error as the loss function. The optimizer is Adam, and gradient descent is used to enable the model to accurately learn the mapping relationship from images to anatomical landmarks, ultimately resulting in a generalizable keypoint detection model. S13. Calculate the joint midline tilt angle based on the real-time key anatomical landmark coordinate set, and use the affine transformation matrix to perform rotation correction on the X-ray and MRI images of the target joint of the patient to be diagnosed, so that the joint midline coincides with the vertical axis of the image, and obtain the corrected X-ray and MRI images. In specific implementation, the above embodiment S13 is as follows: In order to eliminate the error caused by incorrect body positioning during shooting, the tilt angle of the joint midline is calculated, the center point of the femur and the center point of the tibia are connected, and the angle θ between the connecting line and the vertical axis of the image is calculated; the calculation formula is: ; If θ = 5 degrees, it means that the knee joint is tilted to the right in the image. The X-ray and MRI images of the target joint of the patient to be diagnosed are rotated counterclockwise by 5 degrees by an affine transformation matrix to make the joint axis vertical. S14. Using the joint center point of the corrected X-ray and MRI images as a reference, the region of interest is cropped according to the preset physical size ratio; the cropped image is scaled to a uniform resolution and histogram equalization is performed to enhance it, thus obtaining the processed X-ray and MRI images. In specific implementation, the above embodiment S14 is as follows: take the corrected joint center point (i.e., the midpoint of the line connecting the femoral center and the tibial center) as the origin (0,0); cut according to the preset physical size ratio: extend upward 10cm to include the femoral condyle, extend downward 10cm to include the tibial plateau, and extend to the left and right 8cm to include the collateral ligament area. The above are the physical dimensions set for the knee joint in this embodiment. In specific implementation, they can be dynamically set according to factors such as joint location. In this embodiment, the image pixel resolution is 10 pixels / mm, and the cropped area is 2000 * 1600 pixels. The cropped image is uniformly scaled to 512 * 512 pixels and enhanced with CLAHE (contrast-limited adaptive histogram equalization) to obtain a standardized multimodal image set. This embodiment ensures that the images input to the neural network have consistent anatomical field of view and texture contrast. The above embodiments connect to the hospital's PACS and EMR systems through a unified interface, and use a targeted feature extraction network to transform image and text information of different dimensions into feature vectors that can be processed by computers. This effectively integrates morphological, functional, and clinical symptom information, laying a multidimensional data foundation for subsequent accurate diagnosis.

[0016] Dual-stream feature extraction and fusion module: configured to construct a dual-stream feature extraction network including structural flow and texture flow branches; input the processed X-ray images and MRI images into the structural flow and texture flow branches respectively to obtain bone morphology features and soft tissue texture features; calculate the correlation matrix between structural flow features and texture flow features in spatial and channel dimensions to generate a multimodal fusion feature vector; this embodiment takes "medial joint space stenosis of the knee" as an example to solve the problem of spatial correspondence between X-ray films and MRI slices; The implementation steps of the dual-stream feature extraction and fusion module in the above embodiments include: S21. Construct a dual-flow feature extraction network containing structural flow and texture flow branches; input the processed X-ray images and MRI images into the structural flow branch and texture flow branch respectively, and obtain the bone morphology feature map F output by the structural flow branch. S The soft tissue texture feature map F output by the texture stream branch T ; In specific implementation, the above embodiment S21 is as follows: the input of the structure flow branch is the processed X-ray image, using ResNet-34 as the backbone network, removing fully connected layers, and is mainly used to extract high-frequency information of bone edges, such as the sharp edges of osteophytes and the width variation of joint spaces; the input of the texture flow branch is the preprocessed MRI slice, using 3D-ResNet-18 as the backbone network, and is used to extract low-frequency texture information of soft tissues, such as the wear surface of cartilage, signal abnormalities of the meniscus, and high-signal areas of bone marrow edema; both the structure flow branch and the texture flow branch are trained using historically processed X-ray images and MRI images; S22, Transfer the skeletal morphology feature map F S and soft tissue texture feature map F T The features are compressed into single-channel feature maps, and a skeletal morphology feature map spatial weight matrix M is generated using the Sigmoid function. S and the spatial weight matrix M of soft tissue texture feature map T ; In specific implementation, the above embodiment S22 is specifically as follows: In this embodiment, the skeletal morphology feature map F S (Size 16*16*512), Soft tissue texture feature map F T (Size 16*16*512); Use 1×1 convolution to convert F S Compressed to a single-channel M1 S The skeletal morphological feature map spatial weight matrix M is obtained after Sigmoid activation. S Similarly, the spatial weight matrix M of the soft tissue texture feature map is obtained. T ; S23. Calculate the skeletal morphology feature map and soft tissue texture feature map F T Based on the spatial overlap under the anatomical view, a cross-modal attention fusion unit is constructed; the correlation matrix of different modal feature maps in the channel dimension and spatial dimension is calculated through the cross-modal attention fusion unit; S24, Based on the spatial weight matrix M of the skeletal morphology feature map S Soft tissue texture feature map spatial weight matrix M T Skeletal morphology feature matrix F 1 S and soft tissue texture feature matrix F 1 T Using the attention matrix Acorss and the feature fusion formula, we obtain the multimodal fused feature vector F. fused ; In specific implementation, the above embodiment S24 is as follows: the feature fusion formula is... ; The implementation steps of the above embodiment S23 include: S231, Transfer the skeletal morphology feature map F S and soft tissue texture feature map F T Reconstructed into skeletal morphological feature matrix F 1 S and soft tissue texture feature matrix F 1 T ; S232, the skeletal morphology feature matrix F 1 S As the query vector, the soft tissue texture feature matrix F 1 T As the key vector; calculate the skeletal morphological feature matrix F. 1 S and soft tissue texture feature matrix F 1 T The product of these components and normalized yields the attention matrix Acorss, which reflects the channel dependencies between different modalities. In specific implementation, the above embodiment S23 specifically involves: reshaping, i.e., flattening the spatial grid of the feature map, arranging the features of each pixel position into a long queue to perform pairwise matching calculations between all position pairs; the formula for calculating the attention matrix is: Among them, F 1T TF represents the soft tissue texture feature matrix 1 T transpose, d k F represents the soft tissue texture feature matrix 1 T The dimension, i.e., the soft tissue texture feature map F T The number of channels; The above embodiments, by constructing a two-stream network and a cross-modal attention fusion mechanism, achieve deep semantic alignment and complementarity between X-ray skeletal structures and MRI soft tissue textures; effectively solving the feature misalignment problem caused by modal heterogeneity in traditional methods; tests show that this fusion method improves the model's detection sensitivity for early joint lesions (such as cartilage wear and bone marrow edema) by approximately 28%, and the fused features achieve a diagnostic accuracy of 93.2% for complex signs such as "narrowing of the medial joint space of the knee" in subsequent atlas reasoning tasks; verifying that cross-modal attention can accurately establish pathological associations such as "narrowing of the interosseous space" and "thinning of the cartilage", providing a complete and highly relevant feature foundation for subsequent anatomical logic-based reasoning.

[0017] Anatomical Atlas Reasoning Module: It is configured to map multimodal fused feature vectors to joint anatomical topology knowledge graph, use the final graph convolutional neural network to mine the spatial adjacency edges and functional association edges between nodes in the anatomical topology knowledge graph, and calculate the contribution of each node to the target diagnostic grading to obtain the diagnostic results of joint lesions and the set of key lesion factors. The implementation steps of the anatomical atlas reasoning module in the above embodiments include the following steps: S31. Define a set of atlas node types; the set of atlas node types includes anatomical structure nodes, pathological manifestation nodes, and clinical sign nodes; In specific implementation, the above embodiment S31 specifically includes: the anatomical structure nodes include the medial femoral condyle, the lateral femoral condyle, the medial tibial plateau, the lateral tibial plateau, the patella, the medial meniscus, and the lateral meniscus; the pathological manifestation nodes include osteophytes and sclerosis; the clinical sign nodes include pain; the pathological manifestations are obtained through electronic medical records, and the clinical sign nodes are obtained through the patient's chief complaint text; S32. Based on the set of graph node types, construct the physical structure of the anatomical topology knowledge graph to obtain the joint anatomical topology knowledge graph; In specific implementation, the above embodiment S32 is as follows: The present invention has a pre-set graph structure configuration file for different joint types; the graph structure configuration file for the knee joint in this embodiment defines the standard number of anatomical nodes, node names and initial adjacency matrix of the knee joint, hip joint and ankle joint; the system automatically loads the corresponding graph structure configuration file to construct the corresponding graph convolutional neural network according to the input image type label; S33. Map the multimodal fusion feature vectors to the joint anatomy topology knowledge graph, and map the feature vectors to specific nodes in the graph according to the anatomical location; establish spatial adjacency edges based on biomechanical transmission relationships, and establish functional association edges based on left-right symmetry structure to obtain the instantiated anatomy topology knowledge graph. In specific implementation, the above embodiment S33 specifically involves: instantiating anatomical structure nodes as the medial femoral condyle, lateral femoral condyle, medial tibial plateau, lateral tibial plateau, patella, medial meniscus, and lateral meniscus; establishing connecting edges: since the femur and tibia are in direct contact and bear weight, strong spatial adjacency edges e1,2 are established; since the meniscus is located between the two, associated edges e2,3 are established; the weights of the edges are dynamically set according to the medical correlation tightness; this mapping method incorporates prior knowledge of human anatomy; S34. Construct an initial graph convolutional neural network based on the joint anatomical topology; collect historical X-ray and MRI images, as well as anatomical structure annotations and graph-level labels, and train the initial graph convolutional neural network. During the training process, combine optimization algorithms to find the weight parameters of the initial graph convolutional neural network and obtain the optimal solution; use the optimal solution as the weight parameters of the initial graph convolutional neural network to obtain the final graph convolutional neural network. In specific implementation, the above embodiment S34 is as follows: the anatomical structure annotation is a region segmentation mask or key point coordinates corresponding to the defined graph nodes, annotated by a radiologist, used to accurately map the multimodal fusion feature map to the initial feature vector of each node; the diagnostic annotation includes the final diagnosis and grading of the entire joint (such as KL grading) and the pathological state of key anatomical sub-regions (such as normal, cartilage wear, bone marrow edema, osteophyte formation). In the above embodiment S34, the steps for finding the weight parameters of the initial graph convolutional neural network and obtaining the optimal solution during the training process include: S341. Construct a chromosome population. Let the size of the chromosome population be p. Then, the chromosome population is represented as: p i Represents the 1st chromosome in the population. One chromosome; each chromosome in the chromosome population is used as the weight parameters of the initial graph convolutional neural network; the maximum number of training iterations is set; S342. After each round of training, predicted classification data is obtained; the similarity between the predicted classification data and the anatomical structure annotations and graph-level labels is calculated to obtain the classification accuracy; the classification accuracy of the initial graph convolutional neural network is set to z1, and the single prediction classification time is set to z2. Based on the classification accuracy z1 and the single prediction classification time z2, the fitness of each chromosome in the chromosome population is set to k, and the fitness formula is obtained; the fitness formula is as follows. ; S343. Begin iteration. In each iteration, calculate the fitness of each chromosome in the chromosome population according to the fitness formula to obtain the chromosome fitness set. Select chromosomes in the chromosome population according to the fitness set to obtain the selected chromosome population. Perform crossover and mutation operations on the chromosomes in the selected chromosome population to obtain the operated chromosome population. S344. Repeat S343. When the maximum number of training iterations is reached, stop the iteration and obtain the optimal solution. S35. Based on the instantiated anatomical topology knowledge graph, the final graph convolutional neural network is used to take the attribute features of the nodes as the initial embedding vectors. After multiple layers of graph convolution, the neighbor node information is aggregated through the graph message passing mechanism, the node representation vector is updated, and after passing through a fully connected classification layer, the diagnosis result of joint lesions is obtained. In the above embodiment S35, the steps for aggregating neighbor node information and updating the node representation vector through multi-layer graph convolution and graph message passing mechanism include: S351. For a target node, identify its set of first-order neighbor nodes; the set of first-order neighbor nodes includes anatomical structure nodes directly connected by spatial adjacency edges. S352. Calculate the feature similarity between the first-order neighbor node and the target node to obtain the edge weight coefficient; S353. The feature vectors of first-order neighbor nodes are weighted and summed based on the edge weight coefficients, and then concatenated with the feature vector of the target node itself. After linear transformation and nonlinear activation, the updated node vector is obtained. In specific implementation, the above embodiment S353 is as follows: The weighted summation formula is as follows: ; where h i (l+1) Let N represent the updated node vector, σ represent the ReLU activation function, and N represent the N+1 node vector. i Let e ​​represent the set of neighboring nodes of node i. i,j h represents the edge weight between node i and node j. i (l) W represents the feature vector of node i in the l-th layer. (l) This represents the weight parameters of the l-th convolutional layer in the optimal weight parameters. For example, the state update of the tibial medial plateau node depends not only on its own features (whether it has bone sclerosis), but also on the features of the medial femoral condyle (whether it has osteophytes) and the features of the medial meniscus (whether it has a tear). If the meniscus is torn and there are osteophytes in the femur, then the probability of cartilage wear on the tibial plateau is extremely high, simulating the doctor's comprehensive reasoning logic. S36. For any node, obtain the influence weight of the node feature vector on the final diagnostic grading probability after passing through the fully connected classification layer; if the influence weight exceeds the preset threshold, the node is determined to be a key lesion node, and its corresponding feature vector and adjacency relationship are extracted to obtain the key lesion factor set. In specific implementation, the above embodiment S36 is as follows: the system predicts that the patient has KL grade 3 (moderate arthritis); in order to find evidence, the system uses the Grad-CAM algorithm to reverse calculate the weight α of each node. c r The calculation formula is: Where r represents the r-th anatomical node (e.g., the knee joint has 7 anatomical nodes), D represents the dimension of the feature vector of each node, d represents the d-th feature dimension in D, and Z H The target category score y represents the diagnostic results of joint lesions. c For the node feature matrix H output by the last layer of the graph convolutional neural network; in this embodiment, the weight of the node tibial plateau is 0.85, and its feature vector contains a significant "subchondral bone sclerosis" pattern; at the same time, it is found that the weight of the node medial femoral condyle is 0.6, which contains osteophyte features; therefore, the key lesion factor set is {factor A: tibial plateau sclerosis, factor B: medial femoral condyle osteophytes}; The above embodiments, by constructing an anatomical topology knowledge graph and combining it with a graph convolutional network, realize the reasoning analysis of complex structural relationships inside joints; transforming a single image classification task into a graph reasoning task based on anatomical logic, it can comprehensively consider the interaction between bones, cartilage and meniscus like a doctor, accurately locate the key anatomical sites and pathological features that lead to disease grading, and greatly improve the interpretability of diagnosis.

[0018] The visualization interpretation generation module is configured to calculate the gradient information of the backpropagation of the dual-stream feature extraction network in S2 based on the diagnosis results of joint lesions, and generate visualized lesion heat maps for different pathological types using the gradient weighted class activation mapping algorithm. This embodiment aims to allow doctors to intuitively see the areas that the AI ​​is interested in, avoiding the black box effect. The implementation steps of the visualization explanation generation module in the above embodiments include: S41. Target category score y based on the diagnostic results of joint lesions c Calculate the gradient of the target category score relative to the feature map of the last convolutional layer of the two-stream feature extraction network; In specific implementation, the above embodiment S41 specifically involves: the target category is KL Grade 3, and the score y is calculated. c Relative to feature map A k The partial derivatives are used to obtain the gradient value Z. AThis gradient value reflects the degree to which each pixel in the feature map contributes to the final classification result; S42. Perform global average pooling on the gradient values ​​to obtain channel importance weights; perform linear weighted superposition of the channel importance weights and the corresponding feature maps, and process them through the ReLU activation function to obtain the coarse-grained localization map L; In specific implementation, the above embodiment S42 is as follows: the formula for global average pooling is: ; where α c k Z represents the channel importance weight, G represents the total number of pixels in the feature map, g1 and g2 represent the two heights and widths in the feature map, and g1 and g2 together constitute a pixel (g1, g2); Ag1g2 This represents the gradient value at pixel (g1, g2); The formula for the ReLU activation function is: ; S43. Upsample the coarse-grained localization map to the original image resolution, and use a color mapping table to map the activation values ​​to RGB color values ​​to generate a visual lesion heat map superimposed on the original image. The above embodiment S44 specifically involves: upsampling the calculated coarse-grained localization map L to 512×512, and normalizing the numerical range to [0,1]; the mapping rules can be dynamically set according to actual needs; in this embodiment, a risk threshold of 0.6 is set, with areas greater than 0.6 mapped as red (high risk) and areas between 0.3 and 0.6 mapped as yellow (medium risk); finally, on the original X-ray, the medial tibial plateau region appears as a deep red thermal patch, intuitively indicating the presence of a major lesion; through gradient-weighted activation mapping technology, the abstract feature weights inside the deep neural network are transformed into a human-visualized heat map; enabling doctors to intuitively verify whether the basis of the AI ​​diagnosis is reasonable (for example, confirming whether the heat map actually covers the osteophyte or sclerotic area, rather than background noise), thereby establishing trust in the AI-assisted diagnostic system; The above embodiments use gradient-weighted class activation mapping technology to transform the abstract feature weights inside the deep neural network into a heat map that is visualized by humans. This processing method allows doctors to intuitively verify whether the basis of the AI ​​diagnosis is reasonable (for example, to confirm whether the heat map actually covers the osteophyte or sclerotic area, rather than background noise), thereby establishing trust in the AI-assisted diagnostic system.

[0019] Intelligent diagnostic report generation module: It is configured to locate the anatomical region based on the visualized lesion heat map, retrieve the set of key lesion factors from the preset pathological description knowledge base, and convert them into natural language descriptions containing the location, nature and severity of the lesion. The descriptions are then overlaid on the original images to obtain a structured diagnostic report. The implementation steps of the intelligent diagnostic report generation module in the above embodiments include: S51. Traverse the high-response areas in the visualized lesion heatmap generated in S4, determine the corresponding anatomical structure nodes, and extract the set of key lesion factors calculated in S3 for that node. S52. Using a pre-set pathological description knowledge base, match the corresponding medical term templates according to the feature types of key lesion factors to obtain extracted feature parameters; the feature types include osteophyte formation, joint space narrowing, cartilage wear and bone marrow edema; fill the extracted feature parameters into the template slots to generate a natural language description; The implementation steps for constructing the pathological description knowledge base in embodiment S52 above include: S521. Collect historical joint imaging diagnostic reports and extract the image description paragraphs and diagnostic conclusions from them; S522. Utilize natural language processing technology to perform entity recognition and relation extraction on the descriptive paragraphs, and construct a set of triplets of <anatomical location, pathological features, degree modifiers>. S523. Conduct frequency statistics and expert review on the triplet set, select high-frequency and standardized medical terminology combinations, establish a standardized key-value pair mapping table, and form a pathological description knowledge base. In specific implementation, the above embodiment S52 is as follows: The system identified "sclerosis" (pathological feature) and a high response value (severe severity) in the medial tibial plateau (anatomical location); the knowledge base was searched and a template was matched: "[location] shows obvious [pathological] changes"; after entering the parameters, the system generated: "obvious subchondral bone sclerosis changes are visible in the medial tibial plateau"; The rules in the pathological description knowledge base of this embodiment are as follows: Rule 1: If node = medial tibial plateau, feature = sclerosis AND degree = severe, the output text is: "Obvious subchondral bone sclerosis is visible on the medial tibial plateau"; Rule 2: If node = medial femoral condyle AND feature = osteophyte AND degree = moderate, the output text is: "Beak-like osteophyte formation is visible at the edge of the medial femoral condyle"; Rule 3: If KL grade = 3, the output conclusion is: "Significantly narrowed joint space, consistent with stage III osteoarthritis (KL Grade 3) changes"; S53. For the anatomical structure nodes determined in S51, extract local features from the multimodal fusion feature vector obtained in S2, compare them with historical local features, calculate the similarity, and obtain the pathological feature type; render the original image according to the pathological feature type to obtain intelligent diagnostic image. In specific implementation, the above embodiment S53 is as follows: For osteophytes at the edge of the medial femoral condyle, based on their high-density characteristics (high brightness on X-ray), it is recommended to use a "thin solid red line" to outline their contours to highlight their sharp edges; for the meniscus tear area shown on MRI, it is recommended to use a "semi-transparent yellow block" to cover it as a warning. S54. Based on the framework of the standard diagnostic report template, fill the structured text description generated in S52 into the "Image View" column, embed the intelligent diagnostic image rendered in S53 into the "Key Image" column, complete the structured assembly of the text and image content, and finally render and export the complete intelligent analysis report. In specific implementation, the above embodiment S54 is specifically as follows: Image overview: a standard anteroposterior film with red / yellow thermogram coverage, which intuitively indicates the lesion area; Structured text: The diagnosis is osteoarthritis of the left knee (KL grade 3). Detailed findings include obvious subchondral bone sclerosis on the medial tibial plateau (confidence 92%), and coracoid osteophyte formation at the edge of the medial femoral condyle (confidence 85%). The joint space is significantly narrower on the medial side than on the lateral side (measured as 2.1 mm on the medial side and 5.5 mm on the lateral side). The above embodiments, by combining a pathological description knowledge base with intelligent chart rendering technology, achieve automated generation of diagnostic reports. The system not only outputs standardized medical text but also automatically selects the most suitable visual annotation style based on the physical characteristics of the lesions (such as the hardness of osteophytes and cartilage defects), generating professional reports that are rich in graphics and highlight key points. In actual deployment tests, the system reduced the average generation time of a single knee joint structured report from approximately 15-20 minutes required manually to within 30 seconds. The generated reports are more standardized than radiological reports, significantly improving the standardization and comparability of results. Feedback from physician surveys shows that reports written with the assistance of this system have a pass rate exceeding 98% in terms of content completeness and key lesion detection rate, greatly reducing the workload of radiologists and improving the standardization level of reports.

[0020] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0021] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. An artificial intelligence-based automatic joint image analysis and diagnosis system, characterized in that, include: Data acquisition and standardization module: It is configured to acquire X-ray and MRI images of the target joint area of ​​the patient to be diagnosed, and perform spatial standardization correction and cropping to obtain processed X-ray and MRI images; Dual-stream feature extraction and fusion module: configured to construct a dual-stream feature extraction network containing structural flow and texture flow branches; the processed X-ray images and MRI images are input into the structural flow branch and texture flow branch respectively to obtain bone morphology features and soft tissue texture features; Calculate the correlation matrix between structure flow features and texture flow features in the spatial and channel dimensions, and generate a multimodal fusion feature vector; Anatomical Atlas Reasoning Module: It is configured to map multimodal fused feature vectors to joint anatomical topology knowledge graph, use the final graph convolutional neural network to mine the spatial adjacency edges and functional association edges between nodes in the anatomical topology knowledge graph, and calculate the contribution of each node to the target diagnostic grading to obtain the diagnostic results of joint lesions and the set of key lesion factors. Visualization and interpretation generation module: It is configured to generate visual lesion heatmaps for different pathological types based on the diagnosis results of joint lesions and to calculate the gradient information of the backpropagation of the dual-stream feature extraction network in S2 using the gradient weighted class activation mapping algorithm. The intelligent diagnostic report generation module is configured to locate the anatomical region based on the visualized lesion heat map, retrieve the key lesion factor set from the preset pathological description knowledge base, and convert it into a natural language description containing the lesion location, lesion nature and severity. This description is then overlaid with the original image to obtain a structured diagnostic report.

2. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 1, characterized in that, The implementation steps of the data acquisition and standardization module include: S11. Use the DICOM protocol interface to read the X-ray and MRI images of the target joint of the patient to be diagnosed from the hospital PACS system; S12. Use ResNet-50 as the backbone network to build the initial keypoint detection model; collect historical joint image data and the coordinates of key anatomical landmarks in historical joint image data to obtain historical joint training data; use the historical joint training data to train the initial keypoint detection model to obtain the final keypoint detection model. The X-ray and MRI images of the target joint of the patient to be diagnosed are input into the final key point detection model to obtain a set of real-time key anatomical landmark coordinates. S13. Calculate the joint midline tilt angle based on the real-time key anatomical landmark coordinate set, and use the affine transformation matrix to perform rotation correction on the X-ray and MRI images of the target joint of the patient to be diagnosed, so that the joint midline coincides with the vertical axis of the image, and obtain the corrected X-ray and MRI images. S14. Using the joint center point of the corrected X-ray and MRI images as a reference, the region of interest is cropped according to the preset physical size ratio; the cropped image is scaled to a uniform resolution and histogram equalization enhancement is performed to obtain the processed X-ray and MRI images.

3. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 1, characterized in that, The implementation steps of the dual-stream feature extraction and fusion module include: S21. Construct a dual-flow feature extraction network that includes a structure flow branch and a texture flow branch; input the processed X-ray image and MRI image into the structure flow branch and the texture flow branch respectively, and obtain the bone morphology feature map output by the structure flow branch and the soft tissue texture feature map output by the texture flow branch. S22. Compress the skeletal morphology feature map and the soft tissue texture feature map into single-channel feature maps respectively, and generate the spatial weight matrix of the skeletal morphology feature map and the spatial weight matrix of the soft tissue texture feature map through the Sigmoid function. S23. Calculate the skeletal morphology feature map and soft tissue texture feature map F T Based on the spatial overlap under the anatomical view, a cross-modal attention fusion unit is constructed; the correlation matrix of different modal feature maps in the channel dimension and spatial dimension is calculated through the cross-modal attention fusion unit; S24. Based on the spatial weight matrix of the skeletal morphology feature map, the spatial weight matrix of the soft tissue texture feature map, the skeletal morphology feature matrix, the soft tissue texture feature matrix, and the attention matrix, a multimodal fusion feature vector is obtained through the feature fusion formula.

4. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 3, characterized in that, The implementation steps of S23 include: S231. Reshape the skeletal morphology feature map and the soft tissue texture feature map into skeletal morphology feature matrices and soft tissue texture feature matrices, respectively. S232. Use the skeletal morphology feature matrix as the query vector and the soft tissue texture feature matrix as the key vector; calculate the product of the skeletal morphology feature matrix and the soft tissue texture feature matrix and normalize it to obtain the attention matrix.

5. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 1, characterized in that, The implementation steps of the anatomical atlas reasoning module include: S31. Define a set of atlas node types; the set of atlas node types includes anatomical structure nodes, pathological manifestation nodes, and clinical sign nodes; S32. Based on the set of graph node types, construct the physical structure of the anatomical topology knowledge graph to obtain the joint anatomical topology knowledge graph; S33. Map the multimodal fusion feature vectors to the joint anatomy topology knowledge graph, and map the feature vectors to specific nodes in the graph according to the anatomical location; establish spatial adjacency edges based on biomechanical transmission relationships, and establish functional association edges based on left-right symmetry structure to obtain the instantiated anatomy topology knowledge graph. S34. Construct an initial graph convolutional neural network based on the joint anatomical topology; collect historical X-ray and MRI images, as well as anatomical structure annotations and graph-level labels, and train the initial graph convolutional neural network. During the training process, combine optimization algorithms to find the weight parameters of the initial graph convolutional neural network and obtain the optimal solution; use the optimal solution as the weight parameters of the initial graph convolutional neural network to obtain the final graph convolutional neural network. S35. Based on the instantiated anatomical topology knowledge graph, the final graph convolutional neural network is used to take the attribute features of the nodes as the initial embedding vectors. After multiple layers of graph convolution, the neighbor node information is aggregated through the graph message passing mechanism, the node representation vector is updated, and after passing through a fully connected classification layer, the diagnosis result of joint lesions is obtained. S36. For any node, obtain the influence weight of the node feature vector on the final diagnostic grading probability after passing through the fully connected classification layer; if the influence weight exceeds the preset threshold, the node is determined to be a key lesion node, and its corresponding feature vector and adjacency relationship are extracted to obtain the key lesion factor set.

6. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 5, characterized in that, The steps in S34 for finding the weight parameters of the initial graph convolutional neural network and obtaining the optimal solution during the training process include: S341. Construct a chromosome population and set the size of the chromosome population; use each chromosome in the chromosome population as the weight parameters of the initial graph convolutional neural network; set the maximum number of training iterations; S342. After each round of training, the predicted classification data is obtained; the similarity between the predicted classification data and the anatomical structure annotation and the graph-level label is calculated to obtain the classification accuracy; the classification accuracy and single prediction classification time of the initial graph convolutional neural network are set; the fitness of each chromosome in the chromosome population is set according to the classification accuracy and single prediction classification time to obtain the fitness formula. S343. Begin iteration. In each iteration, calculate the fitness of each chromosome in the chromosome population according to the fitness formula to obtain the chromosome fitness set. Select chromosomes in the chromosome population according to the fitness set to obtain the selected chromosome population. Perform crossover and mutation operations on the chromosomes in the selected chromosome population to obtain the operated chromosome population. S344. Repeat S343. When the maximum number of training iterations is reached, stop the iteration and obtain the optimal solution.

7. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 5, characterized in that, The steps in S35, which involve aggregating neighbor node information through multi-layer graph convolution and updating the node representation vector via graph message passing, include: S351. For a target node, identify its set of first-order neighbor nodes; the set of first-order neighbor nodes includes anatomical structure nodes that are directly connected through spatial adjacency edges. S352. Calculate the feature similarity between the first-order neighbor node and the target node to obtain the edge weight coefficient; S353. The feature vectors of first-order neighbor nodes are weighted and summed based on the edge weight coefficients, and then concatenated with the feature vector of the target node itself. After linear transformation and nonlinear activation, the updated node vector is obtained.

8. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 1, characterized in that, The implementation steps of the visualization explanation generation module include: S41. Target category score y based on the diagnostic results of joint lesions c Calculate the gradient of the target category score relative to the feature map of the last convolutional layer of the two-stream feature extraction network; S42. Perform global average pooling on the gradient values ​​to obtain channel importance weights; then perform linear weighted superposition of the channel importance weights and the corresponding feature maps, and process them through the ReLU activation function to obtain a coarse-grained localization map. S43. Upsample the coarse-grained localization map to the original image resolution, and use a color mapping table to map the activation values ​​to RGB color values ​​to generate a visual lesion heat map superimposed on the original image.

9. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 1, characterized in that, The implementation steps of the intelligent diagnostic report generation module include: S51. Traverse the high-response areas in the visualized lesion heatmap generated in S4, determine the corresponding anatomical structure nodes, and extract the set of key lesion factors calculated in S3 for that node. S52. Using a pre-set pathological description knowledge base, match the corresponding medical term templates according to the feature types of key lesion factors; the feature types include osteophyte formation, joint space narrowing, cartilage wear and bone marrow edema; fill the extracted feature parameters into the template slots to generate natural language descriptions; S53. For the anatomical structure nodes determined in S51, extract local features from the multimodal fusion feature vector obtained in S2, compare them with historical local features, calculate the similarity, and obtain the pathological feature type; render the original image according to the pathological feature type to obtain intelligent diagnostic image. S54. Based on the framework of the standard diagnostic report template, fill the structured text description generated in S52 into the "Image View" column, embed the intelligent diagnostic image rendered in S53 into the "Key Image" column, complete the structured assembly of the text and image content, and finally render and export the complete intelligent analysis report.

10. The artificial intelligence-based automatic joint image analysis and diagnosis system according to claim 1, characterized in that, The implementation steps for constructing the pathological description knowledge base in S52 include: S521. Collect historical joint imaging diagnostic reports and extract the image description paragraphs and diagnostic conclusions from them; S522. Utilize natural language processing technology to perform entity recognition and relation extraction on the descriptive paragraphs, and construct a set of triplets of <anatomical location, pathological features, degree modifiers>. S523. Conduct frequency statistics and expert review on the triplet set, select high-frequency and standardized medical terminology combinations, establish a standardized key-value pair mapping table, and form a pathological description knowledge base.

Images