Analysis method, system and device for large-scale medical images, computer storage medium and computer program product

Abstract

The invention provides an analysis method, system and device for large-scale medical images, a computer storage medium and a computer program product. The method is applied to a central server in a distributed federated learning architecture. The method comprises the following steps: acquiring an MPR feature vector and a DICOM metadata feature vector which are uploaded by an edge node Yi and obtained based on a tomography medical image set of the same sequence Qi; obtaining a comprehensive feature vector through a bidirectional cross attention mechanism; and processing the comprehensive feature vector by using a neural network model based on a Transform architecture to obtain an identification result of the focus, and obtaining an analysis report of the large-scale medical image corresponding to the edge node cluster based on a set of identification results obtained by all tomography medical images based on the same sequence on all edge nodes in the edge node cluster. According to the scheme, on the premise that data privacy is guaranteed, the processing efficiency and accuracy of large-scale medical images are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to methods, systems, devices, computer storage media, and computer program products for analyzing large-scale medical images. Background Technology

[0002] In today's large-scale medical imaging screening applications, medical imaging data is experiencing explosive growth, with its volume increasing exponentially. For example, a regional health screening program can generate tens of thousands of computed tomography (CT) images daily. Faced with such a massive data volume, traditional manual analysis methods have revealed serious efficiency bottlenecks. Specifically, the diagnostic process for a single CT image takes 15-30 minutes, a processing speed far from meeting the stringent timeliness requirements of large-scale screening programs. Furthermore, prolonged and continuous image reading can easily lead to visual fatigue for doctors, significantly increasing the rate of missed diagnoses of minute lesions and affecting the accuracy of diagnostic results.

[0003] Meanwhile, while existing automated analysis methods possess some capability in processing large-scale data, their processing efficiency still falls short of the actual needs of large-scale medical image screening. More critically, existing automated analysis methods heavily rely on manual design for feature extraction. This approach not only consumes significant human, material, and time resources but also struggles to comprehensively and accurately capture complex feature information in image data, thus limiting the accuracy and reliability of diagnosis to some extent.

[0004] Therefore, how to improve the processing efficiency and accuracy of large-scale medical images is a technical problem that urgently needs to be solved in the field of medical image analysis. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide methods, systems, devices, computer storage media, and computer program products for analyzing large-scale medical images, which can improve the processing efficiency and accuracy of large-scale medical images.

[0006] In a first aspect, embodiments of the present invention provide a method for analyzing large-scale medical images, applied to a central server in a distributed federated learning architecture. The distributed federated learning architecture includes an edge node cluster and the central server. The edge node cluster is deployed in different computed tomography (CT) medical image acquisition institutions. The analysis method includes: acquiring multi-planar reconstruction image queue (MPR) feature vectors obtained from a set of CT medical images based on the same sequence Qi uploaded by edge node Yi, and applying these vectors to digital imaging and communications in medical fields. Medicine (DICOM) metadata feature vector; the edge node Yi is any edge node in the edge node cluster, and the sequence Qi is any set of tomographic medical images in the set obtained by grouping edge node Yi according to the same sequence; the MPR feature vector and the DICOM metadata feature vector are fused through a bidirectional cross-attention mechanism to generate a comprehensive feature vector containing lesion semantics and spatial details; the comprehensive feature vector is processed using a neural network model based on the Transform architecture to obtain the lesion identification result, and the identification result is sent to the edge node Yi; based on the set of identification results obtained from all tomographic medical images based on the same sequence on all edge nodes in the edge node cluster, an analysis report of large-scale medical images corresponding to the edge node cluster is obtained.

[0007] This scheme is based on a distributed federated learning architecture. Medical images are acquired through edge nodes, while the raw data remains at each medical imaging institution. Edge nodes only upload MPR feature vectors and DICOM metadata feature vectors derived from the raw data, avoiding the cross-institutional transmission of sensitive medical data and meeting medical data privacy protection regulations. Simultaneously, the participation of edge nodes in data processing reduces the computational burden on the central server, facilitating efficient analysis of large-scale medical images. By utilizing a bidirectional cross-attention mechanism to fuse MPR feature vectors (spatial details) and DICOM metadata feature vectors (scanning parameters), and combining this with the model processing capabilities of the Transformer architecture, multimodal information is effectively integrated to improve lesion identification accuracy. Therefore, this scheme significantly improves the processing efficiency and accuracy of medical images from large populations while ensuring data privacy.

[0008] In conjunction with the first aspect, in the first possible implementation of the first aspect, before fusing the MPR feature vector and the DICOM metadata feature vector through a bidirectional cross-attention mechanism to generate a comprehensive feature vector containing lesion semantics and spatial details, the method further includes, when the dimensions of the MPR feature vector and the DICOM metadata feature vector are not aligned, performing layer normalization, standardizing feature distribution, and projecting them to the same dimensional space on the MPR feature vector and the DICOM metadata feature vector respectively, to obtain dimensionally aligned MPR feature vector and DICOM metadata feature vector.

[0009] Before feature fusion, the above scheme performs layer normalization, standardization, and dimensional projection on the MPR feature vector and DICOM metadata feature vector when their dimensions are not aligned. This eliminates the dimensional heterogeneity between the two, ensuring that the features are effectively fused in the same dimensional space. This improves the efficiency of feature interaction, provides higher-quality input data for the subsequent bidirectional cross-attention mechanism, and enhances the model's ability to extract and analyze medical image features.

[0010] In conjunction with the first aspect, in a second possible implementation of the first aspect, the bidirectional cross-attention mechanism includes: using the MPR feature vector as a query and the DICOM metadata feature vector as a key and value, generating an MPR attention output guided by metadata semantics to suppress noise interference on lesion edges; and using the DICOM metadata feature vector as a query and the MPR feature vector as a key and value, generating a DICOM attention output that integrates anatomical details to locate the spatial position of the lesion.

[0011] The above scheme clarifies the specific implementation of the bidirectional cross-attention mechanism. It uses MPR feature vectors to guide and suppress noise interference at the lesion edges, while simultaneously utilizing DICOM metadata feature vectors to locate the spatial position of the lesion, achieving complementary advantages between the two modalities. This mechanism effectively reduces the impact of image noise, strengthens the capture of the spatial relationship between anatomical structures and lesions, and improves the accuracy of lesion identification.

[0012] In conjunction with the first aspect, in a third possible implementation of the first aspect, the fusion includes: generating dynamic weights through a feature gating layer to adaptively adjust the contribution of MPR features and the DICOM metadata features, wherein, when identifying lesions smaller than a preset threshold, the spatial detail weight of the MPR features is enhanced, and the scanning parameter weight of the DICOM metadata is enhanced in noise-sensitive scenarios.

[0013] The aforementioned scheme generates dynamic weights through a feature gating layer, adaptively adjusting the contribution of MPR features and DICOM metadata features based on lesion size and scene requirements. It enhances the weights of spatial details for small lesions and strengthens the weights of scanning parameters in noisy environments. This mechanism enables the model to flexibly adapt to the analysis needs of different types of medical images, significantly improving its ability to identify lesions in complex images.

[0014] In conjunction with the first aspect, in the fourth possible implementation of the first aspect, an analysis report of large-scale medical images corresponding to the edge node cluster is obtained based on the set of recognition results obtained from all tomographic medical images based on the same sequence on all edge nodes in the edge node cluster. This report includes: performing statistical analysis based on the set of recognition results to calculate at least one of the following indicators: lesion detection rate, misdiagnosis rate, and the influence coefficient of different equipment parameters on the recognition results; and / or performing statistical analysis based on the set of recognition results to generate visualization charts, including heat maps of lesion distribution in different institutions and scatter plots of the correlation between equipment parameters and recognition accuracy.

[0015] The above scheme involves the central server performing multi-dimensional statistical analysis and generating visual charts. By calculating indicators such as lesion detection rate, misdiagnosis rate, and equipment parameter influence coefficient, as well as generating lesion distribution heat maps and parameter-accuracy correlation scatter plots, the medical image analysis results are transformed into intuitive and understandable decision-making basis. This helps medical institutions optimize scanning plans, evaluate diagnostic effects, and provide reference information and effective support for clinical and public health decision-making.

[0016] In conjunction with the first aspect, in the fifth possible implementation of the first aspect, the distributed federated learning architecture further includes: the secure communication protocol between each edge node in the edge node cluster and the central server adopts a TLS 1.3 encrypted channel with a key length ≥ 2048 bits.

[0017] The above scheme employs a TLS 1.3 encrypted channel with a key length of ≥2048 bits in the distributed federated learning architecture, providing strong security for data transmission between edge nodes and the central server. This ensures the privacy and security of medical image data during transmission, complying with medical data privacy protection regulations. Simultaneously, it maintains efficient data communication while ensuring security, achieving a balance between security and efficiency.

[0018] Secondly, embodiments of the present invention provide an analysis system for large-scale medical images. The medical image analysis system employs a distributed federated learning architecture, comprising: an edge node cluster and a central server. The edge node cluster is deployed in different computed tomography (CT) medical image acquisition institutions. Each edge node in the edge node cluster is used to acquire CT medical images and, based on the locally acquired CT medical images, obtain an MPR feature vector and a DICOM metadata feature vector corresponding to a set of CT images based on the same sequence. The central server is used to acquire the MPR feature vector and DICOM metadata feature vector obtained from the set of CT medical images based on the same sequence uploaded by the edge node Yi. The edge node Yi is part of the edge node cluster. In the context of any edge node, the sequence Qi is any set of tomographic medical images obtained by grouping edge node Yi according to the same sequence; the central server is further configured to fuse the MPR feature vector and the DICOM metadata feature vector through a bidirectional cross-attention mechanism to generate a comprehensive feature vector containing lesion semantics and spatial details; and to process the comprehensive feature vector using a neural network model based on the Transform architecture to obtain the lesion identification result, and send the identification result to the edge node Yi; and to obtain an analysis report of large-scale medical images corresponding to the edge node cluster based on the set of identification results obtained from all tomographic medical images based on the same sequence on all edge nodes in the edge node cluster.

[0019] Thirdly, embodiments of the present invention provide an analysis device for large-scale medical images. The medical image analysis device serves as a central server in a distributed federated learning architecture, which includes an edge node cluster and the central server. The edge node cluster is deployed in different computed tomography (CT) medical image acquisition institutions. The medical image analysis device includes a processor configured to execute computer-executable instructions; and a memory storing one or more computer-executable instructions. When executed by the processor, the computer-executable instructions implement the steps of the large-scale medical image analysis method as described in the first aspect or any possible implementation of the first aspect.

[0020] Fourthly, embodiments of the present invention provide a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method for analyzing large-scale medical images as described in the first aspect or any possible implementation thereof.

[0021] Fifthly, embodiments of the present invention provide a computer program product, including a computer program that, when executed by a processor, implements the method for analyzing large-scale medical images as described in the first aspect or any possible implementation thereof.

[0022] In a sixth aspect, embodiments of the present invention provide a chip system applied to an electronic device. The chip system includes one or more processors, which are configured to invoke computer instructions to cause the electronic device to perform the method for analyzing large-scale medical images as described in the first aspect or any possible implementation thereof.

[0023] Understandably, the technical effects achieved by the large-scale medical image analysis system described in the second aspect, the large-scale medical image analysis device described in the third aspect, the computer storage medium described in the fourth aspect, the computer program product described in the fifth aspect, and the chip system described in the sixth aspect are similar to the technical effects achieved by the corresponding technical means in the large-scale medical image analysis method described in the first aspect, and will not be repeated here. Attached Figure Description

[0024] Figure 1A This is a flowchart illustrating a method for analyzing large-scale medical images according to an embodiment of the present invention. Figure 1B This is a flowchart illustrating a method for analyzing large-scale medical images according to another embodiment of the present invention; Figure 2 This is a schematic diagram of the process of obtaining DICOM metadata feature vectors from edge nodes in one embodiment of the present invention; Figure 3A This is a schematic diagram of the process of obtaining MPR feature vectors from edge nodes in one embodiment of the present invention; Figure 3B This is a schematic diagram of the architecture of a deep learning model for processing MPR image queues in one embodiment of the present invention; Figure 4A This is a schematic diagram of a model architecture for fusing MPR feature vectors and DICOM metadata feature vectors according to an embodiment of the present invention; Figure 4B yes Figure 4A It includes a clear diagram of the areas corresponding to labels 401, 402, and 403; Figure 4C yes Figure 4A It includes a clear diagram of the areas corresponding to labels 404, 405, and 406; Figure 4D yes Figure 4A It includes a clear diagram of the area indicated by the 407 label; Figure 5A This is a schematic diagram of the structure of a neural network model based on the Transformer architecture in one embodiment of the present invention; Figure 5B yes Figure 5A The document includes a clear diagram of the area corresponding to identifier 501; Figure 5C yes Figure 5A A clear diagram showing the areas corresponding to markings 5021 / 5022 / 5023; Figure 5D yes Figure 5A A clear diagram showing the areas corresponding to markings 503, 504, and 505; Figure 6 This is a schematic diagram of the structure of a device for analyzing large-scale medical images provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0026] It should be understood that the term "multiple" in this invention refers to two or more. In the description of this invention, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, terms such as "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply differences.

[0027] With the rapid development of medical technology, medical imaging plays an increasingly crucial role in disease diagnosis, treatment planning, and disease monitoring. Computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) are widely used in clinical diagnosis. However, current technologies face numerous significant challenges in processing and analyzing large-scale medical images. On the one hand, the imaging equipment and scanning parameters used by different medical institutions vary greatly, resulting in significant differences in the format, quality, and feature representation of the acquired medical image data. For example, images generated by different brands of CT equipment have different resolutions, contrasts, and imaging effects on tissues and organs. When it is necessary to conduct comprehensive analysis of large-scale medical images from multiple institutions, this data heterogeneity greatly increases the difficulty of analysis and reduces the accuracy and reliability of the analysis results. On the other hand, traditional medical image analysis methods largely rely on doctors manually interpreting images, which is not only inefficient but also highly susceptible to subjective factors such as the doctor's experience and fatigue. Faced with the ever-increasing volume of medical image data, manual interpretation is insufficient to meet the clinical demands for rapid and accurate diagnosis. Although artificial intelligence technology has been applied in medical image analysis in recent years, such as deep learning-based image recognition algorithms, these algorithms often require large amounts of labeled data for training, and medical image annotation is time-consuming, labor-intensive, and its accuracy is difficult to guarantee. Furthermore, medical imaging data contains a wealth of sensitive patient information, making data privacy and security paramount during data transmission and sharing. Especially when multiple institutions collaborate on large-scale medical image analysis, achieving efficient collaborative analysis while ensuring data privacy becomes a pressing issue. Current data collaboration solutions either lack sufficient security or suffer from significantly reduced data processing efficiency due to complex encryption mechanisms. In conclusion, developing a large-scale medical image analysis method that can effectively integrate multi-source heterogeneous medical image data, improve analysis efficiency and accuracy, and ensure data privacy and security has significant practical significance and clinical application value.

[0028] This invention provides a method for analyzing large-scale medical images, which can improve the processing efficiency and accuracy of large-scale medical images.

[0029] To more clearly illustrate the method for analyzing large-scale medical images described in this invention, a detailed explanation is provided below with reference to specific embodiments. Assume there exists a regional medical alliance composed of multiple hospitals, each acting as a peripheral node, jointly participating in the analysis of large-scale medical images within the region to assist in the early screening and diagnosis of diseases.

[0030] Figure 1A This is a flowchart illustrating a method for analyzing large-scale medical images according to an embodiment of the present invention. The method is applied to a central server in a distributed federated learning architecture. The distributed federated learning architecture includes an edge node cluster and the central server. The edge node cluster is deployed in different computed tomography (CT) medical image acquisition institutions (e.g., multiple hospitals in a certain region). Figure 1AAs shown, the method for analyzing large-scale medical images may include steps S101 to S104.

[0031] S101. Obtain the MPR feature vector and DICOM metadata feature vector of the set of tomographic medical images based on the same sequence Qi uploaded by edge node Yi; edge node Yi is any edge node in the edge node cluster, and sequence Qi is any set of tomographic medical images in the set obtained by grouping edge node Yi according to the same sequence.

[0032] For example, Hospital A (as edge node Y1) acquires a large number of computed tomography (CT) images during its daily clinical practice. Hospital A's information system groups and organizes CT images based on the same sequence Q1 according to factors such as scan time and patient information. For instance, for chest CT images of the same sequence, images belonging to the same examination item are grouped together based on scan time sequence and patient medical record number. Hospital A utilizes a locally deployed feature vector generation module to generate both Multi-Planar Reconstruction Image Queue (MPR) feature vectors and DICOM (Digital Imaging and Communication in Medicine) metadata feature vectors for a set of medical images belonging to the same sequence. The MPR feature vectors primarily reflect detailed information such as the spatial location and shape of lesions in the images; the DICOM metadata feature vectors contain semantic content such as scanning equipment parameters and basic patient information. After feature extraction, Hospital A uploads these feature vectors to the central server via a secure communication protocol (e.g., using a TLS 1.3 encrypted channel with a key length ≥ 2048 bits).

[0033] In some possible implementations, such as Figure 1B As shown, after step S101, the following steps may also be included: S1011. When the MPR feature vector and the DICOM metadata feature vector are not aligned in dimension, perform layer normalization, standardize the feature distribution, and project them to the same dimension space for the MPR feature vector and the DICOM metadata feature vector respectively to obtain dimension-aligned MPR feature vector and DICOM metadata feature vector.

[0034] For example, after receiving the MPR feature vector and DICOM metadata feature vector uploaded by hospital A, the central server first checks their dimensions. If misalignment is found, for example, the MPR feature vector has a dimension of 1*128 while the DICOM metadata feature vector has a dimension of 1*64, the central server uses a layer normalization method to process the MPR and DICOM metadata feature vectors separately, making their data distribution more stable. Next, through feature distribution standardization, the data features are further standardized. Finally, a projection algorithm is used to project both onto the same dimensional space, resulting in dimension-aligned MPR and DICOM metadata feature vectors, both with a dimension of 1*64. The central server then uses a bidirectional cross-attention mechanism to fuse these dimension-aligned feature vectors. Calculations are performed using the MPR feature vector as the query and the DICOM metadata feature vector as the key and value. For example, through a series of matrix operations and attention weight allocation, an MPR attention output guided by metadata semantics is generated. This process can suppress noise interference on lesion edges because information such as scan parameters in the DICOM metadata helps determine which are genuine lesion edge features and which are noise interference. Similarly, using DICOM metadata feature vectors as queries and MPR feature vectors as keys and values, a DICOM attention output incorporating anatomical details is generated for precise spatial location of lesions. For example, spatial detail information in the MPR feature vectors can more accurately determine the specific location of lesions within the human body structure described by the DICOM metadata. Subsequently, the central server generates dynamic weights through a feature gating layer. For the identification of tiny lesions smaller than a preset threshold (e.g., 5 mm), the spatial detail weights of the MPR features are automatically enhanced to highlight the fine structural features of these lesions. In noise-sensitive scenarios, such as image areas near metallic implants, the scanning parameter weights of the DICOM metadata are enhanced, effectively reducing the impact of noise on the analysis results. Through this dynamic weight adjustment, a comprehensive feature vector containing both lesion semantics and spatial details is generated.

[0035] S102. By fusing MPR feature vectors and DICOM metadata feature vectors through a bidirectional cross-attention mechanism, a comprehensive feature vector containing lesion semantics and spatial details is generated.

[0036] In some possible implementations, the bidirectional cross-attention mechanism may include: using the MPR feature vector as the query and the DICOM metadata feature vector as the key and value, generating an MPR attention output guided by metadata semantics to suppress noise interference on lesion edges; and using the DICOM metadata feature vector as the query and the MPR feature vector as the key and value, generating a DICOM attention output that incorporates anatomical details to locate the spatial position of lesions.

[0037] In some possible implementations, the fusion includes: generating dynamic weights through a feature gating layer to adaptively adjust the contribution of MPR features and DICOM metadata features. Specifically, when identifying lesions smaller than a preset threshold, the spatial detail weight of the MPR features is enhanced, and the scanning parameter weight of the DICOM metadata is enhanced in noise-sensitive scenarios.

[0038] S103. The comprehensive feature vector is processed using a neural network model based on the Transform architecture to obtain the lesion identification result, and the identification result is sent to the edge node Yi.

[0039] For example, the central server uses a neural network model based on the Transform architecture to process the comprehensive feature vector. The model employs a multi-layered self-attention mechanism and a feedforward neural network structure to deeply mine and analyze lesion information within the comprehensive feature vector, ultimately obtaining lesion identification results, such as determining whether a tumor exists in the image and the nature of the tumor (benign or malignant). The central server then sends these identification results back to Hospital A.

[0040] S104. Based on the set of recognition results obtained from all tomographic medical images based on the same sequence on all edge nodes in the edge node cluster, obtain an analysis report of the large-scale medical images corresponding to the edge node cluster.

[0041] In some possible implementations, statistical analysis can be performed based on the set of identification results to calculate at least one of the following indicators: lesion detection rate, misdiagnosis rate, and the influence coefficient of different equipment parameters (such as CT slice thickness, MRI sequence type, etc.) on the identification results; and / or, statistical analysis can be performed based on the set of identification results to generate visualization charts, including heat maps of lesion distribution in different institutions and scatter plots of the correlation between equipment parameters and identification accuracy.

[0042] For example, other hospitals within the region (such as Hospital B, Hospital C, and other edge nodes) also follow the same process, uploading their respective sets of medical image feature vectors to the central server and receiving the recognition results. The central server then performs statistical analysis based on the set of recognition results obtained from all tomographic medical images of the same sequence on all edge nodes in the edge node cluster.

[0043] The analysis process includes calculating lesion detection rates, such as the proportion of images accurately detecting lesions among all analyzed images; calculating misdiagnosis rates, i.e., the proportion of images incorrectly identifying lesions; and analyzing the impact coefficients of different equipment parameters on recognition results, such as studying the relationship between tube voltage and tube current parameters of different CT equipment and tumor recognition accuracy. Simultaneously, it generates visualization charts, such as creating heat maps of lesion distribution across different institutions to visually display the distribution density of lesions within each hospital's area; and generating scatter plots showing the correlation between equipment parameters and recognition accuracy, clearly presenting the trend of how changes in equipment parameters affect recognition accuracy. These statistical analysis results and visualization charts constitute a large-scale medical image analysis report corresponding to the regional medical alliance, providing strong data support for medical institutions to optimize scanning protocols and improve diagnostic capabilities.

[0044] This implementation is based on a distributed federated learning architecture. Medical images are acquired through edge nodes, while the raw data remains at each medical image acquisition institution. Edge nodes only upload MPR feature vectors and DICOM metadata feature vectors derived from the raw data, avoiding the cross-institutional transmission of sensitive medical data and meeting medical data privacy protection regulations. Simultaneously, the participation of edge nodes in data processing reduces the computational burden on the central server, facilitating efficient analysis of large-scale medical images. By utilizing a bidirectional cross-attention mechanism to fuse MPR feature vectors (spatial details) and DICOM metadata feature vectors (scanning parameters), and combining this with the model processing capabilities of the Transformer architecture, multimodal information is effectively integrated to improve lesion identification accuracy. Therefore, this solution significantly improves the processing efficiency and accuracy of medical images from large-scale populations while ensuring data privacy.

[0045] The DICOM feature vectors obtained from edge node pairs of tomographic medical images belonging to the same sequence can be referenced. Figure 2 As shown, it includes steps S201 to S205.

[0046] S201. Based on medical images, load the DICOM file sequence corresponding to the image set belonging to a sequence obtained from tomographic scanning, and arrange them according to the scanning order to obtain the initial image file queue.

[0047] S202. Read DICOM files from a specified position in the initial image file queue to form a fixed-length key image file queue.

[0048] S203. Read the sequence-level DICOM metadata and image-level DICOM metadata used to represent key features from the key image file queue, arrange them according to preset rules, and obtain a sequence DICOM metadata queue with fixed length and fixed information items at each position.

[0049] S204. Input the sequence DICOM metadata queue into the deep learning network, which includes a feature preprocessing and classification layer, a feature encoding layer, a feature fusion layer, a context modeling layer, and an output layer.

[0050] It should be noted that deep learning networks obtain the sequence DICOM metadata queue through the input layer.

[0051] S205. Trigger the deep learning network to transform the sequence DICOM metadata queue into a fixed-dimensional feature vector.

[0052] It should be noted that imaging examinations generally use DICOM files for data transmission and storage. DICOM is an international standard in the field of medical imaging (ISO 12052), which defines the format, transmission protocol and storage architecture of medical imaging data, and is widely used in the storage, transmission and sharing of medical imaging information.

[0053] According to the DICOM standard, a single patient imaging examination corresponds to a "Study" object, which typically contains one or more Series objects, each of which usually includes multiple Image objects. Conventional sequences of computed tomography (CT), MRI, PET, and other medical imaging techniques are characterized by continuous scanning of specific anatomical regions of the human body, acquiring a set of images with highly consistent spatial location and scanning direction, conforming to specific patterns. Through advanced 3D reconstruction algorithms, these 2D images can be precisely reconstructed into 3D volumetric images, providing more comprehensive and three-dimensional imaging information for precision medical diagnosis.

[0054] Different sequences correspond to different scanning sites or imaging parameters. In the storage, retrieval, processing, and analysis of medical images, it is crucial to accurately identify the scanning site (such as the head, chest, abdomen, pelvis, etc.) corresponding to each sequence.

[0055] Currently, the identification of scanned areas mainly relies on relevant label information in the DICOM header file, such as study descriptions and series descriptions. However, this descriptive information often lacks standardization due to differences between hospitals, equipment manufacturers, or operators, and may even contain erroneous information, resulting in low accuracy in scanned area identification. Furthermore, relying solely on these text labels for identification cannot fully utilize the rich visual information contained in the images themselves.

[0056] The DICOM file format includes a header information section. This section conforms to the DICOM standard and uses a standard tag list to record and store additional information for each image file, i.e., metadata information, including the examination type, scan parameters, and scanned area. The DICOM header information, or metadata, for different images in a sequence consists of two parts: one part is information common to all images in the sequence, i.e., the parts that are the same for all images in the sequence; the other part is information unique to each image, i.e., each image is different. This differing information is mainly spatial location information, usually arranged and changing according to certain rules, such as an arithmetic progression.

[0057] The DICOM header information set, or metadata set, of different images in a sequence contains a large amount of examination-related information, such as examination type, scan parameters, scan site, and slice spatial location. If this information can be transformed into feature vectors that neural networks can receive and process, especially fixed-length one-dimensional feature vectors, it will make intelligent processing of images within the sequence (such as disease detection and lesion identification) possible.

[0058] Since a sequence may contain hundreds or even thousands of images, directly selecting the DICOM header information set of all images as the feature vector would result in an excessively long feature vector containing a large amount of redundant information. Furthermore, the variable number of images in a sequence also leads to variable feature vector lengths. Deep learning networks, due to their inherent characteristics, require both concise and redundant input feature vectors, while maintaining a fixed length. Therefore, existing technologies limit the processing of DICOM files by deep learning networks. This application addresses this by removing redundant information from DICOM metadata and establishing a standardized data processing mechanism to unify feature vector lengths, thus meeting the requirements of deep learning networks for concise and fixed-dimensional input data.

[0059] It should be noted that step S205 aims to convert the sequence DICOM metadata queue into a high-dimensional feature vector, effectively capturing the complex patterns and relationships in the DICOM metadata through a deep learning network, thereby generating a DICOM metadata feature representation for subsequent analysis.

[0060] The DICOM metadata encoder employs a deep neural network based on the Transformer architecture, specifically designed to handle the heterogeneous characteristics of medical image DICOM metadata. This encoder can simultaneously process categorical, continuous, textual, and missing value features, generating feature vectors that highly represent sequential DICOM metadata information through multi-level feature extraction and self-attention mechanisms.

[0061] The final output DICOM metadata feature vector is a high-dimensional vector (e.g., 256-dimensional) containing rich semantic information of the sequence DICOM metadata. It can effectively represent key characteristics of medical images such as scanning parameters, equipment information, and patient location, laying the foundation for subsequent medical image analysis.

[0062] This feature vector will serve as an important component of multimodal feature fusion, and will be fused with image information from medical images, enabling its use in scenarios such as automatic identification of scanned areas.

[0063] The functions and processing procedures of each layer in this deep learning network will be explained one by one below.

[0064] (1) Preprocessing and classification layer The input sequence DICOM metadata queue is first classified and processed according to data type. Categorical features (such as scan parameters, device type, etc.) are transformed through one-hot encoding or embedding technology; continuous features (such as pixel values, scan spacing, etc.) are processed using standardization or Min-Max normalization; textual features (such as inspection description, sequence name, etc.) are processed using sequence encoding methods; missing values ​​are processed using special masking to preserve the semantic meaning of missing information.

[0065] (2) Feature coding layer Categorical features are converted into dense vector representations through an embedding layer (nn.Embedding). Continuous features are processed by linear projection and batch normalization layers to adjust the dimensionality and numerical distribution. Textual features are extracted using a dedicated Transformer encoder to extract semantic information. Masked features are encoded with positional markers to ensure that the model can identify the location and pattern of missing values.

[0066] (3) Feature fusion layer The encoded features of each type are concatenated to form a complete metadata representation. A multi-head attention mechanism is applied to enhance the interaction between features. The attention weights are dynamically adjusted to adjust the importance of each metadata field, highlighting key information related to the identification of the scanned part.

[0067] (4) Context modeling layer The first feedforward network layer uses linear transformation and GELU activation function to extract high-level feature patterns. Layer normalization maintains training stability and accelerates convergence. The second feedforward network layer further extracts abstract features and applies layer normalization again to ensure stable feature distribution.

[0068] (5) Output layer Global feature pooling summarizes the entire sequence information to generate a fixed-dimensional representation. A fully connected layer adjusts the feature dimension to the required size, batch normalization ensures stable feature distribution, and a Dropout layer (p=0.1) prevents overfitting. Finally, the DICOM metadata feature vector is output with a preset dimension (e.g., 256 or 512).

[0069] The MPR feature vectors obtained from edge nodes can be obtained by using... Figure 3A The process is carried out in the steps shown (S301 to S306).

[0070] S301. Acquire raw DICOM data based on computed tomography (CT) technology. The raw DICOM data can originate from scans of a specified area of ​​the patient using computed tomography equipment such as CT, MRI, or PET. Each scan sequence contains a set of spatially consistent and continuous two-dimensional tomographic images. For example, a chest C-scan sequence may contain 500-1000 cross-sectional images, each carrying anatomical, tissue, or metabolic information from different layers of the same scanned area.

[0071] S302. Construct 3D volumetric data based on the original DICOM data. According to the spatial coordinate information (such as Image Position (Patient) and Image Orientation (Patient) labels) in the DICOM file, sort and integrate the data according to the spatial position of the scanned slices. For example, arrange the transverse images in ascending order of the Z-axis coordinate, and the coronal images in ascending order of the Y-axis coordinate, ensuring that adjacent slices are continuous and without misalignment in 3D space. Fill the pixel gaps using interpolation algorithms (such as cubic spline interpolation) to construct a complete 3D volumetric data cube, whose spatial resolution is consistent with the original tomographic images, reflecting the 3D distribution characteristics of human tissue.

[0072] S303. Perform multi-plane reconstruction based on three-dimensional volume data to generate an MPR image queue.

[0073] The MPR image queue includes at least: coronal MPR images, sagittal MPR images, and transverse MPR images. For ease of description, this application uses the example of an MPR image queue including coronal MPR images, sagittal MPR images, and transverse MPR images. In some possible implementations, the MPR image queue can be arranged in the following order: coronal MPR images, sagittal MPR images, and transverse MPR images.

[0074] In some possible implementations, multi-planar reconstruction based on 3D volumetric data is used to generate an MPR image queue. This includes: slicing along the coronal, sagittal, and transverse axes based on 3D volumetric data to generate a 2D image sequence with orthogonal planes, where slice thickness and spacing are configurable. For example, slice thickness and spacing can be configured manually, automatically by the inspection equipment, or partially automatically and partially manually. For instance, parameter values ​​can be automatically determined based on a pre-trained deep learning model. In oncology radiotherapy departments using standardized scanning protocols, multiple follow-up CT scans of the same patient can use the same slice thickness parameters. In emergency rooms, when rapidly processing CT scans of patients with cerebral hemorrhage, the system can automatically select the optimal slice thickness parameters to highlight the hemorrhage area. Hybrid configuration applications include CT scans of children; for example, MPR parameters can be dynamically adjusted according to the pediatric patient's age, while allowing doctors to fine-tune them.

[0075] S304. Input the MPR image queue into the deep learning model for processing. After the MPR image queue is processed sequentially through the backbone feature extraction module, the attention enhancement module, and the in-plane multi-scale feature fusion module, a multi-scale feature map corresponding to each MPR image in the MPR image queue is obtained.

[0076] The deep learning model includes: a backbone feature extraction module with convolutional blocks, an attention enhancement module, and an in-plane multi-scale feature fusion module, a multi-plane interaction module, and a feature fusion and encoding module, all comprising a feature pyramid network. In some possible implementations, the attention module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule uses global average pooling and a multilayer perceptron to weight channel features, while the spatial attention submodule uses convolution and a non-linear activation function to enhance spatial features. The non-linear activation function can be any of the following: Sigmoid, ReLU, or Swish.

[0077] S305. Use the multi-plane interaction module to perform cross-plane association and fusion of multi-scale feature maps to obtain a multi-plane fused feature map with a two-dimensional structure.

[0078] In some possible implementations, the multi-plane interaction module calculates the similarity matrix between the multi-scale feature maps corresponding to each MPR image in the MPR image queue, and adjusts the attention weight values ​​of the collaborative diagnostic features based on the similarity matrix and according to the preset attention weight enhancement rules.

[0079] S306. The feature fusion and encoding module is used to perform global pooling and fully connected encoding on the feature map fused from the multi-plane plane, and the output includes a fixed-length MPR feature vector.

[0080] In some possible implementations, the workflow of each module in this deep learning module can be referenced as follows: Figure 3B As shown.

[0081] The MPR image sequence obtained after processing by the MPR image encoder includes: coronal MPR, sagittal MPR, and transverse MPR. Then, the coronal MPR, sagittal MPR, and transverse MPR are respectively input into the backbone feature extraction module (ResNet or EfficientNet) in the deep learning module, which includes convolutional blocks. Then, in parallel, corresponding to a set of convolutional blocks 1, 2, 3, and 4, features are extracted step by step from shallow to deep layers, capturing low-level texture to high-level semantic information.

[0082] The input to the backbone feature extraction module can be a single-plane MPR image (such as a coronal lung image), which extracts features at different levels (from texture to semantics) from the MPR image and outputs feature maps of different depths (shallow layers contain edge textures, and deep layers contain organ semantics).

[0083] For example, network structures such as ResNet can be used to extract features step by step through multiple convolutional layers: shallow convolutions identify lines and edges (e.g., the outline of lung lobes); deep convolutions identify the overall shape of the organ (e.g., the outline of the entire lung). For instance, after inputting a coronal lung image, the backbone feature extraction module first finds the lines of the lung edges (shallow layer), and then combines these lines to identify the overall shape of the left and right lungs (deep layer).

[0084] The attention enhancement module adds a Convolutional Block Attention Module (CBAM) after each intermediate layer (Convolutional Block 2, Convolutional Block 3, and Convolutional Block 4). The CBAM module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule uses global average pooling and a multilayer perceptron to weight channel features, while the spatial attention submodule uses convolution and non-linear activation functions to enhance spatial features. This adaptively emphasizes important feature channels and spatial locations, improving the model's ability to focus on key anatomical structures. In simpler terms, it tells the model "where to look," strengthening key region features. Its input is the output (feature map) of the backbone feature extraction module. The output is a weighted feature map (with key regions highlighted). Specifically, the channel attention submodule determines which feature channels (e.g., the "lung nodule texture" channel) are more important; the spatial attention submodule determines which locations in the feature map (e.g., the upper lobe region of the lung) require more attention. For example, if a lung nodule is present in an image, CBAM will assign higher weights to the "nodule texture channel" and "spatial location of the nodule", allowing the model to focus on analyzing that area and ignore irrelevant structures such as ribs.

[0085] The in-plane multi-scale feature fusion module includes three Feature Pyramid Networks (FPNs). Each FPN module (coronal, sagittal, and transverse) independently processes the features of that plane, outputting a feature map that considers both the overall picture and the details of that plane. Coronal FPN: Integrates large-scale (e.g., overall lung morphology) and small-scale (e.g., nodular spiculation) features of the coronal plane; Sagittal FPN: Integrates large-scale (e.g., the positional relationship between the heart and lungs) and small-scale (e.g., interstitial abnormalities) features from the sagittal plane; Transverse FPN: Integrates large-scale (e.g., the layered structure of the thoracic cavity) and small-scale (e.g., nodular lobulation) features from the transverse plane.

[0086] The three feature maps output by FPN are from the in-plane multi-scale feature fusion module, one for each plane, integrating information from different scales within that plane. For example, when analyzing lung CT scans, the in-plane multi-scale feature fusion module simultaneously focuses on the "morphology of the entire lung" (large scale) and "small nodules of 2mm in the lung" (small scale), avoiding missed diagnoses due to nodules being too small.

[0087] The multi-plane interaction module connects features from three planes to form a three-dimensional understanding. The inputs to the multi-plane interaction module include FPN feature maps from the coronal plane, sagittal plane, and transverse plane. The output includes cross-plane fused feature maps (e.g., the correlation features of "coronal nodule spiculation + sagittal nodule location"). Specifically, the multi-plane interaction module can use attention interaction to strengthen the malignancy correlation weight between related features from the three planes, such as when "spiculation" in the coronal plane and "lobulation" in the transverse plane appear simultaneously. For example, looking at the transverse plane alone might suggest a benign nodule, but combining the spiculation features from the coronal plane, the multi-plane interaction module will determine that the probability of malignancy is higher when "spiculation + lobulation" appears simultaneously, thus avoiding misjudgment from a single plane.

[0088] The feature fusion and encoding module compresses cross-plane features into a diagnostic vector. The input to this module can include the feature map output from the multi-plane interaction module. The output is a one-dimensional diagnostic vector (e.g., [0.9, 0.1] represents a 90% probability of malignancy). Specifically, the module can include a global pooling module and a fully connected layer module. The global pooling module extracts global features from each plane (e.g., whether there are abnormalities in the entire lung); the fully connected layer maps the features to diagnostic probabilities. For example, the feature fusion and encoding module integrates features from three planes and outputs a numerical vector. For instance, a value close to 1 indicates a high probability of malignancy, and a value close to 0 indicates a high probability of benignity, providing doctors with quantitative diagnostic information.

[0089] After obtaining the MPR feature vector and the DICOM metadata feature vector, the central server processes them to obtain a comprehensive feature vector. In some possible implementations, the deep learning network shown in Figure 4 can be used for this processing. This deep learning network may include: an input layer 401, a feature preprocessing and classification layer 402, a cross-attention layer 403, a feature enhancement layer 404, a feature gating layer 405, a feature integration layer 406, and an output layer 407.

[0090] The following is combined with Figure 4A The function and processing of each layer in a deep learning network will be explained one by one. It should be noted that... Figure 4B , Figure 4C and Figure 4D They are Figure 4A Part of it, because Figure 4A There's a lot of content, and when displayed on the same page, the images aren't clear enough. Figure 4B , Figure 4C and Figure 4D For reference Figure 4A The specific details.

[0091] First, the MPR feature vector and the DICOM metadata feature vector are obtained through the input layer 401.

[0092] The feature preprocessing layer 402 performs standardization and dimension alignment operations on the MPR and DICOM feature vectors, respectively. When the MPR and DICOM feature vectors have different numbers of elements and are dimensionally misaligned, layer normalization, standardization of feature distribution, and projection onto the same dimensional space are performed on the MPR and DICOM metadata feature vectors, respectively, to obtain dimension-aligned MPR and DICOM metadata feature vectors. Specifically, layer normalization (LayerNorm) eliminates the distribution differences between feature dimensions within a single sample (such as differences in numerical range, mean, and variance of different dimensions). Standardization adjusts the mean and variance of each feature dimension in the entire dataset, eliminating global distribution differences between feature dimensions. Then, a linear projection layer maps the two types of features with different dimensions to the same dimensional space (e.g., unified to 256 dimensions), enabling multimodal features to have the basic conditions for subsequent cross-modal interaction. For example, suppose the original MPR feature vector is 128-dimensional, and the values ​​of each dimension are scattered due to differences in image reconstruction algorithms (e.g., "nodule diameter" is 0-5cm, "edge smoothness" is 0-1); the DICOM metadata is 64-dimensional, with "age" ranging from 0-100 and "radiation dose" ranging from 0-500mGy. After layer normalization, the mean of each dimension of the MPR feature approaches 0 and the variance approaches 1, and the same applies to the DICOM feature; during linear projection, MPR is converted into a 256-dimensional vector through a 128×256 weight matrix, and DICOM is converted through a 64×256 weight matrix.

[0093] The cross-attention layer 403 constructs a bidirectional cross-attention mechanism, enabling MPR and DICOM features to engage in bidirectional "query-association-fusion". In the first stage, MPR features are used as the query and DICOM features as the key and value. Scaled Dot-Product Attention is used to calculate feature association weights, uncovering the focus of MPR spatial features on DICOM semantic information. In the second stage, DICOM features are used as the query and MPR features as the key and value, enabling DICOM semantics to guide MPR spatial features, completing the bidirectional flow and initial fusion of cross-modal information. For example, if the MPR feature "irregular nodule margin (value 0.8)" is used as a query, and its dot product is calculated with DICOM features such as "age > 60 years (value 0.7)" and "smoking history ≥ 20 years (value 0.9)", then after scaling (dividing by the square root of 256 to obtain a value of 16) and using Softmax to obtain weights (e.g., "age" weight 0.4, "smoking history" weight 0.6), the DICOM values ​​are weighted and summed to allow the MPR feature to incorporate the clinical semantics of "advanced age + long-term smoking". Similarly, if the DICOM feature "family history of lung cancer (value 0.8)" is used as a query, and associated with the MPR features "nodule lobulation sign (value 0.7)" and "spiculation sign (value 0.6)", after weighted fusion, the DICOM feature incorporates spatial morphological information.

[0094] In some possible implementations, the bidirectional cross-attention mechanism includes: using the MPR feature vector as the query and the DICOM metadata feature vector as the key and value, generating an MPR attention output guided by metadata semantics to suppress noise interference on lesion edges; and using the DICOM metadata feature vector as the query and the MPR feature vector as the key and value, generating a DICOM attention output that integrates anatomical details to locate the spatial position of lesions. For example, in lung CT image analysis, if an image is blurred due to equipment parameters, and the value of "nodule edge sharpness" in the MPR feature vector is only 0.3, it indicates that the nodule edge looks very blurry. In this case, the MPR feature vector is treated as the "questioner," and the DICOM metadata feature vector is treated as the "responder." The "responder" records information such as scan slice thickness and filtering algorithm. After calculating the correlation between the two, it is found that the scan slice thickness information is particularly important, and it is given a high score (weight) of 0.7. Combined with the information it provides that "thin slice scanning helps to see the edges clearly," the nodule edge sharpness value is adjusted to 0.5, making the edge information more accurate and reducing noise interference. Conversely, if the location of the patient's lesion recorded in the DICOM metadata is unclear, with a value of 0.2, the DICOM metadata feature vector is treated as the "questioner," and the MPR feature vector as the "responder." The MPR feature vector records information such as the specific location of the nodule in the lung lobe and its distance from blood vessels. After calculation, the information on "the location of the nodule in the lung lobe" is found to be the most useful, and it is given a high score (weight) of 0.8. With reference to this information, the clarity value of the lesion location is increased to 0.7 in order to more accurately locate the lesion.

[0095] Feature enhancement layer 404 is used to deepen feature representation through a combination of "multi-head mechanism + residual connection + feedforward network + layer normalization". Multi-head attention splits the cross-attention output into multiple subspaces for parallel processing, capturing multi-dimensional feature associations; residual connection directly transmits the original features, avoiding gradient vanishing and maintaining information integrity; feedforward network (FFN) extracts high-order feature patterns through nonlinear transformation (such as ReLU); layer normalization stabilizes the data distribution. The multi-stage collaboration enhances feature discriminative power, providing more representative features for subsequent classification. For example, the 256-dimensional features output by cross-attention are split into eight 32-dimensional subspaces by the multi-head mechanism. Subspace 1 focuses on the correlation between "nodule edge and age", while subspace 2 explores the relationship between "nodule density and smoking history". Residual connections add the original MPR / DICOM features (such as "nodule diameter" value of 0.5) to attention-enhanced features (value of 0.6) to preserve basic information. The feedforward network first maps the features to 512 dimensions (linear transformation + ReLU) and then compresses them back to 256 dimensions to enhance the expression of "malignant feature combinations (such as irregular edges + advanced age + long-term smoking)". The final layer normalization ensures that the feature distribution of each subspace is stable and outputs enhanced features that are easier to distinguish between benign and malignant.

[0096] Feature gating layer 405 is used to construct an adaptive gating mechanism to dynamically adjust the contribution ratio of MPR and DICOM enhanced features. For MPR and DICOM enhanced features, gating weights are generated through Sigmoid activation (with a value range constrained to [0,1]). Weight generation is driven by the feature's own attributes (such as the edge sharpness of MPR and the age value of DICOM) and the current diagnostic scenario (such as low-dose CT scan, conventional CT scan, or different disease types), automatically adapting to the differences in the importance of modal information to diagnosis under different scenarios. For example, in low-dose CT, MPR spatial details are easily interfered with by noise, so gating will reduce their weight; in conventional CT, MPR details are reliable, so the weight will be increased. This solves the problem of unbalanced contributions from multimodal features, guiding the model to focus on information with greater diagnostic value. For example, in the scenario of pulmonary disease diagnosis, if the input is MPR features from low-dose CT (due to high noise, the "nodule edge smoothness" dimension value is 0.3, indicating low reliability) and DICOM metadata ("patient age 70 years old" dimension value is 0.9, indicating clear information), the gating mechanism will be more effective. During gating, the gating weights are calculated for MPR enhanced features: after sigmoid activation, the weight output for "nodule edge features" is 0.3; for DICOM enhanced features, the weight for "age > 60 years" is 0.7 after sigmoid. This reduces the influence of unreliable spatial features and enhances clinical semantic information. If a conventional CT input is used, the MPR feature "nodule lobulation sign" has a dimension value of 0.8 (clear details), and the gating weight is increased to 0.8; the DICOM feature weight is adjusted to 0.2, flexibly adapting to the diagnostic needs of different scanning scenarios.

[0097] In some possible implementations, the fusion includes: generating dynamic weights through a feature gating layer to adaptively adjust the contribution of MPR features and DICOM metadata features. Specifically, when identifying lesions smaller than a preset threshold, the spatial detail weight of the MPR features is enhanced; and in noise-sensitive scenarios, the scanning parameter weight of the DICOM metadata is enhanced. For example, the preset threshold could be a lesion diameter of 3mm, etc., as a criterion. When the minimum circumscribed circle diameter of the lesion is less than 3mm, the spatial detail feature vector of the MPR features can be assigned a weight coefficient of no less than 0.7; when the noise index in the DICOM metadata exceeds 15%, the weight ratio of the scanning parameter feature vector of the DICOM metadata is increased to above 0.6, such as 0.65.

[0098] The feature integration layer 406 generates a comprehensive feature vector through a dual-path fusion strategy. The concatenation path directly connects the gated MPR and DICOM features, reducing dimensionality and preserving the original associations of the multimodal features. The weighted path calculates modality weights through attention, highlighting key information. The fusion of the two path results in a comprehensive feature vector that simultaneously contains lesion spatial details and semantic information, providing a foundation for the standardization process of the output layer. For example, if the gated MPR features (containing "edge irregularity 0.8") and the DICOM features (containing "age 0.7") are concatenated and dimensionality reduced, resulting in a fused expression with a certain dimension value of "0.8 (spatial) + 0.7 (semantic)," this fused expression is ultimately standardized into a unit vector by the output layer for use in downstream tasks.

[0099] Output layer 407 is designed to perform final processing on the results of the feature integration layer, generating a high-quality comprehensive feature vector. First, a feature transformation layer uses a linear transformation to adjust the dimensionality of the feature vector, making it suitable for the dimensionality requirements of subsequent tasks. Then, L2 normalization is used to scale the feature vector to a unit length, eliminating differences in feature scale between samples and ensuring numerical consistency and stability. After these two steps, a comprehensive feature vector containing lesion semantics and spatial details is finally output, providing a standardized feature representation for subsequent analysis and judgment tasks. For example, in the diagnosis of lung diseases, the feature integration layer outputs a 256-dimensional feature vector, which integrates spatial details of nodules from the MPR image (such as nodule edge irregularity, lobulation, density, etc.) and lesion semantic information from DICOM metadata (such as patient age, smoking history, family history, etc.). If a subsequent task requires a 64-dimensional feature vector, the output layer uses a linear transformation layer with a 256×64 weight matrix to convert the 256-dimensional feature vector into a 64-dimensional vector. The resulting 64-dimensional feature vector has different numerical ranges for each dimension, such as [2.5, 0.3, 1.8, ...]. To avoid the impact of numerical scale differences on subsequent applications, the output layer performs L2 normalization on it. First, the L2 norm of the vector is calculated, i.e. A specific numerical value is obtained; then, each element in the vector is divided by this L2 norm to make the vector length unit length, resulting in a standardized vector such as [2.5 / calculation result, 0.3 / calculation result, 1.8 / calculation result, ...]. The final output, a 64-dimensional unit-length comprehensive feature vector, preserves the spatial details of lung lesions in the MPR image while incorporating semantic information from DICOM metadata, serving as input for further analysis, research, or other processing tasks.

[0100] After obtaining the comprehensive feature vector, the comprehensive feature vector is processed using a neural network model based on the Transformer architecture to output the identification result of the target lesion.

[0101] This step, based on a neural network model with a Transformer architecture, processes the comprehensive feature vector and outputs the identification results of the scanned areas in the medical images. See the Transformer architecture for details. Figure 5A It should be noted that, Figure 5B , Figure 5C and Figure 5D They are Figure 5A Part of it, because Figure 5A There's a lot of content, and when displayed on the same page, the images aren't clear enough. Figure 5B , Figure 5C and Figure 5D For reference Figure 5A The specific details.

[0102] The processing flow of a neural network model based on the Transformer architecture is described below.

[0103] The input processing layer 501, acting as a "pre-normalization center" for multimodal feature processing and the first step in target lesion identification, specifically normalizes multimodal features containing target lesion information (such as lesion-related scanning parameters in DICOM metadata and spatial features of lesions in MPR images). Through the feature embedding layer, semantic features such as "lesion layer thickness" and "contrast agent injection parameters" in DICOM and spatial features such as "lesion edge contour" and "density distribution" in MPR images are uniformly mapped to a dimension that the model can efficiently compute (e.g., 64-dimensional). The positional encoding layer injects sequence identifiers into the feature sequences, enabling the model to recognize the logical relationship of "scan parameter description preceding lesion image features" (e.g., the sequential dependency between "high-resolution scan parameters" and "lesion detail features"). The input normalization layer eliminates differences in feature dimensions (e.g., the numerical range of lesion diameter "5mm" differs from that of edge smoothness "0.8"), ensuring that subsequent networks can fairly process all features related to the target lesion, thus laying a solid data foundation for accurate identification.

[0104] For example, in a lung nodule identification scenario, the input multimodal features include: DICOM metadata: scan slice thickness 1mm (related to the display of small lesions), contrast agent type (affecting lesion enhancement features); MPR image features: lobulation sign of the nodule's edge in the coronal plane (value 0.7), and axial density value (40HU). The input processing layer upscales the scan slice thickness scalar to 64 dimensions, embeds the contrast agent type as a 64-dimensional vector, and maintains the nodule edge and density features at 64 dimensions, uniformly outputting a standardized feature sequence with positional encoding (slice thickness → contrast agent → edge → density), enabling the model to focus on feature combinations related to lung nodules (target lesions), such as "thin slice scan + lobulation sign". In another example, for brain tumor identification, the input features include: DICOM metadata: scan sequence (T1 enhancement, related to tumor enhancement), patient age (affecting the probability of tumor type); MPR image features: the degree of ring enhancement of the tumor on the T1 enhanced image (value 0.9), and its boundary with surrounding tissue (value 0.6). After passing through input processing layer 501, the T1 enhanced sequence is encoded into a 64-dimensional vector. After age standardization, it is projected to 64 dimensions. The enhancement level and boundary features are kept in 64 dimensions, generating a feature sequence with position encoding (scan sequence → age → enhancement → boundary), ensuring that key features of the target lesion (brain tumor) such as "T1 enhancement + ring enhancement" are processed preferentially.

[0105] like Figure 5A As shown, the Transformer encoder 502 consists of three cascaded Transformer blocks (5021, 5022, 5023). This layer, through multiple Transformer blocks (5021 / 5022 / 5023), is used to uncover hidden correlations between features related to the target lesion. Each Transformer block uses a multi-head self-attention mechanism to dynamically interact the "scan parameters" in the DICOM metadata with the "lesion morphology" features in the MPR image (e.g., automatically discovering the negative correlation between "low-dose scan parameters" and "blurred lesion edge features"). Residual connections and layer normalization ensure that the basic features of the target lesion are not lost, and the feedforward network extracts the deep logic of "scan parameters → image quality → lesion discernibility". After multiple iterations, the output feature sequence can accurately represent the "semantic-spatial" association pattern of the target lesion (e.g., "high-resolution scan + lesion lobulation sign = malignant tumor tendency").

[0106] For example, in one instance, when identifying liver tumors, the input feature sequence is processed by Transformer encoder 1 5021: multi-head self-attention calculates the correlation between "DICOM contrast agent injection rate" and "MPR tumor enhancement peak," finding that the enhancement peak is significant when the injection rate is high (weight 0.8), establishing a correlation between "contrast agent parameters → tumor blood supply characteristics"; Transformer encoder 2 5022 further refines the feature pattern of "fast in and fast out enhancement + high injection rate = liver cancer," preserving the original tumor edge features through residual connections (value 0.7); in the feature sequence output by Transformer encoder 3 5023, the abstract expression of "liver cancer feature combination" is enhanced (value 0.9), providing deep interactive features for subsequent identification of target lesions (liver cancer). In another example, for a ground-glass nodule in the lung (the target lesion), the Transformer encoder 502 works as follows: Transformer encoder 1 calculates the correlation between "DICOM scan tube voltage 120kV" and "MPR nodule density -600HU", finding that density measurement is more accurate under high voltage (weight 0.7); Transformer encoder 2 5022 abstracts the combination of "tube voltage + density + nodule diameter 5mm" into a "pure ground-glass nodule" feature through a feedforward network (value 0.8); In the feature sequence output by Transformer encoder 3 5023, the "pure ground-glass nodule" related features are given high weight, suppressing normal lung tissue features (weight 0.3), forming a specific feature representation of the target lesion.

[0107] The feature extraction layer 503 is a crucial step in mining the features of the target lesion. Its role is to filter out information closely related to the target lesion from the multimodal feature sequence output by the Transformer encoding layer. Through a global attention pooling mechanism, the correlation between each feature location and the target lesion is automatically calculated, giving high weight to lesion region features and weakening background and irrelevant tissue features. Then, fully connected layers, GELU activation function, and Dropout layer are used to further compress feature dimensions, enhance feature expressive power, and prevent overfitting. Finally, a global summary feature vector containing key information such as the core morphology, density, and location of the target lesion is output, laying the foundation for accurate lesion identification.

[0108] For example, in a lung cancer screening scenario, the target lesion is a lung nodule. The input multimodal feature sequence includes features such as the edge morphology and density distribution of the nodule in the MPR image, as well as information such as scan parameters and patient history from DICOM metadata. The feature extraction layer uses global attention pooling to increase the weight of features such as spiculation and lobulation of the nodule edge to 0.8, while the weight of texture features of normal lung tissue is only 0.2. After a fully connected layer and GELU activation, the feature combination pattern of "irregular edge + uneven density = suspected malignant nodule" is strengthened, and the final output is a 512-dimensional vector containing key features of the nodule, which helps to determine the benign or malignant nature of the nodule. In another example, for the identification of brain hemorrhage lesions, the target lesion is the intracranial hemorrhage area. The input features include the signal intensity and location information of the hemorrhage lesion in the MRI image, as well as patient age and underlying diseases from DICOM metadata. The feature extraction layer uses global attention pooling to increase the weight of high signal features and hematoma morphology features of the hemorrhage lesion on T2-weighted images to 0.7, while the weight of ordinary signal features of the surrounding brain tissue is set to 0.3. After dimensionality reduction by the fully connected layer and GELU activation, the correlation feature of "sudden high signal + elderly hypertensive patient = cerebral hemorrhage" is highlighted, and the output feature vector can accurately characterize the hemorrhage lesion, assisting doctors in quickly locating and assessing the hemorrhage.

[0109] The classification head 504 is used to transform the globally summarized features output by the feature extraction layer into probability scores that can be directly used to determine the type of lesion. It first compresses the feature dimensions progressively through multiple classification mapping layers (e.g., 512-dimensional → 256-dimensional → 128-dimensional), filtering out redundant information irrelevant to target lesion identification. Simultaneously, it enhances the non-linear expressive power of the features using the GELU activation function (e.g., strengthening the combined association of "lesion enhancement pattern + patient age"), and uses a Dropout layer to randomly discard some features to improve the model's robustness against interference. Finally, the output layer maps the features to a dimension consistent with the number of target lesion categories (e.g., outputting 4 dimensions if 4 types of lesions are identified), and uses batch normalization to unify the feature distribution, outputting a matching score for each category, providing a quantitative basis for clinically determining the specific type of the target lesion.

[0110] For example, in the identification of benign and malignant lung nodules, the 512-dimensional feature vector output by the feature extraction layer includes information such as "nodule lobulation intensity 0.7" and "patient's 30-year smoking history". After compression by the 512→256-dimensional mapping layer, the GELU activation highlights the feature combination of "lobulation + long-term smoking = malignant tendency"; after purification by the 256→128-dimensional mapping layer, the output layer is mapped to 2 dimensions (benign / malignant), and after batch normalization, the output matching score [0.3, 0.7] is given, indicating that the target lesion (lung nodule) is more likely to be malignant. In another example, for the identification of target lesions in the brain (such as tumor / hemorrhage / infarction), the classification head receives a 512-dimensional feature vector (including "T1-weighted image low signal intensity 0.8" and "patient's history of hypertension"). After being compressed to 128 dimensions through multi-layer mapping, GELU activation strengthens the association feature of "low signal + high blood pressure = cerebral hemorrhage"; the output layer is mapped to 3 dimensions, and after batch normalization, the output score is [0.1, 0.8, 0.1], which shows that the target lesion (abnormal area in the brain) has the highest matching degree for cerebral hemorrhage.

[0111] The output layer 505 generates the identification results of the target lesion. It transforms the lesion category matching score output by the classification head 504 into a probabilistic identification result with clinical guidance significance. Using the Softmax activation function, the score is normalized to a probability distribution between 0 and 1, clearly showing the probability that the target lesion belongs to each category. Combined with temperature scaling technology, the concentration of the probability distribution can be flexibly adjusted to make the results more consistent with actual diagnostic needs. Finally, the probability value of each target lesion category is output, presenting the identification results in an intuitive and quantitative way, providing a reliable basis for doctors' diagnostic and treatment decisions.

[0112] For example, in the diagnosis of thyroid nodules, the target lesion is a nodule within the thyroid gland, which may be benign, malignant, or cystic. The 3D matching score output by the classification head is [0.2, 0.6, 0.2]. After Softmax activation, it is transformed into a probability distribution [0.18, 0.65, 0.17], meaning the probability of the thyroid nodule being benign is 18%, malignant is 65%, and cystic is 17%. If temperature scaling (temperature set to 0.5) is used to further enhance the probability difference, the output becomes [0.1, 0.85, 0.05], clearly indicating that the nodule is highly likely to be malignant, and further fine-needle aspiration biopsy is recommended. In another example, for the identification of target lesions in the skeletal system, such as bone tumors, fractures, and osteomyelitis, the classification head outputs a 4D matching score [0.1, 0.3, 0.5, 0.1] (corresponding to different types of skeletal lesions). The probability obtained after Softmax activation is [0.09, 0.27, 0.54, 0.1], indicating that the target lesion is most likely osteomyelitis; the probability is fine-tuned to [0.08, 0.25, 0.6, 0.07] by temperature scaling (temperature set to 0.8), further highlighting the diagnostic tendency of osteomyelitis.

[0113] This scheme offers significant advantages in target lesion identification. The input processing layer ensures effective input of lesion-related DICOM semantics and MPR spatial features by unifying the dimensions and distribution of multimodal features. The Transformer encoding layer utilizes a self-attention mechanism to deeply mine hidden correlations between lesion features, strengthening cross-modal interaction between "scanning parameters and lesion morphology." The feature extraction layer focuses on core lesion features through global attention, suppressing background interference. The classification head and output layer accurately map features to probabilistic results of lesion categories, improving identification reliability. For example, in lung nodule identification, it can automatically associate "thin-slice scan parameters" with "nodule lobulation sign," quickly determining benign or malignant. This scheme avoids human intervention bias through an adaptive weighting mechanism and captures complex pathological features using a hierarchical structure, effectively improving the efficiency and accuracy of target lesion identification in medical images, providing quantitative support for precise clinical diagnosis.

[0114] It should be noted that in some possible implementations of this invention, the target lesion is not pre-inputted or manually specified, but is gradually determined through the automated analysis of medical images by the model. For example, the MPR feature vector contains morphological and spatial location information of anatomical structures (such as organ contours and tissue density differences), while the DICOM metadata feature vector records scanning parameters and patient information (such as high-resolution scan slice thickness and contrast agent injection parameters). These data implicitly contain potential clues about the lesion (such as abnormal density areas and morphological distortions). The bidirectional cross-attention mechanism fuses two types of features through dynamic weights. For example, in lung CT, MPR features capture the lobulation sign of nodules, while DICOM metadata is associated with thin-slice scanning parameters (improving the display accuracy of small lesions). The interaction between the two can enhance the ability to locate "abnormal structures." The neural network model based on the Transformer architecture extracts the semantic information (such as "ground-glass nodule" and "enhancing tumor") and spatial location (such as "right upper lobe") of the lesion from the comprehensive feature vector through multi-layer feature interaction and abstraction. The final output recognition result is the target lesion and its type automatically detected by the model, without the need for manual pre-input of the lesion location or range.

[0115] This solution achieves end-to-end detection of target lesions from raw medical image data through fully automated processing of "feature encoding - cross-modal fusion - deep recognition", avoiding the subjectivity and limitations of manual annotation.

[0116] This invention also provides a system for analyzing large-scale medical images. The medical image analysis system adopts a distributed federated learning architecture, including: an edge node cluster and a central server; wherein, the edge node cluster is deployed in different computed tomography (CT) medical image acquisition institutions, and any edge node in the edge node cluster is used to acquire CT medical images, and to obtain MPR feature vectors and DICOM metadata feature vectors corresponding to a set of CT images based on the same sequence based on the locally acquired CT medical images; the central server is used to acquire the MPR feature vectors and DICOM metadata feature vectors obtained from the set of CT medical images based on the same sequence Qi uploaded by the edge node Yi; the edge node Yi is any edge node in the edge node cluster, and the sequence Qi is any set of CT medical images in the set obtained by grouping edge node Yi according to the same sequence; The central server is also used to fuse MPR feature vectors and DICOM metadata feature vectors through a bidirectional cross-attention mechanism to generate a comprehensive feature vector containing lesion semantics and spatial details; and to process the comprehensive feature vector using a neural network model based on the Transform architecture to obtain the lesion identification result and send the identification result to the edge node Yi; and to obtain an analysis report of large-scale medical images corresponding to the edge node cluster based on the set of identification results obtained from all tomographic medical images based on the same sequence on all edge nodes in the edge node cluster.

[0117] Figure 6 This is a schematic diagram of the structure of a large-scale medical image analysis device provided in an embodiment of the present invention. The medical image analysis device 600 serves as the central server in a distributed federated learning architecture. The distributed federated learning architecture includes an edge node cluster and the central server. The edge node cluster is deployed in different computed tomography medical image acquisition institutions. The medical image analysis device 600 includes a processor 602 configured to execute computer-executable instructions; and a memory 601 storing one or more computer-executable instructions. When the computer-executable instructions are executed by the processor 602, they implement the steps of the large-scale medical image analysis method described in any of the preceding method embodiments.

[0118] This invention also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements any of the embodiments of the method described above for analyzing large-scale medical images.

[0119] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the embodiments described in the preceding embodiments of the method for analyzing large-scale medical images.

[0120] This invention also provides a chip system for use in an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform any of the methods described in the preceding embodiments of the method for analyzing large-scale medical images.

[0121] Understandably, the beneficial effects that the above-mentioned large-scale medical image analysis system, large-scale medical image analysis device, computer storage medium, computer program product, and chip system can achieve can be referred to the beneficial effects described in the method embodiments, and will not be repeated here.

[0122] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0123] The above-described embodiments are optional embodiments provided by this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

1. A method for analyzing large-scale medical images, applied to a central server in a distributed federated learning architecture, the distributed federated learning architecture comprising: The edge node cluster and the central server, wherein the edge node cluster is deployed in different computed tomography medical image acquisition institutions, characterized in that the analysis method includes: The MPR feature vector and DICOM metadata feature vector are obtained from the set of tomographic medical images based on the same sequence Qi uploaded by the edge node Yi; the edge node Yi is any edge node in the edge node cluster, and the sequence Qi is any set of tomographic medical images in the set obtained by grouping edge node Yi according to the same sequence. By fusing the MPR feature vector and the DICOM metadata feature vector through a bidirectional cross-attention mechanism, a comprehensive feature vector containing lesion semantics and spatial details is generated. The comprehensive feature vector is processed using a neural network model based on the Transform architecture to obtain the lesion identification result, and the identification result is sent to the edge node Yi; Based on the set of recognition results obtained from all tomographic medical images based on the same sequence on all edge nodes in the edge node cluster, an analysis report of large-scale medical images corresponding to the edge node cluster is obtained.

2. The method for analyzing large-scale medical images according to claim 1, characterized in that, Before fusing the MPR feature vector and the DICOM metadata feature vector through a bidirectional cross-attention mechanism to generate a comprehensive feature vector containing lesion semantics and spatial details, the method further includes, When the MPR feature vector and the DICOM metadata feature vector are not aligned in dimension, the MPR feature vector and the DICOM metadata feature vector are respectively subjected to layer normalization, standardized feature distribution, and projected to the same dimension space to obtain dimension-aligned MPR feature vector and DICOM metadata feature vector.

3. The method for analyzing large-scale medical images according to claim 2, characterized in that, The bidirectional cross-attention mechanism includes: Using the MPR feature vector as the query and the DICOM metadata feature vector as the key and value, an MPR attention output guided by metadata semantics is generated to suppress noise interference on lesion edges; and using the DICOM metadata feature vector as the query and the MPR feature vector as the key and value, a DICOM attention output incorporating anatomical details is generated to locate the spatial position of the lesion.

4. The method for analyzing large-scale medical images according to claim 3, characterized in that, The fusion includes: Dynamic weights are generated through a feature gating layer to adaptively adjust the contribution of MPR features and DICOM metadata features. Specifically, when identifying lesions smaller than a preset threshold, the spatial detail weight of MPR features is enhanced, and the scanning parameter weight of DICOM metadata is enhanced in noise-sensitive scenarios.

5. The method for analyzing large-scale medical images according to any one of claims 1 to 4, characterized in that, The set of recognition results obtained from all tomographic medical images based on the same sequence on all edge nodes in the edge node cluster yields an analysis report of the large-scale medical images corresponding to the edge node cluster, including: Statistical analysis is performed based on the set of identification results to calculate at least one of the following indicators: lesion detection rate, misdiagnosis rate, and the influence coefficient of different equipment parameters on the identification results; and / or, Statistical analysis is performed on the set of identification results to generate visual charts, including heat maps of lesion distribution in different institutions and scatter plots showing the correlation between equipment parameters and identification accuracy.

6. The method for analyzing large-scale medical images according to any one of claims 1 to 4, characterized in that, The distributed federated learning architecture also includes: The secure communication protocol between each edge node in the edge node cluster and the central server uses a TLS 1.3 encrypted channel with a key length of ≥2048 bits.

7. A system for analyzing large-scale medical images, characterized in that, The medical image analysis system adopts a distributed federated learning architecture, including: an edge node cluster and a central server; wherein, The edge node cluster is deployed in different computed tomography medical image acquisition institutions. Each edge node in the edge node cluster is used to acquire computed tomography medical images and obtain MPR feature vectors and DICOM metadata feature vectors corresponding to the computed tomography image set based on the locally acquired computed tomography medical images. The central server is used to obtain the MPR feature vector and DICOM metadata feature vector of the set of tomographic medical images based on the same sequence Qi uploaded by the edge node Yi; the edge node Yi is any edge node in the edge node cluster, and the sequence Qi is any set of tomographic medical images in the set obtained by grouping edge node Yi according to the same sequence. The central server is further configured to fuse the MPR feature vector and the DICOM metadata feature vector through a bidirectional cross-attention mechanism to generate a comprehensive feature vector containing lesion semantics and spatial details; and to process the comprehensive feature vector using a neural network model based on the Transform architecture to obtain the lesion identification result, and send the identification result to the edge node Yi; and to obtain an analysis report of large-scale medical images corresponding to the edge node cluster based on the set of identification results obtained from all tomographic medical images based on the same sequence on all edge nodes in the edge node cluster.

8. An analysis device for large-scale medical images, wherein the medical image analysis device serves as a central server in a distributed federated learning architecture, the distributed federated learning architecture comprising: The edge node cluster and the central server, wherein the edge node cluster is deployed in different computed tomography (CT) medical image acquisition institutions, characterized in that the medical image analysis device comprises: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method for analyzing large-scale medical images as described in any one of claims 1-6.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for analyzing large-scale medical images as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for analyzing large-scale medical images as described in any one of claims 1-6.

Images