A big data-based stereoscopic imaging method and system for anterior mediastinal surgery

Abstract

The present invention belongs to the field of medical imaging technology, and specifically relates to a method and system for stereoscopic imaging of anterior mediastinal surgery based on big data. The invention performs parallel preprocessing of multimodal medical imaging data through a distributed computing framework, effectively improving the speed and efficiency of data processing. By comparing and analyzing different baseline image data, overlapping areas and independent areas can be determined, and the overlapping areas can be partitioned accordingly. In the three-dimensional reconstruction stage, the independent areas and overlapping sub-partitions are reconstructed in three dimensions, generating a three-dimensional image model of the independent area and a fused three-dimensional image model of the overlapping sub-partition. The spatial topological structure and surface geometric features of the image data are fully considered to ensure the accuracy of the reconstructed image. In the image reconstruction stage, the three-dimensional image model of the independent area is combined with the three-dimensional image model of the overlapping sub-partition to generate a complete stereoscopic image of the anterior mediastinal surgery, providing a strong guarantee for the precise implementation of the surgery.

Description

Technical Field

[0001] The present invention belongs to the field of medical imaging technology, and in particular relates to a big data-based stereoscopic imaging method and system for anterior mediastinal surgery. Background Art

[0002] As the complexity of anterior mediastinal surgery continues to increase in clinical practice, the requirements for medical imaging technology are also increasing. Traditional medical imaging technology can often only provide two-dimensional plane image information, which makes it difficult for surgeons to intuitively and comprehensively understand the three-dimensional structure of the patient's anterior mediastinum area, thereby increasing the risk and difficulty of the operation. Therefore, the development of three-dimensional stereoscopic imaging technology is of great significance to improving surgical safety and accuracy.

[0003] While several 3D imaging methods have been proposed in existing medical imaging technologies, these methods often rely solely on a single imaging modality, such as 3D reconstruction using CT or MRI. However, different imaging modalities have their own strengths and weaknesses in reflecting tissue structure and pathological features. A single imaging modality struggles to fully and accurately capture the anterior mediastinum's 3D structure and pathological characteristics, and the resulting 3D images may contain errors and incompleteness. This paper proposes a big data-based 3D imaging method for anterior mediastinal surgery to address these issues. Summary of the Invention

[0004] The purpose of the present invention is to provide a big data-based stereoscopic imaging method and system for anterior mediastinal surgery, which can integrate the advantages of different medical imaging modalities and improve the accuracy and completeness of stereoscopic imaging of anterior mediastinal surgery.

[0005] The technical solutions adopted by the present invention are as follows:

[0006] A big data-based stereoscopic imaging method for anterior mediastinal surgery, comprising:

[0007] Collect multimodal medical image data, perform parallel preprocessing on the multimodal medical image data through a distributed computing framework, and output the preprocessed multimodal image data as reference image data;

[0008] Comparing and analyzing different reference image data, determining overlapping areas and independent areas between different reference image data, and partitioning the overlapping areas to obtain multiple overlapping sub-partition images;

[0009] Directly perform three-dimensional reconstruction on the image in the independent area to generate a three-dimensional image model of the independent area;

[0010] Conduct confidence assessment on each overlapping sub-region image, and fuse the overlapping sub-region images based on the confidence assessment results to obtain a fused overlapping sub-region 3D image model;

[0011] The 3D image models of the independent regions were combined with the 3D image models of the overlapping sub-regions to generate a complete 3D image of the anterior mediastinum for surgery.

[0012] In a preferred embodiment, the multimodal image data includes binocular vision images, CT images, MRI images, PET images and ultrasound images.

[0013] In a preferred embodiment, the step of parallel preprocessing of multimodal medical imaging data using a distributed computing framework includes:

[0014] Segmenting multimodal medical image data into multiple sub-image data blocks, wherein each sub-image data block carries a modality type label and spatial coordinate metadata;

[0015] Allocate each sub-image data block to a cluster node, and perform denoising, image enhancement, and normalization processing in parallel on each cluster node;

[0016] Perform spatial consistency verification on the sub-image data blocks pre-processed by each cluster node, and reprocess the sub-image data blocks whose coordinate deviation exceeds the preset threshold;

[0017] The sub-image data blocks that have passed the spatial consistency verification are reorganized and output as reference image data.

[0018] In a preferred embodiment, the step of comparing and analyzing different reference image data to determine overlapping areas and independent areas between the different reference image data includes:

[0019] The reference image data under different modalities are dynamically deformed based on the spatial coordinate metadata and mapped to a unified spatial coordinate system;

[0020] Perform layer-by-layer similarity comparison on the registered reference image data, and identify the image areas with similarity higher than the similarity threshold as overlapping areas;

[0021] The image areas that do not meet the similarity threshold are determined as independent areas, and the spatial position and range of the independent areas are marked in a unified spatial coordinate system;

[0022] The connection boundary between the overlapping area and the independent area is smoothed to eliminate the edge jaggedness between the overlapping area and the independent area.

[0023] In a preferred embodiment, the step of partitioning the overlapping area to obtain a plurality of overlapping sub-partition images includes:

[0024] The initial partition boundaries are preliminarily divided according to the anatomical characteristics of blood vessels and nerves in the overlapping area, and the initial sub-partitions with spatial coordinate marks are generated;

[0025] Perform morphological processing on the partition boundaries of the initial sub-partitions to eliminate irregular edge protrusions on the partition boundaries;

[0026] Extracting the texture distribution features and gradient change features of each initial sub-partition, and calibrating the partition boundaries of adjacent initial sub-partitions based on the texture distribution features and gradient change features, and merging the adjacent initial sub-partitions into an overlapping sub-partition when the feature fit of the adjacent initial sub-partitions is higher than a preset fit threshold, otherwise, they are retained as independent overlapping sub-partitions;

[0027] Mutation detection is performed on the texture distribution characteristics and gradient change characteristics in each overlapping sub-partition to identify the texture mutation points and gradient mutation points, and the overlapping sub-partitions are subjected to secondary partitioning processing based on the texture mutation points and gradient mutation points until the texture distribution characteristics and gradient change characteristics in all overlapping sub-partitions reach the preset uniformity standard.

[0028] In a preferred embodiment, the step of performing secondary partitioning on the overlapping sub-partitions according to the texture mutation points and the gradient mutation points includes:

[0029] Obtain overlapping sub-regions where texture mutation points or gradient mutation points exist, and record them as secondary partition sub-regions;

[0030] Statistical analysis is performed on the texture mutation points and gradient mutation points within the secondary partition sub-region to determine the density distribution of the texture mutation points and gradient mutation points;

[0031] The secondary partition boundary is delineated in the secondary partition sub-region according to the density distribution of texture mutation points and gradient mutation points, and the secondary partition sub-region is divided according to the secondary partition boundary to form two independent overlapping sub-regions;

[0032] The spatial position of the overlapping sub-partitions after segmentation is calibrated to maintain the three-dimensional spatial connection relationship between adjacent overlapping sub-partitions after segmentation.

[0033] In a preferred embodiment, the steps of performing confidence assessment on each overlapping sub-regional image and fusing the overlapping sub-regional images according to the confidence assessment results include:

[0034] Obtain confidence evaluation criteria for overlapping sub-partition images under different modalities, and calculate confidence scores for corresponding overlapping sub-partition images based on the confidence evaluation criteria under each modality;

[0035] The confidence scores of overlapping sub-region images under different modalities are compared with the preset grading intervals. There are three grading intervals, corresponding to high confidence, medium confidence, and low confidence respectively.

[0036] Under high confidence, the overlapping sub-region images with the largest confidence score are directly selected for fusion processing;

[0037] Under medium confidence, each overlapping sub-region image is compared pixel by pixel, the pixel difference between adjacent overlapping sub-region images is calculated, and the overlapping sub-region image with the smallest pixel difference is selected as the fusion benchmark;

[0038] Under low confidence, the overlapping sub-region images are collected again, and the confidence of the re-collected overlapping sub-region images is evaluated until overlapping sub-region images with high or medium confidence are obtained, and then the fusion processing is performed;

[0039] The confidence evaluation criteria include:

[0040] The confidence score of binocular vision images is determined by parallax consistency and depth information accuracy.

[0041] The confidence score of CT images is determined by the clarity and tissue contrast values;

[0042] The confidence score of MRI images is determined based on the stability of signal intensity changes;

[0043] PET images determine confidence scores based on the distribution of imaging concentration in metabolically active areas;

[0044] Ultrasound images determine confidence scores based on the uniformity of the sound wave reflection signal.

[0045] In a preferred embodiment, the step of combining the three-dimensional image model of the independent region with the three-dimensional image model of the overlapping sub-region to generate a complete anterior mediastinal surgical stereoscopic image includes:

[0046] Based on a unified spatial coordinate system, the 3D spatial topological structure of independent regions and overlapping sub-regions is extracted, and the surface geometric features of adjacent regions are aligned;

[0047] Smooth transition between the boundary between independent areas and overlapping sub-areas;

[0048] The independent region 3D image model after smooth transition processing and the overlapping sub-region 3D image model are superimposed and combined in a unified spatial coordinate system to output a combined model;

[0049] The combined model is reverse-mapped to the original image layer, and the structural consistency between the combined model and the original image data is verified by structural coincidence. If the verification is passed, the combined model is determined to be the final anterior mediastinal surgical stereoscopic image. If the verification is not passed, the combined model is modified until the structural consistency requirements are met.

[0050] The present invention further provides a big data-based stereoscopic imaging system for anterior mediastinal surgery, which uses the above-mentioned big data-based stereoscopic imaging method for anterior mediastinal surgery, including:

[0051] A data acquisition module, the data acquisition module is used to acquire multimodal medical image data, perform parallel preprocessing on the multimodal medical image data through a distributed computing framework, and output the preprocessed multimodal image data as reference image data;

[0052] An image partitioning module is used to compare and analyze different reference image data, determine overlapping areas and independent areas between different reference image data, and partition the overlapping areas to obtain multiple overlapping sub-partition images;

[0053] A first reconstruction module, configured to directly perform three-dimensional reconstruction on the image in the independent region to generate a three-dimensional image model of the independent region;

[0054] a second reconstruction module, configured to perform confidence evaluation on each overlapping sub-region image and fuse the overlapping sub-region images according to the confidence evaluation results to obtain a fused overlapping sub-region three-dimensional image model;

[0055] An image reconstruction module is used to combine the three-dimensional image model of the independent area with the three-dimensional image model of the overlapping sub-region to generate a complete anterior mediastinal surgical stereoscopic image.

[0056] And, an electronic device, comprising:

[0057] at least one processor;

[0058] and a memory communicatively coupled to the at least one processor;

[0059] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the above-mentioned big data-based anterior mediastinal surgery stereoscopic imaging method.

[0060] The technical effects achieved by the present invention are:

[0061] The present invention uses a distributed computing framework to perform parallel preprocessing of multimodal medical imaging data, effectively improving the speed and efficiency of data processing. By comparing and analyzing different benchmark image data, it can accurately determine overlapping areas and independent areas, and perform fine partitioning of overlapping areas, providing a reliable foundation for subsequent 3D reconstruction and image fusion. In the 3D reconstruction stage, the independent areas and overlapping sub-partitions are respectively reconstructed in three dimensions to generate a 3D image model of the independent area and a fused 3D image model of the overlapping sub-partition, fully considering the spatial topological structure and surface geometric features of the image data to ensure the accuracy and authenticity of the reconstructed image. In the image reconstruction stage, the 3D image model of the independent area is combined with the 3D image model of the overlapping sub-partition to generate a complete anterior mediastinal surgical stereo image. Through steps such as smooth transition processing and structural coincidence verification, the structural consistency of the combined model with the original image data is ensured, providing a strong guarantee for the precise implementation of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic flow chart of the method of the present invention;

[0063] Figure 2 It is a schematic diagram of binocular camera depth calculation of the present invention;

[0064] Figure 3 Schematic diagram of binocular camera field of view fusion of the present invention;

[0065] Figure 4 It is a schematic diagram of the system modules of the present invention;

[0066] Figure 5 It is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0067] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0068] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0069] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive of other embodiments.

[0070] See also Figure 1 As shown, the present invention provides a big data-based anterior mediastinal surgery stereoscopic imaging method, comprising:

[0071] S1. Collect multimodal medical image data, perform parallel preprocessing on the multimodal medical image data through a distributed computing framework, and output the preprocessed multimodal image data as reference image data;

[0072] In step S1, when image acquisition of the anterior mediastinal surgical area is required, high-precision medical image acquisition equipment is used to acquire multimodal medical image data from different imaging devices, where the multimodal image data includes binocular vision images, CT images, MRI images, PET images, and ultrasound images. Then, a distributed computing framework is used to perform parallel preprocessing on the multimodal medical image data to improve processing efficiency, so that the multimodal medical image data can be quickly converted into reference image data that can be used for subsequent analysis. The step of performing parallel preprocessing on the multimodal medical image data using the distributed computing framework includes:

[0073] Segmenting multimodal medical image data into multiple sub-image data blocks, wherein each sub-image data block carries a modality type label and spatial coordinate metadata;

[0074] Allocate each sub-image data block to a cluster node, and perform denoising, image enhancement, and normalization processing in parallel on each cluster node;

[0075] Perform spatial consistency verification on the sub-image data blocks pre-processed by each cluster node, and reprocess the sub-image data blocks whose coordinate deviation exceeds the preset threshold;

[0076] Recombining the sub-image data blocks that have passed the spatial consistency verification and outputting them as reference image data;

[0077] Specifically, in the process of parallel preprocessing of multimodal medical imaging data using a distributed computing framework, first, the acquired multimodal medical imaging data is subdivided into multiple smaller sub-image data blocks. During this process, each sub-image data block is assigned a modality type label to correspond to the imaging method of the sub-image data block, such as CT, MRI, etc. At the same time, each sub-image data block is also accompanied by spatial coordinate metadata to record the position information of the sub-image in the original image. The marked sub-image data blocks are then distributed to a cluster composed of multiple computing nodes. On each node of the cluster, a series of key image processing operations are performed in parallel, including noise removal, image quality enhancement, and labeling. Standardization processing is performed to improve the quality of image data and provide clearer and more consistent images for subsequent analysis and diagnosis. After each cluster node completes preprocessing, the sub-image data blocks will be verified for spatial consistency. The purpose is to ensure that the relative position relationship of each sub-image data block remains unchanged during the segmentation and processing process. If the coordinate deviation is found to exceed the preset threshold, the sub-image data blocks with larger deviations will be reprocessed to ensure the accuracy of the data. Finally, once all sub-image data blocks have passed the spatial consistency verification, the reorganization processing will begin, that is, the verified sub-image data blocks will be recombined into a complete reference image data, which will serve as the basis for further analysis and diagnosis.

[0078] S2. Comparing and analyzing different reference image data, determining overlapping areas and independent areas between the different reference image data, and partitioning the overlapping areas to obtain multiple overlapping sub-partition images;

[0079] In step S2, after the benchmark image data is output, the different benchmark image data are compared and analyzed to determine the overlapping areas and independent areas between the different benchmark image data. For the overlapping areas, corresponding partitioning processing is performed. Specifically, the overlapping areas are divided into multiple overlapping sub-partitioned images to facilitate subsequent processing. The steps of comparing and analyzing the different benchmark image data to determine the overlapping areas and independent areas between the different benchmark image data include:

[0080] The reference image data under different modalities are dynamically deformed based on the spatial coordinate metadata and mapped to a unified spatial coordinate system;

[0081] Perform layer-by-layer similarity comparison on the registered reference image data, and identify the image areas with similarity higher than the similarity threshold as overlapping areas;

[0082] The image areas that do not meet the similarity threshold are determined as independent areas, and the spatial position and range of the independent areas are marked in a unified spatial coordinate system;

[0083] Smoothing the connection boundary between the overlapping area and the independent area to eliminate the jagged edges between the overlapping area and the independent area;

[0084] Specifically, when analyzing different reference image data, it is first necessary to perform dynamic deformation compensation on the reference image data under different modalities based on the corresponding spatial coordinate metadata to ensure that the reference image data can be mapped to a unified spatial coordinate system, thereby ensuring that the image data under different modalities can accurately correspond to the same anatomical structure. The dynamic deformation compensation process will take into account various factors, such as respiratory motion, heartbeat, and geometric differences between different imaging devices, so as to adjust the reference image data under different modalities to make them consistent in spatial position. Subsequently, the reference image data after the registration processing is compared layer by layer. The similarity comparison can be achieved by calculating the intensity difference, gradient similarity, or structural correlation between each pixel or voxel. Based on this, image regions with similarity higher than a preset similarity threshold can be identified. These image regions will be recorded as overlapping regions. For image regions that do not meet the similarity threshold, they will be directly determined as independent regions, and the spatial position and range of the independent regions will be marked in the unified spatial coordinate system. Finally, the connecting boundary between the overlapping region and the independent region will be smoothed to eliminate edge jaggedness that may appear between the overlapping region and the independent region, ensuring the consistency and accuracy of the image data.

[0085] Secondly, the steps of partitioning the overlapping area to obtain multiple overlapping sub-partition images include:

[0086] The initial partition boundaries are preliminarily divided according to the anatomical characteristics of blood vessels and nerves in the overlapping area, and the initial sub-partitions with spatial coordinate marks are generated;

[0087] Perform morphological processing on the partition boundaries of the initial sub-partitions to eliminate irregular edge protrusions on the partition boundaries;

[0088] Extracting the texture distribution features and gradient change features of each initial sub-partition, and calibrating the partition boundaries of adjacent initial sub-partitions based on the texture distribution features and gradient change features, and merging the adjacent initial sub-partitions into an overlapping sub-partition when the feature fit of the adjacent initial sub-partitions is higher than a preset fit threshold, otherwise, they are retained as independent overlapping sub-partitions;

[0089] Perform mutation detection on the texture distribution characteristics and gradient change characteristics in each overlapping sub-partition, identify the texture mutation points and gradient mutation points, and perform secondary partitioning on the overlapping sub-partitions based on the texture mutation points and gradient mutation points until the texture distribution characteristics and gradient change characteristics in all overlapping sub-partitions meet the preset uniformity standard;

[0090] Specifically, when processing overlapping areas, a preliminary division is first performed based on the anatomical orientation characteristics of the blood vessels and nerves in the overlapping areas, thereby generating initial sub-partitions with spatial coordinate markers. In order to improve the accuracy of the partitioning, this embodiment performs morphological processing on the partition boundaries of the initial sub-partitions, aiming to eliminate the irregular edge protrusions of each partition boundary, making the partition boundaries smoother and more regular. After the morphological processing, the texture distribution characteristics and gradient change characteristics of each initial sub-partition are extracted. Based on the texture distribution characteristics and gradient change characteristics of the initial sub-partitions, the partition boundaries of adjacent initial sub-partitions are calibrated. When the feature fit of adjacent initial sub-partitions is higher than a preset fit threshold, the adjacent initial sub-partitions are merged into an overlapping sub-partition. On the contrary, if the fit is lower than the threshold, the adjacent initial sub-partitions are retained as independent overlapping sub-partitions. The calculation formula for the feature fit of adjacent initial sub-partitions is:

[0091] ;

[0092] Where, represents the feature fit of adjacent initial sub-partitions, and represents the texture feature vector of the adjacent initial sub-partition, and represents the gradient magnitude vector of the adjacent initial sub-partitions, represents the covariance calculation function, and represents the standard deviation of the texture feature vectors corresponding to adjacent initial sub-partitions, and represents the standard deviation of the gradient magnitude vectors corresponding to adjacent initial sub-partitions, and Represent the weight coefficients of texture distribution characteristics and gradient change characteristics, and satisfy + =1. In addition, in order to further improve the accuracy of overlapping sub-partitions, mutation detection will be performed on the texture distribution characteristics and gradient change characteristics in each overlapping sub-partition. Texture mutation points and gradient mutation points can be identified through mutation detection.

[0093] Among them, the detection function of texture mutation point is:

[0094] ;

[0095] Where, represents the local texture feature value, Indicates the window size, express × The mean value in the neighborhood window, when When the texture mutation intensity is greater than the preset texture mutation threshold, it will be marked is the texture mutation point;

[0096] The detection function of the gradient mutation point is:

[0097] ;

[0098] Where, represents the gradient mutation intensity, represents the gradient amplitude field;

[0099] Texture mutation points and gradient mutation points may represent the boundaries between different tissues or structures, or they may be abnormalities caused by artifacts or other factors in the imaging process. Therefore, based on the texture mutation points and gradient mutation points, the overlapping sub-partitions will be secondary partitioned to ensure that the texture distribution and gradient change characteristics within each overlapping sub-partition meet the preset uniformity standards, so that the final stereoscopic imaging results can be more accurate and detailed, providing doctors with a more reliable diagnostic basis.

[0100] Next, the step of performing secondary partitioning on the overlapping sub-partitions according to the texture mutation points and the gradient mutation points includes:

[0101] Obtain overlapping sub-regions where texture mutation points or gradient mutation points exist, and record them as secondary partition sub-regions;

[0102] Statistical analysis is performed on the texture mutation points and gradient mutation points within the secondary partition sub-region to determine the density distribution of the texture mutation points and gradient mutation points;

[0103] The secondary partition boundary is delineated in the secondary partition sub-region according to the density distribution of texture mutation points and gradient mutation points, and the secondary partition sub-region is divided according to the secondary partition boundary to form two independent overlapping sub-regions;

[0104] Perform spatial position calibration on the overlapping sub-partitions after segmentation to maintain the three-dimensional spatial connection between adjacent overlapping sub-partitions after segmentation;

[0105] In the above, when performing secondary partitioning processing on overlapping sub-partitions, it is first necessary to identify and obtain overlapping sub-regions with texture mutation points or gradient mutation points, and record them as secondary partition sub-regions. Then, corresponding statistical analysis is performed on the texture mutation points and gradient mutation points in the secondary partition sub-region to determine the density distribution of these mutation points. Then, based on the density distribution of texture mutation points and gradient mutation points, the secondary partition boundary is delineated within the secondary partition sub-region, and the secondary partition sub-region can be effectively segmented to form two independent overlapping sub-partitions. Finally, the spatial position of the segmented overlapping sub-regions is calibrated to ensure that the adjacent overlapping sub-regions after segmentation can maintain a three-dimensional spatial connection relationship.

[0106] S3, directly performing three-dimensional reconstruction on the image in the independent region to generate a three-dimensional image model of the independent region;

[0107] In step S3, the image in the independent area will be directly reconstructed in three dimensions to generate a three-dimensional image model of the independent area. Specifically, the three-dimensional reconstruction process can be achieved through surface rendering or volume rendering technology. Surface rendering focuses on extracting the surface of the anatomical structure of interest from the image data, and approximating the shape and position of the surface through polygonal meshes and other forms, thereby generating a realistic three-dimensional image model. Volume rendering technology focuses more on rendering the entire image data volume, and simulates the interaction process of light passing through the image data volume by considering the optical properties of each voxel, such as transparency, color, and scattering coefficient, to ultimately generate a three-dimensional image that can reflect internal details. Specifically, the appropriate three-dimensional reconstruction technology can be selected according to the specific characteristics of the independent area and imaging requirements.

[0108] S4, performing confidence assessment on each overlapping sub-region image, and fusing the overlapping sub-region images based on the confidence assessment results to obtain a fused overlapping sub-region three-dimensional image model;

[0109] In step S4, a confidence assessment is performed on each overlapping sub-regional image, and the confidence assessment results are used to guide the fusion processing of the overlapping sub-regional images. The purpose of the fusion processing is to integrate multiple overlapping sub-regional images into a unified, boundary-free three-dimensional image model to reduce uncertainty during surgery. The steps of performing confidence assessment on each overlapping sub-regional image and fusing the overlapping sub-regional images based on the confidence assessment results include:

[0110] Obtain confidence evaluation criteria for overlapping sub-partition images under different modalities, and calculate confidence scores for corresponding overlapping sub-partition images based on the confidence evaluation criteria under each modality;

[0111] The confidence scores of overlapping sub-region images under different modalities are compared with the preset grading intervals. There are three grading intervals, corresponding to high confidence, medium confidence, and low confidence respectively.

[0112] Under high confidence, the overlapping sub-region images with the largest confidence score are directly selected for fusion processing;

[0113] Under medium confidence, each overlapping sub-region image is compared pixel by pixel, the pixel difference between adjacent overlapping sub-region images is calculated, and the overlapping sub-region image with the smallest pixel difference is selected as the fusion benchmark;

[0114] Under low confidence, the overlapping sub-region images are collected again, and the confidence of the re-collected overlapping sub-region images is evaluated until overlapping sub-region images with high or medium confidence are obtained, and then the fusion processing is performed;

[0115] The confidence evaluation criteria include:

[0116] The confidence score of binocular vision images is determined by parallax consistency and depth information accuracy.

[0117] The confidence score of CT images is determined by the clarity and tissue contrast values;

[0118] The confidence score of MRI images is determined based on the stability of signal intensity changes;

[0119] PET images determine confidence scores based on the distribution of imaging concentration in metabolically active areas;

[0120] Ultrasound imaging determines the confidence score based on the uniformity of the sound wave reflection signal;

[0121] Specifically, when performing confidence assessment on each overlapping sub-regional image, it is first necessary to clarify the confidence evaluation criteria for the overlapping sub-regional images under different modalities. Then, based on the confidence evaluation criteria under each modality, the confidence scores of the corresponding overlapping sub-regional images are calculated one by one to ensure that each image has a clear confidence value. In this embodiment, the confidence assessment includes the evaluation of multiple pairs of binocular vision images, CT images, MRI images, PET images, and ultrasound images.

[0122] When evaluating the confidence of binocular vision images, the binocular cameras synchronously collect and solidify their respective reference speckle images. and The binocular camera consists of two independent cameras with the same performance indicators (same optical lens and image sensor), which are arranged horizontally on both sides of the laser speckle projector. The optical axis of the camera is parallel to the optical axis of the laser speckle projector and is on the same baseline. It receives the coded pattern emitted by it. The focal length of the camera image sensor is , the baseline distance from the camera to the laser speckle projector is , the camera image sensor dot distance parameter is ;

[0123] Binocular cameras synchronously capture input images containing coded patterns and , and preprocess the input image. The preprocessing module includes distortion correction to simplify the subsequent binocular matching process. The reference speckle image and the real-time input speckle image sequence are all processed by the same preprocessing module to ensure the same speckle characteristics.

[0124] Binocular camera synchronization is achieved through the synchronization signal of a dedicated chip, and image acquisition is performed simultaneously. The chip's internal row memory adjusts the data by pixel or row to ensure the alignment of the left and right camera data. The internal and external parameters of the camera and projector are calibrated using Zhang Zhengyou's method. The infrared camera calibration requires an auxiliary infrared light source to illuminate the calibration plate. The calibration error is corrected in the depth calculation through parallax compensation and actual platform measurement.

[0125] After preprocessing, the input image and Detect the shadow areas cast by the target objects and mark them as and The shadow area is caused by the edge of the target object blocking the projection of the laser speckle projector, resulting in an area without a coded pattern in the camera image. The detection method is based on the number of feature points in the image block. In an input image block of a certain size, the number of speckles is detected. If the number is less than a predetermined threshold, it is determined to be a projected shadow area. At the same time, the area that is not within the effective distance range of the laser speckle projector irradiation and the camera reception is also treated as a shadow;

[0126] The offset, i.e. the disparity vector, is generated using two modes of block matching disparity compensation:

[0127] Binocular block matching mode, input image and Match each other and calculate the X-axis offset Or Y-axis offset ;

[0128] Self-matching mode, input image and and their reference speckle images respectively and Match and calculate the X-axis offset , Or Y-axis offset , ;

[0129] Reference Figure 2 , depth calculation based on binocular matching: select offset or (Select when arranging horizontally , select when arranging vertically ), combined with the focal length , baseline distance 2 and point spacing parameters , calculate the depth information of the center point of the projected image block according to the formula. Taking the horizontal offset as an example, the depth calculation formula is: , is the optimal offset of the input image block in the X direction (absolute value, number of pixels);

[0130] Depth calculation based on free matching: choosing the offset , or , (Select when arranging horizontally , ), combined with the known distance of the reference speckle image ,focal length , baseline distance and point spacing parameters , calculate the input image according to the formula and The depth information of the center point of the projected image block at the same position and , taking the horizontal offset as an example, the depth calculation formula is:

[0131] , ;

[0132] Utilizing depth information and Combined shadow areas and Compensate the depth information and output the final depth value of the center point of the projected image block ;

[0133] like Figure 3 As shown, classification is performed according to the position of the center point:

[0134] If the center point is in the non-intersection area 1 of the left view, select As ;

[0135] If the center point is in the non-intersection area 3 of the right view, select As ;

[0136] If the center point is in the intersection area 2 of the left and right views: non-shaded area: if and (express may be an error value), select or As Otherwise, select ;

[0137] Shaded area: If Region, select for , if in Region, select As ;

[0138] This is a specific method, and the actual application may not be limited to this;

[0139] When calculating the depth information of the entire image, move the center point of the projected image block to the next pixel in the same row and repeat the above process. Calculate the depth information of the entire image point by point in order from left to right and from top to bottom. Then, match the depth information with the preset confidence score table to determine the binocular vision image confidence score.

[0140] When evaluating the confidence of CT images, the confidence score is determined based on the evaluation of the CT image clarity and tissue contrast values. The higher the clarity and contrast, the higher the confidence score. The calculation formula for the CT image confidence score is: , where represents the confidence score of CT image, and Represents the preset weight coefficient, Indicates the total number of pixels in the image, Indicates the The gradient magnitude of each pixel, and Represents the average gray value of two different tissues (used to reflect contrast);

[0141] When evaluating the confidence of MRI images, the confidence score is determined based on the stability of the signal strength changes. The specific calculation formula is: , where represents the MRI image confidence score, Indicates the standard deviation of the signal strength in the selected area. Indicates the mean signal strength in the selected area;

[0142] When evaluating the confidence of PET images, the confidence score is determined by analyzing the imaging concentration distribution in metabolically active areas. The specific calculation formula is: , where represents the PET image confidence score, represents the average imaging concentration in metabolically active areas, It represents the average contrast concentration of the non-metabolic background area. The greater the concentration difference, the higher the corresponding PET image confidence score.

[0143] When evaluating the confidence of ultrasound images, the confidence score is determined based on the uniformity of the sound wave reflection signal. The specific calculation formula is: , where represents the ultrasound image confidence score, represents the total number of divided local areas, Indicates the The standard deviation of the reflected signal in a local area, Indicates the The average value of the reflected signal of each local area is calculated, and then the confidence score of the overlapping sub-partition images under different modes is compared with the preset grading interval. The grading interval is set to three different levels, corresponding to high confidence, medium confidence and low confidence. Specifically, the high confidence interval indicates that the image quality is very high and can be directly selected. The medium confidence interval indicates that the image quality is medium and needs further processing, while the low confidence interval indicates that the image quality is poor and cannot be used. In the case of high confidence, the overlapping sub-partition image with the largest confidence score is directly selected for fusion processing. In the case of medium confidence, It is necessary to perform a detailed pixel-by-pixel comparison of each overlapping sub-partition image, calculate the pixel difference between adjacent overlapping sub-partition images, and select the overlapping sub-partition image with the smallest pixel difference as the fusion benchmark to reduce the error in the fusion process. In the case of low confidence, due to the substandard image quality, it is necessary to re-execute the acquisition of the overlapping sub-partition image and perform a new round of confidence assessment on the re-acquired overlapping sub-partition image. This process will be repeated until a high-confidence or medium-confidence overlapping sub-partition image is obtained, and then the fusion processing can be continued to ensure the reliability of the final fusion result.

[0144] S5, combining the three-dimensional image model of the independent region with the three-dimensional image model of the overlapping sub-region to generate a complete three-dimensional image of the anterior mediastinum surgery;

[0145] In step S5, after the three-dimensional image models of the independent region and the overlapping sub-region are output, they are combined accordingly to generate a complete stereoscopic image of the anterior mediastinum surgery. The step of combining the three-dimensional image model of the independent region with the three-dimensional image model of the overlapping sub-region to generate a complete stereoscopic image of the anterior mediastinum surgery includes:

[0146] Based on a unified spatial coordinate system, the 3D spatial topological structure of independent regions and overlapping sub-regions is extracted, and the surface geometric features of adjacent regions are aligned;

[0147] Smooth transition between the boundary between independent areas and overlapping sub-areas;

[0148] The independent region 3D image model after smooth transition processing and the overlapping sub-region 3D image model are superimposed and combined in a unified spatial coordinate system to output a combined model;

[0149] The combined model is reverse-mapped to the original image layer, and the structural consistency between the combined model and the original image data is verified by structural coincidence. If the verification is passed, the combined model is determined to be the final anterior mediastinal surgical stereoscopic image. If the verification is not passed, the combined model is modified until the structural consistency requirements are met.

[0150] Specifically, when combining the three-dimensional image model of the independent area and the three-dimensional image model of the overlapping sub-partition, the three-dimensional spatial topological structure information of the independent area and the overlapping sub-partition is first extracted based on the unified spatial coordinate system, and the surface geometric features between adjacent areas are ensured to be accurately aligned, so as to ensure that each area can be seamlessly connected when combined to avoid geometric dislocation or deviation. Then, corresponding smooth transition processing is performed on the connection boundary between the independent area and the overlapping sub-partition, aiming to eliminate the abruptness at the boundary and make the image model more visually coherent, thereby improving the overall quality of the surgical stereoscopic image. After that, the three-dimensional image model of the independent area and the three-dimensional image model of the overlapping sub-partition that have undergone smooth transition processing are placed in the unified spatial coordinate system. The combined model is accurately superimposed and combined under the standard system to generate a preliminary combined model and output the combined model. Then, the generated combined model is reversely mapped to the original image layer. The structural consistency between the combined model and the original image data is verified by comparing and analyzing the structural coincidence. If the combined model passes the verification of structural consistency, it can be determined that the combined model is the final anterior mediastinal surgical stereo image. If the combined model fails to pass the verification of structural consistency, it needs to be corrected accordingly. The correction process includes readjusting the alignment of independent areas and overlapping sub-partitions, or further optimizing the boundary smooth transition processing, until the combined model fully meets the requirements of structural consistency, ensuring that it can accurately reflect the stereo image information of the anterior mediastinal surgical area.

[0151] See also Figure 4 A big data-based anterior mediastinal surgery stereoscopic imaging system, using the above-mentioned big data-based anterior mediastinal surgery stereoscopic imaging method, includes:

[0152] A data acquisition module is used to acquire multimodal medical imaging data, perform parallel preprocessing on the multimodal medical imaging data through a distributed computing framework, and output the preprocessed multimodal imaging data as reference imaging data;

[0153] The image partitioning module is used to compare and analyze different reference image data, determine the overlapping areas and independent areas between different reference image data, and partition the overlapping areas to obtain multiple overlapping sub-partition images;

[0154] A first reconstruction module, the first reconstruction module is used to directly perform three-dimensional reconstruction on the image in the independent area to generate a three-dimensional image model of the independent area;

[0155] The second reconstruction module is used to perform confidence evaluation on each overlapping sub-region image and fuse the overlapping sub-region images according to the confidence evaluation results to obtain a fused overlapping sub-region three-dimensional image model;

[0156] The image reconstruction module is used to combine the three-dimensional image model of the independent area with the three-dimensional image model of the overlapping sub-partition to generate a complete three-dimensional image of the anterior mediastinum surgery.

[0157] In the above, the main function of the data acquisition module is to collect multimodal medical imaging data, that is, to collect medical imaging data of multiple modalities including CT, MRI, etc., and then input the collected multimodal medical imaging data into the distributed computing framework. The parallel processing capability of the framework is used to perform parallel preprocessing on the multimodal medical imaging data. The preprocessing process includes steps such as data cleaning and standardization to ensure the quality and consistency of the data. Finally, the multimodal imaging data after preprocessing will be output as reference image data for use by subsequent modules. The main responsibility of the image partitioning module is to perform detailed comparative analysis of different reference image data to determine the overlapping areas between different reference image data and their respective independent areas. On this basis, these overlapping areas will be finely partitioned. The area is processed and divided into multiple overlapping sub-partition images. The first reconstruction module directly reconstructs the image in the independent area on the premise that the corresponding image of the independent area meets the preset standard, thereby generating a three-dimensional image model of the independent area. The main task of the second reconstruction module is to perform confidence assessment on each overlapping sub-partition image, and generate a confidence assessment result by comprehensively evaluating the quality, clarity and other factors of the image. Based on the assessment result, the overlapping sub-partition images are accurately fused to finally obtain the fused overlapping sub-partition three-dimensional image model. The role of the image reconstruction module is to organically combine the three-dimensional image model of the independent area with the fused overlapping sub-partition three-dimensional image model to generate a complete anterior mediastinal surgical stereo image, providing doctors with a comprehensive and intuitive surgical reference.

[0158] Refer to 5. An electronic device, the electronic device comprising:

[0159] at least one processor;

[0160] and a memory communicatively coupled to the at least one processor;

[0161] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the above-mentioned big data-based anterior mediastinal surgery stereoscopic imaging method.

[0162] The processor of the above-mentioned electronic device can be a central processing unit (CPU), a graphics processing unit (GPU) or a digital signal processor (DSP), etc. At the same time, the memory, as the core storage component of the electronic device, is used to store the operating system, application programs and various data and programs required for executing the big data-based anterior mediastinal surgery stereoscopic imaging method. The memory can be a random access memory (RAM), a read-only memory (ROM), a flash memory (Flash Memory), etc. In addition, the electronic device can also include an arithmetic unit, an input device, an output device, a network interface, etc., wherein the arithmetic unit is used to perform various arithmetic and logical operations, input devices such as a keyboard and a mouse are used to receive user input instructions, and output devices such as a display and a printer are used to display processing results or print relevant information. The network interface is used to realize the communication connection between the electronic device and other devices or networks to facilitate data transmission and sharing.

[0163] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0164] The foregoing is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained herein shall, unless otherwise specified or limited, be implemented in accordance with conventional means in the art.

Claims

1. A big data-based stereoscopic imaging method for anterior mediastinal surgery, characterized by: include: Collect multimodal medical image data, perform parallel preprocessing on the multimodal medical image data through a distributed computing framework, and output the preprocessed multimodal image data as reference image data; Comparing and analyzing different reference image data, determining overlapping areas and independent areas between different reference image data, and partitioning the overlapping areas to obtain multiple overlapping sub-partition images; Directly perform three-dimensional reconstruction on the image in the independent area to generate a three-dimensional image model of the independent area; Conduct confidence assessment on each overlapping sub-region image, and fuse the overlapping sub-region images based on the confidence assessment results to obtain a fused overlapping sub-region 3D image model; The three-dimensional image model of the independent area is combined with the three-dimensional image model of the overlapping sub-region to generate a complete three-dimensional image of the anterior mediastinum surgery; The step of comparing and analyzing different reference image data to determine overlapping areas and independent areas between the different reference image data includes: The reference image data under different modalities are dynamically deformed based on the spatial coordinate metadata and mapped to a unified spatial coordinate system; Perform layer-by-layer similarity comparison on the registered reference image data, and identify the image areas with similarity higher than the similarity threshold as overlapping areas; The image areas that do not meet the similarity threshold are determined as independent areas, and the spatial position and range of the independent areas are marked in a unified spatial coordinate system; Smoothing the connection boundary between the overlapping area and the independent area to eliminate the jagged edges between the overlapping area and the independent area; The step of partitioning the overlapping area to obtain a plurality of overlapping sub-partition images includes: The initial partition boundaries are preliminarily divided according to the anatomical characteristics of blood vessels and nerves in the overlapping area, and the initial sub-partitions with spatial coordinate marks are generated; Perform morphological processing on the partition boundaries of the initial sub-partitions to eliminate irregular edge protrusions on the partition boundaries; Extracting the texture distribution features and gradient change features of each initial sub-partition, and calibrating the partition boundaries of adjacent initial sub-partitions based on the texture distribution features and gradient change features, and merging the adjacent initial sub-partitions into an overlapping sub-partition when the feature fit of the adjacent initial sub-partitions is higher than a preset fit threshold, otherwise, they are retained as independent overlapping sub-partitions; Mutation detection is performed on the texture distribution characteristics and gradient change characteristics in each overlapping sub-partition to identify the texture mutation points and gradient mutation points, and the overlapping sub-partitions are subjected to secondary partitioning processing based on the texture mutation points and gradient mutation points until the texture distribution characteristics and gradient change characteristics in all overlapping sub-partitions reach the preset uniformity standard.

2. The method for stereoscopic imaging of anterior mediastinum surgery based on big data according to claim 1, characterized in that: The multimodal image data includes binocular vision images, CT images, MRI images, PET images and ultrasound images.

3. The method for stereoscopic imaging of anterior mediastinum surgery based on big data according to claim 1, characterized in that: The step of parallel preprocessing of multimodal medical imaging data using a distributed computing framework includes: Segmenting multimodal medical image data into multiple sub-image data blocks, wherein each sub-image data block carries a modality type label and spatial coordinate metadata; Allocate each sub-image data block to a cluster node, and perform denoising, image enhancement, and normalization processing in parallel on each cluster node; Perform spatial consistency verification on the sub-image data blocks pre-processed by each cluster node, and reprocess the sub-image data blocks whose coordinate deviation exceeds the preset threshold; The sub-image data blocks that have passed the spatial consistency verification are reorganized and output as reference image data.

4. The method for stereoscopic imaging of anterior mediastinum surgery based on big data according to claim 1, characterized in that: The step of performing secondary partitioning on the overlapping sub-partitions according to the texture mutation points and the gradient mutation points includes: Obtain overlapping sub-regions where texture mutation points or gradient mutation points exist, and record them as secondary partition sub-regions; Statistical analysis is performed on the texture mutation points and gradient mutation points within the secondary partition sub-region to determine the density distribution of the texture mutation points and gradient mutation points; The secondary partition boundary is delineated in the secondary partition sub-region according to the density distribution of texture mutation points and gradient mutation points, and the secondary partition sub-region is divided according to the secondary partition boundary to form two independent overlapping sub-regions; The spatial position of the overlapping sub-partitions after segmentation is calibrated to maintain the three-dimensional spatial connection relationship between adjacent overlapping sub-partitions after segmentation.

5. The method for stereoscopic imaging of anterior mediastinum surgery based on big data according to claim 1, characterized in that: The steps of performing confidence assessment on each overlapping sub-regional image and fusing the overlapping sub-regional images according to the confidence assessment results include: Obtain confidence evaluation criteria for overlapping sub-partition images under different modalities, and calculate confidence scores for corresponding overlapping sub-partition images based on the confidence evaluation criteria under each modality; The confidence scores of overlapping sub-region images under different modalities are compared with the preset grading intervals. There are three grading intervals, corresponding to high confidence, medium confidence, and low confidence respectively. Under high confidence, the overlapping sub-region images with the largest confidence score are directly selected for fusion processing; Under medium confidence, each overlapping sub-region image is compared pixel by pixel, the pixel difference between adjacent overlapping sub-region images is calculated, and the overlapping sub-region image with the smallest pixel difference is selected as the fusion benchmark; Under low confidence, the overlapping sub-region images are collected again, and the confidence of the re-collected overlapping sub-region images is evaluated until overlapping sub-region images with high or medium confidence are obtained, and then the fusion processing is performed; The confidence evaluation criteria include: The confidence score of binocular vision images is determined by parallax consistency and depth information accuracy. The confidence score of CT images is determined by the clarity and tissue contrast values; The confidence score of MRI images is determined based on the stability of signal intensity changes; PET images determine confidence scores based on the distribution of imaging concentration in metabolically active areas; Ultrasound images determine confidence scores based on the uniformity of the sound wave reflection signal.

6. The method for stereoscopic imaging of anterior mediastinum surgery based on big data according to claim 1, characterized in that: The step of combining the three-dimensional image model of the independent region with the three-dimensional image model of the overlapping sub-region to generate a complete anterior mediastinal surgical stereoscopic image includes: Based on a unified spatial coordinate system, the 3D spatial topological structure of independent regions and overlapping sub-regions is extracted, and the surface geometric features of adjacent regions are aligned; Smooth transition between the boundary between independent areas and overlapping sub-areas; The independent region 3D image model after smooth transition processing and the overlapping sub-region 3D image model are superimposed and combined in a unified spatial coordinate system to output a combined model; The combined model is reverse-mapped to the original image layer, and the structural consistency between the combined model and the original image data is verified by structural coincidence. If the verification is passed, the combined model is determined to be the final anterior mediastinal surgical stereoscopic image. If the verification is not passed, the combined model is modified until the structural consistency requirements are met.

7. A big data-based stereoscopic imaging system for anterior mediastinal surgery, characterized by: The method for anterior mediastinal surgery stereoscopic imaging based on big data according to any one of claims 1 to 6 comprises: A data acquisition module, the data acquisition module is used to acquire multimodal medical image data, perform parallel preprocessing on the multimodal medical image data through a distributed computing framework, and output the preprocessed multimodal image data as reference image data; An image partitioning module is used to compare and analyze different reference image data, determine overlapping areas and independent areas between different reference image data, and partition the overlapping areas to obtain multiple overlapping sub-partition images; A first reconstruction module, configured to directly perform three-dimensional reconstruction on the image in the independent region to generate a three-dimensional image model of the independent region; a second reconstruction module, configured to perform confidence evaluation on each overlapping sub-region image and fuse the overlapping sub-region images according to the confidence evaluation results to obtain a fused overlapping sub-region three-dimensional image model; An image reconstruction module is used to combine the three-dimensional image model of the independent area with the three-dimensional image model of the overlapping sub-region to generate a complete anterior mediastinal surgical stereoscopic image.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the big data-based anterior mediastinal surgical stereoscopic imaging method according to any one of claims 1 to 6.

Images