Three-dimensional volume data reconstruction method, device, equipment and computer storage medium

Abstract

Embodiments of the present application provide a three-dimensional volume data reconstruction method, device, equipment and computer storage medium; the reconstruction method comprises: obtaining initial three-dimensional volume data, the initial three-dimensional volume data includes a missing region, the confidence of the voxel in the missing region is less than or equal to a preset threshold; the initial three-dimensional volume data is processed using a preset mask prediction model, a three-dimensional shape mask corresponding to a region of interest in the initial three-dimensional volume data is generated, the three-dimensional shape mask includes a to-be-completed region, and the to-be-completed region is at least a part of the missing region; the initial three-dimensional volume data is interpolated and completed based on the to-be-completed region in the three-dimensional shape mask, and intermediate three-dimensional volume data is obtained; a three-dimensional volume data reconstruction operation is performed based on the intermediate three-dimensional volume data and the three-dimensional shape mask, and target three-dimensional volume data is obtained. Based on the three-dimensional shape mask, the three-dimensional volume data is reconstructed, and since the three-dimensional shape mask can accurately determine the region of interest, the accuracy of the reconstruction operation is improved.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a method, apparatus, device, and computer storage medium for reconstructing three-dimensional volume data. Background Technology

[0002] In the current medical field, magnetic resonance imaging (MRI) technology is widely used. During actual acquisition, to reduce scanning time and motion artifacts, a small number of sparsely sampled two-dimensional MRI slices are often acquired. These sparse two-dimensional MRI slices are then directly stacked according to the acquisition sequence to obtain reconstructed three-dimensional volume data. This method of obtaining three-dimensional volume data can result in phenomena such as banded artifacts, inter-slice voids, and discontinuities in anatomical structures, leading to inaccurate reconstruction results. Summary of the Invention

[0003] This application provides a method, apparatus, device, and computer storage medium for reconstructing three-dimensional volume data, which can perform high-quality reconstruction operations on three-dimensional volume data and effectively ensure the quality and accuracy of the target three-dimensional volume data acquisition.

[0004] In a first aspect, embodiments of the present invention provide a method for reconstructing three-dimensional volume data, including: Acquire initial three-dimensional volume data, which includes missing regions, wherein the confidence level of voxels in the missing regions is less than or equal to a preset threshold. The initial 3D volume data is processed using a preset mask prediction model to generate a 3D shape mask corresponding to the region of interest in the initial 3D volume data. The 3D shape mask includes a region to be completed, and the region to be completed is at least a part of the missing region. Based on the region to be filled in the three-dimensional shape mask, the initial three-dimensional volume data is interpolated and filled to obtain intermediate three-dimensional volume data; Based on the intermediate 3D volume data and the 3D shape mask, a 3D volume data reconstruction operation is performed to obtain the target 3D volume data.

[0005] Secondly, embodiments of the present invention provide a three-dimensional volume data reconstruction apparatus, comprising: The first acquisition module is used to acquire initial three-dimensional volume data, which includes missing regions, and the confidence of voxels in the missing regions is less than or equal to a preset threshold. The first processing module is used to process the initial three-dimensional volume data using a preset mask prediction model to generate a three-dimensional shape mask corresponding to the region of interest in the initial three-dimensional volume data. The three-dimensional shape mask includes a region to be completed, and the region to be completed is at least a part of the missing region. The first processing module is used to interpolate and complete the initial three-dimensional volume data based on the region to be completed in the three-dimensional shape mask to obtain intermediate three-dimensional volume data; The first processing module is used to perform a reconstruction operation on the three-dimensional volume data based on the intermediate three-dimensional volume data and the three-dimensional shape mask to obtain the target three-dimensional volume data.

[0006] Thirdly, embodiments of the present invention provide an electronic device, including: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the method for reconstructing three-dimensional volume data described in the first aspect is implemented.

[0007] Fourthly, embodiments of the present invention provide a computer storage medium for storing a computer program, which, when executed by a computer, implements the method for reconstructing three-dimensional volume data as described in the first aspect.

[0008] Fifthly, embodiments of the present invention provide a computer program product, comprising: a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause one or more processors to perform the steps in the three-dimensional volume data reconstruction method described in the first aspect.

[0009] The method, apparatus, device, and computer storage medium for reconstructing three-dimensional volume data provided in this embodiment acquire initial three-dimensional volume data including missing regions, process the initial three-dimensional volume data using a preset mask prediction model to generate a three-dimensional shape mask corresponding to the region of interest in the initial three-dimensional volume data, and then perform interpolation to complete the initial three-dimensional volume data based on the region to be completed in the three-dimensional shape mask to obtain intermediate three-dimensional volume data. The reconstruction operation of the three-dimensional volume data is then performed based on the intermediate three-dimensional volume data and the three-dimensional shape mask to obtain the target three-dimensional volume data, thereby effectively realizing the accurate reconstruction operation of the three-dimensional volume data. Specifically, when performing interpolation to complete the initial three-dimensional volume data based on the region to be completed in the three-dimensional shape mask, since the three-dimensional shape mask can accurately determine the region of interest in the initial three-dimensional volume data, the above-mentioned interpolation to complete operation can accurately complete the region of interest, thereby improving the efficiency and accuracy of three-dimensional volume data reconstruction. Furthermore, since the interpolation completion operation only applies to the region of interest in the initial 3D volume data, which is a part of the initial 3D volume data, this reduces the computational load of the interpolation completion operation to a certain extent, thereby improving computational efficiency, reducing the waste of computational resources, and also improving the practicality of the method, which is conducive to market promotion and application. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic diagram of a scenario for a method for reconstructing three-dimensional volume data provided as an exemplary embodiment of this application; Figure 2 A flowchart illustrating a method for reconstructing three-dimensional volume data, provided as an exemplary embodiment of this application; Figure 3 A schematic diagram of a process for obtaining initial three-dimensional volume data is provided for an exemplary embodiment of this application; Figure 4 A schematic diagram illustrating another process for generating a three-dimensional shape mask corresponding to a region of interest in initial three-dimensional volume data, provided as an exemplary embodiment of this application; Figure 5 A schematic diagram of another method for reconstructing three-dimensional volume data provided for an exemplary embodiment of this application; Figure 6 A flowchart illustrating a method for predicting a three-dimensional shape mask, provided as an exemplary embodiment of this application; Figure 7 A flowchart illustrating an exemplary embodiment of this application provides a method for interpolating and completing initial three-dimensional volume data based on a three-dimensional shape mask; Figure 8 A schematic diagram of a process for obtaining target three-dimensional volume data is provided for an exemplary embodiment of this application; Figure 9 A schematic diagram of the structure of a three-dimensional volume data reconstruction apparatus provided for an exemplary embodiment of this application; Figure 10 To and Figure 9 The illustrated embodiment provides a schematic diagram of the electronic device corresponding to the three-dimensional volume data reconstruction device. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] It should be noted that, in the case of user information involved in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0013] The various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards. Furthermore, the technical solutions provided in the embodiments of this application can employ deep learning models with relatively large parameter scales. The large model is merely an example, and the embodiments of this application do not limit the number of model parameters supported by the deep learning model used, aiming to meet actual needs. The deep learning models involved in the embodiments of this application can be artificial intelligence-based language models (LM) or multimodal models (MM).

[0014] Additionally, it should be noted that when user interaction operations or triggering operations are involved in the embodiments of this application, these operations include, but are not limited to, various interaction methods such as touch operations, gesture operations, voice operations, head movement operations, and eye movement operations. Touch operations include, but are not limited to, click operations, double-click operations, long-press operations, swipe operations, pinch operations, or mouse hover operations. Swipe operations include, but are not limited to, straight-line swipes and curved-line swipes.

[0015] Terminology definition: Three-dimensional volume data: data used to describe objects or phenomena in three-dimensional space.

[0016] Voxel: short for volume element, refers to the smallest unit of numerical data in three-dimensional space.

[0017] To facilitate understanding of the three-dimensional volume data reconstruction method, apparatus, device, and computer storage medium provided in the embodiments of this application, the relevant technologies are briefly described below: Magnetic resonance imaging (MRI) is a commonly used medical imaging technique, currently widely applied in clinical diagnosis of various sites, including the nervous system, cardiovascular system, and abdomen. For example, in clinical diagnosis and subsequent quantitative analysis, doctors typically need to use structurally intact, isotropic three-dimensional MRI data to locate lesions, measure volume, visualize in three dimensions, and plan preoperatively.

[0018] However, when clinical diagnosis requires the use of three-dimensional MRI volumetric data, limitations such as scan time, patient cooperation, motion artifacts, and scan sequence settings often make it difficult to directly acquire high-resolution and comprehensive three-dimensional MRI volumetric data. Especially for fetuses, children, and critically ill patients who cannot remain still for extended periods, in order to reduce the impact of scan time and motion artifacts, only a small number of thick slices or single-stack, sparsely sampled two-dimensional MRI slice sequences are often acquired. However, these sparse slices suffer from spatial problems such as sparse sampling, incomplete coverage, and large interslice intervals.

[0019] Currently, related technologies generally solve the above problems by directly stacking the acquired two-dimensional MRI slice sequences to obtain coarse three-dimensional volume data, and then using nearest neighbor interpolation, linear interpolation, spline interpolation or B-spline interpolation to perform interpolation and completion operations on the coarse three-dimensional volume data to obtain the final three-dimensional MRI volume data presented to the doctor.

[0020] However, the aforementioned methods struggle to accurately reconstruct intricate anatomical structures with complex boundary shapes and internal textures (e.g., brain structures). Furthermore, when the distance between slices in a 2D MRI sequence is large, simple interpolation methods can easily introduce severe blurring and structural distortion in the intermediate regions, resulting in low accuracy in 3D MRI volume data reconstruction. Moreover, when significant motion artifacts or noise interference exist in the 2D MRI slice sequence, interpolation strategies may amplify these artifacts or introduce false structures, further reducing the accuracy of 3D MRI volume data reconstruction.

[0021] To address the aforementioned technical problems, embodiments of this application provide a method, apparatus, device, and computer storage medium for reconstructing three-dimensional volume data, as detailed in the appendix. Figure 1As shown, the execution entity of this 3D volume data reconstruction method can be a 3D volume data reconstruction device 200, which can be implemented as a local server, a cloud server, or an edge server. Specifically, when the 3D volume data reconstruction device 200 is implemented as a cloud server, the 3D volume data reconstruction method can be executed in the cloud. Several computing nodes (cloud servers) can be deployed in the cloud, each with computing, storage, and other processing resources. In the cloud, multiple computing nodes can be organized to provide a certain service; of course, a single computing node can also provide one or more services. The cloud can provide this service by providing an external service interface, which users can call to use the corresponding service. Service interfaces include Software Development Kits (SDKs), Application Programming Interfaces (APIs), etc.

[0022] The 3D volume data reconstruction device 200 is communicatively connected to the client 100. The client 100 is used by users to trigger the 3D volume data reconstruction operation. The client 100 can be any computing device with a certain information interaction capability. Specifically, the client 100 can be a mobile phone, a personal computer (PC), a tablet computer, a configuration application, etc. Furthermore, the basic structure of the client 100 may include at least one processor. The number of processors depends on the client's configuration and type. The client 100 may also include memory, which can be volatile, such as Random Access Memory (RAM), or non-volatile, such as Read-Only Memory (ROM), flash memory, etc., or both types. The memory typically stores the operating system (OS), one or more applications, and may also store program data. In addition to the processing unit and memory, the client 100 also includes some basic configurations, such as a network card chip, an I / O bus, a display component, and some peripheral devices. Optionally, some peripheral devices may include, for example, a keyboard, a mouse, a stylus, a printer, etc. Other peripheral devices are well known in the art and will not be described in detail here.

[0023] A 3D volume data reconstruction device 200 refers to a device capable of reconstructing 3D volume data in a network virtual environment. It typically refers to a device that utilizes a network for information planning and 3D volume data reconstruction. Physically, the 3D volume data reconstruction device 200 can be any device capable of providing computing services and performing corresponding 3D volume data reconstruction operations, such as a processor or server. The 3D volume data reconstruction device 200 mainly comprises a processor, hard disk, memory, and system bus, and its architecture is similar to that of a general-purpose computer.

[0024] In the above embodiment, the 3D volume data reconstruction device 200 and the client 100 are connected via a network, which can be a wireless or wired network connection. If the 3D volume data reconstruction device 200 and the client 100 are connected via a communication connection, the mobile network standard can be any one of 2G (Global System for Mobile Communications GSM), 2.5G (General Packet Radio Service GPRS), 3G (Wideband Code Division Multiple Access (WCDMA), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), 4G (Long Term Evolution LTE), 4G+ (Enhanced Long Term Evolution LTE+), Global Microwave Access Interoperability (WiMax), 5G, 6G, etc.

[0025] In this embodiment, the client 100 is used by a user to generate a 3D volume data reconstruction request to trigger a 3D volume data reconstruction operation. To enable the 3D volume data reconstruction operation, the 3D volume data reconstruction request can be sent to the 3D volume data reconstruction device 200, so that the 3D volume data reconstruction device 200 can perform the corresponding 3D volume data reconstruction operation based on the 3D volume data reconstruction request. After the 3D volume data reconstruction device 200 processes the 3D volume data reconstruction request, the client 100 can receive the target 3D volume data corresponding to the reconstruction request returned by the 3D volume data reconstruction device 200.

[0026] The 3D volume data reconstruction device 200 is used to receive a 3D volume data reconstruction request sent by the client 100, obtain initial 3D volume data including missing regions based on the 3D volume data reconstruction request, process the initial 3D volume data using a preset mask prediction model, generate a 3D shape mask corresponding to the region of interest in the initial 3D volume data, and interpolate and complete the initial 3D volume data based on the region to be completed in the 3D shape mask to obtain intermediate 3D volume data. Finally, the 3D volume data reconstruction operation is performed based on the intermediate 3D volume data and the 3D shape mask to obtain the target 3D volume data, and the target 3D volume data corresponding to the reconstruction request is returned to the client 100.

[0027] In this embodiment, initial 3D volume data including missing regions is acquired, and a preset mask prediction model is used to process the initial 3D volume data to generate a 3D shape mask corresponding to the region of interest in the initial 3D volume data. The region of interest in the initial 3D volume data is accurately determined. Then, based on the region to be completed in the 3D shape mask, the initial 3D volume data is interpolated and completed to obtain intermediate 3D volume data that has been accurately interpolated and completed. Based on the intermediate 3D volume data and the 3D shape mask, the 3D volume data is reconstructed to obtain the target 3D volume data. This effectively realizes the accurate reconstruction of 3D volume data and also improves the practicality of the method, which is conducive to market promotion and application.

[0028] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0029] Figure 2 A flowchart illustrating a method for reconstructing three-dimensional volume data, provided as an exemplary embodiment of this application; see attached diagram. Figure 2 As shown, this embodiment provides a method for reconstructing three-dimensional volume data. The execution subject of this method is a three-dimensional volume data reconstruction device. This three-dimensional volume data reconstruction device can be implemented as software or a combination of software and hardware. When the three-dimensional volume data reconstruction device is implemented as hardware, it can specifically be various electronic devices capable of performing three-dimensional volume data reconstruction operations. In some instances, the three-dimensional volume data reconstruction device can be implemented as an application terminal, server, cloud server, etc. When the three-dimensional volume data reconstruction device is implemented as software, it can be installed in the aforementioned electronic devices. Specifically, the three-dimensional volume data reconstruction method provided in this embodiment may include: Step S201: Obtain initial three-dimensional volume data, which includes missing regions, and the confidence of voxels in the missing regions is less than or equal to a preset threshold.

[0030] Step S202: Process the initial 3D volume data using a preset mask prediction model to generate a 3D shape mask corresponding to the region of interest in the initial 3D volume data. The 3D shape mask includes the region to be completed, which is at least a part of the missing region.

[0031] Step S203: Based on the region to be completed in the 3D shape mask, interpolate and complete the initial 3D volume data to obtain intermediate 3D volume data.

[0032] Step S204: Reconstruct the three-dimensional volume data based on the intermediate three-dimensional volume data and the three-dimensional shape mask to obtain the target three-dimensional volume data.

[0033] The specific implementation methods and principles of each of the above steps are explained in detail below: Step S201: Obtain initial three-dimensional volume data, which includes missing regions, and the confidence of voxels in the missing regions is less than or equal to a preset threshold.

[0034] The initial 3D volumetric data is used to characterize the target organ's data information in 3D space. It's important to note that this initial 3D volumetric data is not ordinary 3D data, but rather 3D volumetric data including missing regions. Specifically, the initial 3D volumetric data includes multiple voxels, each with its own confidence level. The confidence levels of different voxels may be the same or different. The confidence level of a voxel characterizes the probability that the voxel is reliable within the initial 3D volumetric data. The confidence levels of each voxel are compared with preset thresholds. For voxels with confidence levels less than or equal to the preset thresholds, the regions corresponding to adjacent voxels are identified as missing regions in the initial 3D volumetric data. Conversely, the regions corresponding to voxels with confidence levels greater than the preset thresholds are identified as non-missing regions.

[0035] When a user has a need to reconstruct 3D volume data, the 3D volume data reconstruction device can obtain initial 3D volume data based on the reconstruction need. This embodiment does not limit the specific implementation method of obtaining initial 3D volume data. In some instances, the initial 3D volume data can be sent from the client to the 3D volume data reconstruction device. Specifically, the client stores the initial 3D volume data. When the user has a need to reconstruct the initial 3D volume data, the client can be triggered to actively send the initial 3D volume data to the 3D volume data reconstruction device, thereby ensuring the accuracy of obtaining the initial 3D volume data.

[0036] In other instances, the initial 3D volume data can also be determined through user interaction. In this case, the 3D volume data reconstruction device can store multiple 3D volume data to be processed. In response to the user's selection operation for any 3D volume data, the initial 3D volume data can be accurately obtained, and then subsequent processing operations can be performed on the initial 3D volume data.

[0037] Step S202: Process the initial 3D volume data using a preset mask prediction model to generate a 3D shape mask corresponding to the region of interest in the initial 3D volume data. The 3D shape mask includes the region to be completed, which is at least a part of the missing region.

[0038] Since the initial 3D volume data is not ordinary 3D volume data, but rather includes 3D volume data with missing regions, which can include regions of interest (ROIs) and regions of non-interest (NOTs), to make the ROIs in the initial 3D volume data more intuitive, a mask prediction model can be used to analyze and process the initial 3D volume data after acquisition to generate 3D shape masks corresponding to the ROIs. The ROI can be an organ region of interest (such as brain tissue) set in the initial 3D volume data, or it can be a local organ region of interest (such as the prefrontal cortex in brain tissue) set in the initial 3D volume data.

[0039] Specifically, the 3D shape mask corresponding to the region of interest can be determined by analyzing and processing the initial 3D volume data based on a pre-trained mask prediction model. Generating the 3D shape mask corresponding to the region of interest can include: determining the pre-trained mask prediction model; inputting the initial 3D volume data into the mask prediction model for processing to obtain the 3D shape mask corresponding to the region of interest output by the mask prediction model, thus improving the accuracy of obtaining the 3D shape mask corresponding to the region of interest. In this case, the 3D shape mask corresponding to the region of interest output by the mask prediction model directly includes the region to be completed. The region to be completed is at least a part of the missing region. One scenario is that the region to be completed can be a part of the missing region, in which case the missing region includes not only the region to be completed but also at least one of the following: irrelevant background regions, regions of no interest, etc.; another scenario is that the region to be completed is the entire missing region, in which case the missing region only includes the region to be completed and does not include other regions.

[0040] In some instances, when processing initial 3D volume data using a pre-defined mask prediction model, the generated 3D shape mask not only corresponds to the region of interest (ROI) in the initial 3D volume data, but also corresponds to the complete initial 3D volume data. Notably, in the 3D shape mask corresponding to the complete initial 3D volume data, the ROI and non-ROI regions are represented differently; for example, the ROI is represented as a white area, and the non-ROI region as a black area, making it easier to intuitively distinguish between the ROI and non-ROI regions in the initial 3D volume data. Specifically, processing the initial 3D volume data using a pre-defined mask prediction model to generate a 3D shape mask corresponding to the complete initial 3D volume data can include: determining a pre-trained mask prediction model; inputting the initial 3D volume data into the mask prediction model for processing to obtain the 3D shape mask corresponding to the complete initial 3D volume data output by the mask prediction model, ensuring the accuracy of the 3D shape mask corresponding to the complete initial 3D volume data.

[0041] Furthermore, the size of the initial 3D volume data can be determined based on the expected size of the target 3D volume data configured by the user or by default. In some instances, to improve the accuracy of 3D volume data reconstruction and reduce errors caused by size changes, the size of the initial 3D volume data can be the same as the size of the target 3D volume data. Moreover, the size of the 3D shape mask can be determined based on the size of the initial 3D volume data. In some instances, the size of the 3D shape mask can be the same as the size of the initial 3D volume data. Then, the 3D shape mask corresponding to the initial 3D volume data can be generated based on the size of the 3D shape mask.

[0042] Step S203: Based on the region to be completed in the 3D shape mask, interpolate and complete the initial 3D volume data to obtain intermediate 3D volume data.

[0043] The initial 3D volume data can include multiple layers of 2D slice data. Each layer in the initial 3D volume data represents a 2D slice, and two adjacent layers represent two adjacent 2D slices. Furthermore, there is a correlation between the size of the initial 3D volume data and the size and number of the included 2D slice data. That is, given a fixed size of the initial 3D volume data, the size and number of any included 2D slices can be determined based on that size. For example, if the initial 3D volume data has a size of 128*128*64, it means that the initial 3D volume data includes 64 slices, i.e., there are 64 2D slices, and each slice has a size of 128 pixels * 128 pixels.

[0044] Since the 3D shape mask includes the region to be completed, in order to reliably acquire relatively complete target 3D volume data after reconstruction, a completion operation can be performed on the region to be completed after obtaining the 3D shape mask corresponding to the region of interest in the initial 3D volume data. Specifically, since the initial 3D volume data is a layered structure data including multiple layers of 2D slice data, a preset method can be used to perform interpolation completion on the region to be completed in each layer of the initial 3D volume data. The preset method can be the nearest neighbor interpolation method or a local smoothing method, etc., to obtain the grayscale values ​​of each voxel in the region to be completed in each layer. The 3D volume data after the interpolation completion operation is used as intermediate 3D volume data, improving the accuracy and precision of the intermediate 3D volume data.

[0045] In some instances, when performing interpolation completion operations, not only can the regions to be completed in each layer be interpolated and completed, but the regions to be completed between two adjacent layers in the initial 3D volume data can also be completed. In this case, for the regions to be completed between two adjacent layers, the distances between each voxel in the region to be completed and the two adjacent layers can be determined first. Then, based on the distances between each voxel and the two adjacent layers, as well as the gray values ​​of the two adjacent layers, interpolation completion operations are performed to obtain intermediate 3D volume data. At this point, each voxel in the region to be completed in the intermediate 3D volume data has a corresponding gray value.

[0046] For example, when the interpolation completion operation is implemented using a linear interpolation algorithm, the two adjacent layers in the initial 3D volume data include layer A and layer B. We can first determine the distances between each voxel in the region to be completed and layer A, and the distances between each voxel in the region to be completed and layer B. Based on the distances, we can calculate the linear interpolation weights. If the distances between each voxel in the region to be completed and layer A are smaller, then layer A has a higher weight. Similarly, layer B has a lower weight. At this point, we can perform a weighted average of the gray values ​​of layer A, the corresponding weights of layer A, the gray values ​​of layer B, and the corresponding weights of layer B to obtain the gray values ​​of each voxel in the region to be completed.

[0047] It is understandable that, in addition to linear interpolation algorithms, interpolation completion operations can also be performed using higher-order interpolation methods based on the obtained distance to obtain the gray values ​​of each voxel in the region to be completed between two adjacent layers. The 3D volume data with completed interpolation completion is then used as intermediate 3D volume data, which improves the accuracy and precision of the intermediate 3D volume data.

[0048] In some instances, when performing interpolation completion on the region to be completed, it can be achieved not only by predicting the grayscale value of the region to be completed, but also by using the target ground truth. In this case, the method for performing interpolation completion on the region to be completed can also include: obtaining the target ground truth; and performing interpolation completion on the region to be completed based on the target ground truth. The target ground truth can be a value specified or input by the user, so that the intermediate 3D volume data for completing the interpolation completion operation better meets the user's needs.

[0049] In other instances, intermediate 3D volume data can be obtained not only through simple interpolation completion operations, but also by combining reliable regions and regions to be completed in a 3D shape mask. In this case, the method in this embodiment may include: determining reliable regions in a 3D shape mask, where the confidence of voxels in the reliable regions is greater than or equal to a preset confidence level; and performing interpolation completion on the initial 3D volume data based on the reliable regions and regions to be completed to obtain intermediate 3D volume data.

[0050] Specifically, after obtaining the 3D shape mask, the confidence level of each voxel in the 3D shape mask can be determined. Then, the confidence level of each voxel is compared with the preset confidence level to obtain multiple voxels with a confidence level greater than or equal to the preset confidence level. The regions corresponding to adjacent voxels among the multiple voxels are taken as reliable regions in the 3D shape mask. In this way, the voxels in the reliable regions have higher confidence levels, so the grayscale values ​​of each voxel are more reliable.

[0051] In some instances, reliable regions in a 3D shape mask can be determined not only by judging the confidence of voxels, but also by analyzing and processing the 3D shape mask based on a pre-trained reliable region prediction model. In this case, determining reliable regions in a 3D shape mask can include: determining a pre-trained reliable region prediction model; inputting the 3D shape mask, including the region to be completed, into the reliable region prediction model for processing, and obtaining the reliable regions in the 3D shape mask output by the reliable region prediction model, thereby improving the accuracy of obtaining reliable regions in the 3D shape mask.

[0052] After determining the reliable regions in the 3D shape mask, interpolation can be performed on the initial 3D volume data based on the reliable regions and the regions to be completed to obtain intermediate 3D volume data. At this point, the positional relationship between the region to be completed and the reliable regions can be determined to obtain at least one adjacent reliable region corresponding to the region to be completed. The number of reliable regions can be one or more. If there is only one reliable region, it means that there is only one adjacent reliable region around the region to be completed; if there are multiple reliable regions, it means that there are multiple reliable regions around the region to be completed. In this case, the positional relationship between the region to be completed and each reliable region is determined, and one or more adjacent reliable regions corresponding to the region to be completed are determined based on the positional relationship.

[0053] After obtaining the adjacent reliable regions corresponding to the region to be completed, multiple voxels can be selected as reference points in the adjacent reliable regions. These reference points can be randomly selected from the adjacent reliable regions, or the confidence levels of each voxel in the adjacent reliable regions can be sorted, and multiple voxels with confidence levels greater than or equal to a preset threshold can be selected as reference points.

[0054] It is worth noting that for any given reference point, the grayscale value corresponding to that reference point is non-null. Specifically, the grayscale value of a reference point in an adjacent reliable region is determined by the NMR slice corresponding to that reference point. When there is one NMR slice corresponding to a reference point, the grayscale value of the NMR slice corresponding to that reference point can be used as the grayscale value of that reference point. When there are multiple NMR slices corresponding to a reference point, the grayscale values ​​of the multiple NMR slices corresponding to that reference point can be calculated by weighted average to obtain the grayscale value of that reference point. Alternatively, when there are multiple NMR slices corresponding to a reference point, the grayscale values ​​of the multiple NMR slices corresponding to that reference point can be compared, and the maximum grayscale value among the multiple NMR slices can be selected as the grayscale value of that reference point.

[0055] After determining the grayscale values ​​of the reference points, the distances between the incomplete pixel in each region to be filled and multiple reference points in adjacent reliable regions can be calculated. Different weights are assigned to these reference points based on their distances; a negative correlation exists between distance and weight, meaning the smaller the distance between the incomplete pixel and a reference point, the greater the weight of that reference point. Then, based on the weights assigned to each reference point, a weighted summation operation is performed on the grayscale values ​​of multiple reference points in adjacent reliable regions to obtain the grayscale values ​​of the incomplete pixel. This yields the region to be filled after interpolation, and the completed 3D volume data is used as intermediate 3D volume data, improving the accuracy and precision of the intermediate 3D volume data.

[0056] Step S204: Reconstruct the three-dimensional volume data based on the intermediate three-dimensional volume data and the three-dimensional shape mask to obtain the target three-dimensional volume data.

[0057] After acquiring the intermediate 3D volume data and the 3D shape mask, the 3D volume data can be reconstructed based on the intermediate 3D volume data and the 3D shape mask. Specifically, since the intermediate 3D volume data includes regions of interest and regions of non-interest, and the 3D shape mask corresponds to the region of interest in the intermediate 3D volume data, the 3D shape mask can highlight the region of interest in the intermediate 3D volume data. In some instances, the 3D shape mask and the intermediate 3D volume data can be directly superimposed to obtain the target 3D volume data quickly, thus effectively ensuring the efficiency of target 3D volume data acquisition.

[0058] In other instances, the target 3D volume data can also be determined by analyzing and processing intermediate 3D volume data and 3D shape masks based on a pre-trained 3D volume data reconstruction model. The 3D volume data reconstruction model is used to reconstruct the 3D volume data. This model can be a Large Language Model (LLM) based on artificial intelligence, implemented using a neural network architecture that can be pre-trained on large amounts of data. In one alternative implementation, the 3D volume data reconstruction model may include an encoder, a decoder, an attention layer, and a feed-forward neural network. The encoder is mainly used to convert input data (usually in sequence form) into vector representation. This process can capture the semantic features of the input data. The decoder is responsible for converting the intermediate representation generated by the encoder into output data (usually in sequence form). The attention layer is a mechanism that allows the model to pay attention to the current position information. The feedforward neural network can perform nonlinear transformations on the output of the self-attention layer to enhance the model's expressive power. All parts work together, enabling the model built on them to perform well in various complex processing tasks, such as natural language processing, computer vision, speech recognition, machine translation, text summarization, and intelligent question answering.

[0059] At this point, obtaining the target 3D volume data may include: determining a pre-trained 3D volume data reconstruction model; inputting intermediate 3D volume data and 3D shape mask into the 3D volume data reconstruction model for processing, and obtaining the target 3D volume data output by the 3D volume data reconstruction model, effectively ensuring the quality of the target 3D volume data acquisition.

[0060] In some instances, the training process of a 3D volume data reconstruction model may include: acquiring the 3D volume data reconstruction model to be trained and historical 3D volume data, with the historical 3D volume data used as the training ground truth for the 3D volume data reconstruction model; processing the historical 3D volume data to obtain historical 2D NMR slice sequences corresponding to the historical 3D volume data; using the 3D volume data reconstruction model to be trained to process the historical 2D NMR slice sequences to determine the training 3D volume data corresponding to the historical 2D NMR slice sequences; and determining the model loss function based on the historical 3D volume data and the training 3D volume data. The model loss function can be determined based on a first loss function and a second loss function. The first loss function characterizes the loss between individual voxels in the historical 3D volume data and individual voxels in the training 3D volume data, while the second loss function characterizes the loss of overall similarity between the historical 3D volume data and the training 3D volume data. Subsequently, the parameters of the 3D volume data reconstruction model to be trained can be updated based on the model loss function to obtain the trained 3D volume data reconstruction model, thereby improving and ensuring the 3D volume data reconstruction effect and quality of the 3D volume data reconstruction model.

[0061] The 3D volume data reconstruction method provided in this embodiment acquires initial 3D volume data including missing regions, processes the initial 3D volume data using a preset mask prediction model to generate a 3D shape mask corresponding to the region of interest in the initial 3D volume data, accurately determining the region of interest in the initial 3D volume data, and then performs interpolation completion on the initial 3D volume data based on the region to be completed in the 3D shape mask to obtain intermediate 3D volume data with accurate interpolation completion. The 3D volume data reconstruction operation is then performed based on the intermediate 3D volume data and the 3D shape mask to obtain the target 3D volume data, thereby effectively realizing the accurate reconstruction of 3D volume data. Specifically, when performing interpolation completion on the initial 3D volume data based on the region to be completed in the 3D shape mask, since the 3D shape mask can accurately determine the region of interest in the initial 3D volume data, the above-mentioned interpolation completion operation can accurately complete the region of interest, thereby improving the efficiency and accuracy of 3D volume data reconstruction. Furthermore, since the interpolation completion operation only applies to the region of interest in the initial 3D volume data, which is a part of the initial 3D volume data, this reduces the computational load of the interpolation completion operation to a certain extent, thereby improving computational efficiency, reducing the waste of computational resources, and also improving the practicality of the method, which is conducive to market promotion and application.

[0062] Figure 3 This application provides a schematic diagram of a process for obtaining initial three-dimensional volume data, based on an exemplary embodiment. Referring to the appendix... Figure 3As shown, to improve the flexibility and practicality of 3D volume data reconstruction, the methods for obtaining initial 3D volume data can include: Step S301: Obtain a two-dimensional nuclear magnetic resonance (NMR) slice sequence, which includes at least two NMR slices.

[0063] Step S302: Determine the three-dimensional voxel mesh used to obtain the initial three-dimensional volume data.

[0064] Step S303: Project the NMR slice sequence into a three-dimensional voxel grid to obtain initial three-dimensional volume data.

[0065] In this context, a three-dimensional voxel grid refers to a uniform three-dimensional grid formed by voxels arranged at fixed intervals. Specifically, to reduce the adverse effects of scanning time and motion artifacts caused by crowd movement, especially for individuals who cannot remain still for extended periods, such as fetuses, children, and critically ill patients, a two-dimensional MRI slice sequence can be obtained first. This sequence includes at least two MRI slices, each corresponding to the same region of interest, and both slices are acquired in the same direction. The acquisition direction can be at least one of the following: top-down acquisition (axial direction), front-to-back acquisition (coronal direction), left-to-right acquisition (sagittal direction), etc. In certain applications, the relationship between at least two MRI slices can also satisfy at least one of the following: the acquisition time interval between the at least two MRI slices is less than a preset threshold, or the physical distance between the at least two MRI slices in the acquisition direction is less than a preset threshold, etc.

[0066] Subsequently, based on the dimensions of each slice in the acquired NMR slice sequence, a three-dimensional voxel grid is determined for acquiring the initial three-dimensional volume data. This ensures that the three-dimensional voxel grid can accommodate the initial three-dimensional volume data. Since the NMR slice sequence corresponds to target three-dimensional volume data, the size of the three-dimensional voxel grid is determined based on the size of the target three-dimensional volume data. The size of the three-dimensional voxel grid can be exactly the same as the size of the target three-dimensional volume data, or it can be larger. After determining the three-dimensional voxel grid, the two-dimensional NMR slice sequence can be projected onto the three-dimensional voxel grid to obtain the initial three-dimensional volume data. This effectively realizes the data conversion from two-dimensional to three-dimensional and improves the acquisition quality of the initial three-dimensional volume data.

[0067] In some instances, after acquiring a two-dimensional NMR slice sequence, each slice in the NMR slice sequence is projected onto a three-dimensional voxel grid according to a preset method, such as splat rendering. In this case, a voxel in the three-dimensional voxel grid may be covered by one or more slices. When a voxel is covered by multiple slices, its grayscale value is determined by combining the grayscale values ​​of the covering slices. For example, a weighted average of the grayscale values ​​of the covering slices can be used to obtain the voxel's grayscale value; or the grayscale values ​​of the covering slices can be compared, and the maximum grayscale value is selected as the voxel's grayscale value. When a voxel in the three-dimensional voxel grid is not covered by any slice, it is marked as a missing voxel. The region corresponding to multiple adjacent missing voxels is the missing region. This allows for the identification of missing or non-missing regions in the initial three-dimensional volume data using the voxel's grayscale value, improving the accuracy of identifying missing regions in the initial three-dimensional volume data.

[0068] In other instances, before projecting the NMR slice sequence onto a 3D voxel grid, each NMR slice in the sequence can be analyzed and processed. Based on a standard 3D coordinate system (x-direction representing the sagittal direction, y-direction representing the coronal direction, and z-direction representing the axial direction), transformation parameters corresponding to each NMR slice are determined. These transformation parameters include slice translation distance, rotation angle, scaling ratio, etc. Alternatively, one NMR slice can be selected as a standard slice. The standard slice can be the one with the fewest transformation operations or any randomly selected slice. Then, the transformation parameters of each other NMR slice in the sequence are determined relative to the standard slice, using it as a reference. A transformation matrix is ​​then generated based on these parameters. The NMR slices in the sequence are then projected onto a 3D voxel grid based on this generated transformation matrix to obtain initial 3D volume data corresponding to the NMR slice sequence, improving the rationality and accuracy of the obtained initial 3D volume data.

[0069] In some other instances, after obtaining a two-dimensional NMR slice sequence, at least two NMR slices from the sequence can be directly stacked to obtain initial three-dimensional volume data. Alternatively, at least two NMR slices from the sequence can be preprocessed before being stacked to obtain initial three-dimensional volume data. Preprocessing operations may include brightness normalization of the at least two NMR slices (adjusting their brightness to a preset range), or cropping, translating, or rotating the NMR slices to achieve initial alignment. This improves the reasonableness and accuracy of the obtained initial three-dimensional volume data.

[0070] To improve the accuracy of this method, after obtaining the two-dimensional nuclear magnetic resonance (NMR) slice sequence, the method in this embodiment may further include: determining the pose parameters corresponding to each NMR slice in the NMR slice sequence; adjusting the pose of the corresponding NMR slice based on the pose parameters to obtain the adjusted NMR slice.

[0071] The pose parameters include at least one of the following: the translation amount, rotation angle, and scaling degree of the MRI slice, etc.

[0072] Specifically, after obtaining the two-dimensional MRI slice sequence, the brightness of each slice in the sequence can first be normalized to ensure that the overall brightness of all slices remains relatively consistent. Secondly, the MRI slices can be cropped to remove irrelevant background areas to a certain extent, preserving organ regions such as brain tissue or regions of interest.

[0073] In addition, coarse registration operations, such as translation or rotation, can be performed on the MRI slices to avoid large-scale misalignment between different MRI slices. Then, a pre-trained pose correction network can be used to process the MRI slices, generating pose parameters corresponding to each slice. This generation of pose parameters can include: determining the pre-trained pose correction network; inputting at least two MRI slices from the sequence into the pose correction network for processing to obtain the pose parameters of each MRI slice relative to a standard 3D coordinate system, output by the network. After obtaining the pose parameters, the MRI slices corresponding to the pose parameters are accurately adjusted according to their respective pose parameters to obtain adjusted MRI slices. These adjusted MRI slices are then projected onto a 3D voxel mesh to obtain initial 3D volume data, improving the rationality and accuracy of the obtained initial 3D volume data.

[0074] In this embodiment, the initial three-dimensional volume data is obtained by projecting the acquired two-dimensional nuclear magnetic resonance slice sequence into a three-dimensional voxel grid. This effectively realizes the conversion of data from two-dimensional to three-dimensional and improves the quality of the acquired initial three-dimensional volume data.

[0075] Figure 4 A schematic flowchart illustrating another method for generating a 3D shape mask corresponding to a region of interest in initial 3D volume data, provided as an exemplary embodiment of this application; based on the above embodiment, refer to the appendix... Figure 4 As shown, 3D shape masks can be generated not only based on pre-trained mask prediction models, but also based on reliability maps. In this case, processing the initial 3D volume data using a pre-set mask prediction model to generate the 3D shape mask corresponding to the region of interest in the initial 3D volume data can include: Step S401: Process the initial 3D volume data using a preset mask prediction model to generate a reliability map corresponding to the region of interest in the initial 3D volume data. Each voxel in the reliability map corresponds to a confidence level.

[0076] The mask prediction model is used to predict 3D shape masks. This model can be a Large Language Model (LLM) based on artificial intelligence. After determining the initial 3D volume data, it can be input into the mask prediction model for analysis and processing. This yields a reliability map, output by the model, showing the region of interest (ROI) within the initial 3D volume data and including the confidence values ​​of each voxel. The reliability map is a type of 3D mask, where the confidence value of each voxel is a fixed value within the range of 0-1. For example, the confidence values ​​of each voxel in the reliability map can be 0.32, 0.47, 0.65, 0.81, 0.93, etc., ensuring the quality of the obtained reliability map.

[0077] Step S402: Based on the confidence level of each voxel in the reliability map, determine the three-dimensional shape mask including the region to be completed.

[0078] After obtaining the reliability map, each voxel in the reliability map can be analyzed and processed based on its confidence level. The final result is a 3D shape mask including the region to be completed. The confidence level of each voxel in the 3D shape mask has only two values: 0 and 1. This means the confidence level of each voxel in the 3D shape mask is a binary representation. Specifically, first, it can be determined whether the confidence level of each voxel in the reliability map is 0, resulting in one or more voxels with a confidence level of 0. If there is one voxel with a confidence level of 0, the grid region containing that voxel is taken as the region to be completed. If there are multiple voxels with a confidence level of 0, the regions corresponding to adjacent voxels with multiple confidence levels of 0 are taken as the region to be completed. Then, the confidence levels of all voxels in the reliability map are binarized to obtain the 3D shape mask including the region to be completed, improving the accuracy of determining the 3D shape mask.

[0079] In some instances, the region to be completed can be determined simply by judging whether the confidence level of the voxels in the reliability map is 0, thus obtaining a 3D shape mask including the region to be completed. Alternatively, the 3D shape mask including the region to be completed can be determined based on the confidence level of each voxel. Specifically, this includes: identifying voxels with confidence levels less than a preset confidence level as all voxels to be completed; determining the region to be completed based on the identified voxels; and determining the 3D shape mask based on the reliability map and the region to be completed.

[0080] Specifically, after obtaining the reliability map, the confidence level of each voxel in the reliability map is compared with a preset confidence level. Based on the comparison results, all voxels in the reliability map with confidence levels lower than the preset confidence level are identified as voxels to be completed. There are multiple voxels to be completed, and the regions corresponding to these multiple voxels are then defined as regions to be completed. The preset confidence level is a pre-configured confidence threshold. When the regions corresponding to multiple voxels to be completed are the same, the number of regions to be completed is one; when the regions corresponding to multiple voxels to be completed are different, the number of regions to be completed is multiple. Then, the confidence levels of all voxels in the reliability map are binarized to obtain a 3D shape mask including the regions to be completed, improving the accuracy of determining the 3D shape mask.

[0081] In this embodiment, by determining the confidence level of each voxel in the reliability map corresponding to the region of interest in the initial 3D volume data, the 3D shape mask including the region to be completed can be determined. Based on the obtained reliability map, the 3D shape mask including the region to be completed is obtained through analysis and processing, which improves the accuracy of determining the 3D shape mask, thereby ensuring the quality of 3D volume data reconstruction and effectively improving the practicality of the method.

[0082] For specific applications, please refer to the appendix. Figure 5 As shown, this application embodiment provides a method for reconstructing three-dimensional volume data. Specifically, when the input information for this method is a sparse MRI two-dimensional slice sequence, taking a brain tissue region as an example, the sparse MRI two-dimensional slice sequence includes at least two MRI slices. The relationship between the at least two MRI slices satisfies at least one of the following: the acquisition time interval between the at least two MRI slices is greater than or equal to a preset threshold; the physical distance between the at least two MRI slices in the acquisition direction is greater than or equal to a preset threshold, etc. In this case, the method for reconstructing three-dimensional volume data may include the following steps: Step 1: Generate initial three-dimensional volume data based on sparse MRI two-dimensional slice sequences.

[0083] Specifically, after acquiring a single-stack or multi-stack sparse MRI 2D slice sequence, the sparse MRI 2D slice sequence can be processed to obtain initial 3D volume data. A single-stack sparse MRI 2D slice sequence refers to a sequence in which all slices are acquired along the same imaging direction. A multi-stack sparse MRI 2D slice sequence includes at least two sparse MRI 2D slice sequences, where slices are acquired along the same imaging direction, but the acquisition directions of the two sparse MRI 2D slice sequences are not the same.

[0084] like Figure 5 As shown, Figure 5 The input data, from top to bottom, consists of sparse MRI two-dimensional slice sequences acquired along the Z-axis (axial), Y-axis (coronal), and X-axis (sagittal) directions. Each sparse MRI two-dimensional slice sequence includes at least two MRI slices, each corresponding to a brain tissue region, and both slices were acquired along the same imaging direction. This initial three-dimensional volume data is used to characterize the brain tissue in three-dimensional space.

[0085] In some instances, after obtaining a sparse MRI 2D slice sequence, preprocessing is required. First, brightness normalization is performed on each slice to ensure overall brightness falls within the same range across different slices. Second, slices with significant irrelevant background areas are cropped to roughly remove these areas, preserving regions of interest such as brain tissue and reducing subsequent computational load. In some cases, coarse registration can also be performed on the slices in the sparse MRI 2D slice sequence. This coarse registration includes simple translation, scaling, or rotation correction operations to mitigate large-scale misalignments.

[0086] After preprocessing the slices in the sparse MRI 2D slice sequence, a pose correction network is obtained. Then, multiple adjacent slices from the sparse MRI 2D slice sequence are input into the pose correction network. The network outputs the pose parameters of each slice relative to a standard 3D coordinate system, including translation, scaling, and rotation angles. Based on the pose parameters output by the network, a corresponding rotation-translation transformation matrix is ​​formed. This matrix describes the precise position of each slice in 3D space.

[0087] Then, based on the size of the sparse MRI 2D slice sequence, a regular voxel grid is established in 3D space. This regular voxel grid can accommodate the initial 3D volume data corresponding to the sparse MRI 2D slice sequence; that is, the size of the regular voxel grid can be exactly the same as or larger than the size of the initial 3D volume data. The preprocessed slices can then be projected onto the regular voxel grid using a rotation and translation transformation matrix. During projection, if multiple slices are projected onto the same voxel location, the gray values ​​of the multiple slices are fused, and the fused result is used as the gray value of that voxel. The fusion method can be averaging, weighted averaging, or selecting the maximum value. If a voxel is not covered by any slice, its location is marked as missing.

[0088] At this point, the three-dimensional volume data projected onto the regular voxel grid through the rotation and translation transformation matrix is ​​the initial three-dimensional volume data corresponding to the sparse MRI two-dimensional slice sequence. It should be noted that... Figure 5 As shown, the initial 3D volume data exhibits a striped structure in some directions and has obvious voids in unsampled areas, and can only be regarded as an incomplete and coarse representation of 3D volume data.

[0089] Step 2: Predict the 3D shape mask.

[0090] In some instances, the process of predicting a 3D shape mask can be as follows: Figure 6As shown, the three-dimensional shape mask prediction network is first trained based on the reference three-dimensional volume data, and the loss function is used to optimize and update the three-dimensional shape mask prediction network. After obtaining the trained three-dimensional shape mask prediction network, the initial three-dimensional volume data obtained in step 1 is input into the three-dimensional shape mask prediction network to obtain the three-dimensional shape mask predicted by the three-dimensional shape mask prediction network corresponding to the initial three-dimensional volume data.

[0091] Specifically, a 3D shape mask prediction network is constructed to predict the 3D shape mask of the initial 3D volume data. This network can employ a 3D convolutional network structure, extracting features from the 3D volume data in 3D space through multiple layers of 3D convolutions and nonlinear activation functions. Furthermore, to obtain information at multiple sizes, downsampling and upsampling modules can be incorporated into the network, forming an encoder-decoder structure to obtain 3D shape masks of different sizes. Additionally, at the end of the network, one or more convolutional layers can be used to map the features of the 3D volume data onto a single-channel output mask, ensuring that the output size of the 3D shape mask matches the size of both the initial and target 3D volume data.

[0092] After constructing the 3D shape mask prediction network to be trained, it needs to be trained. At this stage, a ground truth mask for the 3D shape mask needs to be determined. The ground truth mask can be obtained by thresholding, semi-automatic segmentation, or manual annotation of reference 3D volume data. The reference 3D volume data can be high-quality 3D MRI volume data obtained from historical moments. Then, the 3D shape mask prediction network to be trained processes the reference 3D volume data, outputting a reference reliability map. The voxel confidence scores in the reference reliability map are then binarized to obtain the reference shape mask. Finally, based on the reference shape mask and the ground truth mask, the parameters of the 3D shape mask prediction network are updated using loss functions such as cross-entropy loss and Dice loss.

[0093] After training, the 3D shape mask prediction network can automatically infer the shape and spatial extent of organ regions or regions of interest based on the initial 3D volume data. Therefore, for the initial 3D volume data obtained in step 1 above, the 3D shape mask prediction network outputs a 3D shape mask (such as...). Figure 5 As shown in the figure, it is possible to distinguish between brain tissue regions and non-brain tissue regions. Specifically, the voxel confidence level in brain tissue regions is higher than a set confidence threshold, while the voxel confidence level in non-brain tissue regions is lower than the set confidence threshold. This allows for the explicit differentiation of reliable observation regions from regions to be completed in three-dimensional space, providing prior information for subsequent interpolation and reconstruction operations.

[0094] Step 3: Perform interpolation to complete the initial 3D volume data based on the 3D shape mask.

[0095] First, voxels in the 3D shape mask with voxel confidence levels higher than or equal to a set confidence threshold are identified, and the regions corresponding to these voxels are designated as reliable observation regions. Voxels in the 3D shape mask with voxel confidence levels lower than the set confidence threshold are identified, and the regions corresponding to these voxels are designated as regions to be completed. For all voxels to be completed within the regions to be completed, reliable voxels (voxels with non-zero grayscale values) within their 3D spatial neighborhood can be selected as interpolation reference points. Specifically, different weights are assigned to the reference points based on their spatial distance from the voxels to be completed; for example, the smaller the spatial distance, the greater the weight of the reference point. Then, the estimated grayscale values ​​of all voxels to be completed can be obtained by summing the grayscale values ​​of multiple reference points according to their weights, thus achieving the interpolation completion operation. Figure 5 As shown, after determining the reliable observation area and the area to be completed, the voxel confidence in the reliable observation area and the area to be completed can be binarized. For example, the confidence of each voxel in the reliable observation area is set to 0, which is represented as a black area, and the confidence of each voxel in the area to be completed is set to 1, which is represented as a white area, so as to more accurately locate the area to be completed.

[0096] In some instances, when performing interpolation completion on a whole element to be completed between two adjacent slices in the initial 3D volume data, the interpolation completion operation can be performed using linear interpolation or higher-order interpolation based on the distances of the whole element to be completed to the two slices.

[0097] In other instances, for local void regions on the same level (i.e., the same slice) in the initial 3D volume data, the gray values ​​of voxels in the local void regions can be estimated using 3D nearest neighbor interpolation or local smoothing methods.

[0098] The three-dimensional shape mask corresponds to the external region of non-brain tissue areas in the initial three-dimensional volume data. The corresponding voxels are uniformly set to background values ​​or preset external grayscale values ​​and do not participate in interpolation calculations.

[0099] In some instances, the process of interpolating and completing the initial 3D volume data based on a 3D shape mask is as follows: Figure 7 As shown, the missing region is first located based on the 3D shape mask of the initial 3D volume data, and a missing region location mask is obtained. This allows the initial 3D volume data to be interpolated and completed using the nearest neighbor method based on the missing region location mask, thus obtaining coarsely completed intermediate volume data. The nearest neighbor method involves obtaining reliable regions adjacent to the missing region and determining the grayscale values ​​of voxels in the missing region based on the grayscale values ​​of voxels in the reliable regions.

[0100] Through the above interpolation completion process, we can obtain Figure 5 The intermediate volumetric data obtained through coarse completion fills in the voids and banded structures in the initial 3D volumetric data, resulting in a denser and more continuous distribution of voxels. Furthermore, the intermediate 3D volumetric data obtained using this method maintains an overall shape consistent with the 3D shape mask, providing a more reliable input for subsequent 3D reconstruction networks.

[0101] Step 4: Reconstruct the intermediate 3D volume data to obtain the target 3D volume data.

[0102] In some instances, the process of obtaining the target 3D volume data can be as follows: Figure 8 As shown, the 3D reconstruction network is first trained based on the reference 3D volume data, and then optimized and updated using a joint loss function, which may include a shape loss function and a reconstruction loss function. After the 3D reconstruction network is trained, the intermediate volume data obtained in step 3 is input into the 3D reconstruction network to obtain the target 3D volume data reconstructed by the 3D reconstruction network.

[0103] Specifically, a 3D reconstruction network to be trained is constructed. This 3D reconstruction network can adopt an encoder-decoder structure. The encoding part extracts multi-scale features through multi-layer 3D convolution and downsampling, while the decoding part restores spatial resolution step by step through upsampling and convolution. In order to make full use of shallow details and deep semantic information, multiple skip connections can be set between encoding and decoding to achieve the fusion of features at different scales.

[0104] After constructing the 3D reconstruction network to be trained, it needs to be trained. High-quality 3D MRI volume data generated at historical time points can be used as reference 3D volume data. Steps 1-3 above are executed to obtain the reference 3D shape mask and reference intermediate 3D volume data corresponding to the reference 3D volume data. The 3D reconstruction network to be trained processes the reference 3D shape mask and reference intermediate 3D volume data corresponding to the reference 3D volume data to output reference target 3D volume data. Then, based on the reference target 3D volume data and the reference 3D volume data, the joint loss function of the 3D reconstruction network is determined, and the parameters of the 3D reconstruction network are updated based on the joint loss function. The joint loss function of the 3D reconstruction network consists of two parts: a shape loss function and a reconstruction loss function. The shape loss function is a loss function based on overall similarity, and the reconstruction loss function is a voxel-based loss function.

[0105] After the 3D reconstruction network is trained, the 3D shape mask or reliability map obtained in step 2 and the intermediate 3D volume data obtained in step 3 can be input into it. At this time, the 3D reconstruction network can stitch the 3D shape mask and the intermediate 3D volume data in the channel dimension and finally output the reconstructed 3D volume data. The reliability map is the 3D mask in which the voxel confidence is not binarized.

[0106] In some instances, an attention mechanism layer is deployed in the 3D reconstruction network. In this case, after the 3D shape mask and intermediate 3D volume data are input into the 3D reconstruction network, the 3D reconstruction network can also fuse the 3D shape mask and intermediate 3D volume data through the attention mechanism layer, and finally output the reconstructed 3D volume data.

[0107] The technical solution provided in this application embodiment preprocesses sparse MRI two-dimensional slice sequences to generate initial three-dimensional volume data. Then, a three-dimensional shape mask prediction network is used to analyze and process the initial three-dimensional volume data to obtain a three-dimensional shape mask corresponding to the initial three-dimensional volume data. Based on this, interpolation and completion operations can be performed on the initial three-dimensional volume data based on the three-dimensional shape mask to obtain intermediate three-dimensional volume data. Finally, a three-dimensional reconstruction network is used to process the three-dimensional shape mask and the intermediate three-dimensional volume data to generate the target three-dimensional volume data. Specifically, the following effects can be achieved: ① The introduction of a three-dimensional shape mask enhances the three-dimensional shape rationality of the reconstruction result. By describing the range and boundary position of organs or regions of interest in three-dimensional space through shape masks, subsequent interpolation and reconstruction processes are carried out under shape constraints, effectively avoiding unreasonable pseudo-structures in non-organ regions, and significantly improving the shape integrity and anatomical rationality of the reconstruction result. ② By utilizing the confidence information of 3D shape masks and their voxels, reliable observation areas and missing areas can be effectively distinguished. During the interpolation stage, the missing voxels are filled in as a priority, while reliable areas retain their original observation values ​​or undergo only slight smoothing. This avoids excessive modification of existing real information and improves the accuracy and stability of the 3D completion results. ③ Combining the 3D shape mask-guided interpolation completion operation with a 3D reconstruction network not only achieves accurate interpolation completion but also fully leverages the advantages of deep learning networks in detail recovery and complex structure modeling. Ultimately, even in complex scenarios such as sparse sampling and motion interference, high-fidelity, structurally continuous, and noise-artifact-free 3D MRI volume data can be obtained.

[0108] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 11, 12, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0109] Figure 9 This application provides a schematic diagram of the structure of a three-dimensional volume data reconstruction device according to an exemplary embodiment; see attached... Figure 9 As shown, this embodiment provides a three-dimensional volume data reconstruction apparatus, which is used to perform the above-described... Figure 2 The method for reconstructing three-dimensional volume data shown herein, specifically, the apparatus for reconstructing three-dimensional volume data may include: The first acquisition module 11 is used to acquire initial three-dimensional volume data, which includes missing regions, and the confidence of voxels in the missing regions is less than or equal to a preset threshold. The first processing module 12 is used to process the initial three-dimensional volume data using a preset mask prediction model to generate a three-dimensional shape mask corresponding to the region of interest in the initial three-dimensional volume data. The three-dimensional shape mask includes the region to be completed, which is at least a part of the missing region. The first processing module 12 is also used to interpolate and complete the initial three-dimensional volume data based on the region to be completed in the three-dimensional shape mask to obtain intermediate three-dimensional volume data. The first processing module 12 is also used to perform a reconstruction operation of the three-dimensional volume data based on the intermediate three-dimensional volume data and the three-dimensional shape mask to obtain the target three-dimensional volume data.

[0110] The three-dimensional volume data reconstruction device in this embodiment can also perform the above-described... Figures 1-4 The description of the embodiments shown is for reference only, and will not be elaborated upon here.

[0111] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 11, 12, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0112] In one possible design, Figure 9 The structure of the 3D volume data reconstruction device shown can be implemented as an electronic device, which can be a controller, personal computer, server, or other similar devices. Figure 10 As shown, the electronic device may include a first processor 21 and a first memory 22. The first memory 22 is used to store data executed by the corresponding electronic device. Figures 1-8 In the illustrated embodiment, the program for the reconstruction method of three-dimensional volume data is provided, and the first processor 21 is configured to execute the program stored in the first memory 22.

[0113] The program includes one or more computer instructions, wherein when executed by the first processor 21, the one or more computer instructions can perform the following steps: acquiring initial three-dimensional volume data, the initial three-dimensional volume data including missing regions, wherein the confidence of voxels in the missing regions is less than or equal to a preset threshold; processing the initial three-dimensional volume data using a preset mask prediction model to generate a three-dimensional shape mask corresponding to the region of interest in the initial three-dimensional volume data, the three-dimensional shape mask including a region to be completed, the region to be completed being at least a part of the missing regions; interpolating and completing the initial three-dimensional volume data based on the region to be completed in the three-dimensional shape mask to obtain intermediate three-dimensional volume data; and reconstructing the three-dimensional volume data based on the intermediate three-dimensional volume data and the three-dimensional shape mask to obtain the target three-dimensional volume data.

[0114] Furthermore, the first processor 21 is also used to perform the aforementioned Figures 1-8 All or part of the steps in the illustrated embodiments.

[0115] The structure of the electronic device may also include a first communication interface 23 for communication between the electronic device and other devices or communication networks.

[0116] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by an electronic device, which includes instructions for executing the above-described... Figures 1-8 The procedure involved in the method for reconstructing three-dimensional volume data in the illustrated embodiment.

[0117] Furthermore, embodiments of the present invention provide a computer program product, comprising: a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause one or more processors to perform the aforementioned... Figures 1-8 The steps in the method for reconstructing three-dimensional volume data in the illustrated embodiment.

[0118] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the objectives of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0119] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0120] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0121] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0122] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reconstructing three-dimensional volume data, characterized in that, include: Acquire initial three-dimensional volume data, which includes missing regions, wherein the confidence level of voxels in the missing regions is less than or equal to a preset threshold. The initial 3D volume data is processed using a preset mask prediction model to generate a 3D shape mask corresponding to the region of interest in the initial 3D volume data. The 3D shape mask includes a region to be completed, and the region to be completed is at least a part of the missing region. Based on the region to be filled in the three-dimensional shape mask, the initial three-dimensional volume data is interpolated and filled to obtain intermediate three-dimensional volume data; Based on the intermediate 3D volume data and the 3D shape mask, a 3D volume data reconstruction operation is performed to obtain the target 3D volume data.

2. The method according to claim 1, characterized in that, Obtain initial 3D volume data, including: Obtain a two-dimensional nuclear magnetic resonance (NMR) slice sequence, wherein the NMR slice sequence comprises at least two NMR slices; Determine the three-dimensional voxel mesh to be used to obtain the initial three-dimensional volume data; The nuclear magnetic resonance slice sequence is projected into the three-dimensional voxel grid to obtain the initial three-dimensional volume data.

3. The method according to claim 2, characterized in that, After obtaining the two-dimensional nuclear magnetic resonance slice sequence, the method further includes: Determine the pose parameters corresponding to each nuclear magnetic resonance slice in the nuclear magnetic resonance slice sequence; The pose of the corresponding MRI slice is adjusted based on the pose parameters to obtain the adjusted MRI slice.

4. The method according to claim 1, characterized in that, The initial 3D volume data is processed using a preset mask prediction model to generate a 3D shape mask corresponding to the region of interest in the initial 3D volume data, including: The initial 3D volume data is processed using a preset mask prediction model to generate a reliability map corresponding to the region of interest in the initial 3D volume data. Each voxel in the reliability map corresponds to a confidence level. Based on the confidence level of each voxel in the reliability graph, a three-dimensional shape mask including the region to be filled is determined.

5. The method according to claim 4, characterized in that, Based on the confidence level corresponding to each voxel, a 3D shape mask including the region to be filled is determined, including: Voxels with confidence levels lower than a preset confidence level are identified as voxels to be filled. Based on the determined total elements to be filled, the region to be filled is determined; Based on the reliability map and the region to be completed, the three-dimensional shape mask is determined.

6. The method according to any one of claims 1-5, characterized in that, Based on the region to be completed in the three-dimensional shape mask, the initial three-dimensional volume data is interpolated and completed to obtain intermediate three-dimensional volume data, including: Determine a reliable region in the three-dimensional shape mask, wherein the confidence level of the voxels in the reliable region is greater than or equal to the preset confidence level; Based on the reliable region and the region to be completed, the initial three-dimensional volume data is interpolated and completed to obtain intermediate three-dimensional volume data.

7. The method according to any one of claims 1-5, characterized in that, The process of reconstructing the 3D volume data based on the intermediate 3D volume data and the 3D shape mask to obtain the target 3D volume data includes: Determine the 3D volume data reconstruction model used to generate 3D volume data; The intermediate three-dimensional volume data is processed using the three-dimensional volume data reconstruction model to generate the target three-dimensional volume data.

8. A device for reconstructing three-dimensional volume data, characterized in that, include: The first acquisition module is used to acquire initial three-dimensional volume data, which includes missing regions, and the confidence of voxels in the missing regions is less than or equal to a preset threshold. The first processing module is used to process the initial three-dimensional volume data using a preset mask prediction model to generate a three-dimensional shape mask corresponding to the region of interest in the initial three-dimensional volume data. The three-dimensional shape mask includes a region to be completed, and the region to be completed is at least a part of the missing region. The first processing module is used to interpolate and complete the initial three-dimensional volume data based on the region to be completed in the three-dimensional shape mask to obtain intermediate three-dimensional volume data; The first processing module is used to perform a reconstruction operation on the three-dimensional volume data based on the intermediate three-dimensional volume data and the three-dimensional shape mask to obtain the target three-dimensional volume data.

9. An electronic device, characterized in that, include: A memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method of any one of claims 1-7.

10. A computer storage medium, characterized in that, Used to store a computer program that, when executed by a computer, implements the method of any one of claims 1-7.

Images