A method and apparatus for thin slice medical image synthesis and volume data delineation

Abstract

The application discloses a kind of thin layer medical image synthesis and volume data delineation method and device, method includes: obtaining two medical image sections to be processed and its corresponding mask image;Two medical image sections to be processed are input target elastic deformation network model, and target synthesis image is obtained;Two medical image sections to be processed and two medical image sections to be processed corresponding mask image are input target elastic deformation network model, and volume data delineation result is obtained;Wherein, by obtaining three adjacent thin layer medical image sections and its corresponding mask image;Training set is constructed;Initial elastic deformation network model is constructed;According to weight coefficient, total loss function is constructed;Training set is input to initial elastic deformation network model and is trained, and target elastic deformation network model is obtained.The application realizes thin layer medical image synthesis and volume data delineation, improves image synthesis accuracy and delineation efficiency.The application can be widely applied in image processing technical field.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for thin-layer medical image synthesis and volume data delineation. Background Technology

[0002] Two-dimensional magnetic resonance imaging (MRI) scans acquire anisotropic volumetric data, leading to the loss of inter-slice directional information. Traditional image acquisition methods can reduce the slice thickness of MRI images by adjusting scanning parameters, but this reduces the signal-to-noise ratio (SNR). Three-dimensional MRI sequences can address the low SNR issue, but the scanning time is longer than that of thick-slice two-dimensional MRI scans. Super-resolution reconstruction methods can reconstruct corresponding high-resolution images from low-resolution images, but the synthesized images lack smoothness in the intra-slice direction. Inter-slice image synthesis methods combine multiple adjacent two-dimensional images to generate a composite image, but the synthesized images also lack smoothness in the inter-slice direction, resulting in low accuracy. Furthermore, medical image datasets used for training deep learning models require a large amount of labeled data. Traditional delineation methods rely on manual layer-by-layer delineation using annotation software, which is time-consuming and inefficient due to the large number of image layers.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] This invention provides a method and apparatus for thin-layer medical image synthesis and volume data delineation, which effectively improves the accuracy of image synthesis and the efficiency of delineation.

[0005] On one hand, embodiments of the present invention provide a method for thin-layer medical image synthesis and volume data delineation, comprising the following steps:

[0006] Obtain two cross-sections of the medical images to be processed and their corresponding mask images;

[0007] When the two medical image sections to be processed are adjacent, the two medical image sections to be processed are input into the target elastic deformation network model to obtain the target synthetic image;

[0008] When the positional relationship between the two medical image sections to be processed is that they are separated by layers, the two medical image sections to be processed and the mask images corresponding to the two medical image sections to be processed are input into the target elastic deformation network model to obtain the volume data delineation result.

[0009] The target elastic deformation network model is obtained through the following steps:

[0010] Obtain three adjacent thin-layer medical image sections and their corresponding mask images;

[0011] A training set is constructed based on the three adjacent thin-layer medical image sections and their corresponding mask images. The training set includes the gold standard image, the medical image sections to be trained, and the mask images to be trained.

[0012] Construct an initial elastic deformation network model;

[0013] Construct the total loss function based on the weighting coefficients;

[0014] According to the total loss function, the training set is input into the initial elastic deformation network model so that the initial elastic deformation network model is trained to obtain the target elastic deformation network model.

[0015] In some embodiments, constructing a training set based on the three adjacent thin-layer medical image sections and their corresponding mask images includes:

[0016] The middle section among the three adjacent thin-layer medical image sections is used as the gold standard image, which is used for medical image synthesis and delineation.

[0017] Downsampling is performed on two sections of the septum between the three adjacent thin-layer medical image sections to obtain the corresponding medical image sections to be trained;

[0018] The mask images of the two cross-sections of the partition are downsampled to obtain the corresponding mask images to be trained.

[0019] In some embodiments, the process of constructing the initial elastic deformation network model includes:

[0020] A deformation field estimation module is constructed, which is used to construct the corresponding deformation field based on the position encoding and two cross sections of the medical image to be trained;

[0021] After the deformation field estimation module, a first deformation fusion module is constructed. The first deformation fusion module is used to perform a first deformation fusion process based on the deformation field to obtain a thin-layer synthetic image.

[0022] A second deformation fusion module is constructed, which is used to perform a second deformation fusion process based on the two mask images to be trained to obtain an annotated synthetic image;

[0023] After the first deformation fusion module and the second deformation fusion module, a two-dimensional segmentation network is constructed. The two-dimensional segmentation network is trained using a loss function that combines cross-entropy and Dice loss.

[0024] In some embodiments, the first deformation fusion process based on the deformation field to obtain a thin-layer composite image includes:

[0025] A two-dimensional mesh is generated based on the size and accuracy of the deformation field;

[0026] Based on the two-dimensional grid, the grid node coordinates are mapped to the cross section of the medical image to be trained to obtain the corresponding image pixel values;

[0027] Based on the deformation field and the grid node coordinates, calculate the positional relationship of the grid nodes before and after deformation;

[0028] Based on the positional relationship of the mesh nodes before and after deformation, several pixels closest to the interpolation point are selected from the cross-section of the medical image to be trained as reference points.

[0029] Calculate the interpolation weights based on the multiple reference points;

[0030] According to the interpolation weight, the image pixel values ​​of multiple reference points are weighted and averaged to obtain the grid node pixel values;

[0031] A deformed image is constructed based on the pixel values ​​of multiple grid nodes;

[0032] The thin-layer composite image is obtained by fusing the image index size and the two deformed images.

[0033] In some embodiments, constructing the total loss function based on the weighting coefficients includes:

[0034] Based on the number of images required to be synthesized between the thin-layer synthetic image, the gold standard image, and adjacent medical image sections, an image reconstruction loss function is constructed.

[0035] Construct task-specific and realistically anatomically structured constraints;

[0036] Construct target smoothing constraints;

[0037] The total loss function is constructed based on the image reconstruction loss function, the task-specific and anatomically realistic constraint term, the target smoothness constraint term, and the weight coefficients.

[0038] In some embodiments, the construction task-specific and anatomically realistic constraints include:

[0039] Based on the thin-layer synthetic image segmentation results, pre-segmented labeled images, and cross-entropy loss function, task-specific constraints are constructed.

[0040] Based on the labeled synthetic image, the pre-segmented labeled image, and the cross-entropy loss function, construct an anatomical structure realism constraint term;

[0041] Based on the task-specific constraints and the anatomical structure realism constraints, construct the task-specific and anatomical structure realism constraints.

[0042] In some embodiments, the construction of the target smoothness constraint term includes:

[0043] Based on the deformation field gradient of the medical image to be trained transformed into a thin-layer synthetic image, an intra-layer smoothing constraint term is constructed;

[0044] Based on the synthesized image of the current layer, the synthesized image of the previous layer, and the synthesized image of the next layer, construct inter-layer smoothing constraints;

[0045] The target smoothing constraint term is constructed based on the intra-layer smoothing constraint term and the inter-layer smoothing constraint term.

[0046] In some embodiments, the expression for the image reconstruction loss function is:

[0047]

[0048] In the formula, loss rec For the image reconstruction loss, I i+t The gold standard image, The thin-layer composite image is defined as n, which is the number of images to be composited between adjacent medical image sections, and ||*||1 is the L1 norm.

[0049] On the other hand, embodiments of the present invention provide a thin-layer medical image synthesis and volume data delineation device, comprising:

[0050] The first module is used to acquire two cross-sections of the medical images to be processed and their corresponding mask images;

[0051] The second module is used to input the two medical image sections to be processed into the target elastic deformation network model to obtain the target synthetic image when the positional relationship between the two medical image sections to be processed is adjacent.

[0052] The third module is used to input the two medical image sections to be processed and the mask images corresponding to the two medical image sections to be processed into the target elastic deformation network model when the positional relationship between the two medical image sections to be processed is a layer, so as to obtain the volume data delineation result.

[0053] The target elastic deformation network model is obtained through the following steps:

[0054] Obtain three adjacent thin-layer medical image sections and their corresponding mask images;

[0055] A training set is constructed based on the three adjacent thin-layer medical image sections and their corresponding mask images. The training set includes the gold standard image, the medical image sections to be trained, and the mask images to be trained.

[0056] Construct an initial elastic deformation network model;

[0057] Construct the total loss function based on the weighting coefficients;

[0058] According to the total loss function, the training set is input into the initial elastic deformation network model so that the initial elastic deformation network model is trained to obtain the target elastic deformation network model.

[0059] On the other hand, embodiments of the present invention provide a computer device, including:

[0060] At least one processor;

[0061] At least one memory for storing at least one program;

[0062] When the at least one program is executed by the at least one processor, the at least one processor implements the method.

[0063] The beneficial effects of this invention are as follows:

[0064] This invention first acquires two medical image sections to be processed and their corresponding mask images. Then, it inputs these two sections into a target elastic deformation network model to obtain a target synthesized image. Next, it inputs the two medical image sections and their corresponding mask images into the target elastic deformation network model to obtain volumetric data delineation results. This achieves thin-layer medical image synthesis and volumetric data delineation, improving image synthesis accuracy and delineation efficiency. Specifically, the target elastic deformation network model first acquires three adjacent thin-layer medical image sections and their corresponding mask images to construct a training set. Then, it constructs an initial elastic deformation network model, constructs a total loss function based on weight coefficients, and finally inputs the training set into the initial elastic deformation network model based on the total loss function. This allows the initial elastic deformation network model to be trained, thereby improving model accuracy through the deformation field estimation module and deformation fusion module within the elastic deformation network model.

[0065] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description and the drawings. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a flowchart of a thin-layer medical image synthesis and volume data delineation method according to an embodiment of the present invention;

[0068] Figure 2 This is a flowchart illustrating how to obtain a target elastic deformation network model according to an embodiment of the present invention;

[0069] Figure 3 This is a schematic diagram of the framework structure of an initial elastic deformation network model according to an embodiment of the present invention;

[0070] Figure 4 This is a schematic diagram of the structure of a thin-layer medical image synthesis and volume data delineation device according to an embodiment of the present invention;

[0071] Figure 5 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0073] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0074] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0076] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0077] Thin-slice medical images: These refer to medical images with high inter-slice resolution (slice thickness ≤ 1 mm) (e.g., thin-slice CT images, thin-slice magnetic resonance (MR) images, etc.). For ease of description, this embodiment uses thin-slice cross-sectional MR (magnetic resonance) images as an example to describe the technical details.

[0078] Medical image volume data: Taking tomographic images scanned in the xoy plane as an example, multiple two-dimensional images in the xoy plane are obtained by scanning at different positions along the z-axis. These multiple two-dimensional images in the xoy plane are stacked into a three-dimensional cuboid along the z-direction. This kind of three-dimensional cuboid medical image data is medical image volume data, which can be mathematically represented by a three-dimensional array.

[0079] Outlining: refers to outlining the region of interest in an image, also known as segmentation.

[0080] In related technologies, the volumetric data of MR images acquired by 2D MR scanning is anisotropic, meaning that the inter-slice resolution (e.g., a slice thickness of 4 mm) is much lower than the intra-slice resolution (e.g., a pixel size of 1 mm × 1 mm), leading to the loss of inter-slice directional information. Adjusting scanning parameters can reduce the slice thickness of the MR image, but this reduces the image signal-to-noise ratio and increases scanning time. Although 3D MR scanning sequences (such as Siemens' 3D-Space sequence) can solve the problem of low image signal-to-noise ratio, the scanning speed is still slower than conventional thick-slice MR. Currently, deep learning-based automatic medical image segmentation models require a large amount of labeled data. When doctors manually delineate the volumetric data of medical images, they need to delineate the volumetric data layer by layer, which is time-consuming due to the large number of layers. A fast method for delineating medical image volumetric data can improve the efficiency of medical image volumetric data annotation and is of great significance. Existing methods for thin-slice medical image synthesis are mainly divided into two categories: methods based on super-resolution reconstruction and methods based on inter-slice image synthesis. Methods based on super-resolution reconstruction refer to reconstructing a corresponding high-resolution image from a low-resolution image. Taking thin-slice cross-sectional image synthesis (denoted as the xoy plane) as an example, the intra-slice resolution of the cross-sectional image is higher than the inter-slice resolution. To improve the resolution along the z-axis, super-resolution reconstruction techniques can be used to transform the problem of synthesizing thin-slice cross-sectional images into super-resolution reconstruction of sagittal (yoz plane) and coronal (xoz plane) images. By synthesizing higher-resolution sagittal and coronal images, the image clarity along the z-axis can be effectively improved. Inter-slice image synthesis methods refer to generating new images by combining multiple adjacent two-dimensional images (slices). These new images can be intermediate images between original slices or images viewed from different angles (such as sagittal and coronal planes). Taking thin-slice cross-sectional image synthesis as an example, inter-slice image synthesis methods aim to synthesize one or more cross-sectional images between two adjacent cross-sectional images, thereby improving inter-slice resolution. Traditional medical image volume data delineation methods typically require manual delineation of all two-dimensional images using annotation software, which is inefficient. Super-resolution reconstruction methods, when synthesizing sagittal and coronal planes, do not consider the semantic information of the transverse sections, resulting in synthesized images lacking smoothness in the transverse direction. This lack of smoothness affects the visual quality of the image and leads to low accuracy in the synthesized image. While methods based on inter-slice image synthesis can obtain relatively smooth thin-slice transverse sections, the synthesized transverse images still lack smoothness in the inter-slice direction due to the absence of smoothness constraints, which is detrimental to subsequent downstream segmentation tasks. Furthermore, due to the lack of anatomical structure constraints, the synthesized images lack anatomical fidelity. Traditional methods for delineating volumetric data in medical images typically require layer-by-layer delineation, which is time-consuming and labor-intensive.

[0081] In view of this, this embodiment synthesizes a cross-sectional image between two adjacent cross-sectional images to improve interlayer resolution. Simultaneously, it synthesizes a mask for the image between two adjacent cross-sectional images to achieve layer-by-layer (or multi-layer) delineation. The interlayer smoothness constraint loss function in this embodiment can solve the problem of insufficient smoothness in the interlayer direction of the synthesized image; by utilizing task-specific and realistic anatomical constraints, the anatomical structure of the synthesized image has high realism; and by synthesizing a mask for the intermediate layer between adjacent cross-sectional images, layer-by-layer (or multi-layer) delineation can be achieved, effectively improving delineation efficiency.

[0082] This application provides a method for thin-layer medical image synthesis and volume data delineation, relating to the field of image processing technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited thereto. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the thin-layer medical image synthesis and volume data delineation method, but is not limited to the above forms.

[0083] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0084] The embodiments of this application will be explained in detail below with reference to the accompanying drawings:

[0085] Figure 1This is an optional flowchart of a thin-layer medical image synthesis and volume data delineation method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S103.

[0086] Step S101: Obtain two cross-sections of the medical images to be processed and their corresponding mask images;

[0087] Step S102: When the positional relationship between two medical image sections to be processed is adjacent, input the two medical image sections to be processed into the target elastic deformation network model to obtain the target synthetic image;

[0088] Step S103: When the positional relationship between the two medical image sections to be processed is layered, input the two medical image sections to be processed and the mask images corresponding to the two medical image sections to be processed into the target elastic deformation network model to obtain the volume data delineation result.

[0089] Steps S101 to S103 as shown in the embodiments of this application realize thin-layer medical image synthesis and volume data delineation, thereby improving the accuracy of image synthesis and delineation efficiency.

[0090] In some embodiments, during steps S101-S103, the target elastic deformation network model of this embodiment can realize thin-layer medical image synthesis and rapid delineation of medical image volume data. For example, when thin-layer medical image synthesis is required, two adjacent sections of the medical image are selected as sections to be processed. The index t of the target synthesized image is set to 0.5. The two adjacent sections to be processed medical images and the index t are input into the target elastic deformation network model to obtain the target synthesized image. This target synthesized image is located between two adjacent thick-layer images to be processed, which can increase the interlayer resolution to twice its original value. If further resolution improvement is needed, the image synthesis process can be repeated multiple times. Furthermore, when rapid volumetric data delineation of medical images is required, the medical images are manually annotated layer by layer to obtain an annotated mask image. The medical image slice to be processed with t=0.5 and the two annotated layers and their corresponding mask images are input into the target elastic deformation network model to obtain the mask of the medical image slice in the middle without manual annotation, thus achieving the purpose of layer-by-layer delineation. This process is performed on all slices to finally obtain the volumetric data delineation result. If multiple layers of delineation are required, the value of the index t of the image to be processed and the number of slices between the two input slices can be changed, and the delineation can be repeated.

[0091] In some embodiments, such as Figure 2 As shown, obtaining the target elastic deformation network model may include, but is not limited to, steps S201 to S205.

[0092] Step S201: Obtain three adjacent thin-layer medical image sections and their corresponding mask images;

[0093] Step S202: Construct a training set based on three adjacent thin-layer medical image sections and their corresponding mask images. The training set includes the gold standard image, the medical image section to be trained, and the mask image to be trained.

[0094] Step S203: Construct the initial elastic deformation network model;

[0095] Step S204: Construct the total loss function based on the weighting coefficients;

[0096] Step S205: Based on the total loss function, input the training set into the initial elastic deformation network model to train the initial elastic deformation network model and obtain the target elastic deformation network model.

[0097] Steps S201 to S205, as shown in the embodiments of this application, obtain the target elastic deformation network model, thereby improving the model accuracy.

[0098] In step S201 of some embodiments, three adjacent thin-layer medical image sections and their corresponding mask images can be obtained from a medical image database. Thin-layer medical image sections and their corresponding mask images can also be obtained in other ways, and are not limited to these.

[0099] In some embodiments, in step S202, constructing a training set based on three adjacent thin-layer medical image sections and their corresponding mask images may include, but is not limited to, the following steps:

[0100] The middle section among three adjacent thin-layer medical image sections is used as the gold standard image, which is used for medical image synthesis and delineation.

[0101] Two sections of the septum in three adjacent thin-layer medical image sections are downsampled to obtain the corresponding medical image sections to be trained.

[0102] The mask images of the two cross-sections of the partition are downsampled to obtain the corresponding mask images to be trained.

[0103] In some embodiments, the middle slice of three adjacent thin-layer medical image slices can be used as the gold standard image, which is used for medical image synthesis and delineation. Then, the two slices of the interlayer between the three adjacent thin-layer medical image slices are downsampled to obtain the corresponding medical image slices to be trained. The mask images of the two interlayer slices are also downsampled to obtain the corresponding mask images to be trained. For example, high-quality medical images, taking MR images (magnetic resonance images) as an example, are collected. Isotropic high-resolution 3D-Space sequence MR images and their pre-annotated images are downsampled along the z-axis (assuming sampling every n voxels along the z-axis) to obtain the corresponding medical image slices to be trained (I0). i Or I i+1 ) and the mask image to be trained (Mask(M) i Or M i+1 ), where I i and I i+1 This represents two adjacent cross-sections (horizontal sections) of the medical image to be trained on the z-axis. The downsampled cross-sections of the medical image to be trained serve as the input to the subsequent elastic deformation network model. More specifically, the 3D-Space sequence MR images before downsampling can be used as the gold standard. i+t This is used to train and evaluate the performance of the elastic deformation network model in this embodiment.

[0104] In some embodiments, the process of constructing the initial elastic deformation network model in step S203 may include, but is not limited to, the following steps:

[0105] A deformation field estimation module is constructed, which is used to construct the corresponding deformation field based on the position encoding and two cross sections of the medical images to be trained.

[0106] After the deformation field estimation module, a first deformation fusion module is constructed. The first deformation fusion module is used to perform first deformation fusion processing based on the deformation field to obtain a thin-layer synthetic image.

[0107] A second deformation fusion module is constructed. The second deformation fusion module is used to perform second deformation fusion processing on two mask images to be trained to obtain an annotated synthetic image.

[0108] After the first deformation fusion module and the second deformation fusion module, a two-dimensional segmentation network is constructed. The two-dimensional segmentation network is trained using a loss function that combines cross-entropy and Dice loss.

[0109] In some embodiments, the framework structure of the initial elastic deformation network model is as follows: Figure 3As shown, the system includes a deformation field estimation module, a first deformation fusion module and a second deformation fusion module. Two adjacent 2-dimensional medical image slices to be trained are taken as input, two deformation fields are obtained through the deformation field estimation module, then the input images are deformed through the first deformation fusion module, and thin-slice synthetic images are obtained after fusion. The deformation field estimation module can be constructed first, and corresponding deformation fields can be constructed through the deformation field estimation module based on position encoding and the two adjacent medical image slices to be trained. Illustratively, a UNet network can be used as the deformation field estimation module, the input of which is the medical image slice to be trained I i , I i+1} and the index t of the image to be synthesized (0<t<1), and outputs the deformation field that transforms I i to the thin-slice synthetic image and the deformation field that transforms I to i+1 wherein h and w represent the height and width of the image respectively. It can be understood that the index t of the image to be synthesized is used to construct position encoding. The [j,k]-th pixel of the thin-slice synthetic image is shifted by F t→i [:,j,k] pixels in the height and width directions to obtain the corresponding pixel on the input image I i (that is, the medical image slice to be trained). In order to make the output deformation field correlated with t, this embodiment adopts position encoding technology, automatically learns the position encoding of t during training, fuses it with image features, and outputs deformation fields F t→i and F t→i+1 through the deformation field estimation module. After the deformation field estimation module, a first deformation fusion module is constructed, and first deformation fusion processing can be performed through the first deformation fusion module according to the deformation fields to obtain the thin-slice synthetic image then a second deformation fusion module is constructed, and second deformation fusion processing can be performed through the second deformation fusion module according to the mask images to be trained corresponding to the two adjacent medical image slices to be trained to obtain the annotated synthetic image After the first deformation fusion module and the second deformation fusion module, a 2D segmentation network is constructed, wherein the 2D segmentation network is trained with a loss function combining cross-entropy and Dice loss.

[0110] In some embodiments, performing first deformation fusion processing according to the deformation fields to obtain a thin-slice synthetic image comprises:

[0111] generating a two-dimensional grid according to the size and precision of the deformation field;

[0112] mapping grid node coordinates to the medical image slices to be trained according to the two-dimensional grid to obtain corresponding image pixel values;​​

[0113] Calculate the positional relationship of the mesh nodes before and after deformation based on the deformation field and mesh node coordinates;

[0114] Based on the positional relationship of the mesh nodes before and after deformation, select several pixels from the cross section of the medical image to be trained that are closest to the interpolation point as reference points;

[0115] Calculate the interpolation weights based on multiple reference points;

[0116] Based on the interpolation weights, the image pixel values ​​of multiple reference points are weighted and averaged to obtain the grid node pixel values;

[0117] Construct a deformed image based on the pixel values ​​of multiple grid nodes;

[0118] A thin-layer composite image is obtained by fusing the image index size and the two deformed images.

[0119] In some embodiments, the deformation process can be implemented using mesh sampling and linear interpolation, with the input medical image section I to be trained. i and I i+1 In the deformation field F t→i and F t→i+1 The image is deformed under the action of [something], thus obtaining the deformed image g(I). i ,F t→i ) and g(I i+1 ,F t→i+1 First, a two-dimensional mesh is generated based on the deformation field size and precision. Each mesh corresponds to a pixel in the cross-section of the medical image to be trained. Then, the mesh node coordinates are mapped to the input cross-section of the medical image to be trained, obtaining the corresponding image pixel values. Next, the positional relationship of the mesh nodes before and after deformation is calculated based on the deformation field and mesh node coordinates. Then, based on the positional relationship of the mesh nodes before and after deformation, several pixels closest to the interpolation point are selected from the cross-section of the medical image to be trained as reference points. For example, four pixels closest to the interpolation point in the input cross-section of the medical image to be trained can be selected as interpolation reference points. Then, interpolation weights are calculated based on multiple reference points. Based on the interpolation weights, the image pixel values ​​of the multiple reference points are weighted and averaged to obtain the mesh node pixel values. A deformed image is then constructed based on the multiple mesh node pixel values. Finally, the two deformed images are fused according to the image index size and the deformed image g(I) to obtain a thin-layer composite image. For example, during the fusion process, the image index t and the deformed image g(I) are fused according to the image index t and the deformed image g(I) i ,F t→i ) and g(I i+1 ,F t→i+1 The images are then fused to obtain a thin-layer composite image. It is understandable that the larger the t-value, the better the thin-layer synthesized image. On the z-axis and I i+1 The closer the distance, the more I i+1 right The greater the contribution, the better the mathematical expression for the fusion process becomes: In the formula, For thin-layer composite images, t is the image index, g(I) i ,F t→i ) and g(I i+1 ,F t→i+1 All of these are deformed images.

[0120] In some embodiments, step S204, constructing the total loss function based on the weight coefficients, may include, but is not limited to, the following steps:

[0121] Step S301: Construct an image reconstruction loss function based on the number of images required to be synthesized between the thin-layer synthesized image, the gold standard image, and adjacent medical image sections;

[0122] Step S302: Construct task-specific and realistically anatomically structured constraints;

[0123] Step S303: Construct the target smoothing constraint term;

[0124] Step S304: Construct the total loss function based on the image reconstruction loss function, the task-specific and anatomically realistic constraint term, the target smoothness constraint term, and the weight coefficients.

[0125] In some embodiments, in steps S301-S304, an image reconstruction loss function can be constructed first based on the number of images required to be synthesized between the thin-layer synthesized image, the gold standard image, and adjacent medical image sections to be trained. The expression for the image reconstruction loss function is: In the formula, loss rec For image reconstruction loss, I i+t For gold standard images, For thin-layer image synthesis, n is the number of images to be synthesized between adjacent medical image sections, and ||*||1 is the L1 norm. Then, task-specific and anatomically realistic constraints are constructed, as well as a target smoothing constraint. Finally, based on the image reconstruction loss function, the task-specific and anatomically realistic constraints, the target smoothing constraint, and the weighting coefficients, a total loss function is constructed. The expression for the total loss function is: loss syn =λ rec loss rec +loss spe_ana +loss smooth In the formula, loss syn For the total loss, λ recThe weighting coefficients of the image reconstruction loss are called loss. rec For image reconstruction loss, loss spe_ana For task-specific and realistic anatomical constraints, loss smooth The objective is to smooth out the constraint terms.

[0126] In some embodiments, the construction of task-specific and anatomically realistic constraints in step S302 may include, but is not limited to, the following steps:

[0127] Based on the thin-layer synthetic image segmentation results, pre-segmented labeled images, and cross-entropy loss function, task-specific constraints are constructed.

[0128] Based on the labeled synthetic image, the pre-segmented labeled image, and the cross-entropy loss function, construct a realistic anatomical structure constraint term;

[0129] Based on task-specific constraints and anatomical-realistic constraints, construct task-specific and anatomical-realistic constraints.

[0130] In some embodiments, task-specific constraints can be constructed first based on the thin-layer synthetic image segmentation results, the pre-segmented labeled image, and the cross-entropy loss function. The expression for the task-specific constraints is as follows: In the formula, loss spe For task-specific constraints, L ce M is the cross-entropy loss function. i+t For pre-segmented and labeled images, The result is a thin-layer synthetic image segmentation, where n is the number of images required to be synthesized between adjacent medical image slices to be trained. Then, based on the labeled synthetic image, the pre-segmented labeled image, and the cross-entropy loss function, an anatomical structure realism constraint term is constructed, where the expression for the anatomical structure realism constraint term is: In the formula, loss ana For realistic anatomical structure constraints. This is for annotating the composite image. More specifically, the formula for calculating the annotation of the composite image is: In the formula, t is the image index, g(M) i ,F t→i M is the mask image to be trained. i In the deformation field F i→t Under the influence of grid sampling and linear interpolation, M is obtained. i The deformed mask image (i.e., the deformed image), g(M) i+1 ,F t→i+1 M is the mask image to be trained. i+1 In the deformation field F t→i+1 Under the influence of grid sampling and linear interpolation, M is obtained. i+1The deformed mask image. Then, based on task-specific constraints and anatomical structure realism constraints, a task-specific and anatomical structure realism constraint is constructed, where the expression for the task-specific and anatomical structure realism constraint is: loss spe_ana =λ spe loss spe +λ ana loss ana In the formula, loss spe_ana For task-specific and anatomically realistic constraints, λ spe λ represents the weighting coefficient of the task-specific constraint term. ana The weighting coefficients are for the anatomical structure realism constraint terms.

[0131] In some embodiments, the construction of the target smoothing constraint term in step S303 may include, but is not limited to, the following steps:

[0132] Based on the deformation field gradient of the medical image to be trained transformed into a thin-layer synthetic image, an intra-layer smoothing constraint term is constructed;

[0133] Based on the synthesized image of the current layer, the synthesized image of the previous layer, and the synthesized image of the next layer, construct inter-layer smoothing constraints;

[0134] Based on the intra-layer smoothing constraint term and the inter-layer smoothing constraint term, construct the target smoothing constraint term.

[0135] In some embodiments, an intra-layer smoothing constraint term can be constructed first based on the deformation field gradient of the medical image section transformed into a thin-layer synthetic image. The expression for the intra-layer smoothing constraint term is as follows: In the formula, loss xy This represents the intra-layer smoothing constraint term, where n is the number of images required to be synthesized between adjacent medical image slices to be trained. For the medical image section I to be trained i Transformation to thin-layer composite image The gradient of the deformation field, For the medical image section I to be trained i+1 Transformation to thin-layer composite image The gradient of the deformation field, ‖*‖1, is the L1 norm. It can be understood that by calculating the L1 norm of the gradient, adjacent pixels within the same layer can have similar deformations, maintaining the smoothness of the thin-layer synthesized image within the layer. Then, based on the synthesized image of the current layer, the synthesized image of the previous layer, and the synthesized image of the next layer, an inter-layer smoothing constraint term is constructed, where the expression for the inter-layer smoothing constraint term is: In the formula, loss z For interlayer smoothing constraints, This is the composite image of the current layer, i.e., the thin composite image of the current layer. This is the composite image of the previous layer, specifically the composite image of the thin layer from the previous layer. This is the next layer of synthesized image, specifically the thin-layer synthesized image. It's understandable that by calculating the norm between adjacent pixels in each layer, and then evaluating and minimizing the gray-level differences between adjacent layers, the consistency and smoothness of the thin-layer synthesized image in the inter-layer direction can be achieved. Then, based on the intra-layer and inter-layer smoothing constraints, a target smoothing constraint is constructed, where the expression for the target smoothing constraint is: loss smooth =λ xy loss xy +λ z loss z In the formula, loss smooth For the objective smoothing constraint term, λ xy λ represents the weighting coefficient of the in-layer smoothing constraint term. z These are the weighting coefficients for the interlayer smoothing constraint term.

[0136] In some embodiments, in step S205, the training set can be input into the initial elastic deformation network model according to the total loss function, so that the initial elastic deformation network model can be trained to obtain the target elastic deformation network model. For example, a 2D segmentation network (UNet) can first be trained using high-resolution medical image slices to be trained and their segmented training mask images from the training set. The 2D segmentation network is trained using a loss function combining cross-entropy and Dice loss. Then, the weights of the 2D segmentation network are fixed, and the initial elastic deformation network model is trained using the total loss function to obtain the target elastic deformation network model.

[0137] In some embodiments, the elastic deformation network model of this embodiment can simultaneously achieve thin-layer medical image synthesis and rapid delineation of medical image volume data. By estimating the elastic deformation field, adjacent medical image sections to be trained are deformed and fused to generate thick-layer medical images, while simultaneously achieving rapid delineation of medical image volume data. This embodiment can effectively improve the efficiency of medical image volume data delineation, while solving the problems of lack of anatomical structure information guidance and insufficient image realism in existing thin-layer medical image synthesis techniques, and the synthesized images have task specificity. This embodiment introduces task-specific and anatomically realistic constraint terms, using manually segmented mask images to guide the network to synthesize thick-layer medical images with realistic anatomical structures, ensuring its task specificity and providing a reliable foundation for subsequent segmentation tasks. This embodiment proposes an interlayer smoothing constraint term, which minimizes the gray-level difference between adjacent layers by calculating the norm between adjacent pixels, making the thin-layer synthesized image consistent and smooth in the interlayer direction, thus improving image quality. Furthermore, the deformation field estimation module of this embodiment can use a UNet network, or a network such as ResNet, Transformer, or Mamba.

[0138] The beneficial effects of implementing the embodiments of the present invention include: Firstly, the embodiments of the present invention acquire two medical image sections to be processed and their corresponding mask images. Then, the two medical image sections to be processed are input into a target elastic deformation network model to obtain a target synthesized image. Next, the two medical image sections to be processed and their corresponding mask images are input into the target elastic deformation network model to obtain volume data delineation results. This achieves thin-layer medical image synthesis and volume data delineation, improving the accuracy of image synthesis and delineation efficiency. Specifically, the target elastic deformation network model first acquires three adjacent thin-layer medical image sections and their corresponding mask images to construct a training set. Then, an initial elastic deformation network model is constructed. Next, a total loss function is constructed based on the weight coefficients. Finally, the training set is input into the initial elastic deformation network model based on the total loss function, allowing the initial elastic deformation network model to be trained. This improves model accuracy through the deformation field estimation module and deformation fusion module in the elastic deformation network model. Simultaneously, this embodiment can simultaneously achieve thin-layer medical image synthesis and rapid annotation of medical image volume data. The thin-layer medical images synthesized in this embodiment have good interlayer smoothness and more realistic anatomical structures. This embodiment can delineate medical image volume data layer by layer (or multiple layers) with high efficiency.

[0139] like Figure 4 As shown, this embodiment of the invention also provides a thin-layer medical image synthesis and volume data delineation device, comprising:

[0140] The first module 801 is used to acquire two medical image cross-sections to be processed and their corresponding mask images;

[0141] The second module 802 is used to input the two medical image sections to be processed into the target elastic deformation network model to obtain the target synthetic image when the positional relationship between the two medical image sections to be processed is adjacent.

[0142] The third module 803 is used to input the two medical image sections to be processed and the mask images corresponding to the two medical image sections to be processed into the target elastic deformation network model when the positional relationship between the two medical image sections to be processed is a layer, so as to obtain the volume data delineation result.

[0143] The target elastic deformation network model is obtained through the following steps:

[0144] Obtain three adjacent thin-layer medical image sections and their corresponding mask images;

[0145] A training set is constructed based on three adjacent thin-layer medical image sections and their corresponding mask images. The training set includes the gold standard image, the medical image section to be trained, and the mask image to be trained.

[0146] Construct an initial elastic deformation network model;

[0147] Construct the total loss function based on the weighting coefficients;

[0148] Based on the total loss function, the training set is input into the initial elastic deformation network model to train the initial elastic deformation network model and obtain the target elastic deformation network model.

[0149] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0150] like Figure 5 As shown, embodiments of the present invention also provide a computer device, including:

[0151] At least one processor 901;

[0152] At least one memory 902 is used to store at least one program;

[0153] When at least one program is executed by at least one processor, such that at least one processor achieves Figure 1 The method shown.

[0154] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0155] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for thin-layer medical image synthesis and volume data delineation, characterized in that, Includes the following steps: Obtain two cross-sections of the medical images to be processed and their corresponding mask images; When the two medical image sections to be processed are adjacent, the two medical image sections to be processed are input into the target elastic deformation network model to obtain the target synthetic image; When the positional relationship between the two medical image sections to be processed is that they are separated by layers, the two medical image sections to be processed and the mask images corresponding to the two medical image sections to be processed are input into the target elastic deformation network model to obtain the volume data delineation result. The target elastic deformation network model is obtained through the following steps: Obtain three adjacent thin-layer medical image sections and their corresponding mask images; A training set is constructed based on the three adjacent thin-layer medical image sections and their corresponding mask images. The training set includes the gold standard image, the medical image sections to be trained, and the mask images to be trained. Construct an initial elastic deformation network model; Construct the total loss function based on the weighting coefficients; According to the total loss function, the training set is input into the initial elastic deformation network model so that the initial elastic deformation network model is trained to obtain the target elastic deformation network model; The construction process of the initial elastic deformation network model includes: A deformation field estimation module is constructed, which is used to construct the corresponding deformation field based on the position encoding and two cross sections of the medical image to be trained; After the deformation field estimation module, a first deformation fusion module is constructed. The first deformation fusion module is used to perform a first deformation fusion process based on the deformation field to obtain a thin-layer synthetic image. A second deformation fusion module is constructed, which is used to perform a second deformation fusion process based on the two mask images to be trained to obtain an annotated synthetic image; After the first deformation fusion module and the second deformation fusion module, a two-dimensional segmentation network is constructed. The two-dimensional segmentation network is trained using a loss function that combines cross-entropy and Dice loss. The first deformation fusion process, based on the deformation field, to obtain a thin-layer composite image includes: A two-dimensional mesh is generated based on the size and accuracy of the deformation field; Based on the two-dimensional grid, the grid node coordinates are mapped to the cross section of the medical image to be trained to obtain the corresponding image pixel values; Based on the deformation field and the grid node coordinates, calculate the positional relationship of the grid nodes before and after deformation; Based on the positional relationship of the mesh nodes before and after deformation, several pixels closest to the interpolation point are selected from the cross-section of the medical image to be trained as reference points. Calculate the interpolation weights based on the multiple reference points; According to the interpolation weight, the image pixel values ​​of multiple reference points are weighted and averaged to obtain the grid node pixel values; A deformed image is constructed based on the pixel values ​​of multiple grid nodes; The thin-layer composite image is obtained by fusing the image index size and the two deformed images.

2. The method according to claim 1, characterized in that, The step of constructing a training set based on the three adjacent thin-layer medical image sections and their corresponding mask images includes: The middle section among the three adjacent thin-layer medical image sections is used as the gold standard image, which is used for medical image synthesis and delineation. Downsampling is performed on two sections of the septum between the three adjacent thin-layer medical image sections to obtain the corresponding medical image sections to be trained; The mask images of the two cross-sections of the partition are downsampled to obtain the corresponding mask images to be trained.

3. The method according to claim 1, characterized in that, The step of constructing the total loss function based on the weighting coefficients includes: Based on the number of images required to be synthesized between the thin-layer synthetic image, the gold standard image, and adjacent medical image sections, an image reconstruction loss function is constructed. Construct task-specific and realistically anatomically structured constraints; Construct target smoothing constraints; The total loss function is constructed based on the image reconstruction loss function, the task-specific and anatomically realistic constraint term, the target smoothness constraint term, and the weight coefficients.

4. The method according to claim 3, characterized in that, The construction task-specific and anatomically realistic constraints include: Based on the thin-layer synthetic image segmentation results, pre-segmented labeled images, and cross-entropy loss function, task-specific constraints are constructed. Based on the labeled synthetic image, the pre-segmented labeled image, and the cross-entropy loss function, construct an anatomical structure realism constraint term; Based on the task-specific constraints and the anatomical structure realism constraints, construct the task-specific and anatomical structure realism constraints.

5. The method according to claim 3, characterized in that, The construction target smoothing constraint term includes: Based on the deformation field gradient of the medical image to be trained transformed into a thin-layer synthetic image, an intra-layer smoothing constraint term is constructed; Based on the synthesized image of the current layer, the synthesized image of the previous layer, and the synthesized image of the next layer, construct inter-layer smoothing constraints; The target smoothing constraint term is constructed based on the intra-layer smoothing constraint term and the inter-layer smoothing constraint term.

6. The method according to claim 3, characterized in that, The expression for the image reconstruction loss function is: ； In the formula, The image reconstruction loss is... The gold standard image, For the thin-layer composite image, The number of images to be synthesized between adjacent medical image sections. It is an L1 norm.

7. An apparatus for implementing the thin-layer medical image synthesis and volume data delineation method as described in any one of claims 1-6, characterized in that, include: The first module is used to acquire two cross-sections of the medical images to be processed and their corresponding mask images; The second module is used to input the two medical image sections to be processed into the target elastic deformation network model to obtain the target synthetic image when the positional relationship between the two medical image sections to be processed is adjacent. The third module is used to input the two medical image sections to be processed and the mask images corresponding to the two medical image sections to be processed into the target elastic deformation network model when the positional relationship between the two medical image sections to be processed is a layer, so as to obtain the volume data delineation result. The target elastic deformation network model is obtained through the following steps: Obtain three adjacent thin-layer medical image sections and their corresponding mask images; A training set is constructed based on the three adjacent thin-layer medical image sections and their corresponding mask images. The training set includes the gold standard image, the medical image sections to be trained, and the mask images to be trained. Construct an initial elastic deformation network model; Construct the total loss function based on the weighting coefficients; According to the total loss function, the training set is input into the initial elastic deformation network model so that the initial elastic deformation network model is trained to obtain the target elastic deformation network model.

8. A computer device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-6.

Images