Lung image prediction method, device and computer-readable storage medium

By acquiring rib images of the target human body in different phases and using a rib elastic registration network to predict changes in lung images, the problem of difficulty in tracking changes in lung shape and position in lung image navigation systems is solved, improving navigation accuracy and reducing computational load.

CN116935002BActive Publication Date: 2026-11-17HANGZHOU BRONCUS MEDICAL CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210334799.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-31
Publication Date
2026-11-17
Estimated Expiration
2042-03-31

AI Technical Summary

Technical Problem

Existing lung image navigation systems, which use static images, cannot accurately track changes in the shape and position of the lungs during human respiration, resulting in reduced navigation accuracy.

Method used

By acquiring rib images of the target human body at different phases, a rib elastic registration network is used to predict changes in lung images. Combined with the rib deformation field, dynamic prediction of lung images is achieved.

Benefits of technology

It improves the accuracy of lung image navigation, simplifies the image acquisition process, reduces computational load, and increases the efficiency of acquiring lung images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116935002B_ABST
    Figure CN116935002B_ABST
Patent Text Reader

Abstract

This invention relates to the field of medical image processing technology and provides a method, device, and computer-readable storage medium for predicting lung images. The method includes: acquiring rib images of a target human body at different phases to obtain a rib image sequence; acquiring the rib deformation field corresponding to the rib image at any phase in the rib image sequence and a rib elastic registration network; and predicting the lung image of the target human body at a target phase based on the acquired lung image of the target human body at any phase and the acquired rib deformation field. This enables accurate prediction of lung image changes during human respiration, thereby improving the accuracy of lung image navigation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and particularly relates to a method, device and computer-readable storage medium for lung image prediction. Background Technology

[0002] In recent years, lung-based intelligent navigation technology has developed rapidly. However, most clinically used image navigation systems currently employ static images of the lungs, meaning they assume the lungs' position and shape are static. In actual lung navigation, the shape and position of the lungs change significantly with respiratory movements. Therefore, using static images for navigation reduces accuracy and makes it difficult to pinpoint the exact location of lesions. Summary of the Invention

[0003] The lung image prediction method, apparatus, and computer-readable storage medium provided in this application can accurately predict changes in lung images during human respiration, thereby improving the accuracy of lung image navigation.

[0004] This application provides a lung image prediction method, the method comprising:

[0005] Acquire rib images of the target human body at different phases to obtain a rib image sequence;

[0006] Based on the rib image at any phase contained in the rib image sequence and the rib elastic registration network, obtain the rib deformation field corresponding to the rib image at any phase.

[0007] Based on the acquired lung images of the target human body at any phase and the obtained rib deformation field, the lung image of the target human body at the target phase is predicted, wherein the difference between any phase and the target phase satisfies a preset condition.

[0008] One embodiment of this application provides a lung image prediction device, the device comprising:

[0009] The image sequence acquisition unit is used to acquire rib images of the target human body under different phases to obtain a rib image sequence.

[0010] The deformation field acquisition unit is used to acquire the rib deformation field corresponding to the rib image at any phase in the rib image sequence and the rib elastic registration network; and

[0011] An image prediction unit is used to predict the lung image of the target human body at a target phase based on the acquired lung image of the target human body at any phase and the obtained rib deformation field, wherein the difference between the any phase and the target phase satisfies a preset condition.

[0012] One aspect of this application also provides an electronic device, including: a memory and a processor;

[0013] The memory stores executable program code;

[0014] The processor coupled to the memory invokes the executable program code stored in the memory to perform the steps of the method described above.

[0015] One aspect of this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when run by a processor, implements the steps of the method described above.

[0016] The embodiments provided in this application acquire rib images of a target human body at different phases to obtain a rib image sequence. Based on the rib image at any phase contained in the rib image sequence and the rib elastic registration network, the rib deformation field corresponding to the rib image at any phase is obtained. Based on the acquired lung image of the target human body at any phase and the obtained rib deformation field, the lung image of the target human body at the target phase is predicted. This enables accurate prediction of changes in lung images during human respiration, obtains the dynamic changes of the lung image of the target human body, and thus improves the accuracy of lung image navigation. Furthermore, compared to directly acquiring lung images at different phases, acquiring rib images at different phases is simpler, and the computational load required for extracting rib images is less. In this application, using rib images to predict the lung image at the target phase can significantly reduce the computational load and improve the efficiency of acquiring the lung image at the target phase. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the implementation of a lung image prediction method according to an embodiment of this application;

[0019] Figure 2A flowchart illustrating the implementation of a lung image prediction method according to an embodiment of this application;

[0020] Figure 3 A flowchart illustrating the training implementation of the rib elastic registration network in a lung image prediction method provided in an embodiment of this application;

[0021] Figure 4 A schematic diagram of the structure of a lung image prediction device provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] See Figure 1 This application provides an embodiment of a lung image prediction method. For ease of explanation, only the parts relevant to this embodiment are shown. Figure 1 As shown, the method specifically includes:

[0025] In step S101, rib images of the target human body under different phases are acquired to obtain a rib image sequence;

[0026] This invention is applicable to computing devices, such as computing devices for medical image processing. When acquiring rib images of a target human body at different phases of respiration, rib images of the target human body at different phases can be directly acquired using a rib image acquisition device to obtain a rib image sequence. Optionally, a 4D lung image scanner can be used to acquire 4D lung images of the target human body. Based on the acquired 4D lung images, 3D lung images of lung morphology and position at different phases are obtained. Then, rib extraction is performed on the 3D lung images to obtain rib images of the target human body at different phases. These individual rib images constitute a rib image sequence. A time axis is added to the 4D lung images acquired by the 4D lung image scanner, so that the acquired 4D lung images can include 3D lung images of the target human body at different respiratory phases arranged in chronological order.

[0027] In one embodiment of this application, when extracting ribs from 3D lung images, threshold analysis, morphological operations, and region growing algorithms can be used to improve the accuracy of rib extraction. Specifically, morphological operations can be used to preprocess the 3D lung images, and then a suitable threshold can be selected. A region growing algorithm can then be used to segment the rib and non-rib regions in each preprocessed 3D lung image to obtain the segmented rib regions. Morphological operations can simplify 3D-CT image data, preserving their basic shape characteristics and removing irrelevant structures. Basic morphological operations include dilation, erosion, opening, and closing operations, which are not limited in this specification.

[0028] In step S102, the rib deformation field corresponding to the rib image in any phase is obtained based on the rib image in any phase of the rib image sequence and the rib elastic registration network.

[0029] In this embodiment of the invention, a rib elastic registration network is used to register the rib image to be registered, so as to obtain the rib deformation field corresponding to the rib image. This rib elastic registration network can be a deep learning-based convolutional neural network, thereby improving the accuracy of the obtained rib deformation field. Specifically, the rib elastic registration network can adopt the encoding and decoding structure of the Unet network. Specifically, the rib elastic registration network can include the same number of downsampling layers and upsampling layers, with each upsampling layer connected to a downsampling layer at the corresponding scale.

[0030] In step S103, based on the lung images of the target human body at any phase and the obtained rib deformation field, the lung image of the target human body at the target phase is predicted.

[0031] In this embodiment of the invention, the target phase can be the next phase of any phase (a certain phase) or the second or third phase after that phase. That is, the difference between any phase and its corresponding target phase satisfies a preset condition. The preset condition can be set as needed, and this specification does not limit it. For example, the preset condition can be that the difference between any phase and the target phase satisfies a preset value. When any phase is phase T(1), the target phase can be phase T(2); or, when any phase is phase T(1), the target phase can be phase T(3); or, when any phase is phase T(1), the target phase can be phase T(4), etc. Of course, the preset condition can also be that any phase, the target phase, and any sample rib image in the training sample set match the first phase and the second phase contained therein. This specification does not limit it. After acquiring the lung image of the target human body in any phase, the lung image of the target human body in the target phase can be predicted based on the acquired lung image of the target human body in any phase and the obtained rib deformation field.

[0032] This application embodiment acquires rib images of a target human body at different phases to obtain a rib image sequence. Based on the rib image at any phase contained in the rib image sequence and the rib elastic registration network, the rib deformation field corresponding to the rib image at any phase is obtained. Based on the acquired lung image of the target human body at any phase and the obtained rib deformation field, the lung image of the target human body at the target phase is predicted. This enables accurate prediction of changes in lung images during human respiration, obtains the dynamic changes of the lung image of the target human body, and thus improves the accuracy of lung image navigation, playing an auxiliary role in surgical path planning and lesion localization in surgical navigation. At the same time, compared with directly acquiring lung images at different phases, the acquisition method of rib images at different phases is simpler, and the computational amount required for the rib image extraction process is less. In this application, using rib images to predict the lung image at the target phase can significantly reduce the computational amount and improve the efficiency of acquiring the lung image at the target phase.

[0033] See Figure 2 This application provides an embodiment of a lung image prediction method. For ease of explanation, only the parts relevant to this embodiment are shown. Figure 2 As shown, the method specifically includes:

[0034] In step S201, rib images of the target human body under different phases are acquired to obtain a rib image sequence;

[0035] This invention is applicable to computing devices, such as computing devices for medical image processing. When acquiring rib images of a target human body at different phases of respiration, rib images of the target human body at different phases can be directly acquired using a rib image acquisition device to obtain a rib image sequence. Optionally, a 4D lung image scanner can be used to acquire 4D lung images of the target human body. Based on the acquired 4D lung images, 3D lung images of the lung morphology and position at different phases can be obtained. Then, rib extraction is performed on the 3D lung images to obtain rib images of the target human body at different phases, and the individual rib images constitute a rib image sequence.

[0036] In one embodiment of this application, when extracting ribs from 3D lung images, threshold analysis, morphological operations, and region growing algorithms can be used to improve the accuracy of rib extraction. Specifically, morphological operations can be used to preprocess the 3D lung images, and then a suitable threshold can be selected. A region growing algorithm can then be used to segment the rib and non-rib regions in each 3D lung image to obtain the segmented rib regions. Morphological operations can simplify 3D-CT image data, preserving their basic shape characteristics and removing irrelevant structures. Basic morphological operations include dilation, erosion, opening, and closing operations, etc., and this specification does not limit the scope of these operations.

[0037] In step S202, the rib image under any phase is subjected to an affine transformation to obtain the transformed rib image;

[0038] In step S203, the rib deformation field corresponding to the rib image under any phase is obtained based on the transformed rib image and the rib elastic registration network.

[0039] In this embodiment, the pre-trained rib elastic registration network can be obtained in the following way: first, a training sample set is obtained, which contains several pairs of sample rib images. Each pair of sample rib images contains a sample rib image under a first phase and a sample rib image under a second phase. The first phase and the second phase can be any phase. The difference between the first phase and the second phase satisfies a preset condition and is sorted according to the time series. The first phase is usually located before the second phase.

[0040] To reduce the training difficulty of the rib elastic registration network and improve the accuracy of the output results obtained after training, affine transformations can be performed on the rib image pairs contained in the training sample set. Affine transformations can be applied to the rib images in the first phase of any rib image pair, maximizing the image similarity between the transformed rib image and the rib image in the second phase. The affine transformation can include operations such as scaling, translation, rotation, shearing, and reflection. For example, assuming a rib image pair contains rib images in phases T(n) and T(n+1), the affine transformation can be used to transform the rib image in phase T(n) to obtain the optimal spatial position parameters that maximize the similarity between the transformed rib image in phase T(n) and the rib image in phase T(n+1). The optimal spatial position parameters can include scaling factors and translation parameters. Then, the sample rib image under phase T(n) after affine transformation can be represented as: a*(sample rib image under phase T(n))+b, where a is the scaling factor and b is the translation parameter.

[0041] Then, the trained rib elastic registration network can be obtained by training the rib elastic registration network based on the rib images of each sample after affine transformation.

[0042] In this embodiment, when obtaining the rib deformation field corresponding to the rib image at any phase based on the rib image at any phase contained in the rib image sequence and the trained rib elastic registration network, the rib image at any phase can be first subjected to an affine transformation according to the above-mentioned optimal spatial position parameters to obtain the transformed rib image. Then, based on the transformed rib image and the rib elastic registration network, the rib deformation field corresponding to the rib image at any phase can be obtained. Thus, through the affine transformation of the rib image at the current phase, the displacement deviation between the pixels corresponding to the rib region contained in the transformed rib image and the rib image at the target phase can be reduced, improving the accuracy of the rib elastic registration network in image registration, and thereby improving the accuracy of the obtained corresponding rib deformation field.

[0043] In one embodiment of this application, when obtaining the rib deformation field corresponding to the rib image at any phase based on the transformed rib image and the rib elastic registration network, the transformed rib image is input into the rib elastic registration network to obtain the rib deformation field of the rib image corresponding to any phase output by the rib elastic registration network. Thus, the rib deformation field between the input rib image and the rib image it is registered with is obtained in real time through the rib elastic registration network, thereby improving the accuracy of the obtained corresponding rib deformation field.

[0044] In another embodiment of this application, when obtaining the rib deformation field corresponding to the rib image at any phase based on the transformed rib image and the trained rib elastic registration network, the deformation field set corresponding to the trained rib elastic registration network can be obtained first. This deformation field set contains the rib deformation fields corresponding to the rib images at each phase. Then, subsequently, it is only necessary to search and match the rib image at any phase within the deformation field set to obtain the rib deformation field corresponding to the rib image at any phase. Thus, by searching for the rib deformation field corresponding to the rib image within the deformation field set, it is not necessary to call the trained rib elastic registration network every time, thereby improving the response speed of rib deformation field acquisition. For example, suppose we acquire sample rib images of several patients in the T(1) phase and T(3) phase respectively, and input the acquired sample rib images into the rib elastic registration network for processing. We can obtain the rib deformation field 1 output by the rib elastic registration network, which is used to transform the rib image in the T(1) phase into the corresponding rib image in the T(3) phase. By combining the rib image of any patient in the T(1) phase and the rib deformation field 1, we can predict the rib image of the patient in the T(3) phase. Meanwhile, assuming that several pairs of sample rib images of patients in the T(5) and T(6) phases are also acquired, and the acquired sample rib image pairs are sequentially input into the rib elastic registration network for processing, the rib deformation field 2 output by the rib elastic registration network can be obtained, which is used to transform the rib image in the T(5) phase into the corresponding rib image in the T(6) phase. By combining the rib image of any patient in the T(5) phase and the rib deformation field 2, the rib image of the patient in the T(6) phase can be predicted. At this time, the deformation field set corresponding to the trained rib elastic registration network can be obtained, which includes rib deformation field 1 and rib deformation field 2.

[0045] If we subsequently obtain a rib image of any patient in phase T(1) and want to predict the rib image of that patient in phase T(3), we can directly find the rib deformation field 1 corresponding to the rib image in phase T(1) from the deformation field set, without having to acquire the CT image of that patient in phase T(3) to obtain the corresponding rib image, which is more convenient and faster. Similarly, if we obtain a rib image of any patient in phase T(5) and want to predict the rib image of that patient in phase T(6), we can directly find the rib deformation field 2 corresponding to the rib image in phase T(5) from the deformation field set, without having to acquire the CT image of that patient in phase T(6) to obtain the corresponding rib image.

[0046] In step S204, the lung image under any phase is subjected to an affine transformation to obtain the transformed lung image;

[0047] In step S205, the lung image of the target human body under the target phase is predicted based on the transformed lung image and the rib deformation field.

[0048] In this embodiment of the application, when predicting the lung image of the target human body at the target phase, the lung image at any phase can also be affine transformed according to the above-mentioned optimal spatial position parameters to obtain the transformed lung image, so as to reduce the difference between the input lung image and the predicted image during prediction. Then, based on the transformed lung image and the rib deformation field, the lung image of the target human body at the target phase is predicted.

[0049] In this embodiment, when acquiring the rib deformation field corresponding to the rib image at any phase, the rib image at any phase is subjected to an affine transformation. Based on the transformed rib image and the rib elastic registration network, the rib deformation field corresponding to the rib image at any phase is obtained. Thus, the affine transformation is performed on the input rib image to be registered when the rib deformation field is obtained, which improves the accuracy of the rib deformation field to be acquired. Similarly, when predicting the lung image of the target human body at the target phase, the lung image at any phase is subjected to an affine transformation. Based on the transformed lung image and the rib deformation field, the lung image of the target human body at the target phase is predicted. Thus, the lung image is also subjected to an affine transformation during prediction, which further improves the accuracy of image prediction.

[0050] See Figure 3 This application provides a method for predicting lung images, specifically the training process of a rib elastic registration network. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0051] like Figure 3 As shown, the method specifically includes:

[0052] In step S301, the training sample set is obtained;

[0053] In this embodiment of the application, the training sample set for training the rib elastic registration network includes several rib image pairs. Each rib image pair includes a rib image under a first phase and a rib image under a second phase. The second phase can be the next phase after the first phase or the second or third phase thereafter. That is, the difference between the first phase and the second phase satisfies a preset condition.

[0054] In step S302, each sample rib image pair is used as input to the rib elastic registration network, the rib elastic registration network is trained, and the parameters of the rib elastic registration network are adjusted until the trained rib elastic registration network and the deformation field set corresponding to the trained rib elastic registration network are obtained.

[0055] In this embodiment, after obtaining the training sample set, the rib elastic registration network is trained based on the rib images of each sample to obtain the trained rib elastic registration network. Specifically, before training the rib elastic registration network, the weight parameters of the rib elastic registration network can be initialized using Xavier initialization to improve the training efficiency of the rib elastic registration network.

[0056] In another embodiment of this application, preferably, when training the rib elastic registration network based on the training sample set, an affine transformation is performed on the first phase sample rib image contained in each sample rib image pair to obtain transformed sample rib image pairs. This ensures that the sample rib image of the first phase, after the affine transformation, has the greatest similarity to the image under the other phase in the sample rib image pair, thereby reducing the training difficulty of the rib elastic registration network for the deformation field of rib image pairs with large displacement offsets. Subsequently, the rib elastic registration network is trained based on the transformed sample rib image pairs to obtain the trained rib elastic registration network, thereby improving the training efficiency of the rib elastic registration network.

[0057] In one embodiment of this application, the loss function used when training the rib elastic registration network can be:

[0058]

[0059] Where f represents the sample rib image in the second phase as a fixed image in the training sample set, m represents the sample rib image in the first phase after transformation as a floating image in the training sample set, φ represents the rib deformation field corresponding to the sample rib image in the first phase output by the rib elastic registration network, and L smooth (φ) represents the smoothing loss of the rib deformation field φ. Representing f and the image The similarity loss between them. However, by applying the rib deformation field φ corresponding to the sample rib image under the first phase to image m, the image can be obtained. This loss function can effectively improve the rib elastic registration network, thereby enhancing the performance of the trained rib elastic registration network.

[0060] The process of training the rib elastic registration network may include the following steps:

[0061] Step a: Construct a sample rib dataset, and use 50% of the sample rib dataset as the training set, 10% as the validation set, and 40% as the test set. The sample rib dataset contains sample rib images at different phases.

[0062] Step b: Normalize the size of the sample rib images contained in the sample rib dataset to a uniform size of 512*512*512.

[0063] Step c: Establish the rib elastic registration network and initialize the weight parameters of the rib elastic registration network using the Xavier initialization method.

[0064] Step d: Input the normalized sample rib images into the rib elastic registration network in sequence to obtain the predicted rib deformation field φ corresponding to each sample rib image output by the rib elastic registration network. The rib deformation field φ can be represented by a matrix.

[0065] Step e: Based on the sample rib images at different phases and the corresponding predicted rib deformation field φ, calculate the loss function L. us The function value of (f, m, φ).

[0066] Step f involves using the calculated loss function value in the error backpropagation algorithm to obtain the optimized weight parameters for the rib elastic registration network. Assume 1000 training rounds are performed on the sample rib dataset, with validation performed on the validation set in each round. After 1000 rounds of training, the rib elastic registration network with the best result obtained on the validation set is tested on the test set to obtain the test results. If the test results meet the preset requirements, the trained rib elastic registration network is obtained.

[0067] See Figure 4 This application provides a structure for a lung image prediction device according to one embodiment. For ease of explanation, only the parts relevant to this embodiment are shown. The device may be a computer terminal, or a software unit or module configured on that computer terminal. Figure 4 As shown, the device includes: an image sequence acquisition unit 41, a deformation field acquisition unit 42, and an image prediction unit 43, wherein:

[0068] Image sequence acquisition unit 41 is used to acquire rib images of the target human body under different phases to obtain a rib image sequence;

[0069] The deformation field acquisition unit 42 is used to acquire the rib deformation field corresponding to the rib image at any phase based on the rib image at any phase contained in the rib image sequence and the rib elastic registration network; and

[0070] The image prediction unit 43 is used to predict the lung image of the target human body at a target phase based on the lung image of the target human body at any phase and the obtained rib deformation field, wherein the difference between any phase and the target phase satisfies a preset condition.

[0071] Optionally, the deformation field acquisition unit 42 may include:

[0072] The first transformation unit is used to perform an affine transformation on the rib image at any phase to obtain the transformed rib image; and

[0073] The deformation field acquisition unit is used to obtain the rib deformation field corresponding to the rib image at any phase based on the transformed rib image and the rib elastic registration network.

[0074] Optionally, in one embodiment of this application, the deformation field obtaining unit may include:

[0075] The first deformation field acquisition sub-unit is used to input the transformed rib image into the rib elastic registration network to obtain the rib deformation field corresponding to any phase of the rib image output by the rib elastic registration network.

[0076] Optionally, in another embodiment of this application, the deformation field obtaining unit may include:

[0077] The set acquisition unit is used to acquire the set of deformation fields corresponding to the rib elastic registration network. This set of deformation fields includes the rib deformation fields corresponding to the rib images at each phase of the rib image sequence; and

[0078] The second deformation field acquisition sub-unit is used to search in the deformation field set to obtain the rib deformation field corresponding to the rib image under any phase.

[0079] Furthermore, the image prediction unit 43 may include:

[0080] The second transformation unit is used to perform an affine transformation on the lung image at any phase to obtain the transformed lung image.

[0081] The prediction subunit is used to predict the lung image of the target human body at the target phase based on the transformed lung image and the rib deformation field.

[0082] Furthermore, in one embodiment of this application, the lung image prediction device may further include:

[0083] A sample set acquisition unit is used to acquire a training sample set, wherein the training sample set contains several pairs of rib images, wherein each pair of rib images contains a rib image in a first phase and a rib image in a second phase, and the difference between the first phase and the second phase satisfies the preset condition; and

[0084] The network training unit is used to train the rib elastic registration network by taking each sample rib image pair as input and adjusting the parameters of the rib elastic registration network until the trained rib elastic registration network and the deformation field set corresponding to the trained rib elastic registration network are obtained.

[0085] Optionally, the network training unit includes:

[0086] The image pair transformation unit is used to perform an affine transformation on the sample rib images of the first phase contained in each sample rib image pair to obtain the transformed sample rib image pairs; and

[0087] The training subunit is used to train the rib elastic registration network by taking the transformed rib image pairs as input and adjusting the parameters of the rib elastic registration network until the trained rib elastic registration network and the corresponding deformation field set are obtained.

[0088] Furthermore, the loss function used when training the rib elastic registration network is:

[0089]

[0090] Where f represents the sample rib image in the training sample set as a fixed image in the second phase, m represents the sample rib image in the training sample set as a transformed floating image in the first phase, φ represents the rib deformation field output by the rib elastic registration network corresponding to the sample rib image in the first phase, and L smooth (φ) represents the smoothing loss of the rib deformation field φ. Let f represent the similarity loss between image f and image m.

[0091] The specific process by which each of the above units achieves its respective function can be found in the relevant content of the above method embodiments, and will not be repeated here.

[0092] See Figure 5 The present application provides a schematic diagram of the hardware structure of an electronic device according to an embodiment.

[0093] For example, the electronic device can be any of various types of computer system devices that are non-movable or movable or portable and perform wireless or wired communication. Specifically, the electronic device can be a desktop computer, server, mobile phone or smartphone (e.g., iPhone™-based, Android™-based phone), portable gaming device (e.g., Nintendo DS™, PlayStation Portable™, Gameboy Advance™, iPhone™), laptop computer, PDA, portable internet device, portable medical device, smart camera, music player and data storage device, other handheld devices and such as watches, headphones, pendants, etc. The electronic device can also be other wearable devices (e.g., such as electronic glasses, electronic clothing, electronic bracelets, electronic necklaces and other head-mounted devices (HMDs)).

[0094] like Figure 5 As shown, the electronic device 50 may include a control circuit, which may include a storage and processing circuit 500. The storage and processing circuit 500 may include a memory, such as a hard disk drive, a non-volatile memory (e.g., flash memory or other electronically programmable erasable memory used to form a solid-state drive), a volatile memory (e.g., static or dynamic random access memory), etc., and this embodiment is not limited thereto. The processing circuit in the storage and processing circuit 500 can be used to control the operation of the electronic device 50. This processing circuit may be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application-specific integrated circuits (ASICs), display driver integrated circuits, etc.

[0095] The storage and processing circuitry 500 can be used to run software in the electronic device 50, such as internet browsing applications, Voice over Internet Protocol (VoIP) telephone calling applications, email applications, media playback applications, operating system functions, etc. This software can be used to perform various control operations, such as image acquisition based on a camera, ambient light measurement based on an ambient light sensor, proximity sensor measurement based on a proximity sensor, information display functions based on status indicators such as LED status lights, touch event detection based on a touch sensor, functions associated with displaying information on multiple (e.g., layered) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals, control operations associated with collecting and processing button press event data, and other functions in the electronic device 50, etc., which are not limited in the embodiments of this application.

[0096] Furthermore, the memory stores executable program code, and the processor coupled to the memory calls the executable program code stored in the memory to execute the lung image prediction method as described in the foregoing embodiments.

[0097] The executable program code includes the following as described above. Figure 4 The lung image prediction device described in the illustrated embodiment includes, for example, the image sequence acquisition unit 41, the deformation field acquisition unit 42, and the image prediction unit 43. The specific processes by which these units perform their respective functions are described in the relevant descriptions of the device embodiments above, and will not be repeated here.

[0098] Furthermore, embodiments of this application also provide a non-transitory computer-readable storage medium that can be configured in the server in the above embodiments. The non-transitory computer-readable storage medium stores a computer program that, when executed by a processor, implements the lung image prediction method described in the foregoing embodiments.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] Those skilled in the art will recognize that the modules / units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0101] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0105] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for predicting lung images, characterized in that, The method includes: Acquire rib images of the target human body at different phases to obtain a rib image sequence; Based on the rib image at any phase in the rib image sequence and the rib elastic registration network, the rib deformation field corresponding to the rib image at any phase is obtained. Specifically, this includes: performing an affine transformation on the rib image at any phase to obtain a transformed rib image; and obtaining the rib deformation field corresponding to the rib image at any phase based on the transformed rib image and the rib elastic registration network. Based on the acquired lung image of the target human body at any phase and the obtained rib deformation field, predict the lung image of the target human body at a target phase, specifically including: performing an affine transformation on the lung image at any phase to obtain a transformed lung image; predicting the lung image of the target human body at the target phase based on the transformed lung image and the rib deformation field, wherein the difference between the any phase and the target phase satisfies a preset condition.

2. The method as described in claim 1, characterized in that, Based on the transformed rib image and the rib elastic registration network, the rib deformation field corresponding to the rib image in any phase is obtained, including: The transformed rib image is input into the rib elastic registration network to obtain the rib deformation field corresponding to any phase of the rib image output by the rib elastic registration network.

3. The method as described in claim 1, characterized in that, Based on the transformed rib image and the rib elastic registration network, the rib deformation field corresponding to the rib image in any phase is obtained, including: Obtain the set of deformation fields corresponding to the elastic registration network of the rib, wherein the set of deformation fields includes the rib deformation fields corresponding to the rib images under each phase of the rib image sequence; Search within the set of deformation fields to obtain the rib deformation field corresponding to the rib image under any given phase.

4. The method as described in claim 1, characterized in that, The training process of the rib elastic registration network is as follows: Obtain a training sample set, which contains several pairs of rib images. Each pair of rib images contains a rib image in a first phase and a rib image in a second phase. The difference between the first phase and the second phase satisfies the preset condition. Each sample rib image pair is used as input to the rib elastic registration network. The rib elastic registration network is trained and its parameters are adjusted until the trained rib elastic registration network and the corresponding deformation field set are obtained.

5. The method as described in claim 4, characterized in that, The rib elastic registration network is trained based on the training sample set to obtain the trained rib elastic registration network, including: For each sample rib image pair, perform an affine transformation on the sample rib image containing the first phase to obtain the transformed sample rib image pairs. The transformed rib images are used as input to the rib elastic registration network. The rib elastic registration network is trained by adjusting its parameters until the trained rib elastic registration network and the corresponding deformation field set are obtained.

6. The method as described in claim 5, characterized in that, The loss function used when training the rib elastic registration network is: in, f This represents the sample rib image in the training sample set as the second phase of a fixed image. m This represents the rib image of the sample in the training sample set after transformation to the first phase of the floating image. φ This represents the rib deformation field output by the rib elastic registration network, corresponding to the sample rib image under the first phase. Represents the rib deformation field φ Smoothing loss, express f and image m Similarity loss between φ.

7. A lung image prediction device, characterized in that, The device includes: The image sequence acquisition unit is used to acquire rib images of the target human body under different phases to obtain a rib image sequence. The deformation field acquisition unit is used to acquire the rib deformation field corresponding to the rib image at any phase in the rib image sequence and the rib elastic registration network. Specifically, this includes: performing an affine transformation on the rib image at any phase to obtain a transformed rib image; and acquiring the rib deformation field corresponding to the rib image at any phase based on the transformed rib image and the rib elastic registration network. An image prediction unit is configured to predict the lung image of the target human body at a target phase based on the acquired lung image of the target human body at any phase and the obtained rib deformation field. Specifically, the unit includes: performing an affine transformation on the lung image at any phase to obtain a transformed lung image; and predicting the lung image of the target human body at the target phase based on the transformed lung image and the rib deformation field, wherein the difference between the any phase and the target phase satisfies a preset condition.

8. An electronic device, characterized in that, include: Memory and processor; The memory stores executable program code; The processor coupled to the memory invokes the executable program code stored in the memory to perform the steps of the method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Prediction method and device for liver respiratory movement model and storage medium

    CN111161333A

  • Method and system for generating a phase contrast image

    US20190090832A1

  • Methods, systems and apparatus for processing medical chest images

    US20210312629A1