Data amplification method, signal imaging method, device, equipment and storage medium

Abstract

The application discloses a data augmentation method, a signal imaging method, a device, equipment and a storage medium. The method comprises the following steps: acquiring a first lung medical image sequence at an end-expiratory moment and a second lung medical image sequence at an end-inspiratory moment; establishing an initial lung contour based on the first lung medical image sequence and establishing a final lung contour based on the second lung medical image sequence; generating lung simulation contours at a plurality of intermediate moments according to the initial lung contour and the final lung contour; adding a simulation signal acquisition device on a chest surface corresponding to each of the lung simulation contours, and receiving a sensing signal returned by the simulation signal acquisition device; and the sensing signal of the lung simulation contour and a sensing image corresponding to the lung simulation contour constitute a set of labeled training data. The technical scheme provided by the application can improve the efficiency of data augmentation based on sensing signal imaging to a certain extent.

Description

Technical Field

[0001] This invention relates to the field of signal imaging technology, specifically to a data amplification method, a signal imaging method, an apparatus, a device, and a storage medium. Background Technology

[0002] In some cases, long-term continuous monitoring of a patient's lung activity is necessary. Furthermore, the diagnosis of various lung diseases also requires lung imaging. Currently, computed tomography (CT) is commonly used in medicine for lung imaging. However, before CT imaging, patients need to be transferred from their ward to the CT scanner and exposed to radiation, making CT unsuitable for long-term monitoring. Electrical impedance tomography (EIT) is radiation-free, non-invasive, and has high temporal resolution, enabling real-time bedside imaging of the patient's lungs. EIT works by injecting current into the skin surface, measuring the voltage between corresponding electrodes in a predetermined sequence, and then using an imaging algorithm to create an image. However, due to the highly nonlinear and pathological nature of EIT imaging, traditional EIT imaging algorithms reconstruct EIT images with low spatial resolution. To improve the spatial resolution of EIT images, many data-driven algorithms have been proposed in recent years; however, to achieve good imaging results, these algorithms require a large amount of data for training. Because real-world EIT data is relatively scarce, most literature uses simulated data to construct datasets. The simulated data is mostly two-dimensional thoracic cavity cross-sections, derived from two-dimensional CT image data or simulated two-dimensional elliptical structures, which differs significantly from actual EIT measurements. Existing three-dimensional EIT simulation methods cannot obtain EIT data of the lungs in various states during respiration from four-dimensional CT image data in one go; they can only simulate the lung state at a certain moment during respiration each time, thus requiring a significant amount of time for model processing and resulting in low simulation efficiency. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a data amplification method, a signal imaging method, an apparatus, a device, and a storage medium, which can improve the efficiency of data amplification based on sensor signal imaging to a certain extent.

[0004] This invention provides a data augmentation method, comprising: acquiring a first lung medical image sequence at the end of expiration and a second lung medical image sequence at the end of inspiration; establishing an initial lung contour based on the first lung medical image sequence and a final lung contour based on the second lung medical image sequence; generating a plurality of intermediate lung simulation contours based on the initial lung contour and the final lung contour; the intermediate moments representing the time between the end of expiration and the end of inspiration; adding simulation signal acquisition devices to the thoracic surfaces corresponding to the plurality of lung simulation contours respectively, and receiving sensor signals returned by the simulation signal acquisition devices; the sensor signals and the sensor images corresponding to the lung simulation contours constitute a set of labeled training data; the simulation signal acquisition device is one of an ultrasound signal acquisition device and a voltage signal acquisition device.

[0005] In one embodiment, generating a plurality of intermediate time-period lung simulation profiles based on the initial lung profile and the final lung profile includes: calculating the intrapleural pressure difference between the end-expiratory time and the end-inspiratory time based on the volume corresponding to the initial lung profile and the final lung profile; generating the intrapleural pressure difference between each intermediate time and the end-expiratory time based on the intrapleural pressure difference; and generating a lung simulation profile characterizing each intermediate time based on the intrapleural pressure difference between each intermediate time and the end-expiratory time.

[0006] In one embodiment, the simulation signal acquisition device is a voltage signal acquisition device. Simulation signal acquisition devices are added to the thoracic surfaces corresponding to several lung simulation contours, and the sensor signals returned by the simulation signal acquisition devices are received. This includes: adding electrode arrays to the thoracic surfaces corresponding to several lung simulation contours; measuring the surface voltage of the thoracic surfaces corresponding to the lung simulation contours; and the surface voltage of the thoracic surfaces corresponding to the lung simulation contours and the conductivity distribution image corresponding to the lung simulation contours constitute a set of labeled training data.

[0007] In one embodiment, measuring the surface voltage of the thoracic cavity surface corresponding to the simulated lung contour includes: sequentially injecting excitation current signals into electrode pairs consisting of two adjacent electrodes in the electrode array; and acquiring the voltage values ​​of each electrode on the electrode array for each electrode pair injected with the excitation current signal to obtain the surface voltage of the simulated lung contour.

[0008] In one embodiment, the data augmentation method further includes: constructing training samples; the training samples include at least a sensor signal of the thoracic surface corresponding to the lung simulation contour and a sensor image corresponding to the lung simulation contour; inputting the sensor signal of the thoracic surface corresponding to the lung simulation contour into an image reconstruction model to obtain a target sensor image corresponding to the lung simulation contour; and updating the image reconstruction model based on the difference between the target sensor image corresponding to the lung simulation contour and the sensor image corresponding to the lung simulation contour.

[0009] In one embodiment, inputting the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour into an image reconstruction model to obtain a target sensing image corresponding to the simulated lung contour includes: using the ratio of the difference between the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour and the initial sensing signal at the end of expiration corresponding to the simulated lung contour to the initial sensing signal as first sensing data; normalizing the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour to obtain second sensing data; fusing the first sensing data and the second sensing data to obtain target sensing data; and inputting the target sensing data into an image reconstruction model to obtain the target sensing image corresponding to the simulated lung contour.

[0010] In one embodiment, the data augmentation method further includes: adding Gaussian random noise within a certain threshold range to the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour to obtain a target sensing signal of the simulated lung contour; correspondingly, training the image reconstruction model based at least on the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour and the sensing image corresponding to the simulated lung contour, the target sensing signal of the simulated lung contour and the sensing image corresponding to the simulated lung contour.

[0011] Another aspect of the present invention provides a signal imaging method, the method comprising: acquiring chest cavity surface sensing data acquired by a signal acquisition device for the lungs added to the surface of the chest cavity; using the difference between the chest cavity surface sensing data and the initial sensing data at the end of expiration within a preset expiratory cycle, and the ratio of the initial sensing data, as first sensing data; normalizing the chest cavity surface sensing data to obtain second sensing data; fusing the first sensing data and the second sensing data to obtain target sensing data; and inputting the target sensing data into an image reconstruction model as described in the above embodiments to obtain a target sensing image.

[0012] In one embodiment, the chest cavity surface sensing data is normalized to obtain second sensing data, including: taking the absolute value of the difference between the initial sensing data at the end of expiration and the final sensing data at the end of inspiration within the preset expiratory cycle as the sensing signal difference; and taking the ratio of the difference between the chest cavity surface sensing data and the initial sensing data at the end of expiration within the preset expiratory cycle to the sensing signal difference as the second sensing data.

[0013] In another aspect, the present invention provides a data augmentation device, comprising: an image data acquisition unit for acquiring a first lung medical image sequence at the end of expiration and a second lung medical image sequence at the end of inspiration; a lung contour establishment unit for establishing an initial lung contour based on the first lung medical image sequence and a final lung contour based on the second lung medical image sequence; a lung contour simulation unit for generating a plurality of intermediate lung simulation contours based on the initial lung contour and the final lung contour; the intermediate moments representing the time between the end of expiration and the end of inspiration; and a sensor signal acquisition unit for adding simulation signal acquisition devices to the thoracic surfaces corresponding to the plurality of lung simulation contours and receiving sensor signals returned by the simulation signal acquisition devices; the sensor signals and the sensor images corresponding to the lung simulation contours constitute a set of labeled training data; the simulation signal acquisition device is one of an ultrasound signal acquisition device and a voltage signal acquisition device.

[0014] Another aspect of the present invention provides a signal imaging device, the signal imaging device comprising: a signal acquisition unit for acquiring thoracic cavity surface sensing data acquired by a signal acquisition device for the lungs added to the surface of the thoracic cavity; a first sensing data construction unit for using the ratio of the difference between the thoracic cavity surface sensing data and the initial sensing data at the end of expiration within a preset expiratory cycle to the initial sensing data as first sensing data; a second sensing data construction unit for normalizing the thoracic cavity surface sensing data to obtain second sensing data; a data fusion unit for fusing the first sensing data and the second sensing data to obtain target sensing data; and a signal imaging unit for inputting the target sensing data into a trained image reconstruction model as described in the above embodiments to obtain a target sensing image.

[0015] In another aspect, the present invention provides an electronic device comprising a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the above-described data amplification method and / or signal imaging method.

[0016] In another aspect, the present invention provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the above-described data amplification method and / or signal imaging method.

[0017] By acquiring CT data from lung CT images at the end of expiration and end of inspiration, and constructing corresponding lung contour data based on this data, digital twin technology is used to predict the dynamic changes of the lung during these two phases. This yields simulated lung contour data at several intermediate moments. Sensor signal measurements are then performed on these simulated lung contour data to obtain corresponding sensor data. This approach, combining the biomechanical model and electromagnetic properties of the lung, allows for the creation of a digital twin model of the lung during the respiratory cycle. This improves the efficiency of lung simulation sensor data amplification, providing a large training dataset for signal-based image reconstruction models. Attached Figure Description

[0018] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:

[0019] Figure 1 A schematic diagram of the steps of a data amplification method according to one embodiment of the present invention is shown;

[0020] Figure 2(a) shows a two-dimensional schematic diagram of a lung movement model in one embodiment of the present invention;

[0021] Figure 2(b) shows a three-dimensional schematic diagram of a lung motion model according to one embodiment of the present invention;

[0022] Figure 3 This invention illustrates a simulated dynamic process of the lungs during inhalation in one embodiment of the invention;

[0023] Figure 4(a) shows the lung potential distribution at the end-expiratory state in one embodiment of the present invention;

[0024] Figure 4(b) shows the lung potential distribution at the end of inspiration in one embodiment of the present invention;

[0025] Figure 5 A schematic diagram of the structure of an image reconstruction model according to one embodiment of the present invention is shown;

[0026] Figure 6 A schematic diagram of the steps of a signal imaging method according to one embodiment of the present invention is shown;

[0027] Figure 7A schematic diagram of a data amplification device according to one embodiment of the present invention is shown;

[0028] Figure 8 A schematic diagram of a signal imaging device according to one embodiment of the present invention is shown;

[0029] Figure 9 A schematic diagram of the structure of an electronic device according to one embodiment of the present invention is shown. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] In the intensive care unit, long-term continuous monitoring of a patient's lung activity is essential. Furthermore, lung examination is necessary for the diagnosis of various lung diseases. Currently, computed tomography (CT) is commonly used in medicine for lung imaging. However, before CT imaging, patients need to be transferred from their ward to the CT scanner and exposed to radiation, making CT unsuitable for long-term monitoring. As an emerging imaging technology, electrical impedance tomography (EIT) is increasingly used in lung monitoring due to its non-radioactive, non-invasive nature and high temporal resolution, enabling real-time bedside imaging of the patient's lungs. EIT works by injecting an electric current into the skin surface, measuring the voltage between corresponding electrodes in a predetermined sequence, and then using an imaging algorithm to create an image. However, because EIT imaging is highly nonlinear and pathological, traditional EIT imaging algorithms reconstruct EIT images with low spatial resolution. To improve the spatial resolution of EIT images, many data-driven algorithms have been proposed in recent years. However, to achieve better imaging results, data-driven algorithms require a large amount of data for training. Because actual EIT data is scarce, most literature uses simulation data to construct datasets. Simulated data is mostly two-dimensional chest cavity cross-sections, derived from two-dimensional CT image data or simulated two-dimensional elliptical structures, which differs significantly from actual EIT measurements. Existing three-dimensional EIT simulation methods cannot obtain EIT data for all lung states during respiration from four-dimensional CT image data in one go; they can only simulate the lung state at a specific moment during respiration, requiring significant time for model processing and resulting in low simulation efficiency. Therefore, a method is needed to improve the efficiency of EIT simulation data construction. This method uses voltage data obtained from electric field simulation and chest contour data obtained through digital twins to form multiple training datasets, providing training data for image reconstruction models that generate lung images based on EIT voltage data.

[0032] Please see Figure 1 One embodiment of this application provides a data augmentation method, which may include the following steps.

[0033] S110: Acquire the first lung medical image sequence at the end of expiration and the second lung medical image sequence at the end of inspiration.

[0034] In this embodiment, by acquiring a first lung medical image sequence at the end of expiration and a second lung medical image sequence at the end of inspiration, the contour of the lungs can be constrained. That is, the lung volume during respiration is greater than or equal to the lung volume at the end of expiration, and less than or equal to the lung volume at the end of inspiration. Therefore, the first lung medical image sequence corresponding to the end of expiration and the second lung medical image sequence corresponding to the end of inspiration can be used as boundary conditions to simulate the lung movement process.

[0035] S120: Establish an initial lung contour based on the first lung medical image sequence and establish a final lung contour based on the second lung medical image sequence.

[0036] In this embodiment, the first lung medical image sequence can be a series of first lung medical images, including multiple lung cross-sections, obtained after a single CT scan of the patient at the end of expiration. Specifically, for example, if a 128-slice CT scanner is used to scan the patient, the first lung medical image sequence can be a series of first lung medical images, including 128 lung cross-sections, obtained after a single CT scan of the patient. A three-dimensional model of the lungs can then be constructed based on these 128 first lung medical images, obtaining the initial lung contour data corresponding to the patient's end of expiration. The second lung medical image sequence can be a series of second lung medical images, including multiple lung cross-sections, obtained after a single CT scan of the patient at the end of inspiration. The first lung medical image sequence at the end of expiration and the second lung medical image sequence at the end of inspiration are obtained from multiple CT scans of the patient within one expiratory cycle.

[0037] S130: Generate a number of intermediate lung simulation contours based on the initial lung contour and the final lung contour; the intermediate moments are used to characterize the time between the end of expiration and the end of inspiration.

[0038] Referring to Figures 2(a) and 2(b), in this embodiment, lung respiratory motion is modeled as a contact problem. Taking the inhalation process as an example, the lung contour in the CT image at the end of inspiration is used as the contact condition constraining lung expansion (as shown in Figures 2(a) and 2(b)). Intrapleural pressure acts on the lung surface, causing the lung to expand until it fills the lung contour at the end of inspiration. During this inspiration, the lung root is assumed to be stationary, and the lung is then considered as a linear, homogeneous, and isotropic elastic tissue, with the biomechanical parameters Young's modulus E and Poisson's ratio v set to 1 kPa and 0.3, respectively. Assuming the lung root is fixed, the surface nodes in this region follow fixed boundary conditions, while other nodes in other regions should follow stress boundary conditions.

[0039] u(x) = 0, x ∈ Γ r00t ∪Γlimniting

[0040] δ(x)n(x)=(P intrapl +P contact )n(x), x∈Γ deformed

[0041] Where u(x) is the displacement vector at point x, and δ(x) and n(x) are the Cauchy stress and external normal vector at point x, respectively. intrapl This represents the intrapleural pressure applied to the surface of the lungs. P contact This represents the contact pressure. Since lung motion is modeled as a frictionless contact problem, this means:

[0042] g≥0

[0043] P contact ≤0

[0044] g·P contact =0

[0045] Where g represents the gap between the lung and the end-inspiratory contour. When the gap is positive, i.e., a gap exists, the contact pressure P... contact The pressure must be zero. Otherwise, the contact pressure will not be zero to limit further lung expansion. The intrapleural pressure difference ΔP between the end of expiration and the end of inspiration for each patient can be:

[0046]

[0047] In the formula, V l It is the volume of the lung outline at the end of inspiration, V i This is the volume of the initial lung contour. To ensure that the lung contour at the end of inspiration during simulation closely approximates the actual lung contour at the end of inspiration (i.e., achieving a volume ratio of at least 0.99), its value is appropriately adjusted after estimating the intrapleural pressure difference.

[0048] Please see Figure 3 The lung motion model established in this way can simulate the complete dynamic changes of the lungs during inspiration based on only CT images at the end of expiration and the end of inspiration.

[0049] S140: Add simulation signal acquisition devices to the thoracic surfaces corresponding to several simulated lung contours, and receive the sensing signals returned by the simulation signal acquisition devices; the sensing signals and the sensing images corresponding to the simulated lung contours constitute a set of labeled training data; the simulation signal acquisition device is one of an ultrasound signal acquisition device and a voltage signal acquisition device.

[0050] In this embodiment, a simulation signal acquisition device is used to acquire sensor signals from the thoracic surface corresponding to the simulated lung contours, thereby obtaining a set of sensor signals corresponding to each simulated lung contour. This set of sensor signals can then be used as input to the neural network model during training, while sensor image data generated based on the dynamic changes of the lungs during inhalation, and constructed from the simulated lung contour data, serves as the label. To achieve better imaging results, the data-driven algorithm requires a large amount of data for training. This method allows for the creation of multiple training datasets, amplifying the signal-based imaging data and providing a large number of labeled training datasets for the data-driven algorithm. The simulation signal acquisition device can simulate the acquisition of ultrasound signals, i.e., attaching a simulated ultrasound probe to the thoracic surface corresponding to the simulated lung contours to measure ultrasound signals, and then using multiple sets of ultrasound signals and the corresponding ultrasound images generated based on the simulated lung contours as the model input and labels for model training. Alternatively, the simulation signal acquisition device can be a simulated electrode array, i.e., attaching an electrode array to the thoracic surface corresponding to the simulated lung contours to measure voltage signals, and then using multiple sets of voltage signals and the conductivity distribution image generated based on the simulated lung contours as the model input and labels for model training.

[0051] In one embodiment, generating a plurality of intermediate time-period lung simulation profiles based on the initial lung profile and the final lung profile may include: calculating the intrapleural pressure difference between the end-expiratory time and the end-inspiratory time based on the volume corresponding to the initial lung profile and the final lung profile; generating the intrapleural pressure difference between each intermediate time and the end-expiratory time based on the intrapleural pressure difference; and generating a lung simulation profile characterizing each intermediate time based on the intrapleural pressure difference between each intermediate time and the end-expiratory time.

[0052] In this embodiment, the intrapleural pressure difference at the end of expiration and the end of inspiration is calculated, and then the intrapleural pressure difference is divided into several equal parts. During lung movement, this can be considered a uniformly changing process; therefore, the pressure change can also be uniform. Specifically, for example, if the intrapleural pressure difference is 100 Pa, dividing it into 20 equal parts will include the intrapleural pressure at 19 intermediate moments, with each moment's pressure being 5 Pa relative to the previous moment's pressure. Finally, the intrapleural pressure at each intermediate moment is calculated based on the pressure at the end of inspiration or expiration, and a simulated lung profile corresponding to each intermediate moment is established accordingly. This method allows for the creation of a lung motion model throughout the entire expiration process, even with only lung medical image sequences at the end of inspiration and expiration. This yields simulated lung contour data at 21 time points. Electrode simulation measurements are then performed on these 21 contour data points to obtain the corresponding voltage measurements. These voltage measurements are then used as input to an image reconstruction model, while the corresponding conductivity distribution images serve as labels for training the model.

[0053] In one embodiment, the simulation signal acquisition device is a voltage signal acquisition device. The simulation signal acquisition device is added to the thoracic cavity surface corresponding to several lung simulation contours, and the sensor signals returned by the simulation signal acquisition device are received. This may include: adding electrode arrays to the thoracic cavity surface corresponding to several lung simulation contours; measuring the surface voltage of the thoracic cavity surface corresponding to the lung simulation contours; and the surface voltage of the thoracic cavity surface corresponding to the lung simulation contours and the conductivity distribution image corresponding to the lung simulation contours constituting a set of labeled training data.

[0054] In this embodiment, an electrode array is used to measure the voltage of each simulated lung contour, thereby generating multiple sets of surface voltage data for each simulated lung contour. Then, one set of surface voltage data can be used as input to the neural network model during training, and conductivity distribution image data generated based on the dynamic changes of the lungs during inspiration is used as a label. To achieve better imaging results, the data-driven algorithm requires a large amount of data for training. This method can construct multiple training datasets, thereby amplifying the EIT imaging data and providing a large number of labeled training datasets for the data-driven algorithm.

[0055] In one embodiment, measuring the surface voltage of the thoracic cavity surface corresponding to the simulated lung contour may include: sequentially injecting excitation current signals into electrode pairs consisting of two adjacent electrodes in the electrode array; and acquiring the voltage values ​​of each electrode on the electrode array for each electrode pair injected with the excitation current signal to obtain the surface voltage of the simulated lung contour.

[0056] Please see Figure 4a and Figure 4b In this embodiment, during the measurement of the surface voltage of the simulated lung contour, the electrode pair consisting of two adjacent electrodes in the electrode array can be used as the excitation current signal. Then, the voltage of the remaining adjacent electrodes can be measured sequentially to obtain the voltage value of the electrode pair as the excitation current. Then, the electrode pair consisting of adjacent electrodes in the electrode array can be used as the excitation current signal again to measure the voltage of the other electrodes. Specifically, for example, an electrode array includes 16 electrodes, numbered 1-16. Electrodes numbered 1 and 2 can be used as the excitation current first, and then voltage measurements can be performed sequentially on electrodes 3 and 4, 4 and 5, ..., 15 and 16. Then, electrodes numbered 2 and 3 can be used as the excitation current, and voltage measurements can be performed sequentially on electrodes 4 and 5, 5 and 6, ..., 16 and 1, and so on. A single voltage measurement yields a total of 208 voltage values, which are used as the surface voltage of the simulated lung contour. When the first electrode pair is excited, the initial potential distribution of the lung during respiration (as shown in Figure 4(a)) and the final potential distribution (as shown in Figure 4(b)) are obtained. Based on this, the dynamic potential distribution of the model during lung movement can be obtained. It should be noted that in this process, the EIT excitation current frequency is 100kHz, and the conductivity and relative permittivity of the lung tissue at the end of expiration are set to 0.26 S / m and 8531.4, respectively, and the conductivity and relative permittivity of the lung tissue at the end of inspiration are set to 0.10 S / m and 4272.5, respectively. The electrical parameters of the lung during inspiration change linearly with volume change:

[0057]

[0058] Among them, H T These are the electrical parameters (conductivity or dielectric constant) at time T. H i and H l These are the electrical parameters at the end of expiration and the end of inspiration, respectively. V T It is the volume of the lung at time T.

[0059] In one embodiment, the data augmentation method may further include: constructing training samples; the training samples include at least a sensor signal of the thoracic surface corresponding to the lung simulation contour and a sensor image corresponding to the lung simulation contour; inputting the sensor signal of the thoracic surface corresponding to the lung simulation contour into an image reconstruction model to obtain a target sensor image corresponding to the lung simulation contour; and updating the image reconstruction model based on the difference between the target sensor image corresponding to the lung simulation contour and the sensor image corresponding to the lung simulation contour.

[0060] In this embodiment, a twin digital model of the lungs for each patient is created by selecting CT images at the end of inspiration and end of expiration from multiple patients' 4D-CT images. Specifically, for example, in the simulation of the lung motion model, the inspiratory process is divided into 21 time points. The first time point T0 represents the end-expiratory state, and the last time point T... 20 This represents the end-inspiratory state. Based on this, 21 training samples can be obtained from medical image sequences at the end of expiration and end of inspiration for a single patient. Therefore, a sufficient training dataset and corresponding labels can be obtained using a small number of patients' lung CT image data. Specifically, for example, in the electrode array-based electrical impedance tomography (EIT) algorithm, a sufficient training dataset for the EIT imaging model (image reconstruction model) can be obtained using a small number of patients' lung CT image data. In this embodiment, U-Net can produce accurate image segmentation results using a small number of training images and is often used in tasks such as biomedical image segmentation. Therefore, image reconstruction of EIT data can be achieved based on the U-Net network. Therefore, the voltage data of the simulated lung contour can be used as the input to the U-Net network model, and the conductivity distribution image corresponding to the simulated lung contour can be used as the label. The mean squared error loss function is used during the training process.

[0061]

[0062] Where N is the number of training samples. F(·) represents the nonlinear function of the neural network. For EIT data, and This is the corresponding conductivity distribution image.

[0063] In one embodiment, inputting the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour into an image reconstruction model to obtain a target sensing image corresponding to the simulated lung contour includes: using the ratio of the difference between the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour and the initial sensing signal at the end of expiration corresponding to the simulated lung contour to the initial sensing signal as first sensing data; normalizing the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour to obtain second sensing data; fusing the first sensing data and the second sensing data to obtain target sensing data; and inputting the target sensing data into an image reconstruction model to obtain the target sensing image corresponding to the simulated lung contour.

[0064] In this embodiment, by processing the obtained sensor signals in a certain way, the instability of the image reconstruction model during use due to individual differences in the acquired sensor signals can be avoided. Specifically, for example, when multiple electrodes are attached to the surface of the patient's chest cavity using an electrode array, surface voltage data can be obtained. Then, the ratio of the difference between the surface voltage data of the lung simulation contour and the initial voltage data at the end of expiration corresponding to the lung simulation contour to the initial voltage is used as the first voltage data; the surface voltage data of the lung simulation contour is normalized to obtain the second voltage data; the first voltage data and the second voltage data are fused to obtain the target voltage data; the target voltage data is input into the image reconstruction model to obtain the target conductivity image corresponding to the lung simulation contour.

[0065] In this embodiment, the method for calculating the first voltage data can be:

[0066]

[0067] In the formula, This is the first dimension of data input to the neural network. U0 is the voltage measurement vector at the initial moment, U... t Let be the voltage measurement vector at time t. The second voltage data can be calculated using the following method:

[0068]

[0069] In the formula, This is the second dimension of the data input to the neural network. U0 is the voltage measurement vector at the initial moment, U... t Let U be the voltage measurement vector at time t. T This is the voltage measurement vector at the final moment of the intake process. Because this calculation method is susceptible to noise interference and extreme values, its value is truncated to the [-3, 3] interval at input. Please refer to... Figure 5The two types of EIT voltage data are input into the IR-Net and converted into 32×32 image format through fully connected layers. These images are then stacked to a 2×32×32 dimension and passed through a deconvolution layer to become 1×64×64 image data. Finally, a U-Net architecture containing three downsampling layers and one upsampling layer outputs the predicted image. In this invention, the conductivity variation is normalized to the range [0,1], therefore, Sigmoid is chosen as the activation function of the final convolutional layer to output the target conductivity image.

[0070] In one embodiment, the data augmentation method may further include: adding Gaussian random noise within a certain threshold range to the surface voltage of the sensing signal of the thoracic surface corresponding to the lung simulation contour to obtain a target surface voltage sensing signal of the lung simulation contour; correspondingly, training the image reconstruction model based at least on the surface voltage of the sensing signal of the thoracic surface corresponding to the lung simulation contour and the conductivity distribution sensing image corresponding to the lung simulation contour, the target surface voltage sensing signal of the lung simulation contour and the conductivity distribution sensing image corresponding to the lung simulation contour.

[0071] In this embodiment, adding Gaussian random noise within a certain threshold range to the returned sensor signal improves the generalization ability of the neural network. Furthermore, when using an electrode array to measure a patient's lungs, the electrode array can be placed at two different heights within the chest cavity for voltage measurement, since in the real world, the electrode array is not fixed in a specific location. Specifically, for example, if the training dataset originally includes lung medical image data from 12 patients at the end of expiration and end of inspiration, and then in the simulation of the lung motion model, the inspiratory process is divided into 21 time points, then the electrode array is placed at different locations on the simulated lung contour at each time point, and 5% Gaussian random noise is added to the voltage data obtained from the electrode arrays placed at different locations, thus obtaining a labeled training dataset of 4368 pairs.

[0072] Please see Figure 6 One embodiment of this application also provides a signal imaging method, which may include the following steps.

[0073] S210: Acquire thoracic surface sensing data obtained by a lung-targeting signal acquisition device added to the thoracic surface.

[0074] S220: The ratio of the difference between the chest cavity surface sensing data and the initial sensing data at the end of the preset expiratory cycle to the initial sensing data is used as the first sensing data.

[0075] S230: Normalize the chest cavity surface sensing data to obtain second sensing data.

[0076] S240: The first sensing data and the second sensing data are fused to obtain the target sensing data.

[0077] S250: Input the target sensing data into the trained image reconstruction model as described in the above embodiment to obtain the target sensing image.

[0078] In this embodiment, the specific functions and effects of the signal imaging method are explained in comparison with other embodiments in this specification, and will not be repeated here. It should be noted that the image reconstruction model described in the embodiments of this specification can be a convolutional neural network model such as Res-Net, or a neural network model such as GAN, in addition to the U-Net network model. This specification does not limit the embodiments in this way.

[0079] In one embodiment, normalizing the chest cavity surface sensing data to obtain second sensing data may include: using the absolute value of the difference between the initial voltage sensing data at the end of expiration and the final voltage sensing data at the end of inspiration within the preset expiratory cycle as the sensing signal difference; and using the ratio of the difference between the chest cavity surface sensing data and the initial voltage sensing data at the end of expiration within the preset expiratory cycle to the sensing signal difference as the second sensing data.

[0080] In this embodiment, since the intrathoracic cavity sensor signal data at the end of inspiration and the end of expiration have two extreme values, the absolute value of the difference between these two values ​​can be used as the sensor signal difference. Then, the ratio of the difference between the intrathoracic cavity surface sensor data and the initial sensor data at the end of the preset expiratory cycle to this sensor signal difference is mapped to the range [0,1]. Normalization can accelerate the speed of gradient descent in finding the optimal solution to some extent. If the machine learning model uses gradient descent to find the optimal solution, normalization is often very necessary; otherwise, convergence is difficult or even impossible.

[0081] Please see Figure 7 One embodiment of this application also provides a data augmentation device, which may include: an image data acquisition unit, a lung contour establishment unit, a lung contour simulation unit, and a voltage measurement unit.

[0082] The image data acquisition unit is used to acquire the first lung medical image sequence at the end of expiration and the second lung medical image sequence at the end of inspiration.

[0083] The lung contouring unit is used to establish an initial lung contour based on the first lung medical image sequence and to establish a final lung contour based on the second lung medical image sequence.

[0084] The lung contour simulation unit is used to generate lung simulation contours at several intermediate moments based on the initial lung contour and the final lung contour; the intermediate moments are used to characterize the time between the end of expiration and the end of inspiration.

[0085] The sensing signal acquisition unit is used to add a simulation signal acquisition device to the thoracic cavity surface corresponding to several simulated lung contours, and to receive the sensing signals returned by the simulation signal acquisition device; the sensing signals and the sensing images corresponding to the simulated lung contours constitute a set of labeled training data; the simulation signal acquisition device is one of an ultrasound signal acquisition device and a voltage signal acquisition device.

[0086] Please see Figure 8 One embodiment of this application also provides a signal imaging device, which may include: a signal acquisition unit, a first sensor data construction unit, a second sensor data construction unit, a data fusion unit, and a signal imaging unit.

[0087] The signal acquisition unit is used to acquire the thoracic surface sensing data obtained by the signal acquisition device for the lungs added to the surface of the thoracic cavity.

[0088] The first sensing data construction unit is used to take the difference between the chest cavity surface sensing data and the initial sensing data at the end of the preset expiratory cycle and the ratio of the initial sensing data as the first sensing data.

[0089] The second sensing data construction unit is used to normalize the thoracic cavity surface sensing data to obtain the second sensing data.

[0090] The data fusion unit is used to fuse the first sensing data and the second sensing data to obtain the target sensing data.

[0091] The signal imaging unit is used to input the target sensing data into the trained image reconstruction model described in the above embodiment to obtain the target sensing image.

[0092] The specific functions and effects of the data amplification device and / or signal imaging device can be explained by referring to other embodiments in this specification, and will not be repeated here. Each module in the data amplification device and / or signal imaging device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0093] In one embodiment, this specification provides a scenario example of a data augmentation method. First, 4D-CT images of 20 patients (numbered from 100 to 119) were collected, with each CT image set containing 10 respiratory stages (0-90%). The present invention uses 0% (end of inspiration) and 50% (end of expiration) CT images of 100-117 patients to establish digital twin models of the lungs of each patient. In the simulation of the lung motion model, the inhalation process is divided into 21 time points. The first time point T0 represents the end of expiration, and the last time point T20 represents the end of inspiration. The EIT excitation current frequency is 100kHz, therefore, according to

[10] , the conductivity and relative permittivity of the lung tissue at the end of expiration are set to 0.26S / m and 8531.4, respectively, and the conductivity and relative permittivity of the lung tissue at the end of inspiration are set to 0.10S / m and 4272.5, respectively. The electrical parameters of the lung during the inhalation process change linearly with volume change:

[0094]

[0095] In the formula, H T These are the electrical parameters (conductivity or dielectric constant) at time T. H i and H l These are the electrical parameters at the end of expiration and the end of inspiration, respectively. V T It is the volume of the lung at time T.

[0096] The training dataset for the neural network was generated from digital twin models of the lungs of 100-112 patients. To increase the amount of data, the following methods were applied:

[0097] 1) The electrode array is placed at two different heights in the chest cavity for voltage measurement. This often happens in the real world because the electrode array is not fixed in one specific location.

[0098] 2) Adding 5% Gaussian random noise to the voltage measurement improves the generalization ability of the neural network.

[0099] 3) Flip the initial image and its corresponding label horizontally, vertically, and centrally symmetrically.

[0100] After data augmentation, the training dataset ultimately contained 4368 labeled pairs. Data from patients 13-14 was used as the validation set, containing 168 labeled pairs. Images from patients 15-17 were used as the test set, containing 252 labeled pairs. Training was performed using the PyTorch framework on an NVIDIA 3070Ti GPU, employing the Adam optimizer with a learning rate of 5e-5 for 1300 epochs.

[0101] On the test set, the Structural Similarity Index Measure (SSIM) and Relative Image Error (RIE) were used to quantitatively compare imaging performance. Images reconstructed using the traditional EIT image reconstruction algorithm, Newton-Raphson (NR), were compared with those reconstructed using the algorithm described in the specification. The results are shown in Table 2. When 5% Gaussian random noise was added, the SSIM of the EIT image reconstructed by the NR algorithm decreased from 0.588 to 0.528, while the RIE increased from 0.537 to 0.617, indicating that its imaging performance was easily affected by noise. In contrast, the imaging results of IR-Net showed significant improvement. It can more accurately reconstruct the conductivity distribution and exhibits better noise resistance and imaging performance.

[0102] Table 2 shows the average SSIM and RIE values ​​of the test set.

[0103]

[0104] The imaging results on the test set verified the feasibility of the imaging framework based on the lung digital twin model and image reconstruction network proposed in this invention.

[0105] The above description is merely a scenario example provided in this specification and is not intended to limit the present invention. Any modifications, equivalent substitutions, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0106] Please see Figure 9 One embodiment of this application also provides an electronic device, which includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the above-described data amplification method and / or signal imaging method.

[0107] The processor can be a central processing unit (CPU). It can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0108] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this invention. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods described in the above embodiments.

[0109] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0110] One embodiment of this application also provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the above-described data amplification method and / or signal imaging method.

[0111] Those skilled in the art will understand that implementing all or part of the processes in the methods described in this specification can be accomplished by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described. Any references to memory, storage, databases, or other media used in the embodiments provided in this specification can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0112] It should be understood that each block of a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0113] This specification describes various embodiments in a progressive manner. Different embodiments focus on describing the parts that differ from other embodiments. Those skilled in the art, upon reading this specification, will realize that the various embodiments and the technical features disclosed in these embodiments can be combined in numerous ways. For the sake of brevity, not all possible combinations of the technical features in the described embodiments are described. However, any combination of these technical features that does not contradict each other should be considered within the scope of this specification.

[0114] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] The various embodiments described in this specification emphasize the parts that differ from other embodiments, and these embodiments can be explained by comparison with each other. Any combination of the various embodiments described in this specification, based on general technical knowledge, is covered within the scope of this specification.

[0116] The above description is merely an embodiment of this invention and is not intended to limit the scope of protection of the claims. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principle of this invention should be included within the scope of the claims.

Claims

1. A data augmentation method, characterized in that, The method includes: Acquire the first lung medical imaging sequence at the end of expiration and the second lung medical imaging sequence at the end of inspiration; An initial lung contour is established based on the first lung medical image sequence, and a final lung contour is established based on the second lung medical image sequence. Based on the initial lung contour and the final lung contour, several intermediate lung simulation contours are generated; the intermediate moments are used to characterize the time between the end of expiration and the end of inspiration. The process of generating lung simulation contours at several intermediate time points based on the initial lung contour and the final lung contour includes: The intrapleural pressure difference between the end of expiration and the end of inspiration is calculated based on the volume corresponding to the initial lung contour and the final lung contour. The intrapleural pressure difference between each intermediate moment and the end-expiratory moment is generated based on the intrapleural pressure difference. A simulated lung profile representing each intermediate time point is generated based on the intrapleural pressure difference between each intermediate time point and the end-expiratory time point. Simulation signal acquisition devices are added to the thoracic surfaces corresponding to several simulated lung contours, and the sensor signals returned by the simulation signal acquisition devices are received; the sensor signals and the sensor images corresponding to the simulated lung contours constitute a set of labeled training data; the simulation signal acquisition devices are voltage signal acquisition devices.

2. The method according to claim 1, characterized in that, The simulation signal acquisition device is a voltage signal acquisition device. Simulation signal acquisition devices are added to the thoracic cavity surfaces corresponding to several simulated lung contours, and the devices receive the sensor signals returned by the simulation signal acquisition devices, including: Electrode arrays are added to the thoracic cavity surfaces corresponding to several of the simulated lung contours; The surface voltage of the thoracic cavity surface corresponding to the simulated lung contour is measured; the surface voltage of the thoracic cavity surface corresponding to the simulated lung contour and the conductivity distribution image corresponding to the simulated lung contour constitute a set of labeled training data.

3. The method according to claim 2, characterized in that, Measuring the surface voltage of the thoracic cavity surface corresponding to the simulated lung contour includes: An excitation current signal is injected sequentially into the electrode pair consisting of two adjacent electrodes in the electrode array; By injecting the excitation current signal into each electrode pair, the voltage value of each electrode on the electrode array is collected to obtain the surface voltage of the simulated lung contour.

4. The method according to claim 1, characterized in that, The method further includes: Construct training samples; the training samples include at least the sensor signals of the thoracic cavity surface corresponding to the simulated lung contour and the sensor images corresponding to the simulated lung contour; The sensor signal of the thoracic cavity surface corresponding to the simulated lung contour is input into the image reconstruction model to obtain the target sensor image corresponding to the simulated lung contour; The image reconstruction model is updated based on the difference between the target sensor image corresponding to the simulated lung contour and the sensor image corresponding to the simulated lung contour.

5. The method according to claim 4, characterized in that, The sensing signal of the thoracic cavity surface corresponding to the simulated lung contour is input into the image reconstruction model to obtain the target sensing image corresponding to the simulated lung contour, including: The first sensing data is the ratio of the difference between the sensing signal on the thoracic surface corresponding to the simulated lung contour and the initial sensing signal at the end of expiration corresponding to the simulated lung contour to the initial sensing signal. The sensing signal of the thoracic cavity surface corresponding to the simulated lung contour is normalized to obtain the second sensing data. The first sensing data and the second sensing data are fused to obtain the target sensing data; The target sensing data is input into the image reconstruction model to obtain the target sensing image corresponding to the simulated lung contour.

6. The method according to claim 4, characterized in that, The method further includes: Gaussian random noise within a certain threshold range is added to the sensing signal of the thoracic cavity surface corresponding to the simulated lung contour to obtain the target sensing signal of the simulated lung contour. Accordingly, the image reconstruction model is trained based at least on the sensing signals of the thoracic cavity surface corresponding to the simulated lung contour and the sensing image corresponding to the simulated lung contour, the target sensing signal of the simulated lung contour and the sensing image corresponding to the simulated lung contour.

7. A signal imaging method, characterized in that, The method includes: Acquire thoracic surface sensing data from a lung-targeting signal acquisition device added to the thoracic surface; The ratio of the difference between the chest cavity surface sensing data and the initial sensing data at the end of the preset expiratory cycle to the initial sensing data is used as the first sensing data. The chest cavity surface sensing data is normalized to obtain the second sensing data. The first sensing data and the second sensing data are fused to obtain the target sensing data; The target sensing data is input into the trained image reconstruction model as described in any one of claims 4-6 to obtain the target sensing image.

8. The method according to claim 7, characterized in that, The chest cavity surface sensing data is normalized to obtain second sensing data, including: The absolute value of the difference between the initial sensing data at the end of expiration and the final sensing data at the end of inhalation within the preset expiration cycle is taken as the sensing signal difference. The ratio of the difference between the chest cavity surface sensing data and the initial sensing data at the end of the preset expiratory cycle to the difference in the sensing signal is used as the second sensing data.

9. A data amplification device, characterized in that, The data amplification device includes: The image data acquisition unit is used to acquire the first lung medical image sequence at the end of expiration and the second lung medical image sequence at the end of inspiration; The lung contouring unit is used to establish an initial lung contour based on the first lung medical image sequence and to establish a final lung contour based on the second lung medical image sequence. A lung contour simulation unit is used to generate lung simulation contours at several intermediate moments based on the initial lung contour and the final lung contour; the intermediate moments are used to characterize the time between the end of expiration and the end of inspiration. The process of generating lung simulation contours at several intermediate time points based on the initial lung contour and the final lung contour includes: The intrapleural pressure difference between the end of expiration and the end of inspiration is calculated based on the volume corresponding to the initial lung contour and the final lung contour. The intrapleural pressure difference between each intermediate moment and the end-expiratory moment is generated based on the intrapleural pressure difference. A simulated lung profile representing each intermediate time point is generated based on the intrapleural pressure difference between each intermediate time point and the end-expiratory time point. The sensing signal acquisition unit is used to add a simulation signal acquisition device to the thoracic cavity surface corresponding to several simulated lung contours, and to receive the sensing signals returned by the simulation signal acquisition device; the sensing signals and the sensing images corresponding to the simulated lung contours constitute a set of labeled training data; the simulation signal acquisition device is a voltage signal acquisition device.

10. A signal imaging device, characterized in that, The signal imaging device includes: The signal acquisition unit is used to acquire the thoracic surface sensing data collected by the lung signal acquisition device added to the surface of the thoracic cavity. The first sensing data construction unit is used to take the difference between the chest cavity surface sensing data and the initial sensing data at the end of the preset expiratory cycle and the ratio of the initial sensing data as the first sensing data. The second sensing data construction unit is used to normalize the thoracic cavity surface sensing data to obtain the second sensing data. The data fusion unit is used to fuse the first sensing data and the second sensing data to obtain the target sensing data; A signal imaging unit is used to input the target sensing data into the trained image reconstruction model as described in any one of claims 4-6 to obtain a target sensing image.

11. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory being used to store a computer program that, when executed by the processor, implements the method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images