Imaging method and device, imaging system, storage medium and program product

Abstract

The invention relates to the technical field of images, in particular to an imaging method and device, an imaging system, a storage medium and a program product. The imaging method comprises the following steps: generating a first image of a to-be-tested sample according to perspective data of rays passing through the to-be-tested sample; generating a second image of the to-be-detected sample according to ultrasonic signal data emitted after the to-be-detected sample is excited by the rays; and generating an image of the sample to be detected according to the first image and the second image.

Description

Technical Field

[0001] This disclosure relates to the field of image technology, and in particular to an imaging method and apparatus, an imaging system, a storage medium, and a program product. Background Technology

[0002] With the development of technology, X-ray imaging is being increasingly widely used in medical, industrial and other fields. X-ray imaging can penetrate the sample in a non-invasive manner to quickly and accurately obtain information about the sample's internal structure or function. Summary of the Invention

[0003] In related technologies, the information obtained from X-ray imaging is relatively limited, resulting in poor imaging accuracy.

[0004] In view of this, the present disclosure provides an imaging method and apparatus, an imaging system, a computer-readable storage medium and a computer program product, which simultaneously perform X-ray imaging and photoacoustic imaging with a single ray, and then couple the images generated by different methods. At the same time, it improves the accuracy of X-ray imaging by utilizing the high penetrability of the ray itself and the high resolution of the X-ray-excited ultrasound signal.

[0005] According to a first aspect of this disclosure, an imaging method is provided, comprising: generating a first image of the sample under test based on fluoroscopic data of rays passing through the sample; generating a second image of the sample under test based on ultrasonic signal data emitted by the sample under test after being excited by the rays; and generating an image of the sample under test based on the first image and the second image.

[0006] In some embodiments, generating a second image of the sample to be tested based on the ultrasonic signal data emitted by the sample after being excited by the X-ray includes: acquiring the ultrasonic signal data emitted by the sample after being excited by the X-ray using a detector corresponding to the sample to be tested, wherein different samples to be tested generate ultrasonic signals with different frequency ranges; and generating a second image of the sample to be tested based on the ultrasonic signal data.

[0007] In some embodiments, acquiring ultrasonic signal data emitted by the sample after it is excited by the radiation using a detector corresponding to the sample to be tested includes: determining a frequency range corresponding to the sample to be tested based on the characteristic length of the sample to be tested and the waveform of the radiation; and determining a detector corresponding to the sample to be tested based on the frequency range.

[0008] In some embodiments, the waveform of the ray is a Gaussian waveform.

[0009] In some embodiments, the imaging method further includes: during the imaging process, adjusting at least one of the intensity, duration, and collimation of the ray based on the fluoroscopic data and the ultrasound signal data.

[0010] In some embodiments, adjusting at least one of the intensity, duration, and collimation of the X-ray based on the fluoroscopy data and the ultrasound signal data includes: determining a first predicted value for X-ray fluoroscopy and a second predicted value for the X-ray-excited ultrasound signal based on the material and shape of the sample to be tested; and adjusting at least one of the intensity, duration, and collimation of the X-ray based on the first predicted value, the second predicted value, the fluoroscopy data, and the ultrasound signal data.

[0011] In some embodiments, generating an image of the sample to be tested based on the first image and the second image includes: determining a first weight of the first image and a second weight of the second image based on at least one of the gradient and contrast of the first image and the second image; and performing weighted fusion of the first image and the second image based on the first weight and the second weight to generate an image of the sample to be tested.

[0012] In some embodiments, the first image includes a plurality of first sub-images, and the second image includes a plurality of second sub-images corresponding one-to-one with the plurality of first sub-images. Weighted fusion of the first image and the second image according to the first weight and the second weight to generate the image of the sample to be tested includes: weighted fusion of the i-th first sub-image and the i-th second sub-image according to the first weight and the second weight to generate the i-th fused sub-image, where i is a positive integer; and generating the image of the sample to be tested based on the plurality of fused sub-images.

[0013] In some embodiments, the plurality of first sub-images have different resolutions, the plurality of second sub-images have different resolutions, and the i-th first sub-image and the i-th second sub-image have the same resolution. Determining the first weight of the first image and the second weight of the second image includes: sorting the plurality of first sub-images and the plurality of second sub-images according to their resolutions to determine a first image sequence and a second image sequence; determining the gradient and local contrast of the plurality of first sub-images based on the differences between adjacent first sub-images in the first image sequence; determining the gradient and local contrast of the plurality of second sub-images based on the differences between adjacent second sub-images in the second image sequence; and determining the first weight and the second weight based on the gradient and local contrast of the plurality of first sub-images and the gradient and local contrast of the plurality of second sub-images.

[0014] In some embodiments, the intensity and duration of the radiation are determined based on at least one of the safety threshold, excitation threshold, and transparency factor of the sample under test.

[0015] In some embodiments: the first image is an X-ray image of the sample to be tested; the second image is an ultrasonic image of the sample to be tested.

[0016] According to a second aspect of this disclosure, an imaging apparatus is provided, comprising: a first image generation module configured to generate a first image of the sample under test based on fluoroscopic data of rays passing through the sample under test; a second image generation module configured to generate a second image of the sample under test based on ultrasonic signal data emitted by the sample under test after being excited by the rays; and a sample image generation module configured to generate an image of the sample under test based on the first image and the second image.

[0017] According to a third aspect of this disclosure, an imaging apparatus is provided, comprising: at least one memory; and at least one processor coupled to the at least one memory, the at least one processor being configured to perform an imaging method as described in any embodiment of this disclosure based on instructions stored in the at least one memory.

[0018] According to a fourth aspect of this disclosure, an imaging system is provided, comprising: an imaging device as described in any embodiment of this disclosure; a radiation source for emitting radiation toward the sample to be tested; a first probe for acquiring fluoroscopic data of the radiation passing through the sample to be tested; and a second probe for acquiring ultrasonic signal data emitted by the sample to be tested after being excited by the radiation.

[0019] In some embodiments, the second probe is deployed around the sample to be tested.

[0020] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the imaging method as described in any embodiment of this disclosure.

[0021] According to a sixth aspect of this disclosure, a computer program product is provided that, when run on a computer, causes the computer to implement the imaging method as described in any embodiment of this disclosure.

[0022] Other features, aspects, and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0023] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0024] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein:

[0025] Figure 1 A schematic flowchart of an imaging method according to an embodiment of the present disclosure is shown;

[0026] Figure 2 A schematic flowchart illustrating the generation of a second image according to some embodiments of the present disclosure is shown;

[0027] Figure 3 A schematic flowchart of an imaging method according to other embodiments of the present disclosure is shown;

[0028] Figure 4 A schematic flowchart illustrating the process of generating an image of a sample to be tested according to some embodiments of the present disclosure is shown;

[0029] Figure 5 A block diagram of an imaging apparatus according to some embodiments of the present disclosure is shown;

[0030] Figure 6 This is a block diagram illustrating an imaging apparatus according to other embodiments of the present disclosure;

[0031] Figure 7 Schematic diagrams of imaging systems according to some embodiments of the present disclosure are shown;

[0032] Figure 8 This is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0033] It should be understood that the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale. Furthermore, the same or similar reference numerals denote the same or similar components. Detailed Implementation

[0034] Various embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The descriptions of the embodiments are merely illustrative and are in no way intended to limit the scope of the disclosure or its application or use. The present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided so that this disclosure will be thorough and complete, and will fully express the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specifically stated, the relative arrangement of components and steps set forth in these embodiments should be interpreted as merely illustrative and not as limiting.

[0035] The terms “first,” “second,” and similar words used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different parts. Words such as “including” mean that the element preceding the word covers the element listed after the word, and do not exclude the possibility of covering other elements as well.

[0036] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0037] All terms used in this disclosure (including technical or scientific terms) have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in a general dictionary, such as a dictionary, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0038] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0039] In traditional imaging methods, the multifaceted properties of X-rays cannot be utilized simultaneously during the imaging process, resulting in poor imaging accuracy.

[0040] In view of this, this disclosure proposes an imaging method that simultaneously performs X-ray imaging and photoacoustic imaging using a single ray, and then couples the images generated by different methods. At the same time, it utilizes the high penetrability of the ray itself and the high resolution of the X-ray-excited ultrasound signal to improve the accuracy of X-ray imaging.

[0041] First, combined Figure 1 The imaging method in this disclosure is described. Figure 1 A schematic flowchart of an imaging method according to an embodiment of the present disclosure is shown.

[0042] like Figure 1 As shown, the imaging method may include: step S1, generating a first image of the sample under test based on the fluoroscopic data of the ray passing through the sample under test; step S3, generating a second image of the sample under test based on the ultrasonic signal data emitted by the sample under test after being excited by the ray; step S5, generating an image of the sample under test based on the first image and the second image.

[0043] In step S1, after emitting rays towards the sample to be tested, for example using a ray sensor, the X-ray transmission data of the sample can be acquired to generate a first image of the sample. The first image can also be understood as a perspective image of the sample. The ray sensor is deployed, for example, behind the sample in the direction of the rays, thereby improving the accuracy of the X-ray transmission data acquisition.

[0044] In some embodiments, the aforementioned rays can be high-energy photon beams, such as X-rays. When the rays are X-rays, the first image can be, for example, an X-ray image of the sample under test. By acquiring X-ray transmission data, the attenuation experienced by the X-rays as they pass through the sample under test can be analyzed to generate an X-ray image of the sample, which serves as the first image.

[0045] It should be understood that the X-rays described above are merely exemplary and not limiting. The rays used in the embodiments of this disclosure may also be other rays, such as... ray.

[0046] In some embodiments, ultrashort pulse X / A X-ray source is used to generate X-rays for irradiating the sample under test. The aforementioned source can produce X-rays with pulse widths of approximately 10 picoseconds and energies ranging from 10 kiloelectron volts to 10 megaelectron volts. A beam of radiation, or a pulse of radiation, operates at a certain repetition frequency.

[0047] By using high-energy rays to image the sample, the penetration characteristics of the rays can be improved, thereby generating a first image that highlights the structure of the sample more quickly and accurately.

[0048] The previous section described how to generate the first image of the sample using the perspective data of the X-ray itself in step S1. Below, we will describe how to generate the second image of the sample using the ultrasonic data generated after the sample is excited by the same X-ray in step S3.

[0049] In step S3, after emitting a ray into the sample, the interaction between the ray and the sample will excite the sample, placing it in an excited state. At this point, the atoms or molecules inside the sample absorb the energy of the ray, and their outer electrons will transition from the ground state to a higher energy level.

[0050] Particles in an excited state are unstable and will spontaneously transition to a lower energy level within a very short time. During this transition, the particle releases excess energy in the form of photons, and the energy of the released photons is exactly equal to the energy difference between the two energy levels.

[0051] When a radiation is emitted onto a sample, the atoms or molecules inside the sample absorb the energy of the radiation, resulting in energy deposition. For example, the interaction between the radiation and the sample can excite the atoms or molecules in the sample, causing their outer electrons to transition from the ground state to a higher energy level, thus putting the sample into an excited state.

[0052] After absorbing the energy of the radiation, the sample's temperature rises rapidly, causing its volume to expand. Subsequently, this expansion process will exert a compressive effect on the surrounding medium, such as air, liquid, or other tissues around the sample.

[0053] Furthermore, since atoms or molecules in the excited state are not stable, they will transition to lower energy levels in a short period of time. During the transition, the particles in the sample release excess energy, causing the sample temperature to drop rapidly, and the sample volume to shrink rapidly after expansion.

[0054] Through repeated expansion and contraction processes, the sample continuously compresses the surrounding medium, generating vibrations and thus forming high-frequency pressure waves, i.e., ultrasonic signals. Furthermore, the different photon absorption coefficients at different locations of the sample determine the spatial distribution of energy deposition, thereby affecting the parameters of the ultrasonic signal. Based on the ultrasonic signal waveform, the three-dimensional energy deposition distribution of the sample can be obtained, allowing for the reconstruction of the sample structure and the generation of a second image of the sample. This second image can, for example, be an ultrasonic image of the sample.

[0055] For example, an ultrasonic sensor can acquire ultrasonic signal data emitted by a sample after it is excited by radiation, thereby generating a second image of the sample. This second image can also be understood as an ultrasonic image of the sample. The ultrasonic sensor can be deployed, for example, on one side of the sample along the radiation direction, or it can be deployed around the sample, thereby improving the accuracy of ultrasonic signal acquisition.

[0056] In some embodiments, the intensity and duration of the radiation are determined based on at least one of a safety threshold, an excitation threshold, and a transmission coefficient of the sample under test. The excitation threshold refers to the radiation energy threshold required to excite the sample and generate an ultrasonic signal. The transmission coefficient refers to the energy attenuation coefficient of the radiation as it passes through the sample.

[0057] For example, while ensuring the safety of the sample under test, the corresponding energy of the X-ray can be selected for imaging the sample based on the sample's excitation threshold and transparency coefficient. This ensures that the X-ray data of the sample and the ultrasonic signal emitted after the sample is excited by the X-ray are more accurate, thereby improving the accuracy of the sample imaging.

[0058] In other words, the intensity of the ray in the imaging method provided in this embodiment can simultaneously meet the requirements of fluoroscopic imaging and ultrasound imaging, thereby realizing simultaneous ray imaging and ultrasound imaging with a single ray. At the same time, the high penetrability of the ray itself and the high resolution of the ultrasound signal excited by the ray improve the accuracy of ray imaging.

[0059] Below, we will combine Figure 2 This section introduces one method for generating a second image. Figure 2A schematic flowchart illustrating the generation of a second image according to some embodiments of the present disclosure is shown.

[0060] like Figure 2 As shown, step S3 above, which generates a second image of the sample to be tested based on the ultrasonic signal data emitted by the sample after being excited by the ray, may further include steps S31 and S32.

[0061] Step S31: Using a detector corresponding to the sample to be tested, acquire the ultrasonic signal data emitted by the sample after being excited by the X-ray. The frequency range of the ultrasonic signals generated by different samples to be tested is different. Step S32: Generate a second image of the sample to be tested based on the ultrasonic signal data.

[0062] Due to differences in the parameters of the sample itself or the radiation parameters, the frequency range of the ultrasonic signal excited after the sample is irradiated by radiation may also be different. For example, samples with larger structures produce ultrasonic signals with a lower frequency range, while samples with smaller structures produce ultrasonic signals with a higher frequency range.

[0063] In the above embodiments, a detector with a frequency range consistent with the ultrasonic signal generated by the sample can be selected to acquire sample data, thereby improving the accuracy of the data.

[0064] In some embodiments, acquiring ultrasonic signal data emitted by the sample after it is excited by the radiation using a detector corresponding to the sample to be tested may include: determining a frequency range corresponding to the sample to be tested based on the characteristic length of the sample to be tested and the waveform of the radiation; and determining a detector corresponding to the sample to be tested based on the frequency range.

[0065] The characteristic length of a test sample refers to the geometric dimension parameter of the target feature within the sample, used to quantify the spatial scale and structural complexity of the sample. For example, the test sample can be a crystal, and to examine the integrity of the crystal, the target feature could be a defect such as a vacancy or dislocation within the crystal. In this case, the typical size of the vacancy in the crystal can be determined as the characteristic length of the test sample, and based on this, a corresponding detector can be selected for ultrasonic detection, thereby improving the accuracy of sample imaging.

[0066] As mentioned earlier, the ultrasonic signal generated by the sample is due to vibrations after being excited by radiation. Therefore, the waveform of the ultrasonic signal usually corresponds to the waveform of the radiation. The waveform of the ultrasonic signal can be determined based on the waveform of the radiation, and thus the frequency range corresponding to the sample under test can be determined.

[0067] In some embodiments, the waveform of the ray can be, for example, a Gaussian waveform. A Gaussian waveform pulse signal can accurately characterize short-lived, high-frequency, and rapidly decaying high-energy rays, balancing the time-domain focusing and frequency-domain smoothness of the ray pulse, thereby improving the accuracy of subsequent data processing.

[0068] The following section will use an example of a Gaussian waveform to illustrate the complete process of determining the detector corresponding to the sample to be tested.

[0069] When the X-ray waveform is Gaussian, the sound pressure signal generated by the sample, i.e., the ultrasonic signal, can be approximated by a cosine function modulated by a Gaussian function. The intensity of the ultrasonic signal received by the detector varies with time as a function of... For example, it can be as shown in formula (1):

[0070] (1)

[0071] In the above formula (1), It can be understood as the Gaussian envelope term of the ultrasonic signal function, where A represents the peak amplitude of the ultrasonic signal, which can be determined, for example, based on the excitation characteristics of the sample. The peak time point characterizing the Gaussian envelope is the moment when the ultrasonic signal intensity reaches its maximum value, which can be determined, for example, based on the moment when the radiation intensity reaches its maximum value. This refers to the time standard deviation of the Gaussian envelope, used to determine the time-domain width of an ultrasound signal. The full width at half maximum (FWHM) of an ultrasound signal can be characterized as... .

[0072] This can be understood as the cosine oscillation term of the ultrasonic signal function, where, Characterizing the oscillation frequency of an ultrasound signal. The initial phase characterizes the ultrasonic signal. These parameters can be determined, for example, based on the frequency and initial phase of the radiation.

[0073] Based on the above function, the length resolution R of the detector can be determined by the full width at half maximum (FWHM) of the ultrasonic signal and the speed of sound propagation. The product can be determined, for example, as shown in formula (2):

[0074] (2)

[0075] By performing a Fourier transform on the ultrasonic signal function, the frequency domain function of the ultrasonic signal can be obtained. For example, as shown in formula (3):

[0076] (3)

[0077] The frequency domain function described above can characterize ultrasonic signals at different frequencies. The energy distribution on the surface, that is, the frequency range of the ultrasonic signal and the intensity at each frequency. The oscillation frequency of the ultrasonic signal. In the frequency domain function, it is represented as the center frequency of the frequency domain function.

[0078] The negative values ​​in the frequency domain function are byproducts of the Fourier transform and have no actual physical meaning. For the positive values ​​of the frequency domain function, the frequency range of the ultrasonic signal generated by the sample can be characterized by the full width at half maximum (FWHM) of the frequency domain function, for example, it can be expressed as: .

[0079] Based on the above expressions for length resolution and frequency domain half-width, the relationship between length resolution and frequency range can be determined as shown in formula (4):

[0080] (4)

[0081] Based on the above correlation, the detector corresponding to the sample can be determined according to the characteristic length of the sample to be tested. For example, when the characteristic length of the sample is 250 micrometers, the sound velocity... With an air speed of 340 m / s, the half-width at half-maximum (WHM) of the frequency corresponding to the sample is 10 MHz. By using a detector within this frequency range, the accuracy of the data acquired by the detector can be improved.

[0082] The above section described the specific process of determining the detector corresponding to the sample and acquiring ultrasonic data. Below, we will return to... Figure 2 This section describes the process of generating an ultrasound image by performing signal inverse decoding based on ultrasound signal data in step S32.

[0083] The formulas for the energy deposition of X-rays and the generation of ultrasonic signals from the sample can be shown, for example, as in formula (5):

[0084] (5)

[0085] In the above formula (5), It refers to the pressure wave function of the ultrasonic signal, which is a function that characterizes the change of the ultrasonic signal in the time and space dimensions. This is the endothermic function of the sample under test, which is related to position r and time t. It should be understood that the intensity of the ultrasonic signal received by each sensor as a function of time is... That is, the pressure wave function The specific performance when r represents the position of each sensor.

[0086] The Laplace operator, also known as the second-order differential operator, is used to describe the spatial curvature of a function. The velocity is the sound velocity, consistent with the above. β represents the coefficient of volumetric thermal expansion of the sample under test. Characterizes the specific heat capacity of the sample under constant pressure.

[0087] As can be seen from the above formula (5), the ultrasonic signal generated by the sample after being excited by radiation is related to the endothermic function. It relates to the first derivative with respect to time t. Therefore, by using higher-energy rays, the heat absorption rate of the sample can be increased, thereby increasing the intensity and resolution of the ultrasonic signal generated after the sample is excited by rays, and thus improving the accuracy of X-ray imaging.

[0088] After the sample is excited by X-rays to generate the above-mentioned ultrasonic waves, the propagation process of the ultrasonic waves follows the wave equation, as shown in equation (6):

[0089] (6)

[0090] Since the wave equation only involves the second derivative of time in the time dimension, it has symmetry with respect to both the positive and negative directions of time. That is, it can be used to determine the forward propagation of a wave as well as to derive the reverse direction of a wave.

[0091] Furthermore, since the three processes of energy deposition initiating ultrasonic signals, ultrasonic signal propagation, and probe signal reception can all be approximated as linear processes, the energy deposition values ​​at each point in the sample under test can be... (N is the total number of spatial grid points of the sample to be tested) and the amplitude of each ultrasonic sensor at each time step. (M is the number of sensors, and T is the total number of time sampling points of the sensors) also has a linear relationship, as shown in formula (7):

[0092] (7)

[0093] In the above formula (7), matrix M includes the linear relationship between the vector h representing the energy deposition value and the vector p representing the sensor measurement value, which can be determined, for example, by the relative positional relationship between the sensor and the sample to be tested.

[0094] Based on the above linear relationship, for example, the energy deposition vector h can be optimized through repeated iterations, and the energy deposition distribution of the sample under test can be obtained by inverse solving the ultrasonic data, thereby reconstructing the sample structure.

[0095] The above text combines Figure 2 The process of generating the second image of the sample in step S3 has been described. Below, we will combine... Figure 3 This section introduces how to achieve dynamic adjustment of the X-rays during the imaging process. Figure 3 A schematic flowchart of an imaging method according to other embodiments of the present disclosure is shown.

[0096] like Figure 3 As shown, in Figure 1 Based on this, the imaging method may further include: step S4, during the imaging process, adjusting at least one of the intensity, duration, and collimation of the ray based on the fluoroscopic data and the ultrasound signal data.

[0097] In other words, as the imaging process proceeds, the radiation can be dynamically adjusted based on the data actually collected by the detector, thereby further achieving the goals of improving imaging accuracy, minimizing radiation dose, or protecting the safety of the sample under test.

[0098] In some embodiments, adjusting at least one of the intensity, duration, and collimation of the X-ray based on the fluoroscopy data and the ultrasound signal data may include: determining a first predicted value for X-ray fluoroscopy and a second predicted value for the X-ray-excited ultrasound signal based on the material and shape of the sample to be tested; and adjusting at least one of the intensity, duration, and collimation of the X-ray based on the first predicted value, the second predicted value, the fluoroscopy data, and the ultrasound signal data.

[0099] In the above embodiments, for example, before X-ray irradiation, a simulation platform can be constructed by simulating X-ray fluoroscopy and sound wave propagation to simulate the material and geometry of the sample to be tested, and to determine the first predicted value of X-ray fluoroscopy and the second predicted value of the ultrasonic signal.

[0100] In actual imaging, the intensity, duration, or collimation of the rays can be adjusted by comparing the predicted and actual values ​​of the above data.

[0101] For example, if the actual value of X-ray fluoroscopy is greater than the predicted value, it means that the structural complexity of the sample under test may be lower than expected. In this case, the intensity of the X-ray can be reduced to minimize the radiation dose while meeting the imaging requirements.

[0102] For example, if the actual value of the ultrasonic signal is less than the predicted value, it means that the excitation degree of the sample under test may be lower than expected. In this case, the intensity of the X-ray can be increased to ensure the accuracy of the imaging.

[0103] It should be understood that the above-described dynamic adjustment methods are merely exemplary. In addition to the methods described above, the dynamic changes of the sample under test can also be monitored, for example, by utilizing the temporal characteristics of X-rays. Through multiple X-ray pulse imaging, the motion of the sample can be visualized during the imaging process, and the X-rays can be dynamically adjusted according to the sample's own motion, such as adjusting the collimation of the X-rays, thereby ensuring accurate X-ray irradiation and improving imaging precision.

[0104] The above text combines Figure 3 The process of dynamically responding to and adjusting to radiation was introduced. We will now return to... Figure 1 Next, we will introduce how to generate an image of the sample to be tested based on the first image and the second image in step S5.

[0105] After acquiring the fluoroscopic and ultrasonic images of the sample, image coupling can be performed to generate the final image of the sample. Since the two images in the imaging method of this embodiment are acquired using the same X-ray, registration of the fluoroscopic and ultrasonic images is unnecessary during image coupling, avoiding errors caused by image registration and thus improving the accuracy of X-ray imaging.

[0106] It should be understood that, in the embodiments of this disclosure, a single X-ray pulse can also be used to generate a fluoroscopic image and an ultrasound image of the sample, and the image of the sample under test can be generated through image coupling. In other words, multimodal X-ray imaging can be completed with only a single irradiation of the sample, reducing the irradiation dose received by the sample while meeting imaging requirements.

[0107] Figure 4 A schematic flowchart illustrating the process of generating an image of a sample to be tested according to some embodiments of this disclosure is shown. Figure 4 As shown, step S5 above, the step of generating an image of the sample to be tested based on the first image and the second image, may further include: step S51 and step S52.

[0108] Step S51: Determine a first weight for the first image and a second weight for the second image based on at least one of the gradient and contrast of the first image and the second image; Step S52: Perform weighted fusion on the first image and the second image based on the first weight and the second weight to generate an image of the sample to be tested.

[0109] In the above embodiments, the weight relationship between the fluoroscopic image and the ultrasound image can be assigned by the gradient and contrast of the two images, thereby performing weighted image fusion.

[0110] Due to the penetrating properties of X-rays, the internal structure of the sample under test can be detected more accurately in the X-ray image, and therefore the gradient in the X-ray image is more accurate.

[0111] Due to the penetrating and reflecting characteristics of ultrasound, when the ultrasound signal excited by X-rays propagates inside the sample, it will produce different reflected signals when it encounters different materials or interfaces. This allows for more accurate detection of the material of the sample in the ultrasound image, resulting in more accurate contrast in the ultrasound image.

[0112] Considering the aforementioned characteristics of images, weights can be assigned based on image gradients and contrast during image fusion. For example, higher weights can be assigned to the first image where the gradient is larger, and higher weights can be assigned to the second image where the local contrast is larger. This allows for a balance of the advantages of both images during fusion, resulting in a more accurate final image.

[0113] Further, the first image may include multiple first sub-images, and the second image may include multiple second sub-images that correspond one-to-one with the multiple first sub-images. Weighted fusion of the first image and the second image according to the first weight and the second weight to generate the image of the sample to be tested may include: weighted fusion of the i-th first sub-image and the i-th second sub-image according to the first weight and the second weight to generate the i-th fused sub-image, where i is a positive integer; and generating the image of the sample to be tested based on the multiple fused sub-images.

[0114] In other words, the fluoroscopic and ultrasound images of the sample can be separated to determine multiple sub-images that correspond one-to-one with each other, and then fused between each pair of sub-images.

[0115] For example, based on the structure of the sample, the sample can be divided into different regions, and the image can be split into sub-images of different regions. Image fusion can be performed separately in each region, thereby reducing the interference of gradient or contrast at the region boundaries on image fusion and improving the accuracy of the sample image.

[0116] Subsequently, the obtained multiple fused sub-images can be stitched together to generate an image of the sample under test, which serves as the final imaging result.

[0117] In some embodiments, the first sub-image and the second sub-image can also be generated separately by downsampling. Taking the first image as an example, the downsampling process can be as shown in formula (8):

[0118] (8)

[0119] In the above formula (8), The pixel value at coordinates (i, j) in the k-th first sub-image is represented. The weights are assigned, where m and n are integers between -2 and 2.

[0120] The first sub-image This can be the original image, i.e., the first image itself. The downsampling process can be understood as weighted summing of pixel values ​​in local regions of the previous layer's image to determine the pixel values ​​in the next layer's image. For example, using the third sub-image... The pixel value at coordinates (5, 5) is based on the second sub-image. The value is determined by a weighted sum of pixel values ​​from coordinates (8, 8) to (12, 12). Downsampling can make... The resolution becomes Half of it is generated, thus gradually generating sub-images of different resolutions.

[0121] The downsampling process generates multiple first sub-images with different resolutions, as well as multiple second sub-images with different resolutions. By using the same m and n as the next parameters, the resolution of the i-th first sub-image and the i-th second sub-image can be made the same, so that subsequent fusion can be performed.

[0122] For the first sub-image and the second sub-image in the above embodiments, determining the first weight of the first image and the second weight of the second image may include: sorting the plurality of first sub-images and the plurality of second sub-images according to their resolutions to determine a first image sequence and a second image sequence; determining the gradient and local contrast of the plurality of first sub-images based on the differences between adjacent first sub-images in the first image sequence; determining the gradient and local contrast of the plurality of second sub-images based on the differences between adjacent second sub-images in the second image sequence; and determining the first weight and the second weight based on the gradient and local contrast of the plurality of first sub-images and the gradient and local contrast of the plurality of second sub-images.

[0123] For sub-images with different resolutions, they can be sorted first according to their resolution to determine the gradient and contrast of the sub-images.

[0124] When generating the first sub-image using the downsampling method described above, for example, the sub-images can be generated directly in the order they were generated. to As the first image sequence, 'a' represents the total number of the first sub-images.

[0125] After determining the image sequence, gradients and contrasts can be determined based on the differences between adjacent sub-images. For example, upsampling can be performed based on the image sequence to determine a sequence of differences between adjacent sub-images, where each term in the difference sequence... As shown in formula (9):

[0126] (9)

[0127] In other words, it can be achieved by supplementing... The number of pixels is used to determine the difference between sub-images. (Refer to the above text.) and Example, difference The values ​​of the midpoint coordinates (8, 8) to (12, 12) can be respectively The pixel values ​​at coordinates (8, 8) to (12, 12) and The difference between pixel values ​​at coordinates (5, 5).

[0128] After determining the difference sequence, the gradient and contrast of each sub-image can be further determined. For each first sub-image... , and its corresponding gradient As shown in formula (10):

[0129] (10)

[0130] In the above formula (10), i and j represent the coordinates of pixels in the image. and The Sobel operator is shown in equation (11):

[0131] (11)

[0132] Using the above method, the gradient of each sub-image can be determined based on the difference between sub-images.

[0133] Furthermore, for the first sub-image and its corresponding contrast As shown in formula (12):

[0134] (12)

[0135] In the above formula (12), This refers to the calculation area window for local contrast, which is the area considered during the contrast calculation process. This refers to the area of ​​the calculation region window. Refers to the area The mean within.

[0136] The first sub-image was used as an example above. For the second sub-image, the gradient and contrast of each second sub-image can be determined in a similar way.

[0137] After determining the gradient and contrast of each first sub-image and the gradient and contrast of each second sub-image, the weights between each pair of corresponding first and second sub-images can be determined.

[0138] As mentioned earlier, fluoroscopic imaging typically has higher gradient parameters, while ultrasound images typically have higher contrast parameters. By using the gradients and contrasts of sub-images, a saliency metric can be formed to characterize the modality specificity of the imaging process. Weights in the image fusion process can then be assigned based on this saliency metric, thereby improving the accuracy of the final fused image. Furthermore, Gaussian smoothing can be applied to the weights to ensure a natural transition during the image fusion process.

[0139] The above describes the imaging method provided in this embodiment. By simultaneously performing X-ray imaging and photoacoustic imaging with a single ray, and then coupling the images generated by different methods, the accuracy of X-ray imaging is improved by utilizing the high penetrability of the X-ray itself and the high resolution of the X-ray-excited ultrasound signal.

[0140] The following is for reference. Figure 5 and Figure 6 An imaging apparatus according to embodiments of the present disclosure is described, which is used to perform any of the embodiments of the imaging methods described above. Figure 5 A block diagram of an imaging apparatus according to some embodiments of the present disclosure is shown.

[0141] like Figure 5 As shown, the imaging device 5 may include: a first image generation module 51, configured to generate a first image of the sample under test based on the fluoroscopic data of the ray passing through the sample under test; a second image generation module 52, configured to generate a second image of the sample under test based on the ultrasonic signal data emitted by the sample under test after being excited by the ray; and a sample image generation module 53, configured to generate an image of the sample under test based on the first image and the second image.

[0142] The first image generation module 51 of the imaging device 5 can be used to perform Figure 1 Step S1. The second image generation module 52 of the imaging device 5 can be used to perform... Figure 1 Step S3. The sample image generation module 53 of the imaging device 5 can be used to perform... Figure 1 Step S5 in the process.

[0143] Figure 6 This is a block diagram illustrating an imaging apparatus according to other embodiments of the present disclosure. Figure 6 As shown, the imaging apparatus 6 includes at least one memory 61; and at least one processor 62 coupled to the at least one memory, the at least one processor being configured to execute the imaging method as described in any embodiment of the present disclosure based on instructions stored in the at least one memory.

[0144] Memory 61 is used to store one or more computer-readable instructions. Memory 61 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), and flash memory. Memory 61 may, for example, store operating systems, applications, bootloaders, databases, and other programs, as well as various applications and various data.

[0145] The processor 62 is configured to execute computer-readable instructions to implement the imaging method described in any of the foregoing embodiments. Specific implementations of each step of the method can be found in the above embodiments, for example... Figures 1 to 3 The steps involved are repeated here, so the details will not be repeated.

[0146] The aforementioned imaging device can simultaneously perform X-ray imaging and photoacoustic imaging using a single ray, and then couple the images generated by different methods. At the same time, it utilizes the high penetrability of the ray itself and the high resolution of the ultrasound signal excited by the ray to improve the accuracy of X-ray imaging.

[0147] The processor 62 can be various processing devices, such as a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The central processing unit (CPU) can be based on x86 or ARM architectures, etc.

[0148] The processor 62 and the memory 61 can communicate with each other directly or indirectly. For example, the processor 62 and the memory 61 can communicate via a network. The network can include wireless networks, wired networks, and / or any combination of wireless and wired networks. The processor 62 and the memory 61 can also communicate with each other via a system bus, which is not limited in this disclosure.

[0149] It should be noted that Figure 6 The components of the imaging device 6 shown are merely exemplary and not limiting; the imaging device 6 may have other components as needed for the actual application. The processor 62 can control other components in the imaging device 6 to perform desired functions.

[0150] The imaging device 6 can be implemented by software, firmware and / or hardware, and can be integrated into a device with the relevant application installed.

[0151] This disclosure also provides an imaging system, including: an imaging device as described in any embodiment of this disclosure; a radiation source for emitting radiation toward the sample to be tested; a first probe for acquiring fluoroscopic data of the radiation passing through the sample to be tested; and a second probe for acquiring ultrasonic signal data emitted by the sample to be tested after being excited by the radiation.

[0152] Figure 7 A schematic diagram of an imaging system according to some embodiments of the present disclosure is shown. Figure 7 As shown, the imaging system may include: a radiation source 71, a sample to be tested 72, a first probe 73, a second probe 74, and an imaging device 75.

[0153] X-ray source 71, for example, can generate ultrashort pulses of X-rays. X-rays are used to achieve X-ray imaging of the sample 72 under test. A first probe 73 can acquire X-ray transmission data, and a second probe 74 can acquire ultrasonic data emitted by the sample after it is excited by X-rays. This data can be collected by the probes and sent to the imaging device 75 for sample imaging and system control.

[0154] like Figure 7 As shown, the second probe 74 can be deployed on one side of the sample and consists of a planar array of multiple broadband ultrasonic sensors. It can be deployed close to the sample or filled with a conductive coupling medium between the ultrasonic sensor array and the sample to detect ultrasonic waves generated by X-ray energy deposition.

[0155] In some embodiments, the second probe 74 can also be deployed around the sample to be tested, for example, by a ring array of multiple broadband ultrasonic sensors, so as to collect ultrasonic waves generated by the sample from all directions and obtain more accurate ultrasonic data.

[0156] In some embodiments, the imaging system may also include a simulation device, for example configured to determine a first predicted value of the X-ray fluoroscopy and a second predicted value of the ultrasound signal based on the material and geometry of the sample under test by simulating X-ray fluoroscopy and sound wave propagation.

[0157] In some embodiments, the imaging system may further include a collimation and focusing unit, for example configured to adjust the collimation of the rays according to the control of the imaging device.

[0158] In some embodiments, the imaging system may further include a synchronous triggering and acquisition system, for example configured to achieve high-precision synchronization of X-ray pulse emission and ultrasound signal acquisition.

[0159] The above is an imaging system provided by the embodiments of this disclosure. It performs X-ray imaging and photoacoustic imaging simultaneously using a single ray, and then couples the images generated by different methods. At the same time, it improves the accuracy of X-ray imaging by utilizing the high penetrability of the ray itself and the high resolution of the X-ray-excited ultrasound signal.

[0160] Figure 8 This is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0161] like Figure 8 As shown, the computer system 8 can be represented in the form of a general computing device. The computer system 8 includes a memory 81, a processor 82, and a bus 80 connecting different system components.

[0162] The memory 81 can be various forms of computer-readable storage media, such as system memory, non-volatile storage media, etc. System memory may store, for example, an operating system, application programs, a bootloader, and other programs. System memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. Non-volatile storage media may store, for example, instructions for performing corresponding embodiments of the imaging method. Non-volatile storage media include, but are not limited to, disk storage, optical storage, flash memory, etc.

[0163] The processor 82 can be implemented using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete hardware components such as discrete gates or transistors. Accordingly, each module can be implemented by executing instructions in the central processing unit (CPU) memory to perform the corresponding steps, or by implementing dedicated circuitry to perform the corresponding steps.

[0164] Bus 80 can use any of the various bus architectures. For example, bus architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, and the Peripheral Component Interconnect (PCI) bus.

[0165] The computer system 8 may also include an input / output interface 83, a network interface 84, and a storage interface 85. These interfaces 83, 84, and 85, as well as the memory 81 and processor 82, can be connected via a bus 80. The input / output interface 83 provides a connection interface for input / output devices such as a monitor, mouse, and keyboard. The network interface 84 provides a connection interface for various networked devices. The storage interface 85 provides a connection interface for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0166] According to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product that, when run on a computer, causes the computer to implement the imaging method described in any of the foregoing embodiments. The computer program product includes computer instructions carried on a computer-readable medium, the computer instructions containing program code for performing the methods shown in the flowcharts.

[0167] Various embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions disclosed herein based on the above description.

[0168] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. An imaging method, comprising: A first image of the sample is generated based on the perspective data of the rays passing through the sample. A second image of the sample is generated based on the ultrasonic signal data emitted by the sample after it is excited by the X-rays. An image of the sample to be tested is generated based on the first image and the second image.

2. The imaging method according to claim 1, wherein, Generating a second image of the sample to be tested based on the ultrasonic signal data emitted after the sample is excited by the radiation includes: Using a detector corresponding to the sample to be tested, the ultrasonic signal data emitted by the sample after being excited by the radiation is obtained. The frequency range of the ultrasonic signal generated by different samples to be tested is different. A second image of the sample to be tested is generated based on the ultrasonic signal data.

3. The imaging method according to claim 2, wherein, Using a detector corresponding to the sample under test, acquiring ultrasonic signal data emitted by the sample under test after being excited by the radiation includes: Based on the characteristic length of the sample to be tested and the waveform of the radiation, determine the frequency range corresponding to the sample to be tested; Based on the frequency range, determine the detector corresponding to the sample to be tested.

4. The imaging method according to claim 2, wherein, The waveform of the ray is a Gaussian waveform.

5. The imaging method according to claim 1, further comprising: During the imaging process, at least one of the intensity, duration, and collimation of the ray is adjusted based on the fluoroscopic data and the ultrasonic signal data.

6. The imaging method according to claim 5, wherein, Adjusting at least one of the intensity, duration, and collimation of the X-ray based on the fluoroscopic data and the ultrasound signal data includes: Based on the material and shape of the sample to be tested, determine the first predicted value of X-ray fluoroscopy and the second predicted value of X-ray-excited ultrasonic signal; Based on the first predicted value, the second predicted value, the fluoroscopic data, and the ultrasound signal data, at least one of the following is adjusted: the intensity, duration, and collimation of the ray.

7. The imaging method according to claim 1, wherein, Generating the image of the sample to be tested based on the first image and the second image includes: A first weight for the first image and a second weight for the second image are determined based on at least one of the gradient and contrast of the first image and the second image. Based on the first weight and the second weight, the first image and the second image are weighted and fused to generate the image of the sample to be tested.

8. The imaging method according to claim 7, wherein, The first image includes multiple first sub-images, and the second image includes multiple second sub-images that correspond one-to-one with the multiple first sub-images. The first image and the second image are weighted and fused according to the first weight and the second weight to generate the image of the sample to be tested, including: Based on the first weight and the second weight, the i-th first sub-image and the i-th second sub-image are weighted and fused to generate the i-th fused sub-image, where i is a positive integer; An image of the sample to be tested is generated based on multiple fused sub-images.

9. The imaging method according to claim 8, wherein, The plurality of first sub-images have different resolutions, the plurality of second sub-images have different resolutions, and the i-th first sub-image and the i-th second sub-image have the same resolution. Determining the first weight of the first image and the second weight of the second image includes: The plurality of first sub-images and the plurality of second sub-images are sorted according to their resolution to determine the first image sequence and the second image sequence; The gradient and local contrast of the plurality of first sub-images are determined based on the difference between adjacent first sub-images in the first image sequence. The gradient and local contrast of the plurality of second sub-images are determined based on the difference between adjacent second sub-images in the second image sequence. The first weight and the second weight are determined based on the gradient and local contrast of the plurality of first sub-images and the gradient and local contrast of the plurality of second sub-images.

10. The imaging method according to claim 1, wherein, The intensity and duration of the radiation are determined based on at least one of the safety threshold, excitation threshold, and transparency coefficient of the sample to be tested.

11. The imaging method according to claim 1, wherein: The first image is an X-ray image of the sample to be tested; The second image is an ultrasonic image of the sample to be tested.

12. An imaging device, comprising: The first image generation module is configured to generate a first image of the sample under test based on the perspective data of the rays passing through the sample under test. The second image generation module is configured to generate a second image of the sample under test based on the ultrasonic signal data emitted by the sample under test after it is excited by the X-ray. The sample image generation module is configured to generate an image of the sample to be tested based on the first image and the second image.

13. An imaging device, comprising: At least one memory; as well as At least one processor coupled to the at least one memory, the at least one processor being configured to perform the imaging method as described in any one of claims 1 to 11 based on instructions stored in the at least one memory.

14. An imaging system, comprising: The imaging apparatus as described in claim 12 or 13; A radiation source, used to emit the radiation toward the sample to be tested; The first probe is used to collect the transmission data of the rays passing through the sample to be tested; The second probe is used to collect ultrasonic signal data emitted by the sample under test after it is excited by the radiation.

15. The imaging system according to claim 14, wherein, The second probe is deployed around the sample to be tested.

16. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the imaging method as described in any one of claims 1 to 11.

17. A computer program product, when run on a computer, causes the computer to implement the imaging method as described in any one of claims 1 to 11.

Images