Data processing method, absorption coefficient image generation method, image generation method, and program product

Abstract

Provided is a data processing method for generating a test image from an object image that is an estimated object using a machine learning model. A test image is input to the machine learning model for estimation. The machine learning model is subjected to machine learning processing that sets the base image and the output image as learning data in pairs. A data processing method generates a test image by performing adjustment processing on a target image based on features of a base image.

Description

Technical Field

[0001] The present invention relates to data processing related to test images input to a machine learning model. Background Art

[0002] When applying test data to a machine learning model trained using learning data to obtain inferred results, if there is a qualitative difference called a domain gap between the environment in which the learning data was collected and the environment in which the test data was collected, the machine learning model will basically not be able to perform as expected on the test data.

[0003] As a method of reducing the domain gap, there are, for example, transfer learning and domain adaptation. In transfer learning, additional learning (fine-tuning) is performed on a machine learning model that has been trained using data collected in a learning environment, using data (data with true values) collected in a testing environment.

[0004] Domain adaptation is a method of learning using both data collected in a learning environment and data collected in a test environment. Specific examples of methods used in domain adaptation include methods using pseudo labels, methods using generative models, and methods that convert the domain of learning data to the domain of the test environment. This method does not require the addition of true value labels to the data in the test environment in advance.

[0005] In the method using pseudo labels, for example, as described in Asymmetric Tri-training for UnsupervisedDomain Adaptation [URL: https: / / arxiv.org / pdf / 1702.08400.pdf], the data of the test environment is first tested using a machine learning model learned using the learning data, and the result is used as a pseudo-true value, and machine learning processing is performed using both the learning data and the data collected in the test environment.

[0006] In the method using a generative model, for example, as described in Adapting Visual Category Models to NewDomains (K.Saenko et al.2010), [URL: https: / / people.bu.edu / bkulis / pubs / saenko_eccv_2010.pdf] or Deep Domain Confusion: Maximizing for Domain Invariance (E.Tzeng et al.2014), [URL: https: / / arxiv.org / pdf / 1412.3474.pdf], machine learning processing is performed in a domain-independent manner using both learning data and data collected in a test environment, and a feature extractor is used that makes the distribution of feature quantities obtained from the learning data and the data collected in a test environment consistent.

[0007] In a method of converting the domain of learning data into the domain of data of a test environment, as described in FDA: Fourier Domain Adaptation for Semantic Segmentation (Y. Yang et al. 2020), [URL: arxiv.org / pdf / 2004.05498.pdf], the domain information of the test environment is used to convert the domain of the learning data into the domain of the test environment, and machine learning processing is implemented using the converted learning data. Summary of the invention

[0008] In both the transfer learning and domain adaptation, a large amount of data collected in the test environment needs to be prepared. Furthermore, when there are multiple test environments, data needs to be prepared for each test environment. Both the transfer learning and domain adaptation require trial and error work.

[0009] The present invention has been made in view of such actual conditions, and an object of the present invention is to provide a technology for alleviating the burden on the user side for reducing the domain gap.

[0010] A data processing method according to one scheme of the present disclosure is a data processing method for generating a test image, which is applied to a machine learning model. The machine learning model is a machine learning model obtained by implementing machine learning processing using a basic image. The data processing method comprises: a step of acquiring an object image in a domain different from a domain in which the basic image is acquired; and a step of generating a test image by implementing adjustment processing on the object image based on features corresponding to the domain in which the basic image is acquired.

[0011] An absorption coefficient image generation method according to one embodiment of the present disclosure comprises: a step of generating an intermediate image including an image related to a tissue region based on a test image generated according to the above-mentioned data processing method; and a step of generating an absorption coefficient image based on the intermediate image and a known absorption coefficient of the tissue region.

[0012] An image generation method according to one scheme of the present disclosure is an image generation method using a machine learning model, in which machine learning processing using basic data is performed on the machine learning model, and the image generation method comprises: a step of acquiring an object image acquired in a domain different from a domain in which the basic data is acquired; a step of generating a test image by performing adjustment processing on the object image based on features corresponding to the domain in which the basic data is acquired; and a step of acquiring an output image by applying the test image to the machine learning model.

[0013] A program product according to one aspect of the present disclosure includes a program that is executed by one or more processors of a computer to cause the computer to implement the above-mentioned data processing method, absorption coefficient image generating method, or image generating method.

[0014] The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The diagram shows the structure of a PET (Positron Emission Tomography) apparatus 1 .

[0016] Figure 2 It is a diagram showing the structure of the radiation detector 3 .

[0017] Figure 3 This is a flowchart of the radiation energy distribution image generation process.

[0018] Figure 4 FIG. 1 is a diagram schematically showing the flow of the radiation energy distribution image generation process.

[0019] Figure 5 This is a diagram used to illustrate an example of generating an intermediate image using a machine learning model.

[0020] Figure 6 This is a diagram for explaining an example of generating an absorption coefficient image through conversion processing.

[0021] Figure 7 This is a diagram for illustrating an example of training of the machine learning model 8.

[0022] Figure 8 This is a diagram for explaining an example of adjustment processing.

[0023] Fig. 9 This is a diagram for explaining another example of the adjustment process.

[0024] Fig.10 It is a diagram for explaining another example of the adjustment process.

[0025] Fig.11 It is a diagram for explaining another example of the adjustment process.

[0026] Fig.12 It is a diagram for explaining another example of the adjustment process.

[0027] Fig.13 This is a diagram for more specifically explaining the adjustment process using the pre-processing mask 640. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, the same reference numerals are given to the same or corresponding parts in the drawings, and their description will not be repeated.

[0029] In the present disclosure, an image after adjustment processing is used as a test image input to a machine learning model. Furthermore, in the present disclosure, as a simple example of an image used to generate a test image, an object image generated using measurement data of a PET device is used. In addition, the adjustment processing involved in the present disclosure is not limited to images generated based on measurement data of a PET device, but can also be applied to any type of image input to a machine learning model (for example, CT (Computed Tomography) images, MRI (Magnetic Resonance Imaging) images, X-ray images, ultrasonic images, microscope images, mass analysis images).

[0030] [Structure of PET device]

[0031] Figure 1 is a diagram showing the structure of the PET apparatus 1. Figure 1 As shown, the PET apparatus 1 is an apparatus for imaging the subject 100 by detecting radiation (gamma rays) generated from the subject 100 due to a radioactive drug administered in advance to the subject 100. The subject 100 is, for example, a human. The radiation (gamma rays) is annihilation radiation generated in the subject 100 due to annihilation of a positron generated from the radioactive drug and an electron possessed by an atom near the positron.

[0032] The PET apparatus 1 is configured to generate a radiation energy distribution image of the subject 100 based on the imaging result of the subject 100. The PET apparatus 1 can image the whole body of the subject 100 or a part of the subject 100 (breast, head, etc.).

[0033] The PET apparatus 1 includes a detector ring 2 surrounding the subject 100. The detector ring 2 is provided in a manner of stacking multiple layers in the body axis direction of the subject 100. A plurality of radiation (gamma ray) detectors 3 are provided inside the detector ring 2. Thus, the detector ring 2 is configured to detect radiation (gamma ray) generated from a radiopharmaceutical in the subject 100.

[0034] The PET device 1 includes a controller 4. In one implementation example, the controller 4 is implemented by a general information processing device having a processor and a memory for non-temporarily storing a given program and / or data. The controller 4 includes a synchronous counting circuit 40 and a processing circuit 41. The synchronous counting circuit 40 and the processing circuit 41 can each be implemented as a dedicated hardware circuit, or can be implemented by the processor executing a given program. In addition, the controller 4 can also be implemented by distributed processing based on multiple computers. At least a part of the multiple computers can also include a computer configured away from the PET device 1. The above program can be circulated as a program product including the above program.

[0035] exist Figure 1 In FIG. 1 , only the wiring from two radiation detectors 3 to the controller 4 (synchronous counting circuit 40 ) is shown, but in reality, the controller 4 is connected to each of the plurality of radiation detectors 3 .

[0036] Figure 2 3 is a diagram showing the structure of the radiation detector 3. Figure 2 The structure of the PET apparatus 1 will be described in further detail.

[0037] like Figure 2 As shown, the radiation detector 3 includes a scintillator module 31, a light guide 32, and a photomultiplier tube (PMT) 33. In the radiation detector 3, the light guide 32 may be omitted. In the radiation detector 3, a sensor other than PMT, such as SiPM (Silicon Photomultiplier) or the like may be used.

[0038] The scintillator module 31 converts radiation (gamma rays) from the subject 100 to which the radiopharmaceutical is administered into light. When the radiopharmaceutical is administered to the subject 100, positrons of the positron emission type RI are annihilated, thereby generating two beams of radiation (gamma rays). Each scintillator element constituting the scintillator module 31 converts radiation (gamma rays) from the subject 100 into light.

[0039] The light guide 32 is optically coupled to the scintillator block 31 and the photomultiplier tube 33 . In the scintillator block 31 , light generated in the scintillator element is diffused, and the light is input to the photomultiplier tube 33 via the light guide 32 .

[0040] The photomultiplier tube 33 multiplies the light inputted via the light guide 32 and converts it into an electric signal. The electric signal is sent to the synchronous counting circuit 40.

[0041] The synchronous counting circuit 40 generates detection signal data (count value) based on the electric signal transmitted from the photomultiplier tube 33 .

[0042] More specifically, the synchronous counting circuit 40 checks the position of the scintillator module 31 and the incident timing of the radiation (gamma ray), and determines the transmitted electrical signal as appropriate data only when the radiation (gamma ray) is incident on two scintillator modules 31 located on both sides of the object 100 (on the diagonal line centered on the object 100) at the same time. That is, the synchronous counting circuit 40 detects that the radiation (gamma ray) is simultaneously observed (i.e., synchronously counted) in the two radiation detectors 3 located on both sides of the object 100 (on the diagonal line centered on the object 100) based on the above-mentioned electrical signal.

[0043] The detection signal data (count value) composed of appropriate data determined by the synchronous counting circuit 40 as synchronous count is sent to the processing circuit 41. The processing circuit 41 generates a radiation energy distribution image in the subject 100 based on the detection of radiation (gamma rays) by the detector ring 2.

[0044] [Generation of radiation energy distribution images]

[0045] Next, refer to Figure 3 , the radiation energy distribution image generation process of the PET device 1 is described. Figure 3 is a flowchart of the radiation energy distribution image generation process. Figure 4 . Figure 4 FIG. 1 is a diagram schematically showing the flow of the radiation energy distribution image generation process.

[0046] The radiation energy distribution image generation process is performed by the processing circuit 41 of the controller 4. In one implementation example, the radiation energy distribution image generation process is started when an operation for starting the process is performed in the PET apparatus 1.

[0047] In step S10, the processing circuit 41 acquires measurement data ( Figure 4 "Measurement Data 4X").

[0048] In step S20, the processing circuit 41 generates an object image ( Figure 4 of “object image 5”).

[0049] The target image may include at least one of the following images: a three-dimensional image, an axial cross-sectional image, a coronal cross-sectional image, a sagittal cross-sectional image, a patch image obtained by cutting out a partial area from a three-dimensional image, a patch image obtained by cutting out a partial area from an axial cross-sectional image, a patch image obtained by cutting out a partial area from a coronal cross-sectional image, and a patch image obtained by cutting out a partial area from a sagittal cross-sectional image.

[0050] A "cross-sectional image" refers to a two-dimensional image of one slice. An "axial cross-sectional image" refers to an image of a cross section orthogonal to the body axis. A "coronal cross-sectional image" refers to an image of a transverse cross section parallel to the body axis. A "sagittal cross-sectional image" refers to an image of a longitudinal cross section parallel to the body axis. The target image may be an image of only one slice or a collection of images of several consecutive slices.

[0051] As the imaging process in step S20, for example, there can be cited an imaging process based on histogramming, an imaging process based on machine learning, or a process including back-projection processing. As an imaging process based on histogramming, a method of adding an event at a position with the highest probability based on TOF (Time Of Flight) information contained in the measurement data can be adopted for imaging. As an imaging process based on machine learning, a method of using a machine learning model that converts measurement data into an object image for imaging can be adopted. As a process including back-projection processing, for example, a simple back-projection process and a reconstruction process can be adopted.

[0052] In step S20, the object image may be generated without performing at least one of the absorption correction process and the scatter correction process. The absorption correction process is a process for correcting the absorption of radiation in the object 100. The scatter correction process is a process for correcting the scattering of radiation in the object 100. In step S20, a non-corrected object image may be generated based on the measurement data without performing at least one of the absorption correction process and the scatter correction process.

[0053] In step S20, image quality conversion processing may not be performed, image quality conversion processing may be performed, or region recognition processing may be performed. In the present embodiment, the target image may also include at least one of an image to which image quality conversion processing is not applied, an image to which image quality conversion processing is applied, and an image to which region recognition processing is applied. As image quality conversion processing, for example, γ correction processing, histogram flattening processing, smoothing processing, and edge detection processing can be used. As image quality conversion processing, processing of applying random noise with a distribution such as uniform distribution, normal distribution, Poisson distribution, and Laplace distribution may also be used. As image quality conversion processing, processing of making the entire image or a specific region of the image a constant multiple may also be used. As region recognition processing, processing of recognizing the outline of the subject 100 in the image may also be used.

[0054] In step S30, the processing circuit 41 performs adjustment processing on the target image, thereby generating a test image ( Figure 4 For more information on the adjustment process for the target image, refer to Figure 8 to Figure 13 Described below.

[0055] Through step S40, the processing circuit 41 applies the test image to the machine learning model for the intermediate image ( Figure 4 The “machine learning model 8”) generates an intermediate image ( Figure 4 The intermediate image is an example of the “inferred result” of the machine learning model.

[0056] As the machine learning model 8, a known model (for example, the “machine learning model 8” described in International Publication No. WO2021 / 260928) can be adopted.

[0057] The machine learning model 8 is a machine learning model that takes the test image 6 as input and takes the intermediate image 7 as output. The machine learning model 8 includes at least one of a machine learning model that takes a three-dimensional image as input, a machine learning model that takes an axial cross-sectional image as input, a machine learning model that takes a coronal cross-sectional image as input, a machine learning model that takes a sagittal cross-sectional image as input, a machine learning model that takes a block image cut out from a three-dimensional image as input, a machine learning model that takes a block image cut out from an axial cross-sectional image as input, a machine learning model that takes a block image cut out from a coronal cross-sectional image as input, and a machine learning model that takes a block image cut out from a sagittal cross-sectional image as input.

[0058] Figure 5 This is a diagram for explaining an example of generating an intermediate image using a machine learning model. Figure 5, for ease of explanation, an example is shown in which a machine learning model outputs an intermediate image corresponding to an axial cross-sectional image based on an input of an axial cross-sectional image as a test image.

[0059] The intermediate image 7 is composed of a combination of N (a finite number) of tissues, such as the brain, bones, skin, muscles, and internal organs, whose absorption coefficients are known. For example, when the measurement data 4X is measurement data of a person's head, the unit (tissue) constituting the image related to the tissue region of the intermediate image 7 includes at least one of the background (outside the subject), cavity (nasal cavity and oral cavity, etc.), soft tissue (brain and skin, etc.), and bone (skull). In addition, for example, when the measurement data 4X is measurement data of a person's breast, the unit (tissue) constituting the image related to the tissue region of the intermediate image 7 includes at least one of the background (outside the subject) and soft tissue.

[0060] like Figure 5 As shown, the intermediate image 7 includes a tissue composition ratio image 71 showing the ratio of tissues contained in each pixel as an image related to the tissue region. The tissue composition ratio image 71 is a multi-channel image in which the ratios of multiple tissues contained in each pixel are used as pixel values. Figure 5 In the example shown, the tissue composition ratio image 71 is an image of a human head, including 4-channel images of background, cavity, soft tissue and bone. The image of the background channel is composed of the proportion of the background contained in each pixel as a pixel value. In addition, the image of the cavity channel is composed of the proportion of the cavity contained in each pixel as a pixel value. In addition, the image of the soft tissue channel is composed of the proportion of the soft tissue contained in each pixel as a pixel value. In addition, the image of the bone channel is composed of the proportion of the bone contained in each pixel as a pixel value. In addition, the pixel values ​​of each image of the 4 channels show the ratio, so the sum of the pixel values ​​of each pixel of the 4 channel images is 1.

[0061] Back to Figure 3 and Figure 4 , through step S50, the processing circuit 41 performs conversion processing on the intermediate image, thereby generating an absorption coefficient image ( Figure 4 “Absorption coefficient image 9”).

[0062] Figure 6 : is a diagram for explaining an example of generating an absorption coefficient image by conversion processing. An example of conversion processing is a linear combination of a tissue composition ratio image 71 using a known absorption coefficient. In this example, each tissue in the tissue composition ratio image 71 is combined with the absorption coefficient of each tissue, thereby generating an absorption coefficient image 9. More specifically, the linear combination processing of the tissue composition ratio image 71 of each tissue using the known absorption coefficient as a coefficient is performed by the following formula (1).

[0063]

[0064] n: Organization label (organization number)

[0065] j: pixel number

[0066] μ j : Absorption coefficient of pixel j

[0067] μ * n : Absorption coefficient of tissue n (known absorption coefficient)

[0068] r nj : Composition ratio of tissue n in pixel j

[0069] In addition, the "composition ratio of the tissue n of the pixel j" used in the formula (1) satisfies the following formula (2).

[0070]

[0071] For example, when the tissue composition ratio image 71 is an image of a human head and includes a four-channel image of background, cavity, soft tissue and bone, the processing circuit 41 uses the commonly known absorption coefficient μ*0 of the background, the absorption coefficient μ*1 of the cavity, the absorption coefficient μ*2 of the soft tissue and the absorption coefficient μ*3 of the bone, and implements linear combination processing of the tissue composition ratio image 71 of each tissue using the known absorption coefficients as coefficients according to the above-mentioned formula (1).

[0072] return Figure 3 and Figure 4 , through step S60, the processing circuit 41 generates a radiation energy distribution image ( Figure 4 In the reconstruction process, at least one of the absorption correction process and the scatter correction process is performed based on the absorption coefficient image 9. In one implementation example, in step S60, the absorption correction process is performed based on the absorption coefficient image 9, and the scatter correction process is performed based on the scatter distribution data acquired based on the absorption coefficient image 9 and the measurement data 4X. In step S60, a quantitative radiation energy distribution image 10 based on the absorption correction process and the scatter correction process is generated.

[0073] Strictly speaking, the absorption coefficient image 9 used in the reconstruction process is an image obtained by performing the processing of eliminating the content of the adjustment process on the absorption coefficient image generated by the conversion process on the intermediate image 7 .

[0074] For example, assume that the adjustment process is a 10° clockwise rotation relative to the target image 5. In this case, the absorption coefficient image 9 used in the reconstruction process is an image obtained by performing a 10° counterclockwise rotation on the absorption coefficient image generated by the conversion process on the intermediate image 7.

[0075] It is assumed that the adjustment process is a shift of 5 pixels in the first direction relative to the target image 5. In this case, the absorption coefficient image 9 used in the reconstruction process is an image obtained by performing a shift of 5 pixels in the second direction (a direction opposite to the first direction) on the absorption coefficient image generated by the conversion process on the intermediate image 7.

[0076] As the reconstruction process, for example, analytical reconstruction and / or successive approximate reconstruction can be used. As the analytical reconstruction process, for example, FBP (Filtered Back Projection) method can be used. As the successive approximate reconstruction process, for example, OSEM (Ordered Subsets Expectation Maximization) method can be used.

[0077] Afterwards, the processing circuit 41 makes Figure 3 The processing is finished.

[0078] [Machine learning model for intermediate images]

[0079] Next, the machine learning model 8 for the intermediate image used in the PET apparatus 1 will be described.

[0080] Figure 7 8 is a diagram for explaining an example of training of the machine learning model 8. Figure 7 In FIG. 8 , a pair of a basic image 6X and an intermediate image 7X is shown as learning data. The machine learning model 8 is generated by performing machine learning processing on the learning model. Figure 7 As shown, one example of the machine learning process to be performed is supervised learning using a plurality of pairs of basic images 6X and intermediate images 7X as learning data. In one implementation example, the machine learning process is performed by the processing circuit 41 of the controller 4.

[0081] The plurality of basic images 6X are each generated by performing imaging processing on the measurement data from the radiation detector 3, and are images corresponding to the above-mentioned "target image 5". The plurality of intermediate images 7X are each images corresponding to the above-mentioned "intermediate image 7".

[0082] The machine learning model 8 includes a deep neural network. The deep neural network of the machine learning model 8 includes convolution processing. That is, the machine learning model 8 includes a deep convolutional neural network. As the deep convolutional neural network of the machine learning model 8, for example, a U-shaped network (U-Net) with jump connections can be used. In addition, as an activation function of the deep convolutional neural network of the machine learning model 8, a Softmax function can be used.

[0083] [Adjustment processing for target images]

[0084] As reference Figure 3 As described above, a pre-processing called adjustment processing is performed on the image applied to the machine learning model 8. That is, the test image 6 input to the machine learning model 8 is generated by performing adjustment processing on the object image 5, and the object image 5 is generated as a target for estimation of the machine learning model 8 (generation of an intermediate image using the machine learning model 8). In the adjustment processing, the features of the basic image 6X used in the generation of the machine learning model 8 are used. A specific example of the adjustment processing is described below.

[0085] <Adjustment Processing (1)>

[0086] Figure 8 This is a diagram for explaining an example of adjustment processing.

[0087] In this example, the position of the head in the basic image 6X is used as a feature of the basic image 6X. More specifically, the position information 600 is generated as a feature of the basic image 6X. The position information 600 indicates the position after the center of gravity positions of the heads in the plurality of basic images 6X are statistically processed. An example of statistical processing is the derivation of the average position. For example, when the image is defined in the XY plane, the X coordinate of the position information 600 is the average value of the X coordinates of the N center of gravity positions, and the Y coordinate of the position information 600 is the average value of the Y coordinates of the N center of gravity positions. In addition, in the statistical processing, as long as the position information 600 indicates the feature of the position of the head in the plurality of basic images 6X, it is not limited to the derivation of the average position. It is also possible that the X coordinate of the position information 600 is the central value of the X coordinates of the N center of gravity positions, and the Y coordinate of the position information 600 is the central value of the Y coordinates of the N center of gravity positions.

[0088] In deriving the features, the region of the head in each basic image 6X may be determined by the user or by pattern recognition.

[0089] Furthermore, in the adjustment process of this example, Figure 8 As shown, a process of parallel shifting all pixels of the target image 5 is performed so that the center of gravity of the target image 5 is moved to the position specified in the position information 600 , thereby generating the test image 6 .

[0090] <Adjustment Processing (2)>

[0091] Fig. 9 This is a diagram for explaining another example of the adjustment process.

[0092] In this example, the shape of the head in the basic image 6X is used as a feature of the basic image 6X. More specifically, size information 610 is generated as a feature of the basic image 6X. The size information 610 indicates a value obtained by statistically processing the size of the head in a given direction in a plurality of basic images 6X. An example of statistical processing is the derivation of an average value. For example, the size information 610 is the average value of the size in a given direction in a plurality of basic images 6X. For example, in the case where an image is defined in an XY plane, the given direction is specified by an axis of a specified plane such as an X-axis or a Y-axis. In addition, as long as the size information 610 indicates the feature of the size in a plurality of basic images 6X, other types of processing such as the derivation of a central value may also be used as statistical processing.

[0093] Furthermore, in the adjustment process of this example, Fig. 9 As shown, the test image 6 is generated by enlarging or reducing the target image 5 so that the size of the head region in the target image 5 in the given direction becomes the value specified by the size information 610 .

[0094] In addition, when both the basic image 6X and the target image 5 are a set of images of a plurality of slices, the size information 610 may be a statistical value of the volume of the head specified by the images of the plurality of slices of the basic image 6X. In this case, in the adjustment process, each of the images of the plurality of slices of the target image 5 is enlarged or reduced so that the volume of the head specified by the images of the plurality of slices of the target image 5 is consistent with the statistical value specified by the size information 610, thereby generating a plurality of slices constituting the test image 6.

[0095] <Adjustment Processing (3)>

[0096] Fig.10 It is a diagram for explaining another example of the adjustment process.

[0097] In this example, as a feature of the basic image 6X, the posture of the head in the basic image 6X is used. More specifically, orientation information 620 is generated as a feature of the basic image 6X. The orientation information 620 indicates a value after statistical processing of the values ​​representing the orientations of the plurality of basic images 6X. An example of statistical processing is the derivation of an average value. For example, when each basic image 6X is defined in the XY plane, ellipse fitting is performed for each basic image 6X. Thus, an ellipse corresponding to the head is estimated. Then, the average value of the slope of the major axis of the ellipse estimated for each of the plurality of basic images 6X with respect to the X-axis or the Y-axis is determined as the orientation information 620. In addition, in the statistical processing, as long as the orientation information 620 represents the feature of the orientation of the head in the plurality of basic images 6X, it is not limited to the derivation of the average value.

[0098] Then, in the adjustment process of this example, Fig.10 As shown, ellipse fitting is performed on the target image 5 to derive the slope of the major axis of the estimated ellipse, and rotation processing is performed on the target image 5 so that the derived slope coincides with the slope indicated by the orientation information 620 , thereby generating the test image 6 .

[0099] <Adjustment Processing (4)>

[0100] Fig.11 It is a diagram for explaining another example of the adjustment process.

[0101] In this example, the contrast in the basic image 6X is used as a feature of the basic image 6X. More specifically, pixel value information 630 is generated as a feature of the basic image 6X. The pixel value information 630 specifies a statistical value of the frequency of each pixel value in the plurality of basic images 6X. An example of a statistical value is an average value, but other types of values ​​such as a median value may be used as long as the pixel value information 630 indicates a feature related to the contrast of the plurality of basic images 6X.

[0102] Furthermore, in the adjustment process of this example, Fig.11 As shown, the contrast of the target image 5 is adjusted so that the frequency distribution of the pixel values ​​of the target image 5 coincides with (or is close to) the distribution of the frequencies specified by the pixel value information 630 , thereby generating the test image 6 .

[0103] <Adjustment Processing (5)>

[0104] Fig.12 It is a diagram for explaining another example of the adjustment process.

[0105] In this example, the ratio of the pixel values ​​of the outer edge and the inner portion of the head in the basic image 6X is used as the feature of the basic image 6X. More specifically, as the feature of the basic image 6X, a pre-processing mask 640 is generated that is defined by the ratio of the pixel values ​​of the brain region of the head to the pixel values ​​of the skin region of the head. The pre-processing mask 640 includes a region 641 corresponding to the brain, a region 642 corresponding to the skin, and a region 643 representing a surplus portion of the outer edge of the head.

[0106] Fig.13 6 is a diagram for more specifically explaining the adjustment process using the pre-processing mask 640. The head mask 650 is a filter arranged at the position of the head in the test image 6. The pre-processing mask 640 is formed by combining the head mask 650, the filter unit 651 for the region 641 (a minimum filter of size 3), the filter unit 652 for the region 642 (a maximum filter of size 5), and the filter unit 653 for the region 643 (a maximum filter of size 7) so that their centers of gravity coincide. The inside of the filter unit 651 constitutes the region 641. The outside of the filter unit 651 and the inside of the filter unit 652 constitute the region 642. The outside of the filter unit 652 and the inside of the filter unit 653 constitute the region 643.

[0107] Fig.13 "X1.0", "Xk", and "X0.1" in ^ represent values ​​integrated with the pixel value of the target image 5 in the adjustment process.

[0108] More specifically, in Fig.13 In FIG. 5 , “X1.0” is indicated in the area 641 and the area 643. This means that in the adjustment process, the pixel values ​​of the pixels located in these areas in the target image 5 are set to 1.0 times.

[0109] exist Fig.13 In FIG. 6 , “X0.1” is indicated in the area outside the area 643. This means that in the adjustment process, the pixel values ​​of the pixels located in this area in the target image 5 are set to 0.1 times.

[0110] exist Fig.13 In FIG. 6 , “Xk” is indicated for the region 642. This means that in the adjustment process, the pixel values ​​of the pixels located in this region in the target image 5 are set to k times.

[0111] The value k is derived according to the following equation (3) using statistical values ​​of a plurality of basic images 6X.

[0112]

[0113] In formula (3), variable k is represented by the product of variable x and "MED(h) / MED(s)". MED(h) represents the median value of the pixel values ​​of the brain region in the plurality of basic images 6X. MED(s) represents the median value of the pixel values ​​of the skin region in the plurality of basic images 6X. The head region in each basic image 6X, and the skin region and brain region in the head can be determined by the user or by pattern recognition.

[0114] The variable x is a constant defined according to the type of radiopharmaceutical used in the measurement. In one implementation example, the value of the variable x is smaller when fluorodeoxyglucose (FDG) is used than when flutemetamol is used.

[0115] An upper limit and a lower limit may be set for the variable k. An example of the lower limit of the variable k is 1.0.

[0116] When the lower limit of the variable k is set to 1.0, the skin region is emphasized in the target image 5 in the adjustment process using the pre-processing mask 640, thereby generating the test image 6. In addition, the degree of emphasis depends on the relationship between the statistical value of the pixel value of the skin region and the statistical value of the pixel value of the brain region in the plurality of basic images 6X. More specifically, when the median value of the pixel value of the skin region in the plurality of basic images 6X is x times or less than the median value of the pixel value of the brain region, in the test image, the pixels of the skin region of the target image 5 are emphasized by setting the pixels to k times.

[0117] In the adjustment process using the pre-processing mask 640, the pixel value of the area outside the area 643 of the target image 5 is set to 0.1 times to generate the test image 6. Thus, the noise of the area other than the head in the target image 5 is removed by the adjustment process.

[0118] In the present embodiment described above, a test image is generated by performing image preprocessing (adjustment processing) applied to a machine learning model on a target image. Then, an intermediate image (an example of an estimated result) is generated by applying the test image to the machine learning model. In the adjustment processing, the features of the basic image used in the machine learning processing for training the machine learning model are used.

[0119] The features of the basic image correspond to the domain in which the basic image is acquired. For example, the position, size, and / or angle of the head in the basic image depends on the positional relationship between the bed (more specifically, the place where the head is placed on the bed) in the facility in which the basic image is acquired and the detector ring 2. In this case, an example of the domain includes the above-mentioned positional relationship in the facility. Furthermore, the features of the basic image correspond to the above-mentioned positional relationship in the facility in which the basic image is acquired.

[0120] By applying the test image generated by performing the adjustment process on the target image as described above to the machine learning model, an image on which the process for reducing the domain gap between the target image and the basic image is performed is applied to the machine learning model. The adjustment process does not require the user to collect a large amount of data in an environment where the target image is collected. Therefore, the data processing method according to the present disclosure can suppress the burden on the user side for reducing the domain gap.

[0121] The learning data used for training the machine learning model 8 is a pair of the basic image 6X and the intermediate image 7X which is a true value label for the basic image 6X. In the learning data, the type of data used in pairs with the true value label is not limited to images.

[0122] For example, the basic image 6X may be data of a type other than an image, and may be data (basic data) acquired in a domain different from the domain in which the target image 5 is acquired. As the basic data, for example, waveform data such as sound and spectrum can be considered. In this case, the noise amount can be considered as a feature of the basic data, and processing for adjusting the noise amount such as noise removal can be considered as the adjustment processing.

[0123] In such a case, Figure 3 In step S30 of the processing, the test image 6 is generated by performing adjustment processing based on the features of the basic data on the target image 5. In step S40, the intermediate image 7 is generated by applying the test image 6 to the machine learning model 8.

[0124] There is no restriction on the timing of generating the features of the basic image. The features of the basic image may also be generated in advance. In one implementation example, data defining the machine learning model and data defining the features of the basic image are pre-stored in the memory of the controller 4. The features of the basic image may also be generated when the intermediate image is generated in step S40.

[0125] [plan]

[0126] It should be understood by those skilled in the art that the above-mentioned multiple exemplary embodiments are specific examples of the following schemes.

[0127] (Item 1) A data processing method involved in one scheme is a data processing method for generating a test image applied to a machine learning model, wherein the machine learning model is a machine learning model obtained by implementing machine learning processing using a basic image, and the data processing method may include: a step of acquiring an object image of a domain different from a domain from which the basic image was acquired; and a step of generating the test image by performing adjustment processing on the object image based on features corresponding to the domain from which the basic image was acquired.

[0128] According to the data processing method described in claim 1, it is possible to reduce the burden on the user side for reducing the domain gap.

[0129] (Item 2) In the data processing method described in Item 1, it may also be that the object image includes an object, the feature is associated with the position of the object in the basic image, and the adjustment processing includes adjusting the position of the object in the test image.

[0130] According to the data processing method described in item 2, the domain gap related to the position of the object can be reduced.

[0131] (Item 3) In the data processing method described in Item 1 or Item 2, it may also be that the object image includes an object, the feature is associated with the shape of the object in the basic image, and the adjustment processing includes adjusting the shape of the object in the test image.

[0132] According to the data processing method described in Item 3, the domain gap related to the shape of the object can be reduced.

[0133] (Item 4) In the data processing method described in any one of Items 1 to 3, it may be that the object image includes an object, the feature is associated with the posture of the object in the basic image, and the adjustment processing includes adjusting the posture of the object in the test image.

[0134] According to the data processing method described in Item 4, the domain gap related to the posture of the object can be reduced.

[0135] (Item 5) In the data processing method described in any one of Items 1 to 4, the feature may be associated with the contrast in the basic image, and the adjustment processing includes adjusting the contrast in the test image.

[0136] According to the data processing method of item 5, the domain gap associated with the difference in contrast between the basic image and the object image can be reduced.

[0137] (Item 6) In the data processing method described in any one of Items 1 to 5, it may be that the object image includes an object, the feature is associated with the ratio of the pixel value between the outer edge and the inside of the object, and the adjustment processing includes emphasizing the outer edge of the object.

[0138] According to the data processing method described in item 6, the domain gap related to the ratio of the pixel value of the outer edge and the inside of the object can be reduced.

[0139] (Item 7) In the data processing method described in Item 6, the object may be an image of a human head, the outer edge corresponds to a skin area of ​​the head, and the inner portion corresponds to a brain area of ​​the head.

[0140] According to the data processing method described in Item 7, the domain gap associated with the head image of a person can be reduced.

[0141] (Item 8) In the data processing method described in Item 6 or Item 7, the adjustment processing may utilize a filter, and the filter is defined by a function of statistical values ​​of pixel values ​​of the outer edge of one or more basic images and statistical values ​​of pixel values ​​of the inner part of one or more basic images.

[0142] According to the data processing method described in Item 8, the domain gap associated with a person's head image can be reduced more reliably.

[0143] (Item 9) In the data processing method described in Item 1, the adjustment process may include removing noise in a region other than the object by using the filter.

[0144] According to the data processing method described in Item 9, a test image with noise removed can be input into the machine learning model.

[0145] (Item 10) In the data processing method described in any one of Items 1 to 9, the target image may be a PET (Positron Emission Tomography) image or a SPECT (Single Photon Emission Computed Tomography) image.

[0146] According to the data processing method described in Item 10, in estimation using PET images or SPECT images, it is possible to reduce the burden on the user side for reducing the domain gap.

[0147] (Item 11) A method for generating an absorption coefficient image according to one scheme may include: a step of generating an intermediate image including an image related to a tissue region based on a test image generated according to the data processing method described in Item 10; and a step of generating an absorption coefficient image based on the intermediate image and a known absorption coefficient of the tissue region.

[0148] According to the absorption coefficient image generation method described in Item 11, it is possible to reduce the burden on the user side for reducing the domain gap in the generation of the absorption coefficient image.

[0149] (Item 12) An image generation method involved in one scheme is an image generation method using a machine learning model, wherein the machine learning model is subjected to machine learning processing using basic data, and the image generation method comprises: a step of acquiring an object image acquired in a domain different from a domain in which the basic data was acquired; a step of generating a test image by performing adjustment processing on the object image based on features corresponding to the domain in which the basic data was acquired; and a step of acquiring an output image by applying the test image to the machine learning model.

[0150] According to the image generation method described in Item 12, it is possible to reduce the burden on the user side for reducing the domain gap.

[0151] (Item 13) A program according to one embodiment of the present invention can be executed by one or more processors of a computer to cause the computer to implement the data processing method according to any one of Items 1 to 10, the absorption coefficient image generating method according to Item 11, or the image generating method according to Item 12.

[0152] According to the procedure described in Item 13, it is possible to suppress the burden on the user side for reducing the domain gap.

[0153] The embodiments disclosed herein are to be considered as illustrative and non-restrictive in all respects. The scope of the present disclosure is not indicated by the description of the above-mentioned embodiments but by the claims, which means that all changes within the meaning and scope equivalent to the claims are included. In addition, each technology in the embodiments can be implemented alone, and can also be implemented in combination with other technologies in the embodiments as much as possible as needed.

Claims

1. A data processing method, which is a data processing method applied to a machine learning model for generating a test image, wherein the machine learning model is a machine learning model obtained by implementing machine learning processing using a basic image, characterized in that: The data processing method comprises: a step of acquiring an object image of a domain different from the domain from which the base image was acquired; and The step of generating the test image by performing adjustment processing on the target image based on the feature corresponding to the domain from which the basic image was acquired.

2. The data processing method according to claim 1, characterized in that: The object image includes an object. The feature is associated with the position of the object in the base image, The adjustment process includes adjusting a position of an object in the test image.

3. The data processing method according to claim 1, characterized in that: The object image includes an object. The feature is associated with the shape of the object in the basic image, The adjustment process includes adjusting the shape of the object in the test image.

4. The data processing method according to claim 1, characterized in that: The object image includes an object. The feature is associated with a posture of an object in the base image, The adjustment process includes adjusting the posture of the object in the test image.

5. The data processing method according to claim 1, characterized in that: The feature is associated with a contrast in the base image, The adjustment process includes adjusting contrast in the test image.

6. The data processing method according to claim 1, characterized in that: The object image includes an object. The feature is associated with the ratio of the pixel values ​​of the outer edge and the inner part of the object. The adjustment process includes emphasizing an outer edge of the object.

7. The data processing method according to claim 6, characterized in that: The object is a human head, The outer edge corresponds to the skin area of ​​the head, The interior corresponds to the brain area of ​​the head.

8. The data processing method according to claim 6, characterized in that: The adjustment process utilizes filters, The filter is defined by a function of statistical values ​​of pixel values ​​at the outer edge of one or more base images and statistical values ​​of pixel values ​​at the inner portion of one or more base images.

9. The data processing method according to claim 8, characterized in that: The adjustment process includes removing noise in a region other than the object by using the filter.

10. The data processing method according to claim 1, characterized in that: The object image is a PET image or a SPECT image, PET refers to positron emission tomography, and SPECT refers to single photon emission computed tomography.

11. A method for generating an absorption coefficient image, characterized in that: have: A step of generating an intermediate image including an image related to a tissue region based on a test image generated by the data processing method according to claim 10; and The step of generating an absorption coefficient image based on the intermediate image and the known absorption coefficient of the tissue region.

12. An image generation method is an image generation method using a machine learning model, characterized in that: The machine learning model is subjected to machine learning processing using basic data, The image generation method comprises: a step of acquiring an image of the object acquired in a domain different from the domain in which the basic data is acquired; A step of generating a test image by performing adjustment processing on the object image based on a feature corresponding to the domain from which the basic data is acquired; as well as The step of obtaining an output image by applying the test image to the machine learning model.

13. A program product comprising a program, characterized in that The program is executed by one or more processors of a computer, thereby causing the computer to implement the data processing method according to claim 1 , the absorption coefficient image generating method according to claim 11 , or the image generating method according to claim 12 .

Images