Data processing method, adsorption coefficient image generation method, image generation method, and program

Abstract

To provide a technology which suppresses a burden of a user side for reduction of a domain gap, in use of a machine learning model.SOLUTION: A data processing method for generating a test image 6 from an object image 5 which is an object of estimation using a machine learning model 8 is provided. The test image 6 is inputted to the machine learning model 8 for the estimation. The machine learning model 8 is subjected to machine learning processing which sets a pair of a basic image and an output image as learning data. A data processing method generates the test image 6, by performing adjustment processing based on a feature of the basic image, with respect to the object image 5.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to data processing relating to test images that are input to a machine learning model. [Background technology]

[0002] When estimation results are obtained by applying test data to a machine learning model trained on learning data, if there is a qualitative difference called a domain gap between the environment in which the learning data was collected and the environment in which the test data was collected, the machine learning model will basically not be able to perform as expected on the test data.

[0003] Approaches to reduce the domain gap include, for example, transfer learning and domain adaptation. In transfer learning, additional learning (fine-tuning) is performed on a machine learning model trained with data collected in a learning environment using data (with correct answers) collected in a test environment.

[0004] Domain adaptation is a method of learning using both data collected in a learning environment and data collected in a test environment. Specific examples of approaches used in domain adaptation include an approach using pseudo-labels, an approach using generative models, and an approach that converts the domain of the training data into the domain of the test environment. This method does not require that the data in the test environment be previously labeled with a correct answer.

[0005] In the approach using pseudo-labels, as described in, for example, Non-Patent Document 1, first, a machine learning model trained with training data is used to test data in a test environment, and the result is used as a pseudo-ground truth, and machine learning processing is performed using both the training data and data collected in the test environment.

[0006] In an approach using a generative model, as described in, for example, Non-Patent Document 2 or Non-Patent Document 3, machine learning processing is performed using both training data and data collected in a test environment, with a feature extractor that matches the distribution of features obtained from the training data and the data collected in a test environment, regardless of the domain.

[0007] In the approach of converting the domain of training data to the domain of data in a test environment, as described in Non-Patent Document 4, the domain of the training data is converted to the domain of the test environment using domain information of the test environment, and machine learning processing is performed using the converted training data. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Asymmetric Tri-training for Unsupervised Domain Adaptation, [URL:https: / / arxiv.org / pdf / 1702.08400.pdf] [Non-Patent Document 2] Adapting Visual Category Models to New Domains (K. Saenko et al. 2010), [URL:https: / / people.bu.edu / bkulis / pubs / saenko_eccv_2010.pdf] [Non-Patent Document 3] Deep Domain Confusion: Maximizing for Domain Invariance(E. Tzeng et al. 2014), [URL: https: / / arxiv.org / pdf / 1412.3474.pdf] [Non-Patent Document 4] FDA: Fourier Domain Adaptation for Semantic Segmentation(Y.Yang et al. 2020, [URL:arxiv.org / pdf / 2004.05498.pdf] Summary of the Invention [Problem to be solved by the invention]

[0009] Both the above-mentioned transfer learning and domain adaptation require the preparation of a large amount of data collected in a test environment. Furthermore, if there are multiple test environments, data preparation is required for each test environment. In the first place, both the above-mentioned transfer learning and domain adaptation require a trial-and-error task of learning.

[0010] The present invention has been devised in view of the above circumstances, and has an object to provide a technique for reducing the burden on the user side in order to reduce the domain gap. [Means for solving the problem]

[0011] A data processing method according to one aspect of the present disclosure is a data processing method for generating a test image that is applied to a machine learning model that has been subjected to machine learning processing using a base image, and includes the steps of acquiring a target image of a domain different from the domain from which the base image was acquired, and generating a test image by applying an adjustment process to the target image based on features corresponding to the domain from which the base image was acquired.

[0012] A method for generating an absorption coefficient image according to one aspect of the present disclosure includes generating an intermediate image including an image of a tissue region based on a test image generated according to the data processing method described above, and generating an absorption coefficient image based on the intermediate image and a known absorption coefficient of the tissue region.

[0013] An image generation method according to one aspect of the present disclosure is an image generation method that uses a machine learning model, where the machine learning model has been subjected to machine learning processing using basic data, and includes the steps of acquiring a target image acquired in a domain different from the domain from which the basic data was acquired, generating a test image by applying an adjustment process to the target image based on features corresponding to the domain from which the basic data was acquired, and acquiring an output image by applying the test image to the machine learning model.

[0014] A program according to an aspect of the present disclosure, when executed by one or more processors of a computer, causes the computer to perform the above-described data processing method, absorption coefficient image generating method, or image generating method. Effect of the Invention

[0015] According to an aspect of the present disclosure, there is provided a technique for reducing the burden on the user for reducing the domain gap. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing the configuration of a PET (Positron Emission Tomography) device 1. [Diagram 2] 2 is a diagram showing a configuration of a radiation detector 3. FIG. [Diagram 3] 13 is a flowchart of a radioactivity distribution image generating process. [Figure 4] FIG. 10 is a diagram illustrating a flow of a radioactivity distribution image generating process. [Diagram 5] FIG. 11 is a diagram for explaining an example of generation of an intermediate image using a machine learning model. [Figure 6] 11A and 11B are diagrams for explaining an example of generation of an absorption coefficient image by a conversion process. [Figure 7] FIG. 1 is a diagram for explaining an example of training of a machine learning model 8. [Figure 8] FIG. 11 is a diagram for explaining an example of an adjustment process. [Figure 9] FIG. 11 is a diagram for explaining another example of the adjustment process. [Figure 10] FIG. 11 is a diagram for explaining still another example of the adjustment process. [Figure 11] FIG. 11 is a diagram for explaining still another example of the adjustment process. [Figure 12] FIG. 11 is a diagram for explaining still another example of the adjustment process. [Figure 13] FIG. 13 is a diagram for explaining the adjustment process using the pre-processing mask 640 in more detail. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference characters and their description will not be repeated.

[0018] In the present disclosure, the image after the adjustment process is used as a test image to be input to the machine learning model. In the present disclosure, a target image generated using measurement data of a PET device is adopted as a mere example of an image to be used to generate a test image. Note that the adjustment process according to the present disclosure is not limited to an image generated from measurement data of a PET device, and can be applied to any type of image (for example, a computed tomography (CT) image, a magnetic resonance imaging (MRI) image, an X-ray image, an ultrasound image, a microscope image, a mass spectrometry image) to be input to the machine learning model.

[0019] [PET equipment configuration] Fig. 1 is a diagram showing the configuration of a PET device 1. As shown in Fig. 1, the PET device 1 is a device that images a subject 100 by detecting radiation (gamma rays) generated from within the subject 100 due to a radiopharmaceutical administered in advance to the subject 100. The subject 100 is, for example, a human. The radiation (gamma rays) is annihilation radiation generated within the subject 100 due to annihilation between a positron generated from the radiopharmaceutical and an electron possessed by an atom in the vicinity of the positron.

[0020] The PET device 1 is configured to generate a radioactivity distribution image of the subject 100 based on the imaging results of the subject 100. Note that the PET device 1 may image the entire body of the subject 100, or may image a part of the subject 100 (such as the breasts and head).

[0021] The PET device 1 includes a detector ring 2 surrounding the subject 100. The detector rings 2 are provided so as to be stacked in a plurality of layers in the body axis direction of the subject 100. A plurality of radiation (gamma ray) detectors 3 are provided inside the detector ring 2. As a result, the detector ring 2 is configured to detect radiation (gamma rays) generated from a radiopharmaceutical within the subject 100.

[0022] The PET device 1 includes a controller 4. In one implementation example, the controller 4 is realized by a general-purpose information processing device having a processor and a memory for non-temporarily storing a given program and / or data. The controller 4 includes a coincidence circuit 40 and a processing circuit 41. Each of the coincidence circuit 40 and the processing circuit 41 may be realized as a dedicated hardware circuit, or may be realized by a processor executing a given program. The controller 4 may be realized by distributed processing using multiple computers. At least a part of the multiple computers may include a computer located remotely from the PET device 1.

[0023] In FIG. 1, only wiring from two radiation detectors 3 to the controller 4 (coincidence counting circuit 40) is shown, but in reality, the controller 4 is connected to each of the multiple radiation detectors 3.

[0024] 2 is a diagram showing the configuration of the radiation detector 3. With further reference to FIG. 2, the configuration of the PET device 1 will be described in further detail.

[0025] 2, the radiation detector 3 includes a scintillator block 31, a light guide 32, and a photomultiplier tube (PMT: Photo Multiplier Tube) 33. In the radiation detector 3, the light guide 32 may be omitted. In the radiation detector 3, a sensor other than a PMT, such as a SiPM (Silicon Photomultiplier), may be used.

[0026] The scintillator block 31 converts radiation (gamma rays) from the subject 100 administered with a radiopharmaceutical into light. When the subject 100 is administered with a radiopharmaceutical, the positrons of the positron-emitting RI are annihilated, generating two rays of radiation (gamma rays). Each scintillator element constituting the scintillator block 31 converts the radiation (gamma rays) from the subject 100 into light.

[0027] The light guide 32 is optically coupled to each of the scintillator block 31 and the photomultiplier tube 33. In the scintillator block 31, light generated in the scintillator elements is diffused, and the light is input to the photomultiplier tube 33 via the light guide 32.

[0028] The photomultiplier tube 33 amplifies the light input via the light guide 32 and converts it into an electrical signal. This electrical signal is sent to the coincidence circuit 40.

[0029] The coincidence circuit 40 generates detection signal data (count value) based on the electrical signal transmitted from the photomultiplier tube 33.

[0030] More specifically, the coincidence circuit 40 checks the positions of the scintillator blocks 31 and the incidence timing of radiation (gamma rays), and determines that the transmitted electrical signal is proper data only when radiation (gamma rays) is simultaneously incident on two scintillator blocks 31 on both sides of the subject 100 (on a diagonal line centered on the subject 100). That is, the coincidence circuit 40 detects, based on the above-mentioned electrical signal, that radiation (gamma rays) has been simultaneously observed (i.e., counted simultaneously) in the two radiation detectors 3 on both sides of the subject 100 (on a diagonal line centered on the subject 100).

[0031] Detection signal data (count value) constituted by appropriate data determined to be coincidence by the coincidence circuit 40 is transmitted to a processing circuit 41. The processing circuit 41 generates an image of radioactivity distribution within the subject 100 based on the detection of radiation (gamma rays) by the detector ring 2.

[0032] [Generation of radioactivity distribution images] Next, a radioactivity distribution image generating process by the PET device 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart of the radioactivity distribution image generating process. In addition, Fig. 4 will also be referred to in the description of the radioactivity distribution image generating process. Fig. 4 is a diagram showing a schematic flow of the radioactivity distribution image generating process.

[0033] The radioactivity distribution image generating process is performed by the processing circuitry 41 of the controller 4. In one implementation example, the radioactivity distribution image generating process is started in response to an operation being performed in the PET device 1 to start the process.

[0034] In step S10, the processing circuitry 41 acquires measurement data from the radiation detector 3 based on the detection of radiation emitted from the subject 100 ("measurement data 4X" in FIG. 4).

[0035] In step S20, the processing circuitry 41 performs imaging processing on the measurement data to generate a target image ("target image 5" in FIG. 4).

[0036] The target image may include at least one of a three-dimensional image, an axial section image, a coronal section image, a sagittal section image, a patch image obtained by cutting out a partial area from a three-dimensional image, a patch image obtained by cutting out a partial area from an axial section image, a patch image obtained by cutting out a partial area from a coronal section image, and a patch image obtained by cutting out a partial area from a sagittal section image.

[0037] A "sectional image" refers to a two-dimensional image of one slice. An "axial image" refers to an image of a slice perpendicular to the body axis. A "coronal image" refers to an image of a transverse slice parallel to the body axis. A "sagittal image" refers to an image of a longitudinal slice parallel to the body axis. The target image may be an image of only one slice, or a set of images of several consecutive slices.

[0038] The imaging process in step S20 may be, for example, imaging by histogramming, imaging by machine learning, or a process including backprojection. The imaging process by histogramming may be a method of imaging by adding an event to the position with the highest probability based on TOF (Time Of Flight) information included in the measurement data. The imaging process by machine learning may be a method of imaging using a machine learning model that converts the measurement data into a target image. The process including backprojection may be, for example, a simple backprojection process and a reconstruction process.

[0039] In step S20, a target image may be generated without performing at least one of the absorption correction process and the scattering correction process. The absorption correction process is a process for correcting for absorption of radiation within the subject 100. The scattering correction process is a process for correcting for scattering of radiation within the subject 100. In step S20, an uncorrected target image may be generated from the measurement data without performing at least one of the absorption correction process and the scattering correction process.

[0040] In step S20, image quality conversion processing may not be performed, image quality conversion processing may be performed, or area classification processing may be performed. In this embodiment, the target image may include at least one of an image to which image quality conversion processing is not applied, an image to which image quality conversion processing is applied, and an image to which area classification processing is applied. As the image quality conversion processing, for example, a gamma correction process, a histogram flattening process, a smoothing process, and an edge detection process may be adopted. As the image quality conversion processing, a process of adding random noise of a distribution such as a uniform distribution, a normal distribution, a Poisson distribution, and a Laplace distribution may be adopted. As the image quality conversion processing, a process of multiplying the entire image or a specific area of ​​the image by a constant may be adopted. As the area classification processing, a process of identifying the contour of the subject 100 in the image may be adopted.

[0041] In step S30, processing circuitry 41 performs adjustment processing on the target image to generate a test image ("test image 6" in FIG. 4). The adjustment processing on the target image will be described later with reference to FIGS.

[0042] In step S40, the processing circuit 41 generates an intermediate image ("intermediate image 7" in FIG. 4) by applying the test image to a machine learning model for intermediate images ("machine learning model 8" in FIG. 4). The intermediate image is an example of an "estimation result" by the machine learning model.

[0043] As the machine learning model 8, a known model (for example, the “machine learning model 8” described in International Publication No. WO2021 / 260928) may be adopted.

[0044] The machine learning model 8 is a machine learning model that receives the test image 6 as an input and outputs the intermediate image 7. The machine learning model 8 includes at least one of a machine learning model that receives a three-dimensional image as an input, a machine learning model that receives an axial cross-sectional image as an input, a machine learning model that receives a coronal cross-sectional image as an input, a machine learning model that receives a sagittal cross-sectional image as an input, a machine learning model that receives a patch image cut out from a three-dimensional image as an input, a machine learning model that receives a patch image cut out from an axial cross-sectional image as an input, a machine learning model that receives a patch image cut out from a coronal cross-sectional image as an input, and a machine learning model that receives a patch image cut out from a sagittal cross-sectional image as an input.

[0045] Fig. 5 is a diagram for explaining an example of generation of an intermediate image using a machine learning model. For convenience of explanation, Fig. 5 shows an example in which the machine learning model outputs an intermediate image corresponding to an axial cross-sectional image in response to an input of the axial cross-sectional image as a test image.

[0046] Intermediate image 7 is composed of a combination of N (finite) tissues with known absorption coefficients, such as the brain, bone, skin, muscle, and internal organs. For example, if measurement data 4X is measurement data of a human head, elements (tissues) constituting an image relating to a tissue region of intermediate image 7 include at least one of background (outside the subject), cavities (such as the nasal cavity and oral cavity), soft tissue (such as the brain and skin), and bone (skull). Also, for example, if measurement data 4X is measurement data of a human breast, elements (tissues) constituting an image relating to a tissue region of intermediate image 7 include at least one of background (outside the subject) and soft tissue.

[0047] As shown in FIG. 5, the intermediate image 7 includes a tissue composition ratio image 71 indicating the ratio of tissue contained in each pixel as an image related to the tissue region. The tissue composition ratio image 71 is a multi-channel image in which the ratio of multiple tissues contained in each pixel is used as a pixel value. In the example shown in FIG. 5, the tissue composition ratio image 71 is an image of a human head, and includes four channel images of background, cavity, soft tissue, and bone. The background channel image is configured with the ratio of background contained in each pixel as a pixel value. The cavity channel image is configured with the ratio of cavity contained in each pixel as a pixel value. The soft tissue channel image is configured with the ratio of soft tissue contained in each pixel as a pixel value. The bone channel image is configured with the ratio of bone contained in each pixel as a pixel value. Since the pixel values ​​of each image of the four channels indicate a ratio, the sum of the pixel values ​​of the four channel images of each pixel is 1.

[0048] Returning to FIGS. 3 and 4, in step S50, the processing circuitry 41 performs a conversion process on the intermediate image to generate an absorption coefficient image ("absorption coefficient image 9" in FIG. 4).

[0049] 6 is a diagram for explaining an example of generation of an absorption coefficient image by conversion processing. One example of conversion processing is linear combination of tissue composition ratio images 71 using known absorption coefficients. In this example, each tissue in the tissue composition ratio image 71 is combined with the absorption coefficient of each tissue to generate an absorption coefficient image 9. More specifically, linear combination processing of the tissue composition ratio images 71 of each tissue is performed using the known absorption coefficient as a coefficient according to the following formula (1).

[0050]

number

[0051] The "composition ratio of tissue n of pixel j" used in formula (1) satisfies the following formula (2).

[0052]

number

[0053] For example, if the tissue composition ratio image 71 is an image of a human head and includes four channel images of background, cavity, soft tissue and bone, the processing circuitry 41 may calculate the background absorption coefficient μ * 0, the absorption coefficient of the cavity μ * 1. Soft tissue absorption coefficient μ * 2, and the bone resorption coefficient μ * 3, a linear combination process is performed on tissue composition ratio images 71 of each tissue using known absorption coefficients as coefficients according to the above formula (1).

[0054] 3 and 4, in step S60, the processing circuitry 41 generates a radioactivity distribution image ("radioactivity distribution image 10" in FIG. 4) by reconstruction processing based on the absorption coefficient image 9 and the measurement data 4X. In the reconstruction processing, at least one of absorption correction processing and scattering correction processing is performed based on the absorption coefficient image 9. In one implementation example, in step S60, absorption correction processing is performed based on the absorption coefficient image 9, and scattering correction processing is performed based on scattering distribution data acquired based on the absorption coefficient image 9 and the measurement data 4X. In step S60, a quantitative radioactivity distribution image 10 is generated by the absorption correction processing and scattering correction processing.

[0055] Strictly speaking, the absorption coefficient image 9 used in the reconstruction process is an image obtained by performing a process to cancel out the contents of the adjustment process on the absorption coefficient image generated by the conversion process on the intermediate image 7 .

[0056] For example, suppose that the adjustment process involves a 10° clockwise rotation of the target image 5. In this case, the absorption coefficient image 9 used in the reconstruction process is an image obtained by rotating the absorption coefficient image generated by the conversion process on the intermediate image 7 by 10° counterclockwise.

[0057] The adjustment process is assumed to involve a shift of five pixels in a first direction relative to the target image 5. In this case, the absorption coefficient image 9 used in the reconstruction process is an image obtained by shifting the absorption coefficient image generated by the conversion process on the intermediate image 7 by five pixels in a second direction (a direction representing the opposite direction to the first direction).

[0058] As the reconstruction process, for example, analytical reconstruction process and / or iterative reconstruction process may be adopted. As the analytical reconstruction process, for example, FBP (Filtered Back Projection) method may be adopted. As the iterative reconstruction process, for example, OSEM (Ordered Subsets Expectation Maximization) method may be adopted.

[0059] Thereafter, the processing circuit 41 ends the processing of FIG.

[0060] [Machine learning model for intermediate images] Next, the machine learning model 8 for intermediate images used in the PET device 1 will be described.

[0061] FIG. 7 is a diagram for explaining an example of training of a machine learning model 8. In FIG. 7, a pair of a base image 6X and an intermediate image 7X is shown as learning data. The machine learning model 8 is generated by performing a machine learning process on a learning model. As shown in FIG. 7, an example of the machine learning process is supervised learning that uses multiple pairs of a base image 6X and an intermediate image 7X as learning data. In one implementation example, the machine learning process is performed by a processing circuit 41 of the controller 4.

[0062] Each of the multiple basic images 6X is generated by performing imaging processing on the measurement data from the radiation detector 3, and is an image corresponding to the above-mentioned "target image 5." Each of the multiple intermediate images 7X is an image corresponding to the above-mentioned "intermediate image 7."

[0063] The machine learning model 8 includes a deep neural network. The deep neural network of the machine learning model 8 includes a convolution process. That is, the machine learning model 8 includes a deep convolutional neural network. As the deep convolutional neural network of the machine learning model 8, for example, a U-shaped network (U-Net) having skip connections can be adopted. In addition, as the activation function of the deep convolutional neural network of the machine learning model 8, a Softmax function can be adopted.

[0064] [Adjustment processing for target image] As described with reference to Fig. 3, a pre-processing called an adjustment process is performed on an image to be applied to the machine learning model 8. That is, the adjustment process is performed on a target image 5 generated as a target for estimation by the machine learning model 8 (generation of an intermediate image using the machine learning model 8), thereby generating a test image 6 to be input to the machine learning model 8. The adjustment process utilizes the features of the basic image 6X used in generating the machine learning model 8. A specific example of the adjustment process will be described below.

[0065] <Adjustment process (1)> FIG. 8 is a diagram for explaining an example of the adjustment process.

[0066] In this example, the position of the head in the basic image 6X is adopted as the feature of the basic image 6X. More specifically, position information 600 is generated as the feature of the basic image 6X. The position information 600 indicates a position obtained by statistically processing the center of gravity positions of the head in a plurality of basic images 6X. An example of the statistical processing is derivation of an average position. For example, when an image is defined on an XY plane, the X coordinate of the position information 600 is the average value of the X coordinates of the N center of gravity positions, and the Y coordinate of the position information 600 is the average value of the Y coordinates of the N center of gravity positions. Note that the statistical processing is not limited to derivation of an average position as long as the position information 600 represents the feature of the head position in a plurality of basic images 6X. The X coordinate of the position information 600 may be the median value of the X coordinates of the N center of gravity positions, and the Y coordinate of the position information 600 may be the median value of the Y coordinates of the N center of gravity positions.

[0067] In deriving the features, the head region in each base image 6X may be identified by a user or by pattern recognition.

[0068] In the adjustment process of this example, as shown in FIG. 8, a process of translating all pixels of the target image 5 is performed so as to move the center of gravity position of the target image 5 to a position specified in the position information 600, thereby generating a test image 6.

[0069] <Adjustment process (2)> FIG. 9 is a diagram for explaining another example of the adjustment process.

[0070] In this example, the shape of the head in the basic image 6X is adopted as the feature of the basic image 6X. More specifically, size information 610 is generated as the feature of the basic image 6X. The size information 610 indicates a value obtained by statistically processing the dimensions of the head in a given direction in a plurality of basic images 6X. An example of the statistical processing is derivation of an average value. For example, the size information 610 is the average value of the dimensions in a given direction in a plurality of basic images 6X. For example, when the image is defined in an XY plane, the given direction is defined by an axis that defines the plane, such as the X axis or the Y axis. Note that the statistical processing may be other types of processing, such as derivation of a median, as long as the size information 610 represents the size feature in a plurality of basic images 6X.

[0071] Then, in the adjustment process of this example, as shown in FIG. 9, the target image 5 is enlarged or reduced so that the dimension of the head area in the target image 5 in the given direction becomes the value specified in the size information 610, thereby generating a test image 6.

[0072] When both the basic image 6X and the target image 5 are sets of images for multiple slides, the size information 610 may be a statistical value of the head volume defined by the images for the multiple slides of the basic image 6X. In this case, in the adjustment process, the multiple slides constituting the test image 6 are generated by enlarging or reducing each of the images for the multiple slides of the target image 5 so that the head volume defined by the images for the multiple slides of the target image 5 matches the statistical value specified by the size information 610.

[0073] <Adjustment process (3)> FIG. 10 is a diagram for explaining still another example of the adjustment process.

[0074] In this example, the head posture in the basic image 6X is adopted as the feature of the basic image 6X. More specifically, orientation information 620 is generated as the feature of the basic image 6X. The orientation information 620 indicates a value obtained by statistically processing values ​​representing the orientations of the multiple basic images 6X. An example of the statistical processing is derivation of an average value. For example, when each basic image 6X is defined on an XY plane, ellipse fitting is performed for each basic image 6X. This estimates an ellipse corresponding to the head. Then, the average value of the inclination of the major axis of the ellipse estimated for each of the multiple basic images 6X with respect to the X-axis or Y-axis is specified as the orientation information 620. Note that the statistical processing is not limited to derivation of an average value as long as the orientation information 620 represents the feature of the head orientation in the multiple basic images 6X.

[0075] In the adjustment process of this example, as shown in FIG. 10, ellipse fitting is performed on the target image 5, the inclination of the long axis of the estimated ellipse is derived, and a rotation process is performed on the target image 5 so that the derived inclination matches the inclination represented by the orientation information 620, thereby generating a test image 6.

[0076] <Adjustment process (4)> FIG. 11 is a diagram for explaining still another example of the adjustment process.

[0077] In this example, contrast in the basic image 6X is adopted as the feature of the basic image 6X. More specifically, pixel value information 630 is generated as the feature of the basic image 6X. The pixel value information 630 specifies a statistical value of the frequency of each pixel value in the multiple basic images 6X. An example of the statistical value is the average value, but other types of values ​​such as a median may be used as long as the pixel value information 630 represents a feature related to the contrast of the multiple basic images 6X.

[0078] In the adjustment process of this example, as shown in FIG. 11, the contrast of the target image 5 is adjusted so that the frequency distribution of the pixel values ​​of the target image 5 matches (or approaches) the frequency distribution defined by the pixel value information 630, thereby generating a test image 6.

[0079] <Adjustment process (5)> FIG. 12 is a diagram for explaining still another example of the adjustment process.

[0080] In this example, the ratio of pixel values ​​between the outer edge and the inside of the head in the basic image 6X is used as the feature of the basic image 6X. More specifically, a pre-processing mask 640 is generated that is defined as the feature of the basic image 6X by the ratio of pixel values ​​of the brain region of the head to pixel values ​​of the skin region of the head. The pre-processing mask 640 includes a region 641 that corresponds to the brain, a region 642 that corresponds to the skin, and a region 643 that represents a marginal portion at the outer edge of the head.

[0081] FIG. 13 is a diagram for explaining the adjustment process using the pre-processing mask 640 in more detail. The head mask 650 is a filter arranged at the position of the head in the test image 6. The pre-processing mask 640 is configured by combining the head mask 650 with a filter element 651 (minimum value filter of size 3) for a region 641, a filter element 652 (maximum value filter of size 5) for a region 642, and a filter element 653 (maximum value filter of size 7) for a region 643 so that they have a common center of gravity. The inside of the filter element 651 constitutes the region 641. The outside of the filter element 651 and the inside of the filter element 652 constitutes the region 642. The outside of the filter element 652 and the inside of the filter element 653 constitutes the region 643.

[0082] Each of "X1.0", "Xk", and "X0.1" in FIG. 13 represents a value that is multiplied with the pixel value of the target image 5 in the adjustment process.

[0083] 13, "X1.0" is added to the area 641 and the area 643. This means that in the adjustment process, the pixel values ​​of the pixels located in these areas in the target image 5 are multiplied by 1.0.

[0084] 13, "X0.1" is added to the area outside the area 643. This means that in the adjustment process, the pixel values ​​of pixels located in this area in the target image 5 are multiplied by 0.1.

[0085] 13, "Xk" is attached to the area 642. This means that in the adjustment process, the pixel value of the pixel located in this area in the target image 5 is multiplied by k.

[0086] The value k is derived according to equation (3), which utilizes statistics of multiple base images 6X:

[0087]

number

[0088] In equation (3), the variable k is expressed as the product of the variable x and "MED(h) / MED(s)". MED(h) represents the median of pixel values ​​of the brain region in multiple basic images 6X. MED(s) represents the median of pixel values ​​of the skin region in multiple basic images 6X. The head region, as well as the skin region and brain region of the head in each basic image 6X may be identified by the user or by pattern recognition.

[0089] The variable x is a constant defined by the type of radiopharmaceutical used in the measurement. In one implementation, the value of the variable x is smaller when fluorodeoxyglucose (FDG) is used than when flutemetamol is used.

[0090] An upper limit and a lower limit may be set for the variable k. An example of the lower limit for the variable k is 1.0.

[0091] When the lower limit of the variable k is set to 1.0, in the adjustment process using the preprocessing mask 640, the skin region is enhanced in the target image 5 to generate the test image 6. The degree of enhancement depends on the relationship between the statistics of the pixel values ​​of the skin region and the statistics of the pixel values ​​of the brain region in the multiple basic images 6X. More specifically, when the median pixel value of the skin region in the multiple basic images 6X is equal to or less than x times the median pixel value of the brain region, the pixels of the skin region in the target image 5 are enhanced in the test image by being multiplied by k.

[0092] In the adjustment process using the pre-processing mask 640, the pixel values ​​of the area outside the area 643 of the target image 5 are multiplied by 0.1 to generate the test image 6. In this way, the adjustment process removes noise from areas other than the head in the target image 5.

[0093] In the present embodiment described above, a test image is generated by performing pre-processing (adjustment processing) of an image to be applied to a machine learning model on a target image. Then, an intermediate image (an example of an estimation result) is generated by applying the test image to the machine learning model. The adjustment processing utilizes the features of the base image used in the machine learning processing for training the machine learning model.

[0094] The characteristics of the basic image correspond to the domain in which the basic image was acquired. For example, the position, size, and / or angle of the head in the basic image depends on the positional relationship between the bed on which the image of the head is taken (more specifically, the location where the head is positioned on the bed) and the detector ring 2 in the facility in which the basic image was acquired. In this case, an example of the domain includes the above-mentioned positional relationship in the facility. The characteristics of the basic image correspond to the above-mentioned positional relationship in the facility in which the basic image was acquired.

[0095] By applying the test image generated by performing the adjustment process on the target image as described above to the machine learning model, an image that has been processed to reduce the domain gap between the target image and the base image is applied to the machine learning model. The adjustment process does not require the user to collect a large amount of data in the environment in which the target image is collected. As a result, the data processing method according to the present disclosure can reduce the burden on the user for reducing the domain gap.

[0096] The learning data used for training the machine learning model 8 was a pair of a basic image 6X and an intermediate image 7X that is a correct label for the basic image 6X. The type of data used in the learning data in pairs with the correct label is not limited to an image.

[0097] For example, the basic image 6X may be data (basic data) of a type other than an image, acquired in a domain different from the domain in which the target image 5 is acquired. As the basic data, for example, waveform data such as sound or spectrum is assumed. In this case, the characteristic of the basic data is assumed to be the amount of noise, and the adjustment process is assumed to be a process for adjusting the amount of noise, such as noise removal.

[0098] 3, an adjustment process based on the features of the base data is performed on the target image 5 to generate a test image 6. In step S40, the test image 6 is applied to a machine learning model 8 to generate an intermediate image 7.

[0099] There is no limitation on the timing when the features of the base image are generated. The features of the base image may be generated in advance. In one implementation, data defining the machine learning model and data defining the features of the base image are pre-stored in the memory of the controller 4. The features of the base image may be generated during the generation of the intermediate image in step S40.

[0100] [Aspects] It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0101] (Section 1) A data processing method according to one embodiment is a data processing method for generating a test image to be applied to a machine learning model that has been subjected to machine learning processing using a base image, and may include a step of acquiring a target image of a domain different from the domain from which the base image was acquired, and a step of generating the test image by applying an adjustment process to the target image based on features corresponding to the domain from which the base image was acquired.

[0102] According to the data processing method described in paragraph 1, the burden on the user for reducing the domain gap can be reduced.

[0103] (Clause 2) In the data processing method described in Clause 1, the target image may include an object, the feature may be related to a position of the object in the base image, and the adjustment process may include adjusting a position of the object in the test image.

[0104] According to the data processing method described in paragraph 2, the domain gap regarding the position of the object can be reduced.

[0105] (Clause 3) In the data processing method described in clause 1 or clause 2, the target image may include an object, the feature may be related to a shape of the object in the base image, and the adjustment process may include adjusting the shape of the object in the test image.

[0106] According to the data processing method described in the third aspect, the domain gap regarding the shape of the object can be reduced.

[0107] (Clause 4) In the data processing method described in any one of clauses 1 to 3, the target image may include an object, the feature may be related to a posture of the object in the base image, and the adjustment process may include adjusting a posture of the object in the test image.

[0108] According to the data processing method described in Section 4, the domain gap regarding the pose of the object can be reduced.

[0109] (Clause 5) In the data processing method described in any one of clauses 1 to 4, the feature may be related to contrast in the base image, and the adjustment process may include adjusting contrast in the test image.

[0110] According to the data processing method described in Section 5, the domain gap regarding the contrast difference between the base image and the target image can be reduced.

[0111] (Clause 6) In the data processing method described in any one of clauses 1 to 5, the target image may include an object, the feature may relate to a ratio of pixel values ​​between an outer edge and an interior of the object, and the adjustment process may include emphasizing the outer edge of the object.

[0112] According to the data processing method described in Section 6, the domain gap related to the ratio of pixel values ​​between the outer edge and the interior of an object can be reduced.

[0113] (Clause 7) In the data processing method described in Clause 6, the object may be an image of a human head, the outer edge may correspond to a skin area of ​​the head, and the interior may correspond to a brain area of ​​the head.

[0114] According to the data processing method described in paragraph 7, the domain gap for human head images can be reduced.

[0115] (Clause 8) In the data processing method described in clause 6 or clause 7, the adjustment process may utilize a filter, and the filter may be defined by a function of statistics of pixel values ​​of the outer edges of one or more of the basic images and statistics of pixel values ​​of the interiors of one or more of the basic images.

[0116] According to the data processing method described in Section 8, the domain gap for human head images can be reduced more reliably.

[0117] (Clause 9) In the data processing method according to Clause 1, the adjustment process may include removing noise in areas other than the object by using the filter.

[0118] According to the data processing method described in paragraph 9, a denoised test image may be input to the machine learning model.

[0119] (Item 10) In the data processing method according to any one of items 1 to 9, the target image may be a Positron Emission Tomography (PET) image or a Single Photon Emission Computed Tomography (SPECT) image.

[0120] According to the data processing method described in Section 10, in estimation using PET images or SPECT images, the burden on the user for reducing the domain gap can be reduced.

[0121] (Clause 11) A method for generating an absorption coefficient image according to one embodiment may comprise the steps of generating an intermediate image including an image of a tissue region based on a test image generated in accordance with the data processing method described in clause 10, and generating an absorption coefficient image based on the intermediate image and a known absorption coefficient of the tissue region.

[0122] According to the absorption coefficient image generating method described in Section 11, it is possible to reduce the burden on the user for reducing the domain gap in generating an absorption coefficient image.

[0123] (Item 12) An image generation method according to one embodiment is an image generation method that uses a machine learning model, the machine learning model having been subjected to machine learning processing using basic data, and includes the steps of acquiring a target image acquired in a domain different from the domain in which the basic data was acquired, generating a test image by applying an adjustment process to the target image based on features corresponding to the domain in which the basic data was acquired, and acquiring an output image by applying the test image to the machine learning model.

[0124] According to the program described in paragraph 12, the burden on the user for reducing the domain gap can be reduced.

[0125] (Clause 13) A program according to one aspect may be executed by one or more processors of a computer to cause the computer to implement the data processing method according to any one of clauses 1 to 10, the absorption coefficient image generating method according to clause 11, or the image generating method according to clause 12.

[0126] According to the program described in paragraph 13, the burden on the user for reducing the domain gap can be reduced.

[0127] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present disclosure is indicated by the claims, not by the description of the above-mentioned embodiments, and is intended to include all modifications within the meaning and scope of the claims. In addition, it is intended that each technology in the embodiments can be implemented alone or, if necessary, in combination with other technologies in the embodiments as far as possible. [Explanation of symbols]

[0128] 1 PET device, 2 detector ring, 3 radiation detector, 4 controller, 31 scintillator block, 32 light guide, 33 photomultiplier tube, 40 coincidence circuit, 41 processing circuit, 71 tissue composition ratio image, 100 subject, 640 preprocessing filter, 641, 642, 643 region, 651, 652, 653 filter element.

Claims

1. A data processing method for generating a test image to be applied to a machine learning model that has been subjected to machine learning processing using a base image, comprising: acquiring a target image of a domain different from the domain from which the base image was acquired; generating the test image by applying an adjustment process to the target image based on features corresponding to the domain from which the base image was obtained.

2. the target image includes an object; the features relate to positions of objects in the base images; The data processing method according to claim 1 , wherein the adjustment process includes adjusting a position of an object in the test image.

3. the target image includes an object; the features relate to shapes of objects in the base images; The data processing method according to claim 1 , wherein the adjustment process includes adjusting a shape of an object in the test image.

4. the target image includes an object; the features relate to a pose of an object in the base image; The data processing method according to claim 1 , wherein the adjustment process includes adjusting a pose of an object in the test image.

5. the feature is related to contrast in the base image; The data processing method according to claim 1 , wherein the adjustment process includes adjusting contrast in the test image.

6. the target image includes an object; the feature relates to a ratio of pixel values ​​between the outer edge and the interior of the object; The data processing method according to claim 1 , wherein the adjustment process includes enhancing an outer edge of the object.

7. the object is a human head, The outer edge corresponds to a skin area of ​​the head, The data processing method according to claim 6 , wherein the inside corresponds to a brain region of the head.

8. The adjustment process utilizes a filter, 8. A data processing method according to claim 6 or claim 7, wherein the filter is defined by a function of statistics of pixel values ​​of the outer edge of one or more of the base images and statistics of pixel values ​​of the interior of one or more of the base images.

9. The data processing method according to claim 8 , wherein the adjustment process includes removing noise in areas other than the object by using the filter.

10. 10. The data processing method according to claim 1, wherein the target image is a Positron Emission Tomography (PET) image or a Single Photon Emission Computed Tomography (SPECT) image.

11. generating intermediate images including images relating to tissue regions based on test images generated according to the data processing method of claim 10; generating an absorption coefficient image based on the intermediate image and a known absorption coefficient of the tissue region.

12. 1. A method for generating an image using a machine learning model, comprising: The machine learning model is subjected to machine learning processing using basic data, acquiring a target image acquired in a domain different from the domain from which the base data was acquired; generating a test image by performing an adjustment process on the target image based on features corresponding to the domain from which the basic data was obtained; and obtaining an output image by applying the test image to the machine learning model.

13. A program that, when executed by one or more processors of a computer, causes the computer to carry out the data processing method according to claim 1, the absorption coefficient image generating method according to claim 11, or the image generating method according to claim 12.

Images