Information processing system, information processing unit, information processing method and program

Abstract

To provide a technology capable of generating a learned model according to acquired measurement data and of proceeding to perform inference.SOLUTION: According to one embodiment, an information processing system is provided. The information processing system includes a learning part and an inference part. The learning part inputs to the learning model a first reconstructed image and a second reconstructed image and generates a learned model such that an output with the first reconstructed image as an input becomes the second reconstructed image. Each of the first reconstructed image and the second reconstructed image is an image reconfigured on the basis of any one of a plurality of projection images included in measurement data. The inference part acquires an output image outputted by inputting to the learned model a third reconstructed image. The third reconstructed image is reconfigured on the basis of any one of the plurality of projection images included in the measurement data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses an X-ray diagnostic apparatus having a processing unit that improves the quality of fourth data by inputting fourth data corresponding to a fourth number of views that is less than the first number of views to a trained model generated by performing machine learning using second data corresponding to a second number of views, which is acquired based on first data corresponding to a first number of views, as input data, and third data corresponding to a third number of views that is greater than the second number of views as output data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-58034 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the invention described in Patent Document 1, data obtained from first data is used to generate a trained model, and data obtained from fourth data different from the first data is used to improve the quality of the reconstructed image data. In other words, the invention described in Patent Document 1 requires that measurement data different from the measurement data used for inference be prepared in order to generate a trained model.

[0005] In view of the above circumstances, the present invention provides a technology that can generate a trained model from acquired measurement data and then perform inference directly. [Means for solving the problem]

[0006] According to one aspect of the present invention, an information processing system is provided. The information processing system includes a learning unit and an inference unit. The learning unit inputs a first reconstructed image and a second reconstructed image into a learning model, thereby generating a trained model in which the output when the first reconstructed image is input is the second reconstructed image. Each of the first reconstructed image and the second reconstructed image is an image reconstructed based on one of multiple projection images included in the measurement data. The inference unit inputs a third reconstructed image into the trained model, thereby obtaining an output image that is an image to be output. The third reconstructed image is an image reconstructed based on one of multiple projection images included in the measurement data.

[0007] According to the present disclosure, a trained model can be generated from acquired measurement data and can then be used for inference. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a system configuration and a hardware configuration of an information processing system 1. FIG. [Figure 2] FIG. 2 is a diagram illustrating an example of functional units included in a processor 21. [Figure 3] FIG. 2 is a diagram illustrating an example of an activity executed by the information processing system 1. [Figure 4] FIG. 10 is a diagram for explaining how decomposed waveform data is obtained from waveform data. [Figure 5] FIG. 10 is a diagram for explaining a method for determining an exhalation region and an inhalation region. [Figure 6] FIG. 1 illustrates a method for determining the inhalation and exhalation of the lungs and the diastole and systole of the heart. [Figure 7] FIG. 1 is a diagram illustrating an example of generation of a trained model. [Figure 8] FIG. 1 is an example of a diagram illustrating the use of a trained model. [Figure 9] FIG. 10 is a diagram showing an example of a screen 4. [Figure 10]FIG. 10 is a diagram showing an example of a screen 5. [Figure 11] FIG. 10 is a diagram showing an image obtained by reconstructing a cross section including the heart of a ferret from all projection images included in the measurement data. [Figure 12] FIG. 12 is a diagram showing an image obtained by reconstructing a third reconstructed image from a projection image including, as a feature, inspiration in the ferret's lungs, which is included in the measurement data, for the same cross section as in FIG. [Figure 13] This figure shows an output image for the same cross section as in Figure 11, output by inputting the third reconstructed image in Figure 12 into a trained model. [Figure 14] FIG. 10 is a diagram showing an image obtained by reconstructing a cross section including the lungs and diaphragm of a ferret from all projection images included in the measurement data. [Figure 15] FIG. 15 is a diagram showing an image obtained by reconstructing a third reconstructed image from a projection image including, as a feature, inspiration in the ferret's lungs, which is included in the measurement data, for the same cross section as in FIG. [Figure 16] This figure shows an output image for the same cross section as in Figure 14, output by inputting the third reconstructed image in Figure 15 into a trained model. DETAILED DESCRIPTION OF THE INVENTION

[0009] [Embodiment] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, various information processing according to an embodiment may realize input and output corresponding to the input. Here, the form of information referenced in such information processing (hereinafter referred to as reference information) is not limited as long as an output is obtained as a result of the input. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression formula constructed using a statistical method), a trained model that has previously trained the correlation between input and output, or a large-scale language model that can output a desired result by inputting a prompt.

[0012] In one embodiment, a "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In one embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations can be performed on a circuit in the broad sense.

[0013] Furthermore, a circuit in the broad sense is a circuit realized by at least an appropriate combination of a circuit, circuitry, processor, memory, etc. The processor may be a general-purpose processor or a dedicated circuit. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0014] 1. System configuration and hardware configuration of information processing system 1 First, the system configuration and hardware configuration of an information processing system 1 of this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the system configuration and hardware configuration of the information processing system 1.

[0015] (Information Processing System 1) The information processing system 1 shown in FIG. 1 can process a plurality of projection images captured by a CT device 3. The information processing system 1 includes an information processing device 2 and a CT (Computed Tomography) device 3. The information processing device 2 and the CT device 3 are configured to be able to communicate with each other via a communication cable or a network. This allows the information processing device 2 and the CT device 3 to transmit and receive various information to each other. Here, a system exemplified as the information processing system 1 is made up of one or more devices or components. Therefore, even the information processing device 2 alone or the CT device 3 alone is included in the system exemplified as the information processing system 1. The information processing device 2 and the CT device 3 are operated, for example, by a user who is the measurer.

[0016] (Information processing device 2) The information processing device 2 is a PC (Personal Computer). The information processing device 2 may be a tablet computer, a smartphone, or the like instead of a PC. The information processing device 2 can process multiple projection images captured by the CT device 3. Specifically, for example, the information processing device 2 is configured to be able to perform arbitrary information processing on measurement data acquired from the CT device 3, control radiation generated by a radiation generator 34, acquire projection images detected by a detector 35, control the movement of a sample holder 36, and control a rotation driver 37. Note that the information processing device 2 only needs to be able to ultimately perform arbitrary information processing related to the CT device 3, and another information processing device may be interposed between the information processing device 2 and the CT device 3. As shown in FIG. 1 , the information processing device 2 includes a processor 21, a storage unit 22, a communication unit 23, an input unit 24, and an output unit 25, and these components are electrically connected via a communication bus within the information processing device 2. The information processing device 2 executes processing according to the embodiment.

[0017] The processor 21 processes and controls the overall operations related to the information processing device 2. The processor 21 is, for example, a central processing unit (CPU). Information processing by a program stored in the storage unit 22 is specifically realized by the processor 21, which is an example of hardware, and can be executed as each functional unit included in the processor 21. Each functional unit included in the processor 21 realizes, for example, the processing shown in FIG. 3, which will be described later. Note that the processor 21 is not limited to being single, and may be implemented with multiple processors 21 for each function. A combination of these may also be used.

[0018] The storage unit 22 stores various pieces of information defined above. This may be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing device 2 executed by the processor 21, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to program operations. The storage unit 22 stores various programs and variables related to the information processing device 2 executed by the processor 21, as well as data and the like used when the processor 21 executes processing based on the programs. The storage unit 22 may be an example of a storage medium.

[0019] The communication unit 23 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as LTE / 3G / 4G / 5G, BLUETOOTH (registered trademark) communication, etc. as needed. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the information processing device 2 may communicate various information from the outside via the communication unit 23.

[0020] The input unit 24 may be included in the housing of the information processing device 2 or may be externally attached. For example, the input unit 24 may be implemented as a touch panel integrated with the output unit 25. A touch panel allows a user to input tapping, swiping, and the like. Of course, a switch button, a mouse, a keyboard, and the like may be used instead of a touch panel. That is, the input unit 24 accepts an input based on an operation performed by the user. The input is transferred as a command signal to the processor 21 via a communication bus, and the processor 21 can execute predetermined control or calculation as necessary.

[0021] The output unit 25 can function as a display device of the information processing device 2. The output unit 25 may be included in the housing of the information processing device 2, or may be externally attached. The output unit 25 displays a screen of a graphical user interface (GUI) that can be operated by the user. This is preferably implemented by selectively using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display depending on the type of the information processing device 2.

[0022] (CT device 3) The CT device 3 is an apparatus capable of irradiating a sample with radiation and acquiring a projection image of the sample from the amount of transmitted radiation. The CT device 3 may include, but is not limited to, a sample rotation type CT device that rotates a sample holder 36, a gantry type CT device that rotates a radiation generator 34 and a detector 35 relative to the sample holder 36, etc. The CT device 3 includes a processor 31, a storage unit 32, a communication unit 33, the radiation generator 34, the detector 35, the sample holder 36, and a rotation drive unit 37, and these components are electrically connected via a communication bus within the CT device 3. The CT device 3 executes processing according to the embodiment. For the processor 31, the storage unit 32, and the communication unit 33 of the CT device 3, please refer to the processor 31, the storage unit 22, and the communication unit 23 of the information processing device 2.

[0023] The radiation generator 34 irradiates radiation toward an area including the sample placed in the sample holder 36. The radiation may include any one of alpha rays, beta rays, gamma rays, X-rays, neutron rays, etc.

[0024] The detector 35 is configured to be able to detect radiation that has passed through a sample placed in the sample holder 36. The detected radiation is analyzed as measurement data by the information processing device 2. The measurement data is data obtained by measurement using the CT device 3. The measurement data includes information indicating the angle at which the image was taken and information on the projection image corresponding to that angle. The detector 35 may be a two-dimensional detector that uses a CCD, an imaging plate, or the like.

[0025] The sample holder 36 is configured to be able to hold a sample stage. The sample holder 36 may be configured to be able to move the sample stage in any direction based on a movement instruction generated by the processor 21 or the processor 31. The sample stage is configured to allow a sample to be placed thereon.

[0026] The rotation drive unit 37 is configured to rotate the sample holder 36 and / or the radiation generator 34 and the detector 35. The rotation drive unit 37 may be configured to include a mechanism capable of adjusting the magnification ratio of the projection image during imaging.

[0027] 2. Functional configuration of the processor 21 of the information processing device 2 2 is a diagram illustrating an example of functional units included in the processor 21. As illustrated in FIG. 2, the processor 21 of the information processing device 2, which is an example of the information processing system 1, includes an input processing unit 210, a data storage unit 211, an output processing unit 212, a condition setting unit 213, a feature acquisition unit 214, a synchronization processing unit 215, a phase identification unit 216, a reconstruction unit 217, a learning unit 218, and an inference unit 219. As described above, information processing by software stored in the storage unit 22 can be specifically realized by the processor 21, which is an example of hardware, and executed as each functional unit (step) included in the processor 21. The processor 21 may execute at least a condition setting step, a feature acquisition step, a synchronization processing step, a phase identification step, a reconstruction step, a learning step, and an inference step.

[0028] The input processing unit 210 accepts or acquires various data input from a user via the input unit 24. The input processing unit 210 also accepts, receives, or acquires various data from the CT device 3 via the communication unit 23.

[0029] The data storage unit 211 stores the acquired data in the storage unit 22.

[0030] The output processing unit 212 transmits various data to the CT device 3 via the communication unit 23. The output processing unit 212 is also configured to control display information to be displayed on the output unit 25. Note that the display information may be visual information itself, such as a screen, an image, an icon, or text, generated in a manner that is visible to the user, or may be rendering information for displaying visual information, such as a screen, an image, an icon, or text, on various terminals.

[0031] The condition setting unit 213 accepts the setting of various conditions.

[0032] The feature amount acquisition unit 214 calculates the feature amount from the projection image.

[0033] The synchronization processing unit 215 identifies, from the plurality of projected images, a plurality of projected images that have common movement characteristics.

[0034] The phase identification unit 216 identifies the projection image to be used to generate the reconstructed image.

[0035] The reconstruction unit 217 reconstructs an image of the sample from the multiple projection images.

[0036] The learning unit 218 executes a process for generating a trained model.

[0037] The inference unit 219 executes processing to output an output image using the trained model.

[0038] The input processing unit 210, data storage unit 211, output processing unit 212, condition setting unit 213, feature amount acquisition unit 214, synchronization processing unit 215, phase identification unit 216, reconstruction unit 217, learning unit 218, and inference unit 219 will be described in detail later.

[0039] 3. Operational flow of information processing system 1 Next, an example of preferable information processing executed by the information processing system 1 of this embodiment will be described. Fig. 3 is a diagram showing an example of an activity executed by the information processing system 1. Note that this activity may include any exception handling not shown. Exception handling includes interrupting the information processing and omitting each process.

[0040] In this embodiment, the plurality of projection images include at least a part of the lungs or heart of a living being as a subject. The living being may include a human being and an animal.

[0041] (Activity A1) First, the input processing unit 210 receives an instruction (hereinafter referred to as a measurement start instruction) from the user via the input unit 24 to set measurement conditions and to start measurement by the CT device 3. The measurement conditions include, for example, the number of images to be captured, the scan speed, the exposure time, the magnification rate of the captured image, etc.

[0042] (Activity A2) Next, the output processing unit 212 transmits the measurement conditions and a measurement start instruction to the CT device 3 via the communication unit 23.

[0043] (Activity A3) Next, the processor 31 of the CT device 3 receives the measurement conditions and a measurement start instruction from the information processing device 2 via the communication unit 33.

[0044] (Activity A4) Next, the CT apparatus 3 acquires measurement data based on the received measurement conditions.

[0045] (Activity A5) Next, the processor 31 of the CT device 3 transmits the measurement data to the information processing device 2 via the communication unit 33.

[0046] (Activity A6) Next, the input processing unit 210 receives the measurement data from the CT device 3 via the communication unit 23.

[0047] (Activity A7) Subsequently, the data storage unit 211 stores the acquired measurement data in the storage unit 22.

[0048] When there is no need to acquire new measurement data (for example, when reconstruction is performed using already acquired measurement data), the information processing system 1 may omit the information processing of activities A1 to A7.

[0049] (Activity A8) Next, the output processing unit 212 causes the output unit 25 to display a screen 4 on which the user can view the projected image. Details of the screen 4 will be described later with reference to FIG.

[0050] (Activity A9) Next, the condition setting unit 213 receives an input of the setting of the range of the ROI on the screen 4 via the input unit 24.

[0051] The "ROI (Region of Interest)" refers to a partial region in a projection image from which feature values ​​are acquired, and is also referred to as a region of interest. In the embodiment, the ROI is a region that includes at least a part of the lungs and heart of a living organism.

[0052] (Activity A10) Next, the feature acquisition unit 214 acquires waveform data based on the range of the ROI and the projection image that have been received. For example, the feature acquisition unit 214 acquires feature amounts for the range of the input ROI for each of the multiple projection images. The feature acquisition unit 214 acquires waveform data by plotting the feature amount of the ROI for each projection image on the vertical axis and the number of frames indicated by the projection image on the horizontal axis. The output processing unit 212 further displays the waveform data on the screen 4.

[0053] The "waveform data" is data expressed as feature quantities obtained from each of a plurality of projection images. In the embodiment, the waveform data is expressed as a waveform as will be described later with reference to FIG.

[0054] The "feature amount" is a value obtained by accumulating the intensity in an image. In this embodiment, the feature amount is a value obtained by accumulating the intensity in an ROI. The feature amount is expressed as x for each of the N projection images, as shown in the following equation 1. The intensity may be, for example, a brightness value, but is not limited to this.

[0055]

number

[0056] (Activity A11) Next, the condition setting unit 213 accepts input of settings for the lag value and singular value selection information from the user via the input unit 24 on the screen 4 where the user can view the waveform data. According to this aspect, it is possible to accept input of settings for the projection image conditions to which the singular spectrum analysis is applied while the user is viewing the projection image, waveform data, etc. Furthermore, it is possible to acquire more appropriate decomposed waveform data from the projection image based on the lag value arbitrarily set by the user.

[0057] The "lag value" is a value used to define the Hankel matrix of singular spectrum analysis from the feature. For example, the number of rows in a Hankel matrix is ​​expressed as L, and the number of columns in a Hankel matrix is ​​expressed as K. L is equal to the lag value, and K is the value obtained by subtracting L and 1 from N, the number of projection images, and can also be expressed as NL-1.

[0058] The "singular value selection information" is information indicating which singular value is to be selected from among the plurality of singular values ​​obtained from the matrix. In this embodiment, the singular value selection information is information indicating which singular value is to be used to obtain decomposed waveform data from among the plurality of singular values ​​obtained from the Hankel matrix.

[0059] (Activity A12) Next, the feature amount acquiring section 214 acquires decomposed waveform data by applying singular spectrum analysis to the feature amounts of the waveform data based on the lag value. More specifically, for example, the feature acquisition unit 214 acquires a Hankel matrix shown in Equation 2 by applying a Hankel transform based on the feature and lag values ​​of Equation 1. The feature acquisition unit 214 applies singular value decomposition to the acquired Hankel matrix to generate a plurality of matrices (X1, X2, ..., X) shown in Equation 3. r ) is acquired. Each matrix is ​​defined as shown in Equation 4 according to the singular value i. The feature acquisition unit 214 acquires trend waveform data by applying an inverse Hankel transform to a matrix indicated by a maximum singular value (i=1) corresponding to the largest singular value among the singular values ​​obtained by the singular spectrum analysis. The feature acquisition unit 214 also acquires vibration waveform data by applying an inverse Hankel transform to a matrix indicated by at least one singular value (e.g., i=2, corresponding to a singular value selected by the singular value selection information) selected from the singular values ​​other than the maximum singular value among the singular values ​​obtained by the singular spectrum analysis. According to this aspect, more appropriate decomposed waveform data can be acquired from the projection image based on the set lag value and singular value.

[0060]

number

number

number

[0061] Here, the decomposed waveform data will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining how the decomposed waveform data is obtained from waveform data. As shown in Fig. 4, the waveform data is decomposed into trend components (trend waveform data), vibration components (vibration waveform data), noise components, etc. as decomposed waveform data. There may be one or more vibration waveform data and noise components. The intensities of all the decomposed waveform data are integrated to form the waveform data.

[0062] The "decomposed waveform data" is waveform data obtained by decomposing waveform data. The decomposed waveform data includes at least trend waveform data, vibration waveform data, and noise components.

[0063] "Trend waveform data" refers to waveform data that has been decomposed into components that indicate the trend of the waveform data. Trend waveform data reflects relatively large behaviors, such as the movement of a living organism's trunk or limbs. Trend waveform data is also referred to as the first component.

[0064] "Vibration waveform data" refers to waveform data that has been decomposed into components that represent the vibration of the waveform data. Vibration waveform data reflects the behavior of periodically moving objects such as the lungs or heart of a living organism. Vibration waveform data is obtained from the second largest singular value and is also called the second component.

[0065] In activities A13 to A16, which will be described next, the synchronization processing unit 215 identifies the features of each of the subjects included in each of the multiple projection images based on the waveform indicated by the decomposed waveform data (based on the waveform indicated by at least one of the trend waveform data and the vibration waveform data). In explaining this information processing, a method for determining the expiration region, inspiration region, expiration, inspiration, diastole, and systole using the trend waveform data and the vibration waveform data will be described with reference to Figures 5 to 7.

[0066] Fig. 5 is a diagram for explaining a method for determining the expiratory and inhalation regions. Fig. 5 includes vibration waveform data. The vibration waveform data indicates vibration components when the intensity indicated by the trend waveform data is set as the baseline (predetermined value is 0).

[0067] The "exhalation region" is a region that can be considered to correspond to the exhalation of a living organism. In this embodiment, the exhalation region is a region where the vibration intensity indicated by the vibration waveform data is lower than 0. In the example of FIG. 5, the exhalation region is a region where the intensity of the vibration waveform data is lower than 0 (regions where the frame number is around 40 to 70, around 90 to 110, around 150 to 180, etc.).

[0068] An "inhalation region" is a region that can be considered to correspond to the inhalation of a living organism. In this embodiment, the inhalation region is a region where the vibration intensity is higher than 0. In the example of FIG. 5, the inhalation region is a region where the intensity of the vibration waveform data is higher than 0 (regions where the frame number is around 0 to 40, around 70 to 90, around 110 to 150, etc.). Note that either an exhalation region or an inhalation region may be assigned to the location where the intensity is 0.

[0069] Next, a method for determining the expiration and inspiration of the lungs and the diastole and systole of the heart from the expiration and inspiration regions determined by the method of Fig. 5 will be described. Fig. 6 is a diagram for explaining a method for determining the expiration and inspiration of the lungs and the diastole and systole of the heart. In Fig. 6, trend waveform data and vibration waveform data reflecting the expiration and inspiration regions determined by the method of Fig. 5 are superimposed on the waveform data. In Fig. 6, the solid line represents the waveform data, the dashed line represents the trend waveform data, and the dotted line represents data obtained by combining the trend waveform data with the vibration waveform data. Furthermore, the trend waveform data in Fig. 6 shows disturbances in the respiratory beat and heart beat.

[0070] The exhalation is a point where the feature amount is minimum among the points of the waveform data corresponding to a plurality of exhalation regions, for example, the points indicated by the circles in FIG. Inhalation is a point where the feature amount is maximum among the points of waveform data corresponding to a plurality of inhalation regions, and is, for example, a point indicated by a triangle in FIG. The diastole is a point where the feature amount becomes a minimum value in each point of the waveform data corresponding to the expiratory region, and is, for example, a point indicated by a diamond in FIG. The systole is a point where the feature amount reaches a maximum value in each point of the waveform data corresponding to the expiratory region, and is, for example, a point indicated by a square in FIG.

[0071] (Activity A13) The synchronization processing unit 215 determines, from the vibrations indicated by the decomposed waveform data, a region where the intensity is lower than a predetermined value as an exhalation region. The synchronization processing unit 215 identifies, at each point in the waveform data corresponding to the multiple exhalation regions determined from the decomposed waveform data, each projection image with the smallest feature amount as a projection image corresponding to the exhalation of the living organism.

[0072] (Activity A14) Next, the synchronization processor 215 determines, from the vibrations indicated by the decomposed waveform data, a region where the intensity is higher than a predetermined value as an inhalation region. The synchronization processor 215 identifies, at each point in the waveform data corresponding to the multiple inhalation regions determined from the decomposed waveform data, each projection image with the maximum feature amount as a projection image corresponding to the inhalation of the living organism.

[0073] (Activity A15) Next, the synchronization processing unit 215 identifies the projection image in which the feature amount reaches a maximum value at a position corresponding to the expiration region of the waveform data as the projection image corresponding to the systole of the heart.

[0074] (Activity A16) Next, the synchronization processing unit 215 identifies the projection image in which the feature amount is a minimum value at a portion corresponding to the exhalation region of the waveform data as the projection image corresponding to the diastole of the heart. Note that the portion corresponding to the exhalation region is a portion where the intensity of the vibration waveform data is smaller than the intensity of the trend waveform data, with the intensity of the trend waveform data being used as a reference. Furthermore, the projection image in which the feature amount is a minimum at each portion of the waveform data corresponding to the exhalation region includes a projection image in which the feature amount is a minimum value. Therefore, the projection image corresponding to exhalation may also be identified as the projection image corresponding to the diastole.

[0075] If there is no need to synchronize inhalation and exhalation, the information processing system 1 may perform only the determination process in the processes of activities A13 and A14 and omit the remaining processes.If there is no need to synchronize at least one of systole and diastole, the information processing system 1 may omit the process of at least one of activities A15 and A16.

[0076] Activities A13 to A16 allow for more appropriate determination of projection images used for reconstructing the lungs and heart from trend waveform data and vibration waveform data obtained by singular spectrum analysis. Furthermore, even when factors such as noise and vibration occur, clearer images can be reconstructed from the projection images.

[0077] In response to instructions from the user who has confirmed the synchronization results, the information processing system 1 may proceed to information processing regarding input of ROI range settings (activity A9), input of lag value and singular value selection information settings (activity A11), and acquisition and synchronization of decomposed waveform data (activity A12).

[0078] (Activity A17) When the information processing of the activities A13 to A16 is completed, the output processing unit 212 may cause the output unit 25 to display screen 5. Details of screen 5 will be described later with reference to FIG.

[0079] (Activity A18) Next, the phase identification unit 216 assigns a projection image to be used for reconstructing each reconstructed image for each feature of the object identified by the synchronization processing unit 215. A reconstructed image is an image reconstructed based on any one of multiple projection images included in the measurement data. The first reconstructed image, the second reconstructed image, and the third reconstructed image are each an example of a reconstructed image. This allows each to be reconstructed using a projection image that has common features of the object, making it possible to output clearer first reconstructed image, second reconstructed image, and third reconstructed image.

[0080] More specifically, for example, the phase identification unit 216 checks the checkboxes in the phase setting area 51 for the multiple projection images including the exhalation of the lungs of the living organism as a feature, identified by the synchronization processing unit 215, as projection images to be used for reconstructing the first reconstructed image. The first reconstructed image is used as training data, as input data for generating a trained model. In an embodiment, the first reconstructed image may be reconstructed from the multiple projection images including the exhalation of the lungs of the living organism as a feature.

[0081] Furthermore, the phase identification unit 216 checks the checkboxes in the phase setting area 51 for the multiple projection images including the diastole of the living organism's heart as a feature identified by the synchronization processing unit 215 as projection images to be used in reconstructing the second reconstructed image. The second reconstructed image is used as training data as output data when generating a trained model. In an embodiment, the second reconstructed image may be reconstructed from multiple projection images including the diastole of the living organism's heart as a feature. As described above, the number of projection images used to reconstruct the first reconstructed image may be equal to a reduced number of the multiple projection images used to reconstruct the second reconstructed image, so that the projection image corresponding to exhalation is also identified as the projection image corresponding to diastole. In this way, the multiple projection images used in the first reconstructed image are included in the multiple projection images used in the second reconstructed image, allowing the second reconstructed image to be reconstructed as an image with reduced effects of noise, streaks, and the like compared to the first reconstructed image. Therefore, by generating a trained model using the first reconstructed image as input and the second reconstructed image as output, a trained model capable of improving image quality can be generated.

[0082] Furthermore, the number of projection images used to reconstruct the first reconstructed image may not necessarily be equal to the number of projection images used to reconstruct the second reconstructed image, but may be less than or equal to the number of projection images used to reconstruct the second reconstructed image. Even if the projection images used to reconstruct the reconstructed images do not have an inclusive relationship, the trained model can learn the structure of the subject by performing the alignment described below. Furthermore, if the data of the projection images used to reconstruct the first reconstructed image and the second reconstructed image are different but the number of projection images of the first reconstructed image and the second reconstructed image are the same, the trained model can learn the structure of the subject other than noise. Furthermore, if the data and number of projection images used to reconstruct the first reconstructed image and the second reconstructed image are the same, information processing may be performed using an algorithm capable of input and output by identity mapping, such as an autoencoder, as the neural network of the trained model.

[0083] The projection images used to reconstruct at least one of the first reconstructed image and the second reconstructed image may be aligned. The alignment may be performed based on a specific part of the living organism, such as the lungs, heart, diaphragm, blood vessels, or ribs. This aligns the projection images used to generate the trained model, making it possible to generate a trained model that can improve image quality with higher accuracy.

[0084] Furthermore, the phase identification unit 216 checks the checkboxes in the phase setting area 51 for the multiple projection images including the lung inhalation of the organism identified by the synchronization processing unit 215 as projection images to be used for reconstructing the third reconstructed image. The third reconstructed image is used as input data for the generated trained model. In an embodiment, the third reconstructed image may be reconstructed from multiple projection images including the lung inhalation as a feature. Alternatively, the third reconstructed image may be reconstructed from multiple projection images including the lung exhalation as a feature. Note that the projection images used for reconstructing the third reconstructed image may also be aligned.

[0085] Furthermore, the phase identification unit 216 may accept checks on a plurality of check boxes corresponding to the first reconstructed image, the second reconstructed image, and the third reconstructed image, which are displayed in the phase setting area 51. The phase identification unit 216 may allocate projection images to be used for reconstructing the first reconstructed image, the second reconstructed image, and the third reconstructed image, based on the check results on the check boxes displayed in the phase setting area 51. This makes it possible to adjust the projection images to be used for the first reconstructed image, the second reconstructed image, and the third reconstructed image, based on the check box operation by the user.

[0086] (Activity A19) Subsequently, the condition setting unit 213 may accept conditions for generating a trained model via the learning condition setting area 52. The conditions for generating a trained model may include at least one hyperparameter among the number of epochs, learning rate, batch size, stride, number of layers, number of units, loss, optimizer, number of filters, and kernel size. This allows the conditions for learning to be accepted for each case, thereby enabling the generation of an optimal trained model.

[0087] The number of epochs is a value indicating how many times a set of images selected from the first reconstructed image and the second reconstructed image is used during learning.

[0088] The learning rate is a value that indicates how much the machine learning parameters are changed at one time. In this embodiment, by increasing the learning rate while decreasing the number of epochs, it is possible to obtain output images of higher quality while shortening the learning time.

[0089] The batch size indicates the amount of data to be divided into the first reconstructed image and processed at one time during training. Instead of the batch size, the number of divisions may be used.

[0090] The stride indicates the step size when the filter moves on the first reconstructed image. In this embodiment, the larger the stride value, the faster the trained model is generated, thereby reducing the information processing load.

[0091] The number of layers indicates the number of input layers, hidden layers, and output layers in a neural network. When using DnCNN, the number of layers is 1 input layer, 15 hidden layers, and 1 output layer.

[0092] The number of units indicates the number of neurons included in each of the input layer, hidden layer, and output layer in the neural network.

[0093] Loss is a function that quantifies the error between the predicted result and the actual value and is used to evaluate the accuracy of a trained model. Functions may include mean squared error, mean absolute error, mean squared logarithmic error, cross entropy error, etc.

[0094] The optimizer represents a method for adjusting parameters to minimize the value of the loss function.

[0095] The number of filters indicates the number of filters.

[0096] The kernel size indicates the size of the filter.

[0097] (Activity A20) Next, the input processing unit 210 receives an instruction to generate a trained model by, for example, receiving a press of the learning execution button 53.

[0098] (Activity A21) Next, the reconstruction unit 217 outputs the first reconstructed image, the second reconstructed image, and the third reconstructed image based on the check result of the check box displayed in the phase setting area 51.

[0099] (Activity A22) Next, the learning unit 218 generates a trained model based on the conditions for generating the trained model received in activity A19. FIG. 7 is an example of a diagram illustrating the generation of a trained model. As shown in FIG. 7, the trained model is trained by inputting the first reconstructed image and the second reconstructed image into the training model, so that the output when the first reconstructed image is input is the second reconstructed image. For example, the training model and the trained model may be generated by a neural network. For example, a CNN (Convolutional Neural Network) such as DnCNN (Denoising Convolutional Neural Network) is used as the neural network of this embodiment. The training model is a model before it is trained so that the output when the first reconstructed image is input is the second reconstructed image.

[0100] Information processing regarding the generation of a trained model and the output of an output image from the trained model may be performed within the information processing device 2, or may be performed within another information processing device such as a server.

[0101] (Activity A23) Next, the input processing unit 210 receives an instruction to acquire an output image using the trained model, for example by receiving a press of the inference execution button 54.

[0102] (Activity A24) Next, the inference unit 219 inputs the third reconstructed image into the trained model to obtain an output image, which is an image to be output. The output processing unit 212 displays the output image on the output unit 25. FIG. 8 is an example of a diagram illustrating the use of the trained model. As shown in FIG. 8, the trained model outputs an output image in response to the input of the third reconstructed image. The output image is an image in which the effects of at least one of noise, streaks, blur, and spatial resolution contained in the third reconstructed image have been reduced. Noise may include the concept of artifacts. As a result, by inputting the third reconstructed image, which uses approximately the same number of projection images for reconstruction as the first reconstructed image, into the generated trained model, the image quality of the third reconstructed image can be improved.

[0103] 4. Screen example Next, screens 4 and 5, the detailed description of which has been omitted while showing FIGS. 9 and 10, will be explained.

[0104] (Screen 4) 9 is a diagram showing an example of screen 4. Screen 4 is a screen on which information related to measurement data is displayed in a manner that is visible to the user. Screen 4 includes a projection image display area 40, a waveform display area 41, a condition reception area 42, a read button 43, a synchronization execution button 44, and a reconstruction execution button 45.

[0105] The projection image display area 40 is an area where the acquired projection image is displayed. To set the ROI, an X-axis and a Y-axis may be defined for the projection image in the projection image display area 40. The projection image display area 40 includes an ROI setting area 400. The ROI setting area 400 displays an area showing the ROI in the projection image. The range of the ROI may be set by any operation, such as a drag-and-drop operation into the ROI setting area 400, a range change operation using a cursor in the ROI setting area 400, or inputting a numerical value into a condition receiving area 42 (described later). The condition setting unit 213 sets the range of the ROI as activity A9 in FIG. 3.

[0106] The waveform display area 41 is an area where waveform data and / or resolved waveform data are displayed. More specifically, for example, the waveform display area 41 may be an area where waveform data is displayed before trend waveform data is acquired, and where the waveform data and trend waveform data are displayed superimposed on each other after the trend waveform data is acquired. Furthermore, the waveform display area 41 may be an area where at least two of the waveform data, trend waveform data, and vibration waveform data are displayed side by side so as to be comparable.

[0107] The condition receiving area 42 is an area for receiving input of settings related to singular spectrum analysis of waveform data. The condition receiving area 42 includes an ROI setting area 420, a lag setting area 421, and a singular value setting area 422.

[0108] The ROI setting area 420 is configured to allow the setting of an ROI as the range for acquiring waveform data from the projection image, and may be configured to allow the setting of an ROI by specifying coordinates along the X-axis and Y-axis defined in the projection image display area 40, for example, as shown in FIG. 9.

[0109] The lag setting area 421 is configured so that a lag value for defining a matrix for singular spectrum analysis can be set, and is configured so that, for example, "100" can be input as the lag value as shown in FIG.

[0110] The singular value setting area 422 is configured to allow setting of which vibration component singular value to use to acquire vibration waveform data, and is an area in which check boxes are displayed to determine which vibration component singular value to use in descending order of the singular value associated with the vibration component (1, 2, 3, etc.), as shown in FIG. 9, for example.

[0111] The read button 43 is a button for reading measurement data and displaying a projection image in the projection image display area 40. For example, in response to pressing of the read button 43, the output processing unit 212 executes activity A8 in FIG.

[0112] The synchronization execution button 44 is a button for executing synchronization processing. In response to pressing of the synchronization execution button 44, for example, the feature acquisition unit 214 starts activity A12 in Fig. 3, and then the synchronization processing unit 215 executes activities A12 to A16 in Fig. 3. When the information processing of activities A12 to A16 in Fig. 3 is completed, the output processing unit 212 may display screen 5 shown in Fig. 10.

[0113] (Screen 5) 10 is a diagram showing an example of screen 5. Screen 5 is a screen related to the generation and use of a trained model. Screen 5 includes a feature waveform area 50, a phase setting area 51, a training condition setting area 52, a training execution button 53, and an inference execution button 54.

[0114] The feature waveform area 50 is an area where data with markers added for each feature of the subject identified by the synchronization processing unit 215 (exhalation, inspiration, and diastole in the example of FIG. 10) is displayed on the waveform data.

[0115] The phase setting area 51 includes a plurality of check boxes. The plurality of check boxes may be provided so that each of a plurality of projection images can be selected to be used for reconstructing the first, second, and third reconstructed images. In the example of FIG. 10 , the phase setting area 51 selects the projection image of frame 8602 to be used for reconstructing the first reconstructed image, the projection images of frames 8602, 8607, and 8612 to be used for reconstructing the second reconstructed image, and the projection image of frame 8604 to be used for reconstructing the third reconstructed image. The phase identification unit 216 executes activity A17 in FIG. 3 in response to an operation of the phase setting area 51.

[0116] The learning condition setting area 52 is an area where learning conditions can be set. The learning condition setting area 52 may include at least one of a model selection area 520, a model detail setting button 521, an epoch number area 522, a learning rate area 523, and a batch size area 524. The learning condition setting area 52 in FIG. 10 includes the epoch number area 522, the learning rate area 523, and the batch size area 524 as conditions for neural network learning, but the displayed area may change depending on the model specified in the model selection area 520. The condition setting unit 213 executes activity A18 in FIG. 3 in response to an operation in the learning condition setting area 52.

[0117] The model selection area 520 is configured to allow selection of a model to be used for learning such as CNN.

[0118] The model detail setting button 521 is a button for setting more detailed conditions for each model. For example, in response to pressing the model detail setting button 521, the output processing unit 212 may further display a screen on which the stride, the number of layers, and the number of units can be set.

[0119] The epoch number area 522 is configured so that the epoch number can be set.

[0120] The learning rate area 523 is configured so that the learning rate can be set.

[0121] The batch size area 524 is configured to allow the batch size to be set.

[0122] The learning execution button 53 is a button for starting generation of a trained model based on the conditions set in the learning condition setting area 52.

[0123] The inference execution button 54 is a button for generating an output image using the trained model generated in response to pressing of the training execution button 53.

[0124] 5. Working Example Next, examples of changes in reconstructed images before and after applying synchronization according to the present disclosure will be described with reference to Figures 11 to 16. The subject in the examples is a ferret.

[0125] First, using FIGS. 11 to 13, we will explain the cases where the present disclosure is applied and not applied to a cross section including a ferret's heart. FIG. 11 is a diagram showing an image of a cross section including a ferret's heart reconstructed from all projection images included in the measurement data. FIG. 12 is a diagram showing an image of the same cross section as FIG. 11, where a third reconstructed image is reconstructed from projection images including the ferret's lung inspiration as a feature included in the measurement data. FIG. 13 is a diagram showing an output image of the same cross section as FIG. 11, output by inputting the third reconstructed image of FIG. 12 into a trained model. Comparing FIGS. 11 and 13, in FIG. 11, reconstruction was performed using all projection images, so the boundary between the heart and space was not clearly discernible, the contour of the heart was blurred, and the reconstructed image was blurred. On the other hand, in FIG. 13, the boundary between the heart and space was relatively easy to discern in the same area as FIG. 11. Furthermore, comparing FIGS. 12 and 13, streaks and noise were observed overall in FIG. 12, but the streaks and noise were significantly reduced in FIG. 13.

[0126] Next, using FIGS. 14 to 16, we will explain the cases where the present disclosure is applied and not applied to a cross section that includes a ferret's lungs and diaphragm, which is a cross section different from those in FIGS. 11 to 13. FIG. 14 is a diagram showing an image of a cross section including the ferret's lungs and diaphragm reconstructed from all projection images included in the measurement data. FIG. 15 is a diagram showing an image of the same cross section as FIG. 14, where a third reconstructed image is reconstructed from projection images including the ferret's lungs' inspiration as a feature included in the measurement data. FIG. 16 is a diagram showing an output image of the same cross section as FIG. 14, output by inputting the third reconstructed image of FIG. 15 into a trained model. Comparing FIGS. 14 and 16, in FIG. 14, reconstruction was performed using all projection images, so the boundary between the diaphragm and space was not clearly recognizable, the contour of the diaphragm was blurred, and the reconstructed image was unclear. On the other hand, in FIG. 16, the boundary between the diaphragm and space was relatively easy to recognize at the same location as FIG. 14. Comparing FIG. 15 with FIG. 16, streaks and noise were observed throughout FIG. 15, but in FIG. 16, the streaks and noise were significantly reduced.

[0127] According to the present disclosure, a trained model can be generated from acquired measurement data and can then be used for inference.

[0128] [others] The program is a program that causes one or more computers to execute each function unit (step), or may be an information processing method executed by the information processing system 1 (or the processor 21 of the information processing device 2).

[0129] In the embodiment, the lag value is described as being input by the user. However, in a modified example, an optimal value may be automatically calculated and used. That is, the feature acquisition unit 214 calculates the period of the frequency including the maximum peak by applying frequency analysis to the waveform data. The feature acquisition unit 214 acquires, as a lag value setting, a value representing the number of data items that at least include the calculated period. According to this embodiment, more appropriate resolved waveform data can be acquired from the projection image based on the lag value set by calculation. Furthermore, because a reasonable lag value can be set by calculation, the user does not have to go through the trouble of searching for a lag value.

[0130] In the embodiment, the subject of the projection image has been described as being the lungs or heart of a living organism, but in modified examples, other moving objects may be included as the subject. For example, in modified examples, the subject of the projection image may include parts of a living organism such as the head or abdomen of a living organism, artificial lungs and hearts, or industrial objects such as operating mechanical parts (e.g., operational tests of mechanical parts such as gears and bearings), fluids (e.g., observation of fluid dynamics within a container), etc.

[0131] In the embodiment, the predetermined analysis method is singular spectrum analysis, and the singular spectrum analysis is an example in which singular value decomposition is applied to a Hankel matrix. In a modified example, the predetermined analysis method may be any method that can acquire decomposed waveform data from waveform data, and may be singular spectrum analysis using a Toeplitz matrix or other methods.

[0132] Alternatively, multiple projection images having common motion characteristics of the subject may be identified without using feature quantities. For example, the synchronization processing unit 215 may use another device in addition to the CT device 3 and identify multiple projection images having common motion characteristics of the subject based on data acquired from that device. The device may include an electrocardiograph, an X-ray device, a PET (Positron Emission Tomography) device, an MRI (Magnetic Resonance Imaging) device, an ultrasound diagnostic device, a bone densitometer, a phonocardiograph, a blood pressure monitor, an endoscope, a displacement meter, etc.

[0133] Each of the first, second and third reconstructed images is not limited to those in the embodiments, and may be reconstructed from a plurality of projected images that include the movement of any subject as a feature.

[0134] In the embodiment, a neural network is used as the learning model and the trained model, but this is not limiting. The learning model and the trained model may include models constructed by a learning method such as supervised learning, unsupervised learning, or self-supervised learning.

[0135] As specific algorithms for machine learning using learning models and trained models, in addition to neural networks, deep learning using nearest neighbor methods, naive Bayes methods, decision trees, support vector machines, etc. may be applied as appropriate.

[0136] The learning model and the trained model may be AI (Artificial Intelligence) equipped with a learning model such as a Transformer including GPT (Generative Pretrained Transformer, including GPT-3.5, GPT-4, etc.), BERT (Bidirectional Encoder Representations from Transformers), BART (Bidirectional and Auto-regressive Transformer), etc., or a language model such as a Recurrent Neural Network (RNN), and may include generative AI.

[0137] The learning model and the trained model may be a general-purpose natural language processing learning model such as a large-scale language model (LLM) trained on a huge amount of data as an artificial intelligence. Such a learning model may include a language model that can handle various tasks without fine-tuning by one-shot learning, few-shot learning, etc.

[0138] The training model and the trained model may use a generative adversarial network (GAN). Furthermore, the trained model may include any generative model algorithm such as a deep convolutional GAN ​​(DCGAN), a conditional GAN, Pix2Pix, a progressive growing GAN (PGGAN), or a StyleGAN.

[0139] The present disclosure may be applied under conditions in which the beat cycle of an organism changes during measurement, for example, when photographing an organism on a heated bed, and may be applied under conditions in which the timing at which anesthesia begins to take effect on the organism, the timing at which anesthesia begins to wear off, etc.

[0140] Furthermore, it may be provided in the following aspects.

[0141] (1) An information processing system comprising a learning unit and an inference unit, wherein the learning unit inputs a first reconstructed image and a second reconstructed image into a learning model to generate a trained model in which the output when the first reconstructed image is input is the second reconstructed image, each of the first reconstructed image and the second reconstructed image being an image reconstructed based on one of a plurality of projection images included in measurement data, and the inference unit inputs a third reconstructed image into the trained model to obtain an output image, which is an image to be output, and the third reconstructed image being an image reconstructed based on one of the plurality of projection images included in the measurement data.

[0142] According to this aspect, a trained model can be generated from the acquired measurement data and can then be used for inference.

[0143] (2) In the information processing system described in (1) above, the number of multiple projection images used to reconstruct the first reconstructed image is less than or equal to the number of multiple projection images used to reconstruct the second reconstructed image.

[0144] According to this aspect, the number of projection images of the first reconstructed image that is input is set to be less than or equal to the number of projection images of the second reconstructed image that is output, and the trained model is trained to learn the structure of the subject.

[0145] (3) The information processing system according to (1) or (2) above, further comprising a condition setting unit, wherein the condition setting unit accepts setting of at least one condition among the number of epochs, learning rate, batch size, stride, number of layers, number of units, loss, optimizer, kernel size, and number of filters when generating the trained model, and the trained model is generated by a neural network.

[0146] According to this aspect, since the learning conditions can be accepted for each case, an optimal trained model can be generated and the third reconstructed image can be output.

[0147] (4) The information processing system according to any one of (1) to (3) above, further comprising a feature acquisition unit and a synchronization processing unit, wherein the feature acquisition unit acquires waveform data indicated as feature amounts obtained from each of a plurality of projection images included in the measurement data, and decomposes the waveform data to obtain decomposed waveform data, and the synchronization processing unit identifies each feature of the subject included in each of the plurality of projection images based on the waveform indicated by the decomposed waveform data.

[0148] According to this aspect, even when factors such as noise and vibration occur, the features of the subject can be identified more accurately.

[0149] (5) The information processing system described in (4) above further comprises a phase identification unit, wherein the phase identification unit assigns projection images to be used for reconstructing the first reconstructed image, the second reconstructed image, and the third reconstructed image for each feature of the subject.

[0150] According to this aspect, each of the reconstruction images is performed using projected images that share common characteristics of the subject, so that clearer first, second, and third reconstructed images can be output.

[0151] (6) The information processing system according to any one of (1) to (5) above, further comprising a phase identification unit, wherein the phase identification unit assigns projection images to be used in the reconstruction of the first reconstructed image, the second reconstructed image, and the third reconstructed image based on the results of checking a plurality of check boxes, and the plurality of check boxes are provided to allow selection of whether to use the projection images for the reconstruction of the first reconstructed image, the second reconstructed image, and the third reconstructed image for each of the plurality of projection images.

[0152] According to this aspect, it is possible to adjust the projection images used for the first reconstructed image, the second reconstructed image, and the third reconstructed image based on an instruction from the user.

[0153] (7) In the information processing system described in any one of (1) to (6) above, the number of projection images used to reconstruct the first reconstructed image is equal to the number of projection images used to reconstruct the second reconstructed image reduced.

[0154] According to this aspect, the multiple projection images used for the first reconstructed image are included in the multiple projection images used for the second reconstructed image, so that a trained model can be generated that can improve image quality with higher accuracy.

[0155] (8) In the information processing system described in any one of (1) to (7) above, the projection images used to reconstruct at least one of the first reconstructed image and the second reconstructed image are aligned.

[0156] According to this aspect, the projected images used to generate the trained model are aligned, making it possible to generate a trained model that can improve image quality with higher accuracy.

[0157] (9) In the information processing system described in any one of (1) to (8) above, the first reconstructed image is reconstructed from a plurality of projection images having a feature of exhalation of the lungs of the living organism, the second reconstructed image is reconstructed from a plurality of projection images having a feature of diastole of the heart of the living organism, and the third reconstructed image is reconstructed from a plurality of projection images having a feature of exhalation of the lungs or inhalation of the lungs.

[0158] According to this aspect, by using the first reconstructed image and the second reconstructed image as training data, it is possible to improve the image quality of the third reconstructed image, which includes exhalation or inhalation of the lungs of a living organism as a feature.

[0159] (10) An information processing device comprising a learning unit and an inference unit, wherein the learning unit inputs a first reconstructed image and a second reconstructed image into a learning model to generate a trained model in which the output when the first reconstructed image is input is the second reconstructed image, each of the first reconstructed image and the second reconstructed image being an image reconstructed based on one of a plurality of projection images included in measurement data, and the inference unit inputs a third reconstructed image into the trained model to obtain an output image to be output, wherein the third reconstructed image is an image reconstructed based on one of the plurality of projection images included in the measurement data.

[0160] (11) An information processing method, comprising: a learning unit inputting a first reconstructed image and a second reconstructed image into a learning model, thereby generating a trained model in which the output when the first reconstructed image is input is the second reconstructed image; each of the first reconstructed image and the second reconstructed image is an image reconstructed based on any one of a plurality of projection images included in measurement data; and an inference unit inputting a third reconstructed image into the trained model, thereby obtaining an output image to be output; the third reconstructed image is an image reconstructed based on any one of the plurality of projection images included in the measurement data.

[0161] (12) A program that functions as a functional unit of the information processing system according to any one of (1) to (9) above. Of course, this is not the case.

[0162] Finally, while various embodiments of the present invention have been described, they are presented by way of example only and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments and modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the accompanying claims. [Explanation of symbols]

[0163] 1: Information processing system 2: Information processing equipment 21: Processor 210: Input processing unit 211: Data storage unit 212: Output processing unit 213: Condition setting section 214: Feature acquisition unit 215: Synchronization processing unit 216: Phase identification section 217:Reconstruction part 218: Learning Department 219: Reasoning Department 22: Storage section 23: Communications Department 24: Input section 25: Output section 3:CT device 31: Processor 32: Storage section 33: Communications Department 34: Radiation Generator 35: Detector 36: Sample holder 37: Rotation drive unit 4: Screen 40: Projected image display area 400:ROI setting area 41: Waveform display area 42: Condition acceptance area 420:ROI setting area 421: Lag setting area 422: Singular value setting area 43: Load button 44: Synchronization execution button 45: Reconfigure execution button 5: Screen 50: Feature waveform area 51: Phase setting area 52: Learning condition setting area 520: Model selection area 521: Model detailed setting button 522: Epoch number area 523: Learning rate area 524: Batch size area 53: Learning execution button 54: Inference execution button

Claims

1. An information processing system, A learning unit and an inference unit are provided, the learning unit inputs the first reconstructed image and the second reconstructed image into a learning model to generate a trained model in which an output when the first reconstructed image is input is the second reconstructed image; each of the first reconstructed image and the second reconstructed image is an image reconstructed based on any one of a plurality of projection images included in the measurement data; the inference unit inputs the third reconstructed image into the trained model to obtain an output image that is an image to be output; the third reconstructed image is an image reconstructed based on any one of the plurality of projection images included in the measurement data. Information processing system.

2. 2. The information processing system according to claim 1, the number of the plurality of projection images used to reconstruct the first reconstructed image is equal to or less than the number of the plurality of projection images used to reconstruct the second reconstructed image; Information processing system.

3. 2. The information processing system according to claim 1, A condition setting unit is further provided, the condition setting unit accepts setting of at least one condition among a number of epochs, a learning rate, a batch size, a stride, a number of layers, a number of units, a loss, an optimizer, a kernel size, and a number of filters when generating the trained model; The trained model is generated by a neural network. Information processing system.

4. 2. The information processing system according to claim 1, further comprising a feature acquisition unit and a synchronization processing unit; The feature amount acquisition unit acquiring waveform data indicated as feature amounts obtained from each of a plurality of projection images included in the measurement data; Decomposing the waveform data to obtain decomposed waveform data; the synchronization processing unit identifies features of the subject included in each of the plurality of projection images based on the waveforms indicated by the decomposed waveform data. Information processing system.

5. 5. The information processing system according to claim 4, Further comprising a phase identification unit, the phase identification unit assigns projection images to be used for reconstructing the first reconstructed image, the second reconstructed image, and the third reconstructed image for each feature of the object. Information processing system.

6. 2. The information processing system according to claim 1, Further comprising a phase identification unit, the phase identification unit assigns projection images to be used for reconstructing the first reconstructed image, the second reconstructed image, and the third reconstructed image based on check results of a plurality of checkboxes; the plurality of check boxes are provided to enable selection of whether to use the plurality of projection images for reconstructing the first reconstructed image, the second reconstructed image, and the third reconstructed image, for each of the plurality of projection images; Information processing system.

7. 2. The information processing system according to claim 1, the number of projection images used to reconstruct the first reconstructed image is equal to a reduced number of projection images used to reconstruct the second reconstructed image; Information processing system.

8. 2. The information processing system according to claim 1, The projection images used to reconstruct at least one of the first reconstructed image and the second reconstructed image are aligned. Information processing system.

9. 2. The information processing system according to claim 1, the first reconstructed image is reconstructed from a plurality of projections that feature exhaled air from the lungs of the living organism; the second reconstructed image is reconstructed from a plurality of projections, the second reconstructed image including a diastole of the heart of the living being as a feature; the third reconstructed image is reconstructed from a plurality of projection images, the projection images including, as a feature, expiration of the lungs or inspiration of the lungs; Information processing system.

10. An information processing device, A learning unit and an inference unit are provided, the learning unit inputs the first reconstructed image and the second reconstructed image into a learning model to generate a trained model in which an output when the first reconstructed image is input is the second reconstructed image; each of the first reconstructed image and the second reconstructed image is an image reconstructed based on any one of a plurality of projection images included in the measurement data; the inference unit inputs the third reconstructed image into the trained model to obtain an output image to be output; the third reconstructed image is an image reconstructed based on any one of the plurality of projection images included in the measurement data. Information processing device.

11. An information processing method, comprising: a learning unit inputting the first reconstructed image and the second reconstructed image into a learning model to generate a trained model in which an output when the first reconstructed image is input is the second reconstructed image; each of the first reconstructed image and the second reconstructed image is an image reconstructed based on any one of a plurality of projection images included in the measurement data; causing an inference unit to input a third reconstructed image into the trained model to obtain an output image to be output; the third reconstructed image is an image reconstructed based on any one of the plurality of projection images included in the measurement data. Information processing methods.

12. A program, The information processing system according to any one of claims 1 to 9 functions as a functional unit. program.

Images