Information processing apparatus, information processing method, and program

Abstract

To reduce a reduction in the accuracy of analysis processing in a machine learning model to which an image to be analyzed is input.SOLUTION: An information processing apparatus according to the present disclosure comprises: image acquisition means that acquires first and second tomographic image groups along first and second axes of a subject; map acquisition means that acquires, for the first and second tomographic image groups, first and second maps indicating the performance of analysis of a feature quantity related to an analysis object; comparison means that compares the first map with the second map for every spatial coordinate including the first and second tomographic image groups; analysis data acquisition means that inputs the first and second tomographic image groups to first and second machine learning models and acquires feature quantities to be output as first and second analysis data groups; and integrated data acquisition means that integrates the first and second analysis data groups to each other to acquire integrated data based on a result of comparison made by the comparison means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, convolutional neural networks (CNNs) have been adopted in image processing applications such as image classification, object detection, and semantic segmentation. CNNs are a type of deep learning technology that repeatedly executes convolution processing, and are capable of performing image processing with high accuracy, as described in Non-Patent Documents 1 and 2, for example. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, "Deep Residual Learning for Image Recognition",<https: / / arxiv.org / abs / 1512.03385> (Released December 10, 2015) [Non-patent document 2] Jonathan Long, Evan Shelhamer, Trevor Darrell, "Fully Convolutional Networks for Semantic Segmentation",<https: / / arxiv.org / abs / 1411.4038> (Released November 14, 2014) Summary of the Invention [Problem to be solved by the invention]

[0004] However, many CNNs have issues due to padding (a process that adjusts the size of the output image by filling in the periphery of the input image with a specified value). For example, CNNs can accurately process objects to be classified or detected that are in the center of the image, but the accuracy drops for objects to be classified or detected that are in the peripheral regions of the image. This is because data that does not contribute to improving accuracy is generated during padding processes, and the generated data is incorporated into the results of the convolution process.

[0005] The technology of the present disclosure has been made in consideration of the above, and aims to reduce a decrease in accuracy of analysis processing in a machine learning model to which an image to be analyzed is input. [Means for solving the problem]

[0006] The information processing device according to the present disclosure includes an image acquisition means for acquiring a first group of tomographic images along a first axis of a subject and a second group of tomographic images along a second axis of the subject, a map acquisition means for acquiring a first map indicating analysis performance of a feature amount related to an analysis object for the first group of tomographic images and acquiring a second map indicating analysis performance of a feature amount related to the analysis object for the second group of tomographic images, a comparison means for comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images, and a comparison means for comparing the first group of tomographic images with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images. The information processing device according to the present disclosure also includes an analysis data acquisition means for inputting the second group of tomographic images into a first machine learning model that outputs feature amounts related to the object to be analyzed, acquiring the feature amounts output from the first machine learning model as a first analysis data group, inputting the second group of tomographic images into a second machine learning model that outputs feature amounts related to the object to be analyzed, and acquiring the feature amounts output from the second machine learning model as a second analysis data group, and integrated data acquisition means for integrating the first analysis data group and the second analysis data group to acquire integrated data based on the comparison result by the comparison means. The apparatus includes an image acquisition means for acquiring a first group of tomographic images along a first axis of the subject and a second group of tomographic images along a second axis of the subject, a map acquisition means for acquiring a first map corresponding to the first group of tomographic images and indicating analysis performance of feature quantities related to the object to be analyzed, and a second map corresponding to the second group of tomographic images and indicating analysis performance of feature quantities related to the object to be analyzed, a comparison means for comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images, and a comparison means for inputting the first group of tomographic images to a first machine learning model that outputs feature quantities related to the object to be analyzed, and an analysis data acquisition means for acquiring feature quantities output from a first analysis data group, inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring feature quantities output from the second machine learning model as a second analysis data group; and an integrated data acquisition means for adjusting the feature quantities of the first analysis data group and the feature quantities of the second analysis data group using the analysis performance of the first map and the analysis performance of the second map, integrating the adjusted first analysis data group and the second analysis data group, and acquiring integrated data based on the comparison result by the comparison means. The information processing device according to the present disclosure also includes an information processing device comprising: an image acquisition means for acquiring a first group of tomographic images along a first axis of a subject and a second group of tomographic images along a second axis of the subject; an analysis data acquisition means for inputting the first group of tomographic images into a first machine learning model that outputs features related to an object to be analyzed, acquiring the features output from the first machine learning model as a first analysis data group, inputting the second group of tomographic images into a second machine learning model that outputs features related to the object to be analyzed, and acquiring the features output from the second machine learning model as a second analysis data group; and an integrated data acquisition means for integrating the first analysis data group and the second analysis data group to acquire integrated data by replacing the features of the first analysis data group at spatial coordinates corresponding to a predetermined region among spatial coordinates including the first group of tomographic images and the second group of tomographic images with the features of the second analysis data group.

[0007] An information processing method according to the present disclosure includes the steps of acquiring a first group of tomographic images along a first axis of a subject and a second group of tomographic images along a second axis of the subject, acquiring a first map corresponding to the first group of tomographic images and indicating analysis performance of a feature amount related to an object to be analyzed, acquiring a second map corresponding to the second group of tomographic images and indicating analysis performance of a feature amount related to the object to be analyzed, comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images, and inputting the second group of tomographic images into a first machine learning model that outputs feature quantities related to the object to be analyzed, acquiring the feature quantities output from the first machine learning model as a first group of analysis data; inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second group of analysis data; and integrating the first group of analysis data and the second group of analysis data to acquire integrated data based on a comparison result between the first map and the second map. An information processing method according to the present disclosure includes the steps of acquiring a first group of tomographic images along a first axis of a subject and a second group of tomographic images along a second axis of the subject, acquiring a first map corresponding to the first group of tomographic images and indicating analysis performance of feature amounts related to an object to be analyzed, acquiring a second map corresponding to the second group of tomographic images and indicating analysis performance of feature amounts related to the object to be analyzed, and comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images. inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the first machine learning model as a first group of analysis data; inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second group of analysis data; and adjusting the feature values ​​and the feature values ​​of the second analysis data group, and integrating the adjusted first analysis data group and the second analysis data group to obtain integrated data based on the comparison result between the first map and the second map. The present disclosure also provides an information processing method comprising the steps of: acquiring a first group of tomographic images along a first axis of a subject and a second group of tomographic images along a second axis of the subject; inputting the first group of tomographic images into a first machine learning model that outputs features related to an object to be analyzed, and acquiring the features output from the first machine learning model as a first analysis data group; inputting the second group of tomographic images into a second machine learning model that outputs features related to the object to be analyzed, and acquiring the features output from the second machine learning model as a second analysis data group; and integrating the first analysis data group and the second analysis data group to acquire integrated data by replacing the features of the first analysis data group at spatial coordinates corresponding to a predetermined region among spatial coordinates including the first group of tomographic images and the second group of tomographic images with the features of the second analysis data group. [Effects of the Invention]

[0008] According to the technology of the present disclosure, it is possible to reduce the decrease in accuracy of the analysis process in a machine learning model to which an image to be analyzed is input. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an information processing device according to a first embodiment; [Figure 2] Diagram showing the relationship between tomographic images and anomaly likelihood maps [Figure 3] A diagram explaining the range of effects of padding on images [Figure 4] Diagram illustrating the effect of padding on the output tensor on the image [Figure 5] Another diagram illustrating the effect of padding on the output tensor image [Figure 6]A flowchart showing an example of a process for setting an analysis performance map. [Figure 7] 1 is a flowchart showing a process for generating integrated analysis data in the first embodiment; [Figure 8] FIG. 10 is a diagram for explaining the processing of image data in the first embodiment; [Figure 9] FIG. 10 is a diagram for schematically explaining the distribution of analytical performance in the first embodiment; [Figure 10] FIG. 10 is a diagram for schematically explaining the distribution of analytical performance in one modified example. [Figure 11] A flowchart showing a process for generating integrated analysis data according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the present disclosure is not limited to the following embodiments and can be modified as appropriate without departing from the spirit of the present disclosure. In the drawings described below, components having the same functions are denoted by the same reference numerals, and their description may be omitted or simplified.

[0011] Hereinafter, with reference to the drawings, each embodiment and each modified example of an information processing device, an information processing method, and a program will be described in detail. Note that the following embodiments can be combined with conventional technology, other embodiments, or modified examples to the extent that no contradiction occurs in the content. Similarly, the following modified examples can be combined with conventional technology, embodiments, or other modified examples to the extent that no contradiction occurs in the content. Furthermore, in the following description, similar components will be given common reference numerals, and duplicated descriptions may be omitted.

[0012] (First embodiment) FIG. 1 is a diagram showing an example of the configuration of an information processing device 100 according to the first embodiment. The information processing device 100 acquires image data including an image to be analyzed, and performs information processing on the image data. The information processing device 100 may display the results of the information processing, store the results of the information processing in association with the image data, or output the results of the information processing to an external device. For example, the information processing device 100 may add an object of interest (object to be analyzed) to the image data. The information processing device 100 may generate data indicating an analysis result regarding whether or not the image is depicted, and display the analysis result on a display device (not shown) connected to the information processing device 100. The information processing device 100 is realized by a computer device such as a server or a workstation, for example.

[0013] The information processing device 100 may be communicably connected to a data management device (not shown) via a network 200 or a communication cable or communication circuit (not shown) in order to acquire image data to be processed and store analysis results of the image data. Such a data management device is a device that stores various data such as image data and analysis results of the image data, and can transmit and receive image data to other devices such as the information processing device 100 that can communicate with the data management device. The data management device may be configured integrally with the information processing device 100 as one of the components of the information processing device 100.

[0014] Furthermore, the image data is a three-dimensional image (i.e., volume data), but there is no particular limitation on the type of image to be analyzed in this embodiment. Examples of image data include those obtained by an OCT (Optical Coherence Tomography) imaging device, a CT (Computer Tomography) imaging device, and the like. Other examples of image data include medical images taken by a Magnetic Resonance Imaging (MRI) device, a Positron Emission Tomography (PET) device, and an ultrasound diagnostic device. Examples of image data include medical images captured by a single-photon emission computed tomography (SPECT) device. Other examples of image data include three-dimensional data of people, animals, and artificial objects acquired by a three-dimensional scanner, and three-dimensional image data generated by stacking two-dimensional image data. Note that this image data is not limited to three-dimensional image data, and may be multidimensional image data with four or more coordinate axes.

[0015] 1, the information processing device 100 includes a communication interface 101, a memory circuit 102, a processing circuit 103, an input interface 104, and a display 105. The information processing device 100 can be communicatively connected to a network 200 via the communication interface 101.

[0016] The communication interface 101 is an interface for communicating image data, analysis results, etc. with other devices. The communication interface 101 is realized by a network communication interface such as a network adapter or a NIC (Network Interface Controller). The communication interface 101 is also realized by a device connection interface such as a USB (Universal Serial Bus), PCI Express, SATA (Serial ATA), or M.2. This may also be done.

[0017] The memory circuitry 102 stores various data and various programs used in the processes executed by the information processing device 100 according to this embodiment. Specifically, the memory circuitry 102 is connected to the processing circuitry 103 and stores image data and analysis results under the control of the processing circuitry 103. The memory circuitry 102 also functions as a work memory that temporarily stores various data used in the processes executed by the processing circuitry 103. The memory circuitry 102 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like.

[0018] The processing circuitry 103 controls the operations of the above-mentioned components of the information processing device 100. For example, the processing circuitry 103 performs various processes in response to instructions received from a user via an input interface 104 connected to the information processing device 100. Alternatively, for example, the processing circuitry 103 The processing circuitry 103 may perform various processes in response to instructions received from a user via the communication interface 101. Alternatively, for example, the processing circuitry 103 may perform various processes after detecting that image data has been stored in the memory circuitry 102. The processing circuitry 103 is realized, for example, by a CPU (Central Processing Unit).

[0019] The processing circuitry 103 includes, for example, an image acquisition circuitry 103a that realizes an image acquisition means for acquiring a group of tomographic images of a subject to be analyzed, and an analysis performance map acquisition circuitry 103b that realizes a map acquisition means for acquiring a map showing the analysis performance of feature quantities related to the object to be analyzed. The processing circuitry 103 further includes an analysis data acquisition circuitry 103c that inputs the group of tomographic images into a machine learning model to acquire a group of analysis data, and an integrated data acquisition circuitry 103d that integrates the group of analysis data to acquire integrated data.

[0020] Here, each processing function realized by the image acquisition circuit 103a, analysis performance map acquisition circuit 103b, analysis data acquisition circuit 103c, and integrated data acquisition circuit 103d of the processing circuit 103 is stored in the storage circuit 102 in the form of a computer-executable program. The processing circuit 103 reads each program from the storage circuit 102 and executes the read program to realize the function corresponding to each program. That is, the processing circuit 103 that reads each program has the functions realized by the image acquisition circuit 103a, analysis performance map acquisition circuit 103b, analysis data acquisition circuit 103c, and integrated data acquisition circuit 103d. As a result, the image acquisition circuit 103a functions as image acquisition means that acquires a first group of tomographic images of the subject along a first axis and a second group of tomographic images of the subject along a second axis. Furthermore, the analysis performance map acquisition circuit 103b functions as map acquisition means that acquires maps indicating the analysis performance of feature quantities related to the analysis target based on the tomographic images. The analysis data acquisition circuit 103c functions as an analysis data acquisition means that inputs the tomographic image group to a machine learning model that outputs feature quantities related to the object to be analyzed based on the comparison result and acquires the feature quantities output from the machine learning model as an analysis data group. The integrated data acquisition circuit 103d functions as an integrated data acquisition means that integrates the analysis data group to acquire integrated data.

[0021] The input interface 104 accepts various instructions and input operations of various information from a user of the information processing device 100. Specifically, the input interface 104 is connected to the processing circuit 103 and converts the input operations received from the user into electrical signals and transmits them to the processing circuit 103. For example, the input interface 104 may be realized by a trackball, a switch button, a mouse, a keyboard, or a touchpad that performs input operations by touching the operation surface. Alternatively, the input interface 104 may be realized by a touchscreen that integrates a display screen and a touchpad, a non-contact input interface using an optical sensor, a voice input interface, or the like. Note that the input interface 104 is not limited to those that include physical operation components such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the information processing device 100 and transmits these electrical signals to the processing circuit 103 is also included as an example of the input interface 104.

[0022] The display 105 displays various data such as image data processed by the information processing device 100 and data based on analysis results. Specifically, the display 105 is connected to the processing circuitry 103 and displays various data received from the processing circuitry 103. For example, the display 105 displays medical images based on image data. For example, the display 105 is realized by a liquid crystal monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, or the like.

[0023] The above is a description of an example of the configuration of the information processing device 100 according to this embodiment. The processing device 100 performs various processes described below to reduce the impact of performance degradation of a convolutional neural network (CNN) in generating analysis data using an image to be analyzed. Examples of various processes performed by the information processing device 100 on image data are described below. More specifically, the information processing device 100 performs an anomaly likelihood map generation process on three-dimensional image data. Note that processes similar to those of this embodiment can also be extended to multidimensional image data such as four-dimensional image data, and the following description may be reinterpreted as multidimensional image data as needed.

[0024] Here, an analysis data acquisition model executed by the analysis data acquisition circuit 103c in the processing executed by the information processing device 100 according to this embodiment will be described.

[0025] The analysis data acquisition model executed by the analysis data acquisition circuit 103c according to this embodiment is an image processing model that applies machine learning technology. The image processing model is a machine learning model that outputs, as analysis data, an abnormality likelihood map that indicates the distribution of abnormality likelihoods (degrees of abnormality) related to the feature quantities of an analysis object depicted in input two-dimensional image data. Here, the abnormality likelihoods related to the feature quantities of the analysis object represent lesions, lacerations, image artifacts, etc. in the image, and the degree of abnormality is expressed, for example, as a probability.

[0026] Furthermore, in this embodiment, the abnormality likelihood map is a map corresponding to the spatial coordinates of the two-dimensional image data input to the analysis data acquisition model, and indicates the analytical performance of the feature quantities related to the analysis target for each spatial coordinate. The abnormality likelihood map can be used to identify the coordinates of areas in the two-dimensional image data where an abnormality is present. The abnormality likelihood map can be visualized, and for example, the processing circuit 103 can generate an image with larger pixel values ​​so that areas with a higher degree of abnormality in the abnormality likelihood map are more noticeable to the user. The abnormality likelihood map generated by the processing circuit 103 may be output to the display 105 or the like and presented to the user.

[0027] Specifically, as shown in FIGS. 2A and 2B, the analysis data acquisition model outputs an abnormality likelihood map Ma100 based on image data Im100 input to the analysis data acquisition model. In the example shown in FIG. 2B, in the abnormality likelihood map Ma100, the portion corresponding to the edema region depicted in the image data Im100, which is a tomographic image of the fundus retinal layer, has a high degree of abnormality and a large pixel value. In the abnormality likelihood map Ma100 of FIG. 2B, a white portion with a large pixel value indicates a higher degree of abnormality. Note that in this embodiment, the degree of abnormality indicated by the abnormality likelihood map is assumed to be a continuous value (e.g., a floating-point number from 0 to 1), but the degree of abnormality may also be binarized or multi-valued to produce an abnormal segment map.

[0028] Furthermore, the analysis data acquisition circuit 103c may execute multiple machine learning models to achieve high performance depending on the type of input image data. Here, attributes that distinguish the type of image data include, for example, the size of the image, the type and settings of the imaging device used to acquire the image data, the type and imaging location of the subject, etc. In this embodiment, the processing targets are two types of tomographic image groups with different depiction tendencies, namely, a first group of tomographic images along a first axis of the subject and a second group of tomographic images along a second axis, which are acquired from three-dimensional image data of the subject to be analyzed by imaging the subject. Furthermore, there are two analysis data acquisition models, a first analysis data acquisition model and a second analysis data acquisition model, corresponding to the respective tomographic image groups. The first analysis data acquisition model is a first machine learning model that outputs feature quantities related to the subject to be analyzed, and the second analysis data acquisition model is a second machine learning model that outputs feature quantities related to the subject to be analyzed.

[0029] The size and number of tomographic images constituting the first group of tomographic images along the first axis of the subject and the second group of tomographic images along the second axis also vary depending on the image size of the three-dimensional image data to be analyzed. Therefore, more analysis data acquisition models than those used in this embodiment may be used. This allows for the selection of an analysis data acquisition model that is expected to have high performance depending on the type of tomographic image. However, in the description of this embodiment, for ease of understanding, it is assumed that the image size of the three-dimensional image data to be analyzed is constant.

[0030] The analysis data acquisition model according to this embodiment includes an image processing algorithm for outputting an anomaly likelihood map as analysis data, which outputs an image corresponding to an input image according to the trends of a dataset used for model training. This method applies image generation processing using machine learning technology. Specifically, this method applies CNNs for image generation processing, such as semantic segmentation processing, style conversion processing, and image quality improvement processing (super-resolution processing and denoising processing) using machine learning technology as disclosed in Non-Patent Document 2.

[0031] The image generation process disclosed in Non-Patent Document 2 is a semantic segmentation process that outputs a segment image indicating the region of an object depicted in an input image. However, by not classifying the tensor output from the final layer of the CNN or by mapping it by additional convolution processing or the like, an image equivalent to the above-mentioned anomaly likelihood map can be obtained by the image generation process.

[0032] The analytical data acquisition model in this embodiment is trained to output image data representing a two-dimensional anomaly likelihood map for two-dimensional image data input to the analytical data acquisition model. The dataset used to train the analytical data acquisition model in this embodiment includes multiple pairs of input data and correct answer data. Here, the input data is image data of the same type as the two-dimensional image data expected to be input to the analytical data acquisition model. The correct answer data is image data of a grayscale image representing an anomaly likelihood map corresponding to the input image or a binary image representing an abnormal segment map.

[0033] Specifically, for example, the input data is image data of a two-dimensional tomographic image constituting an OCT image depicting a three-dimensional fundus retinal layer acquired by an OCT imaging device, and the correct answer data is image data of a grayscale image in which the degree of abnormality in the distribution of fundus retinal layer abnormalities such as edema depicted in the tomographic image is expressed by the magnitude of pixel values.

[0034] Furthermore, the analysis data acquisition model does not need to be implemented in the information processing device 100. For example, the analysis data acquisition circuit 103c may use an analysis data acquisition model implemented in an analysis device (not shown) via the network 200 to realize the processes of steps S108 and S109 shown in FIG. 7 (described later). Alternatively, for example, the analysis data acquisition circuit 103c may receive configuration data and parameter data related to the analysis data acquisition model stored in a data management device (not shown) via the network 200. In this case, the analysis data acquisition circuit 103c recreates the analysis data acquisition model in the information processing device 100 using the received data, and realizes the processes of steps S108 and S109 shown in FIG. 7.

[0035] Generally, CNNs, including the image processing model disclosed in Non-Patent Document 2, are often designed to perform padding before convolution processing so that the spatial size between the input tensor and output tensor of the convolution processing does not change. One of the reasons for this design is that the residual feature extraction processing disclosed in Non-Patent Document 1 One reason is that the tensor size must remain constant in the CNN being implemented, allowing the input and output tensors of processing blocks, including convolutions, to be added together. Another reason is that the spatial size of the tensor changes with each convolution, requiring interpolation that negatively impacts accuracy due to tensor pooling, combining, and scaling.

[0036] In addition, a fixed value such as 0 is generally set in the padding region in the padding process. A specific description will be given using FIGS. 3A, 3B, and 4. Assume that a convolution process with a kernel size of 3 × 3 (width × height) is performed on a tensor Te100, as shown in FIG. 3A. In this case, a tensor Te101 is generated by padding the tensor Te100 with 1 × 1 (one pixel above, below, left, and right) so that the spatial size of the tensor Te102 after the convolution process is the same as that of the tensor Te100, and this tensor Te101 is used as the input tensor for the convolution process. In this case, a fixed value is often set for one pixel around the tensor Te101, but this fixed value is set regardless of the state of the original tensor Te100. Therefore, the peripheral portions of the tensor Te102, which are affected by this fixed value, may not have features that contribute to high analysis accuracy calculated, compared to the central portion.

[0037] Furthermore, when convolution processing is performed multiple times in a CNN, pixels of a tensor affected by fixed values ​​due to padding affect neighboring pixels, etc., as a result of the convolution processing. As a result, the range of influence of the fixed values ​​expands from the tensor's peripheral areas toward its center as the convolution processing is repeated. Compare tensor Te102 (3 × 3 pixels) with tensor Te105 (5 × 5 pixels), which is a convolution processing including padding performed on tensor Te103, which has a spatial size larger than tensor Te100, as shown in Figure 3B. In this case, the influence of fixed values ​​due to padding on tensor Te105, which has a larger spatial size than tensor Te102, is smaller relative to the spatial size of the tensor. Specifically, the degree of influence of fixed values ​​due to padding on spatial size is calculated by dividing the number of affected pixels by the spatial size. Therefore, for tensor Te102, it is 8÷(3×3)=approximately 89%, and for tensor Te105, it is 16÷(5×5)=64%, and the influence of fixed values ​​becomes relatively smaller when the space size is large.

[0038] As shown in Figure 4, tensor Te110 is input to CNN Ne110, a feature extractor that repeatedly performs convolution and pooling processes, and tensor Te111 is obtained as the output of CNN Ne110. The central part of tensor Te111 represents high-quality features (dark areas in the figure) that are not affected by fixed values ​​due to padding or are relatively less affected by fixed values ​​compared to the peripheral parts, contributing to high analysis performance. Meanwhile, the peripheral parts of tensor Te111 represent low-quality features (light areas in the figure) that are repeatedly affected by fixed values ​​due to padding and therefore have relatively poor analysis performance. Therefore, in machine learning models equipped with CNNs that repeatedly perform convolution and pooling processes, one method for obtaining highly accurate analysis results is to utilize features that are less affected by fixed values ​​due to padding.

[0039] The analysis data acquisition model according to this embodiment is an image processing model that applies machine learning technology and performs image generation processing using CNN. The analysis data acquisition model includes at least one convolution process, and performs padding before the convolution process so that the spatial size (width, height) of the input tensor and output tensor does not change. Therefore, due to the padding process, in the image data representing the generated anomaly likelihood map, the central part of the image has high analysis performance (i.e., high reliability of anomaly likelihood) and the peripheral part of the image has low analysis performance (i.e., low reliability of anomaly likelihood).

[0040] To address the problem that the analytical performance of the anomaly likelihood map output by the analytical data acquisition model varies depending on the spatial coordinates, the analytical data acquisition circuit 103c includes an analytical performance map, which is definition data regarding the level of analytical performance corresponding to the spatial coordinates of the anomaly likelihood map. In this embodiment, as described above, the input to the analytical data acquisition model is two-dimensional image data, and therefore the anomaly likelihood map and analytical performance map are also two-dimensional data. Furthermore, the analytical performance map may be output to the display 105 or otherwise made available to the user of the information processing device 100 as reference information regarding the analytical performance of the analytical data acquisition circuit 103c.

[0041] The analysis performance map is defined based on the degree of influence of fixed values ​​due to padding processing associated with convolution processing on image data representing the anomaly likelihood map output by the analysis data acquisition model. A specific explanation will be given using Figure 5. When the input tensor Te120, which is a tensor formed from the two-dimensional image data input to the analysis data acquisition model, is input to the CNN Ne120 included in the analysis data acquisition model, an output tensor Te121 indicating the anomaly likelihood is obtained. Note that the output tensor Te121 is output with the same spatial size as the input tensor 120 due to upsampling processing and other processes included in the CNN Ne120.

[0042] Here, the output tensor Te121 is affected by fixed values ​​due to padding associated with the convolution process included in the CNN Ne120, and the analysis performance of the anomaly likelihood (i.e., the reliability of the analysis) varies depending on the position within the tensor. That is, in the spatial coordinate system of the output tensor Te121, the analysis performance is higher toward the center because the influence of fixed values ​​due to padding associated with the convolution process is smaller, and the analysis performance is lower toward the periphery because the influence of fixed values ​​is greater. The analysis performance map has the same spatial size as the output tensor Te121. Furthermore, the analysis performance map is defined so that high values ​​represent areas with high analysis performance and low values ​​represent areas with low analysis performance, corresponding to the degree of influence of fixed values ​​due to padding associated with the convolution process on the output tensor Te121. The analysis performance map may be pre-stored in a storage area (not shown) of the analysis data acquisition circuit 103c in the form of, for example, image data or tensor data, corresponding to the type of analysis data acquisition model.

[0043] That is, in this embodiment, as described above, the analysis data acquisition circuit 103c executes two machine learning models, the first analysis data acquisition model and the second analysis data acquisition model. Therefore, two analysis performance maps, the first analysis performance map and the second analysis performance map, corresponding to the respective analysis data acquisition models, may be stored in advance in a storage area of ​​the analysis data acquisition circuit 103c.

[0044] As a method for setting a set of values ​​corresponding to the distribution of analytical performance that constitutes the analytical performance map, for example, arbitrary values ​​are set so that the value decreases as the distance (Euclidean distance, Manhattan distance, etc.) from the center of the analytical performance map (e.g., the central position) increases. This method takes advantage of the fact that in the output tensor of a typical CNN, the peripheral parts tend to be more affected by fixed values ​​due to padding processing associated with convolution processing than the central parts, resulting in lower analytical performance. However, if it is known that the analytical performance does not follow this trend due to the design of the CNN, it is preferable to adjust the values ​​of the analytical performance map according to the CNN's unique tendency. In other words, an analytical performance map may be used in which large values ​​are set for parts of the CNN output tensor that are less affected by fixed values ​​due to padding processing associated with convolution processing, and small values ​​are set for parts that are more affected by fixed values.

[0045] Furthermore, for example, as described above, a method may be used in which the influence of fixed values ​​due to padding processing accompanying the convolution processing of the CNN included in the analysis data acquisition model is quantified. In this method, the values ​​of the analysis performance map are set based on the results of inputting dummy tensor data into a dummy CNN having the same network structure as the CNN. Generation of the analysis performance map using this method A specific example of the processing will be described with reference to the flowchart shown in Fig. 6. The processing of the flowchart in Fig. 6 is, as an example, processing that can be executed by the processing circuitry 103 of the information processing device 100. However, the processing shown in Fig. 6 may also be executed by an external device of the information processing device 100, and the processing circuitry 103 may acquire the generated analytical performance map from the external device via the communication interface 101.

[0046] In step S1001, the processing circuit 103 fixes the parameters constituting the CNN to generate a dummy CNN. This results in a machine learning model initialized with predetermined parameters. A specific example of parameter fixing is fixing the weight parameters of the kernel of a convolution process involving zero padding to a constant value depending on the kernel size and the number of filters. More specifically, if the kernel size is 3 × 3, all weight parameters are set to 1.0 ÷ (3 × 3) and the bias parameter is set to 0. If the number of filters is N, the weight parameters are further divided by N. This prevents the ratio of the influence of the fixed values ​​set in the padding region of the padding process on the output tensor of the convolution process from varying depending on the region to be convolved. Another parameter fixing process may be fixing the weight parameters of a linear regression process to a constant value depending on the number of nodes. Specifically, if the number of input nodes is M, all weight parameters are set to 1.0 ÷ M, the bias parameter is set to 0, or if the number of output nodes is N, the weight parameters are further divided by N. This makes it possible to prevent large fluctuations in values ​​between the input tensor and output tensor in the linear regression process.

[0047] In step S1002, the processing circuit 103 inputs data having predetermined values ​​for each spatial coordinate to the dummy CNN. For example, the processing circuit 103 inputs dummy tensor data having all non-zero values ​​such as 1 to the dummy CNN generated in step S1001, and acquires the output tensor data.

[0048] In step S1003, the processing circuitry 103 generates an analysis performance map using the tensor data output in step S1002. The processing circuitry 103 may generate the analysis performance map by performing linear or nonlinear calculations to normalize the values ​​constituting the output tensor data so that the minimum value is 0 and the maximum value is 1. An analysis performance map composed of normalized values ​​is particularly useful when handling input image data of different spatial sizes or CNNs of different structures. Specifically, it is possible to compare the effects of fixed values ​​due to padding processing in corresponding output tensors, or to multiply the output tensor by the value of the analysis performance map to reduce unreliable values ​​in the output tensor.

[0049] Furthermore, as a method for generating an analytical performance map that is independent of the configuration of the analytical data acquisition model, first, one or more image data are input into the analytical data acquisition model to collect output data, and statistical values ​​are obtained for a group of pairs of ground truth data and output data corresponding to each image data. Then, based on the statistical values, an analytical performance map is generated in which larger values ​​are set for portions from which output data closer to the ground truth data is obtained. Specifically, for example, an error map is created for each pair in the group of pairs by calculating the absolute error or squared error between each pixel corresponding to the ground truth data and the output data. Then, an analytical performance map is generated in which values ​​are set by averaging or specifying the most frequent value for the multiple error map groups in corresponding pixel coordinate units.

[0050] FIG. 7 is a flowchart showing an example of the flow of processing executed by the information processing device 100 according to this embodiment. In the description of this embodiment, the processing shown in FIG. 7 is assumed to be executed when the user operates the input interface 104 to give an instruction to start the processing. The image data to be analyzed is a three-dimensional OCT image acquired by an OCT imaging device, and this image data is stored in advance in the storage circuitry 102. The following description will be given in accordance with the steps of the flowchart shown in Fig. 7. Note that the order of the steps may be changed as long as no contradiction occurs in the content.

[0051] In step S101, the image acquisition circuitry 103a, as an image acquisition unit, acquires a first group of tomographic images along a first axis of the subject and a second group of tomographic images along a second axis of the subject from a three-dimensional OCT image of the subject to be analyzed. Here, the first axis and the second axis may be axes extending in any direction, and may even coincide. In the description of this embodiment, for ease of understanding, the first axis and the second axis are assumed to be axes perpendicular to each other.

[0052] Specifically, in the example of image data of the OCT image shown in Figures 8A and 8B, the three-dimensional axes that are orthogonal to each other and coincide with the direction in which the pixel groups constituting the image data to be analyzed are defined as the X-axis, Y-axis, and Z-axis, respectively. In this case, the first axis is the Y-axis, the second axis is the X-axis, and the third axis is the Z-axis. The first tomographic image group is one or more tomographic images that constitute the image data to be analyzed and are parallel to the XZ plane (the analysis data, which is two-dimensional data constituting the first analysis data group acquired in step S108, also corresponds to the XZ plane). The second tomographic image group is one or more tomographic images that constitute the image data to be analyzed and are parallel to the YZ plane (the analysis data, which is two-dimensional data constituting the second analysis data group acquired in step S109, also corresponds to the YZ plane).

[0053] In step S102, the analysis performance map acquisition circuit 103b, as a map acquisition means, acquires a first map indicating the analysis performance of feature quantities related to the analysis target for the first group of tomographic images. Specifically, the analysis performance map acquisition circuit 103b maps the first analysis performance map, which is two-dimensional data provided in the analysis data acquisition circuit 103c, onto the first group of tomographic images to generate a first three-dimensional analysis performance map. The analysis performance map acquisition circuit 103b maps the first analysis performance map so that it corresponds to pixel coordinates at which each of the tomographic images, which is two-dimensional image data constituting the first group of tomographic images, is located. Here, the pixel coordinates match the pixel coordinates at which each of the analysis data, which is two-dimensional data constituting the first group of analysis data acquired in step S108, is located.

[0054] In step S103, the analysis performance map acquisition circuit 103b, as a map acquisition means, acquires a second map indicating the analysis performance of feature quantities related to the object to be analyzed for the second group of tomographic images. Specifically, the analysis performance map acquisition circuit 103b maps the second analysis performance map, which is two-dimensional data, so that it corresponds to the pixel coordinates at which each of the tomographic images, which are two-dimensional image data constituting the second group of tomographic images, is located. This matches the pixel coordinates at which each of the analysis data, which are two-dimensional data constituting the second group of analysis data acquired in step S109, is located. In this way, the analysis performance map acquisition circuit 103b generates a second three-dimensional analysis performance map.

[0055] In step S104, the analysis performance map acquisition circuit 103b maps the first and second three-dimensional analysis performance maps into the pixel coordinate space of the three-dimensional OCT image to be analyzed. Here, the two-dimensional analysis performance maps constituting each of the first and second three-dimensional analysis performance maps correspond to any of the two-dimensional tomographic images that are part of the three-dimensional OCT image to be analyzed. Therefore, there are mutually matching spatial coordinates among the first and second three-dimensional analysis performance maps and the three-dimensional OCT image to be analyzed.

[0056] That is, at each coordinate of the pixel group that constitutes the three-dimensional OCT image to be analyzed, the values ​​that indicate the analytical performance that constitute the first three-dimensional analytical performance map and the values ​​that indicate the analytical performance that constitute the second three-dimensional analytical performance map can be compared. At each coordinate of the pixel group that constitutes the image, it is possible to determine which model has higher analytical performance based on the magnitude of the value indicating analytical performance between the first analytical data acquisition model and the second analytical data acquisition model.

[0057] There may be cases where a value indicating the analytical performance of one of the first and second three-dimensional analytical performance maps cannot be obtained at the coordinates of pixels constituting the 3D OCT image to be analyzed. One example of such a case is when the analysis data acquisition circuit 103c does not analyze all or part of the tomographic image group due to the specifications or operating conditions of the information processing device 100, resulting in missing values ​​constituting the analysis data and analytical performance map. In such a case, the processing circuit 103 may determine that the analytical performance of the analysis data acquisition model corresponding to the other analytical performance map, in which no missing values ​​are present, is higher without comparing the maps. Alternatively, when a value indicating the analytical performance of one map cannot be obtained, another method may be adopted. For example, with respect to the analytical performance of the analysis data acquisition model corresponding to the analytical performance value of the other map, if the value indicating the analytical performance is equal to or greater than a predetermined threshold, the analytical performance of the model may be determined to be high, and if the value is less than the threshold, the analytical performance of the model may be determined to be low. Alternatively, when a value indicating the analytical performance of both maps cannot be obtained, one predetermined model (e.g., the first analytical data acquisition model) may be determined to have higher analytical performance.

[0058] In step S105, the integrated data acquisition circuit 103d, as a comparison unit, compares the value indicating the analytical performance constituting the first analytical performance map with the value indicating the analytical performance constituting the second analytical performance map for each spatial coordinate constituting the OCT image to be analyzed. If the value indicating the analytical performance constituting the corresponding first three-dimensional analytical performance map is equal to or greater than the value indicating the analytical performance constituting the second three-dimensional analytical performance map, the process proceeds to step S106. On the other hand, if the value indicating the analytical performance constituting the corresponding first three-dimensional analytical performance map is less than the value indicating the analytical performance constituting the second three-dimensional analytical performance map, the process proceeds to step S107.

[0059] In step S106, the integrated data acquisition circuit 103d schedules a job to set the abnormality likelihood of the first analysis data group at the coordinates of the integrated analysis data corresponding to the spatial coordinates of the comparison target, based on the comparison result of step S105. Also, in step S107, the integrated data acquisition circuit 103d schedules a job to set the abnormality likelihood of the second analysis data group at the coordinates of the integrated analysis data corresponding to the spatial coordinates of the comparison target, based on the comparison result of step S105.

[0060] The above process completes the comparison and job scheduling process for each spatial coordinate of the pixel group that constitutes the 3D OCT image to be analyzed, which determines whether the value from the first analysis data group or the second analysis data group will be used for each pixel (each spatial coordinate) that constitutes the integrated analysis data, which is the 3D anomaly likelihood map.

[0061] 9A and 9B show the analysis performance of a typical CNN in the coordinate group of each pixel constituting the integrated analysis data. As shown in the three-dimensional analysis performance map Ma110 in FIG. 9A, the analysis performance is high in the central portion (dark portion in the figure) of the end face perpendicular to the first and second axes in the image data to be analyzed. That is, the reliability of the abnormality likelihood, which is the analysis data, is high in this central portion. On the other hand, the analysis performance is low in the peripheral portion (light portion in the figure) of the end face perpendicular to the first and second axes in the image data to be analyzed. That is, the reliability of the abnormality likelihood, which is the analysis data, is low in this peripheral portion.

[0062] In this embodiment, each tomographic image of the two-dimensional tomographic image group is used as an analysis data acquisition model. and obtains a set of two-dimensional analysis data as an output. Let us assume that the three-dimensional OCT image to be analyzed is processed by a CNN configured by convolution processing with a three-dimensional kernel. In this case, as shown in the three-dimensional analysis performance map Ma120 in FIG. 9B, the peripheral portions of the output three-dimensional anomaly likelihood map, i.e., the portions near the six end faces, are significantly affected by fixed values ​​due to the three-dimensional padding processing, and analysis performance is likely to be low. In other words, when three-dimensional OCT images are processed by the analysis data acquisition model of this embodiment, analysis performance in the central portions of the end faces perpendicular to the first and second axes is higher than when processed by a CNN configured by convolution processing with a three-dimensional kernel.

[0063] As described above, in this embodiment, based on the comparison result between the first three-dimensional analysis performance map and the second three-dimensional analysis performance map, it is determined which abnormality likelihood of the first analysis data group or the second analysis data group to adopt for each pixel of the integrated analysis data. Here, if a predetermined condition is assumed under which the same result (information regarding which abnormality likelihood to adopt) is obtained each time integrated analysis data is generated, the integrated analysis data generated by executing the above processing in advance may be stored in the storage circuitry 102. This allows the processing of steps S102 to S107 to be omitted. As an example, the predetermined condition is that the first axis and the second axis of the 3D OCT image to be analyzed are in predetermined directions, and the size and number of the tomographic images constituting the first tomographic image group and the size and number of the tomographic images constituting the second tomographic image group are predetermined values.

[0064] In step S108, the analysis data acquisition circuit 103c, as analysis data acquisition means, inputs a first group of tomographic images along a first axis in the spatial coordinate system of the OCT image, which is the image data to be analyzed, shown in FIG. 8A, into a first analysis data acquisition model. Then, the analysis data acquisition circuit 103c acquires a first analysis data group, which is an image data group representing an abnormality likelihood map, output from the first analysis data acquisition model. Also, in step S109, the analysis data acquisition circuit 103c, as analysis data acquisition means, inputs a second group of tomographic images along a second axis in the spatial coordinate system of the OCT image, which is the image data to be analyzed, shown in FIG. 8B, into a second analysis data acquisition model. Then, the analysis data acquisition circuit 103c acquires a second analysis data group, which is an image data group representing an abnormality likelihood map, output from the second analysis data acquisition model.

[0065] In step S110, the integrated data acquisition circuit 103d, as integrated data acquisition means, sets a value indicating the abnormality likelihood of the corresponding pixel coordinate in the first or second analysis data group to each pixel constituting the integrated analysis data, which is a three-dimensional abnormality likelihood map, thereby acquiring the integrated analysis data.

[0066] In the processing of steps S108 to S110, the information processing device 100 may perform processing such as multiplying the corresponding analytical performance values ​​in coordinate space in order to reduce low-reliability abnormality likelihoods constituting each analytical data or emphasize high-reliability abnormality likelihoods. Specifically, for example, there is a method of multiplying the value of each pixel indicating the abnormality likelihood constituting the first analytical data group and the second analytical data group by the value indicating the analytical performance constituting the first analytical performance map and the second analytical performance map at the corresponding spatial coordinates. Another method is to multiply the value of each pixel indicating the abnormality likelihood constituting the integrated analytical data by the value indicating the analytical performance constituting the first analytical performance map or the second analytical performance map at the corresponding spatial coordinates.

[0067] As a result, the information processing device 100 according to this embodiment can obtain analysis results that reduce the impact of the degradation of analysis performance in the peripheral parts of the image caused by padding processing, which occurs when processing the three-dimensional image data to be analyzed using CNN.

[0068] The integrated analysis data (the above-mentioned three-dimensional anomaly likelihood map) that is the analysis result may be output to the display 105. The integrated analysis data may also be output to a data management device (not shown) as management target data via the network 200 or a communication cable or communication circuit (not shown). When the integrated analysis data is output as management target data, it does not have to be output to the display 105.

[0069] Modifications of the above embodiment will be described below. In the following description, the same configurations and processes as those of the information processing device 100 described above will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0070] (First Modification) In the first embodiment, the first and second tomographic image groups are acquired from the same three-dimensional image data. However, the first and second tomographic image groups may be acquired from different three-dimensional image data. However, if the imaging ranges of these different image data are not common, i.e., if there are no common spatial coordinates in these image data, the first and second tomographic image groups cannot be used to compare analytical performance. Therefore, the first and second tomographic image groups cannot be used to reduce the impact of degradation in analytical performance due to padding processing in the machine learning model. In this modified example, one of two three-dimensional image data sets with a common imaging range, acquired by imaging the same subject area twice, is used as the first tomographic image group, and the other is used as the second tomographic image group. Specific examples of image data used in this modified example include two three-dimensional image data sets acquired by scanning the same subject from two approximately orthogonal directions using an OCT imaging device or a magnetic resonance imaging device.

[0071] The information processing device 100 according to this modification executes the above-described processing based on the two sets of three-dimensional image data. As a result, the information processing device 100 according to this modification can generate integrated analysis data, which is a three-dimensional anomaly likelihood map that reduces the degradation of analysis performance in the peripheral parts of the image caused by the padding processing of the machine learning model.

[0072] In the information processing device 100 according to this modification, the first group of tomographic images and the second group of tomographic images are acquired from different image data. This makes it possible to improve the reliability of the second group of analysis data by using the first group of analysis data when the reliability of the second group of analysis data is low because the resolution of the second group of tomographic images is lower than that of the first group of tomographic images, for example.

[0073] (Second Modification) In the first embodiment, the first analysis data acquisition model and the second analysis data acquisition model executed by the analysis data acquisition circuit 103c are two different analysis data acquisition models. On the other hand, in this modification, the second analysis data acquisition model is substituted for the first analysis data acquisition model. Alternatively, the first analysis data acquisition model is substituted for the second analysis data acquisition model. In other words, the analysis data acquisition circuit 103c executes only one analysis data acquisition model for two tomographic image groups.

[0074] At this time, the analysis data acquisition circuit 103c substitutes the first analysis performance map for the second analysis performance map and executes the processing of the flowchart in Fig. 7. Alternatively, the analysis data acquisition circuit 103c substitutes the second analysis performance map for the first analysis performance map and executes the processing of the flowchart in Fig. 7. In other words, the analysis data acquisition circuit 103c uses only one analysis performance map for two tomographic image groups.

[0075] Alternatively, when a first group of tomographic images along a first axis in the image data is analyzed using a first analysis data acquisition model, a second group of tomographic images along a second axis is analyzed using the same first analysis data acquisition model. In some cases, it is known that the analytical performance differs between when the data is analyzed using the data acquisition model and when the data is analyzed using the data acquisition model. In such cases, the values ​​indicating the analytical performance constituting the first analytical performance map or the second analytical performance map may be adjusted in advance.

[0076] Specifically, for example, assume that a first analysis data acquisition model is trained using a dataset consisting of images with features similar to those depicted in a first group of tomographic images but dissimilar to those depicted in a second group of tomographic images. This trains the first analysis data acquisition model to improve its analysis performance for feature quantities related to the analysis target for the first group of tomographic images. In this case, the analysis performance of the first analysis data acquisition model tends to be higher when the first group of tomographic images is input than when the second group of tomographic images is input. In such a case, adjustments are made, such as multiplying each value indicating analysis performance constituting the second analysis performance map used by the second analysis data acquisition model by a predetermined value, such as 0.5. This is expected to result in the effect of the abnormality likelihood group constituting the first analysis data group acquired from the first group of tomographic images being adopted over the abnormality likelihood group constituting the second analysis data group acquired from the second group of tomographic images in the integrated analysis data, which is a three-dimensional abnormality likelihood map.

[0077] Alternatively or additionally, the second analysis data acquisition model may be trained using a dataset composed of images having features similar to those depicted in the second group of tomographic images but dissimilar to those depicted in the first group of tomographic images. This allows the second analysis data acquisition model to be trained to improve its analytical performance for feature quantities related to the analysis target for the second group of tomographic images. In this case, the analytical performance of the second analysis data acquisition model tends to be higher when the second group of tomographic images is input than when the first group of tomographic images is input. In such a case, the values ​​indicating analytical performance constituting the first analysis performance map may be adjusted in the same manner as described above.

[0078] Therefore, according to the information processing device 100 of this modified example, analysis of multiple groups of tomographic images can be performed using one analysis data acquisition model, and, for example, the cost of preparing the analysis data acquisition model can be further reduced compared to the first embodiment.

[0079] (Third Modification) In the first embodiment, when the process of step S110 is completed, integrated analysis data that is a three-dimensional abnormality likelihood map is generated. In this modification, the processing circuitry 103 may generate the generated integrated analysis data as a two-dimensional abnormality likelihood map that is projected with an arbitrary axis direction (third axis direction) in the image data to be analyzed as the line of sight direction.

[0080] As an example, as shown in FIG. 9 , an axis orthogonal to both the first and second axes is set as the gaze direction, which serves as the third axis. In this case, the area in the gaze direction where a degradation in analysis performance due to padding processing, which occurs in CNN processing, is likely to occur is reduced. Examples of projection of the integrated analysis data include minimum value projection, maximum value projection, average value projection, weighted addition of minimum value projection and maximum value projection, and volume rendering, regarding the abnormal likelihood group in the gaze direction. The projected two-dimensional abnormal likelihood map may be output to the display 105. The projected two-dimensional abnormal likelihood map may also be output as management target data to a data management device (not shown) via the network 200 or a communication cable or communication circuit (not shown). When the projected two-dimensional abnormal likelihood map is output as management target data, the projected two-dimensional abnormal likelihood map does not necessarily have to be output to the display 105.

[0081] Furthermore, based on the three-dimensional analysis performance map, it is also possible to select the anomaly likelihoods that constitute the integrated analysis data and generate a two-dimensional anomaly likelihood map to be projected. The first analytical performance map and the second analytical performance map are mapped to correspond to the coordinate space of the first analytical data group and the second analytical data group related to the image data to be analyzed. Next, a three-dimensional analytical performance map is generated that can be determined by adopting, for each coordinate in the coordinate space, either the first analytical performance map or the second analytical performance map, whichever has the larger value indicating analytical performance. Then, in the generated three-dimensional analytical performance map, a coordinate group having a value indicating analytical performance equal to or greater than a predetermined threshold is identified, and the above two-dimensional abnormality likelihood map is generated using the abnormality likelihood that constitutes the integrated analytical data corresponding to that coordinate group.

[0082] If an anomaly likelihood map cannot be generated due to a group of coordinates having values ​​indicating analytical performance below a predetermined threshold in the generated three-dimensional analytical performance map, an image or message indicating that there are no analytical results may be generated. Alternatively, each anomaly likelihood value constituting the integrated analytical data may be multiplied by a value indicating analytical performance of the corresponding coordinate in the analytical performance map, thereby adjusting the area with high analytical performance (reliability) to be the main component.

[0083] In addition, when projecting the integrated analysis data for the three-dimensional image data to be analyzed as a two-dimensional abnormality likelihood map, the analysis performance map may also be generated as a two-dimensional map, and a projection similar to the projection described above may be performed.

[0084] As a specific example, there is a method using a two-dimensional projection analysis performance map, which is a projection image obtained by projecting a three-dimensional analysis performance map with the third axis as the line of sight, as shown in Figures 10A and 10B. First, a first projection analysis performance map Ma150 and a second projection analysis performance map Ma160 are mapped to the coordinate space of the integrated analysis data, into which the integrated analysis data is projected with the third axis as the line of sight. Next, for each coordinate in the coordinate space, a two-dimensional third projection analysis performance map is generated using either the first projection analysis performance map Ma150 or the second projection analysis performance map Ma160, whichever has the larger value indicating analysis performance. Then, a coordinate group in the third projection analysis performance map having a value indicating analysis performance equal to or greater than a predetermined threshold is identified, and an abnormality likelihood map is generated that includes only the abnormality likelihood indicated by the projected integrated analysis data corresponding to that coordinate group.

[0085] As another example, first, a first projection analysis performance map Ma150 and a second projection analysis performance map Ma160 are mapped to correspond to the coordinate space of the first projection analysis data and the second projection analysis data. Next, at each coordinate in the coordinate space, the analytical performance values ​​constituting the first projection analysis performance map Ma150 and the second projection analysis performance map Ma160 are compared, and an abnormality likelihood map is generated that employs the abnormality likelihood indicated by the projection analysis data corresponding to the larger value. Note that in this example, integrated analysis data is not used, and therefore the processes from steps S104 to S107 and step S110 of the first embodiment are omitted.

[0086] As described above, the information processing device 100 according to this modification generates integrated analysis data as a projection image that is a two-dimensional abnormality likelihood map, thereby assisting in the discovery of abnormalities depicted in medical images, for example.

[0087] (Fourth Modification) In step S109 in the first embodiment, the analysis data acquisition circuit 103c inputs each image of the second tomographic image group into the second analysis data acquisition model to acquire the second analysis data group. On the other hand, in this modification, tomographic images to be input into the second analysis data acquisition model are selected from the second tomographic image group so that the amount of second analysis data not used in the integrated analysis data is reduced.

[0088] Specifically, in accordance with the jobs scheduled in steps S106 and S107 in the first embodiment, the analysis data constituting the second analysis data group is aggregated. It is possible to identify analysis data that will not be used for the integrated analysis data. Therefore, among the second group of tomographic images, tomographic images corresponding to analysis data that will not be used for the integrated analysis data do not need to be input into the second analysis data acquisition model in step S109.

[0089] As described above, according to the information processing device 100 of this modified example, the calculation cost associated with generating the integrated analytical data can be reduced more than in the first embodiment by omitting the process of generating analytical data that is not used in the integrated analytical data.

[0090] (Fifth Modification) The information processing device 100 according to the first embodiment executes processing using a first axis and a second axis in a spatial coordinate system of three-dimensional image data to be analyzed. On the other hand, the information processing device 100 according to this modification may execute processing using a third axis different from the first axis and the second axis. Here, it is preferable that the third axis is an axis in a direction substantially perpendicular to the first axis and the second axis.

[0091] In this modification, the analysis data acquisition circuit 103c further executes a third analysis data acquisition model, and further uses a third analysis performance map corresponding to the third analysis data acquisition model.

[0092] Next, the processing executed by the information processing device 100 in this modification will be described with reference to FIG. 11. In step S1101, the image acquisition circuitry 103a acquires a first group of tomographic images along a first axis of the subject, a second group of tomographic images along a second axis of the subject, and a third group of tomographic images along a third axis of the subject from the three-dimensional OCT images to be analyzed. Next, in step S1102, the analysis performance map acquisition circuitry 103b maps the first analysis performance map onto the first group of tomographic images to generate a first three-dimensional analysis performance map. Also, in step S1103, the analysis performance map acquisition circuitry 103b maps the second analysis performance map onto the second group of tomographic images to generate a second three-dimensional analysis performance map. Furthermore, in step S1104, the analysis performance map acquisition circuitry 103b maps the third analysis performance map onto the third group of tomographic images to generate a third three-dimensional analysis performance map.

[0093] In S1105, the analysis performance map acquisition circuit 103b maps the first analysis performance map, the second analysis performance map, and the third analysis performance map onto the pixel coordinate space of the three-dimensional OCT image to be analyzed.

[0094] In step S1106, the integrated data acquisition circuit 103d compares the values ​​indicating the analytical performance that make up the first to third three-dimensional analytical performance maps for each spatial coordinate that makes up the three-dimensional OCT image to be analyzed. The integrated data acquisition circuit 103d then identifies the analytical performance map with the highest analytical performance value among the first to third analytical performance maps. Next, in step S1107, the integrated data acquisition circuit 103d schedules a job to set the abnormal likelihood of the first to third analytical data groups corresponding to the identified first to third analytical performance maps at the coordinates of the integrated analytical data corresponding to the spatial coordinates to be compared. Therefore, if the value of the first analytical performance map is highest, the integrated data acquisition circuit 103d schedules a job to set the abnormal likelihood of the first analytical data group corresponding to the pixel coordinates for which the analytical performance comparison was performed at the coordinates of the integrated analytical data corresponding to the pixel coordinates for which the analysis performance comparison was performed. Furthermore, if the value of the second analysis performance map is the highest, the integrated data acquisition circuit 103d schedules a job to set the abnormal likelihood of the second analysis data group corresponding to the pixel coordinates where the analysis performance comparison was performed to the coordinates of the integrated analysis data corresponding to the pixel coordinates where the analysis performance comparison was performed. Furthermore, if the value of the third analysis performance map is the highest, the integrated data acquisition circuit 103d schedules a job to set the abnormal likelihood of the third analysis data group corresponding to the pixel coordinates where the analysis performance comparison was performed to the coordinates of the integrated analysis data corresponding to the pixel coordinates where the analysis performance comparison was performed. Schedule a job to

[0095] If there are multiple highest values ​​among the three analytical performance values ​​to be compared, job priorities may be set in advance and jobs may be scheduled according to the set priorities. That is, cases may arise in which all three values ​​are the same, or the values ​​in the first analytical performance map and the second analytical performance map are both the highest, or the values ​​in the first analytical performance map and the third analytical performance map are both the highest. In such cases, the job priorities are set in advance so that the job for setting the anomaly likelihood of the first analytical data group is given priority over other jobs. As a result, the integrated data acquisition circuit 103d schedules a job for setting the anomaly likelihood of the first analytical data group corresponding to the pixel coordinates for which the analytical performance comparison was performed, for the coordinates of the integrated analytical data corresponding to the pixel coordinates.

[0096] Furthermore, there may be cases where both the second analytical performance map value and the third analytical performance map value are the highest. In this case, the job priority is set in advance so that the job for setting the abnormality likelihood of the second analytical data group takes precedence over the job for setting the abnormality likelihood of the third analytical data group. As a result, the integrated data acquisition circuit 103d schedules a job for setting the abnormality likelihood of the second analytical data group corresponding to the pixel coordinates for which the analytical performance comparison was performed, at the coordinates of the integrated analytical data corresponding to the pixel coordinates.

[0097] Then, in step S1108, the analysis data acquisition circuit 103c inputs the first to third tomographic image groups into the corresponding first to third analysis data acquisition models. The analysis data acquisition circuit 103c then acquires the first to third analysis data groups, which are image data groups representing an abnormality likelihood map output from the first to third analysis data acquisition models. In step S1109, the integrated data acquisition circuit 103d, as integrated data acquisition means, sets a value indicating the abnormality likelihood of the corresponding pixel coordinates in the first to third analysis data groups to each pixel constituting the integrated analysis data, which is a three-dimensional abnormality likelihood map. This allows the integrated data acquisition circuit 103d to acquire the integrated analysis data.

[0098] As a result, the information processing device 100 according to this modification can reduce the performance degradation of the CNN in a wider range than in the first embodiment. Specifically, in addition to the effects of the first embodiment, the analysis performance of the central part of the end face perpendicular to the third axis in the integrated analysis data tends to improve.

[0099] In this modification, a procedure in which a third axis is added to the first embodiment has been shown, but in a similar manner, further spatial coordinate axes may be added to widen the range in which the performance degradation of the CNN can be reduced. In other words, in addition to the three spatial coordinate axes (first axis, second axis, and third axis) described in this modification, four or more spatial coordinate axes may be considered.

[0100] (Sixth Modification) In the first embodiment, values ​​indicating analytical performance constituting a first three-dimensional analytical performance map (hereinafter referred to as "first analytical performance values") are compared with values ​​indicating analytical performance constituting a second three-dimensional analytical performance map (hereinafter referred to as "second analytical performance values"). In the first embodiment, the first analytical performance value and the second analytical performance value are compared, and the abnormality likelihood of the analytical data group corresponding to the higher analytical performance value is adopted as the integrated analytical data. On the other hand, as in this modified example, the abnormality likelihood may be adjusted by weighting the first analytical performance value and the second analytical performance value.

[0101] Specifically, for example, the value of the first analytical performance corresponding to a certain spatial coordinate i in the integrated analytical data is R i1 and the second analytical performance value is R i2 and the abnormal likelihood in the first analysis data set is A i1 , the abnormal likelihood in the second analysis data set is A i2 In this case, the abnormal likelihood A i may be determined by the following weighted average formula (1): A i =(R i1 ×A i1 +R i2 ×A i2 )÷(Ri1 +R i2 )···(1)

[0102] As another example, the abnormal likelihood A i may be determined by the following formula (2) to adjust or emphasize the analysis performance depending on the level of analysis performance. A i =R i1 ×A i1 +R i2 ×A i2 ···(2)

[0103] The above formula is an example of calculating the abnormality likelihood of the integrated analysis data, and a formula modified as appropriate depending on the operating conditions of the information processing device 100, the characteristics of the analysis data acquisition model, etc. may be used.

[0104] As described above, information processing device 100 according to this modification can determine the value of the abnormality likelihood in the integrated analysis data by taking into consideration the values ​​indicating the analysis performance that constitute the multiple analysis performance maps. This can reduce the impact of degradation of analysis performance in the image peripheral portions due to padding processing that occurs when processing three-dimensional image data to be analyzed using CNN, for example, and improve the robustness of the image analysis processing.

[0105] (Seventh Modification) In this modification, a set of values ​​indicating the abnormality likelihood in a predetermined partial region of each analytical data constituting the first analytical data group is replaced with a set of values ​​indicating the abnormality likelihood constituting the second analytical data group, which corresponds to the spatial coordinates of the image data to be analyzed. Therefore, based on the comparison result, the integrated data acquisition circuit 103d does not acquire a portion of the second analytical data group that will not be used in integrating the first analytical data group and the second analytical data group. In this case, it is preferable that the predetermined partial region is a region with low analytical performance in each analytical data constituting the first analytical data group, and a region with high analytical performance in each analytical data constituting the second analytical data group whose spatial coordinates correspond to the predetermined partial region.

[0106] Specifically, a group of values ​​indicating the abnormal likelihood in the peripheral portion or part of the image where analytical performance is likely to deteriorate in each analysis data constituting the first analysis data group is replaced with a group of values ​​indicating the abnormal likelihood constituting the second analysis data group, which corresponds to the spatial coordinates of the image data to be analyzed. Examples of the part of the analysis data include only the left and right peripheral portions of the image, only the top and bottom peripheral portions, etc. Information indicating the region including the group of values ​​indicating the abnormal likelihood to be replaced may be stored in advance in the memory circuitry 102 of the information processing device 100. Alternatively, the region to be replaced may be determined based on the size of each analysis data constituting the first analysis data group. When the region to be replaced is determined based on the size of the analysis data, for example, when the width of the analysis data is 100%, the region to be replaced may be a peripheral region extending 25% from the left edge and a peripheral region extending 25% from the right edge of the left and right edges of the image.

[0107] In this modification, it is not necessary to execute the processing related to the analytical performance map described in the first embodiment. That is, in the information processing device 100 according to this modification, the processing from step S102 to step S107 can be omitted. In addition, in step S110, the information processing device 100 replaces a group of values ​​indicating the abnormal likelihood in a predetermined region of each analytical data constituting the first analytical data group with a group of values ​​indicating the abnormal likelihood constituting the second analytical data group.

[0108] As another method for replacing values ​​indicating the likelihood of abnormality of the analytical data, a group of values ​​calculated by a formula such as Equation (1) or Equation (2) described in the sixth variant above can be used instead of a group of values ​​indicating the likelihood of abnormality of each analytical data constituting the second analytical data group.

[0109] As described above, according to the information processing device 100 of this modified example, it is possible to reduce the degradation of analysis performance in the peripheral parts of an image caused by padding processing, which occurs when processing image data to be analyzed using CNN, without using an analysis performance map.

[0110] (Other embodiments) The above-described configuration and / or processing of the information processing device 100 may be modified without being limited to the above-described embodiments and modifications. For example, the information processing device 100 may perform the same processes on multidimensional image data of four or more dimensions as it performs on three-dimensional image data. That is, for example, to calculate a four-dimensional anomaly likelihood map for the four-dimensional image data to be analyzed, analysis data for each of the three-dimensional image data groups acquired along two or more spatial coordinate axes of the four-dimensional image data may be acquired. Then, the acquired analysis data groups may be integrated according to the three-dimensional analysis performance map corresponding to each analysis data.

[0111] Furthermore, the components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of each device can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0112] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0113] The methods described in the embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. This control program can also be recorded on a non-transitory computer-readable recording medium such as a hard disk, floppy disk (FD), CD-ROM, magneto-optical disk (MO), or DVD, and read from the recording medium and executed by the computer.

[0114] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims.

[0115] Furthermore, the disclosed technology can be embodied as, for example, a system, a device, a method, a program, or a recording medium (storage medium), etc. Specifically, it may be applied to a system consisting of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or it may be applied to an apparatus consisting of a single device.

[0116] The present invention provides a system or apparatus via a network or a storage medium with a program that causes a computer to execute each step of the image processing method according to the above-described embodiment. Such a program is configured to be read and executed by one or more processors in the computer of the system or device, or may be implemented by a circuit (e.g., ASIC) that implements one or more functions.

[0117] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) an image acquisition means for acquiring a first set of tomographic images along a first axis of a subject and a second set of tomographic images along a second axis of the subject; a map acquisition means for acquiring a first map indicating the analytical performance of a feature amount related to an object to be analyzed for the first group of tomographic images, and acquiring a second map indicating the analytical performance of a feature amount related to the object to be analyzed for the second group of tomographic images; a comparison means for comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images; an analysis data acquisition means for inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the first machine learning model as a first analysis data group; and inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second analysis data group; an integrated data acquisition means for integrating the first analysis data group and the second analysis data group to acquire integrated data based on the comparison result by the comparison means; An information processing device comprising: (Configuration 2) 2. The information processing device according to configuration 1, wherein the first machine learning model and the second machine learning model perform padding processing and convolution processing on the tomographic image of the subject. (Configuration 3) 3. The information processing apparatus according to configuration 1 or 2, wherein the image acquisition means acquires the first group of tomographic images and the second group of tomographic images from the same three-dimensional image. (Configuration 4) The information processing device according to configuration 1 or 2, characterized in that the integrated data acquisition means acquires integrated data based on the comparison result by integrating the first analysis data group and the second analysis data group based on the comparison result. (Configuration 5) 3. The information processing apparatus according to configuration 1 or 2, wherein the image acquisition means acquires the first group of tomographic images and the second group of tomographic images from different three-dimensional images of the subject. (Configuration 6) 5. The information processing device according to any one of configurations 1 to 4, wherein the first machine learning model and the second machine learning model are the same machine learning model. (Configuration 7) the first machine learning model is trained to improve analysis performance of feature quantities related to the analysis object for the first group of tomographic images; the second machine learning model is trained to improve analysis performance of feature quantities related to the analysis object for the second group of tomographic images. 6. The information processing device according to any one of configurations 1 to 5. (Configuration 8) The information processing device according to any one of configurations 1 to 6, wherein the analysis data acquisition means, based on the comparison result, does not acquire a portion of the second analysis data group that the integrated data acquisition means will not use to integrate the first analysis data group and the second analysis data group. (Configuration 9) The information processing device according to configuration 8, wherein the portion of the second analysis data group is data of the second analysis data group corresponding to spatial coordinates where the analysis performance indicated by the first map is higher than the analysis performance indicated by the second map. (Configuration 10) 10. The information processing device according to any one of configurations 1 to 9, wherein the first analysis data group and the second analysis data group are data groups indicating a distribution of the degree of abnormality regarding the feature amount of the object to be analyzed in the first tomographic image group and the second tomographic image group. (Configuration 11) 11. The information processing apparatus according to any one of configurations 1 to 10, further comprising image generating means for generating a projection image by projecting the integrated data in an arbitrary axial direction of the subject. (Configuration 12) The information processing device according to configuration 11, wherein the image generation means generates the projection image for each of the spatial coordinates using a group of analysis data corresponding to the map with higher analytical performance out of the first map and the second map. (Configuration 13) the image acquisition means acquires a third group of tomographic images of the subject along a third axis; the map acquisition means acquires a third map indicating analysis performance of feature quantities related to the object to be analyzed, the third map corresponding to the third group of tomographic images; the comparison means compares the first map, the second map, and the third map for each of the spatial coordinates; the analysis data acquisition means inputs the third group of tomographic images into a third machine learning model that outputs feature quantities related to the object to be analyzed, and acquires the feature quantities output from the third machine learning model as a third group of analysis data; The integrated data acquisition means integrates the first analysis data group, the second analysis data group, and the third analysis data group to acquire the integrated data based on the comparison result. 13. The information processing device according to any one of configurations 1 to 12. (Configuration 14) 14. The information processing device according to any one of configurations 1 to 13, wherein the integrated data acquisition means adjusts the integrated data according to the analytical performance indicated by the first map and the second map for each spatial coordinate. (Configuration 15) 15. The information processing device according to any one of configurations 1 to 14, wherein the map acquisition means weights the analytical performance of the first map using statistical values ​​obtained based on the correct answer data and output data related to the first machine learning model, and weights the analytical performance of the second map using statistical values ​​obtained based on the correct answer data and output data related to the second machine learning model. (Configuration 16) 15. The information processing device according to any one of configurations 1 to 14, wherein the map acquisition means acquires the first map using an output of the first machine learning model when data in which the values ​​for each spatial coordinate are predetermined values ​​is input to the first machine learning model initialized with predetermined parameters, and acquires the second map using an output of the second machine learning model when data in which the values ​​for each spatial coordinate are predetermined values ​​is input to the second machine learning model initialized with predetermined parameters. (Configuration 17) an image acquisition means for acquiring a first set of tomographic images along a first axis of a subject and a second set of tomographic images along a second axis of the subject; a map acquisition means for acquiring a first map indicating analysis performance of feature quantities related to an object to be analyzed, corresponding to the first group of tomographic images, and acquiring a second map indicating analysis performance of feature quantities related to the object to be analyzed, corresponding to the second group of tomographic images; a comparison means for comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images; an analysis data acquisition means for inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the first machine learning model as a first analysis data group; and inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second analysis data group; an integrated data acquisition means for adjusting the feature amounts of the first analysis data group and the feature amounts of the second analysis data group using the analysis performance of the first map and the analysis performance of the second map, and integrating the adjusted first analysis data group and the adjusted second analysis data group to acquire integrated data based on the comparison result by the comparison means; An information processing device comprising: (Configuration 18) an image acquisition means for acquiring a first set of tomographic images along a first axis of a subject and a second set of tomographic images along a second axis of the subject; an analysis data acquisition means for inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to an analysis object, acquiring the feature quantities output from the first machine learning model as a first analysis data group, inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the analysis object, and acquiring the feature quantities output from the second machine learning model as a second analysis data group; an integrated data acquisition means for acquiring integrated data by integrating the first analysis data group and the second analysis data group by replacing feature amounts of the first analysis data group at spatial coordinates corresponding to a predetermined region among spatial coordinates including the first tomographic image group and the second tomographic image group with feature amounts of the second analysis data group; An information processing device comprising: (Method 1) acquiring a first set of slice images of a subject along a first axis and a second set of slice images of the subject along a second axis; acquiring a first map indicating analysis performance of feature quantities related to an object to be analyzed corresponding to the first group of tomographic images, and acquiring a second map indicating analysis performance of feature quantities related to the object to be analyzed corresponding to the second group of tomographic images; comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images; inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the first machine learning model as a first group of analysis data; inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second group of analysis data; Integrating the first analysis data group and the second analysis data group to obtain integrated data based on a comparison result between the first map and the second map; An information processing method comprising: (Method 2) acquiring a first set of slice images of a subject along a first axis and a second set of slice images of the subject along a second axis; acquiring a first map indicating analysis performance of feature quantities related to an object to be analyzed corresponding to the first group of tomographic images, and acquiring a second map indicating analysis performance of feature quantities related to the object to be analyzed corresponding to the second group of tomographic images; comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images; The first group of tomographic images is subjected to a first machine learning model that outputs feature amounts related to the object to be analyzed. inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring feature quantities output from the second machine learning model as a second group of analysis data; adjusting the feature amounts of the first analysis data group and the feature amounts of the second analysis data group using the analysis performance of the first map and the analysis performance of the second map, and integrating the adjusted first analysis data group and the adjusted second analysis data group to obtain integrated data based on a comparison result between the first map and the second map; An information processing method comprising: (Method 3) acquiring a first set of slice images of a subject along a first axis and a second set of slice images of the subject along a second axis; inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to an object to be analyzed, and acquiring the feature quantities output from the first machine learning model as a first group of analysis data; inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second group of analysis data; a step of integrating the first analysis data group and the second analysis data group to obtain integrated data by replacing feature amounts of the first analysis data group at spatial coordinates corresponding to a predetermined region among spatial coordinates including the first tomographic image group and the second tomographic image group with feature amounts of the second analysis data group; An information processing method comprising: (program) A program for causing a computer to execute each step of the information processing method according to any one of Methods 1 to 3. [Explanation of symbols]

[0118] 100 information processing device, 103 processing circuit, 103a image acquisition circuit, 103b analytical map acquisition circuit, 103c analysis data acquisition circuit, 103d integrated data acquisition circuit

Claims

1. an image acquisition means for acquiring a first set of tomographic images of a subject along a first axis and a second set of tomographic images of the subject along a second axis; a map acquisition means for acquiring a first map indicating analytical performance of feature quantities related to an object to be analyzed for the first group of tomographic images, and acquiring a second map indicating analytical performance of feature quantities related to the object to be analyzed for the second group of tomographic images; a comparison means for comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images; an analysis data acquisition means for inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to the object to be analyzed, acquiring the feature quantities output from the first machine learning model as a first analysis data group, inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second analysis data group; an integrated data acquisition means for integrating the first analysis data group and the second analysis data group to acquire integrated data based on the comparison result by the comparison means; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the first machine learning model and the second machine learning model perform padding processing and convolution processing on the tomographic image of the subject.

3. 3. The information processing apparatus according to claim 1, wherein the integrated data acquisition means acquires integrated data based on the comparison result by integrating the first analysis data group and the second analysis data group based on the comparison result.

4. 3. The information processing apparatus according to claim 1, wherein the image acquisition means acquires the first group of tomographic images and the second group of tomographic images from the same three-dimensional image.

5. 3. The information processing apparatus according to claim 1, wherein the image acquisition means acquires the first group of tomographic images and the second group of tomographic images from different three-dimensional images of the subject.

6. The information processing apparatus according to claim 1 , wherein the first machine learning model and the second machine learning model are the same machine learning model.

7. the first machine learning model is trained to improve analysis performance of feature quantities related to the analysis object for the first group of tomographic images; the second machine learning model is trained to improve analysis performance of feature quantities related to the analysis object for the second group of tomographic images; 3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

8. The information processing device described in claim 1 or 2, characterized in that the analysis data acquisition means does not acquire a portion of the second analysis data group that the integrated data acquisition means will not use to integrate the first analysis data group and the second analysis data group based on the comparison result.

9. 9. The information processing device according to claim 8, characterized in that the portion of the second analysis data group is data of the second analysis data group corresponding to spatial coordinates where the analysis performance indicated by the first map is higher than the analysis performance indicated by the second map.

10. 3. The information processing device according to claim 1, wherein the first analysis data group and the second analysis data group are data groups indicating a distribution of the degree of abnormality regarding the feature amount of the object to be analyzed in the first tomographic image group and the second tomographic image group.

11. 3. The information processing apparatus according to claim 1, further comprising an image generating unit that generates a projection image by projecting the integrated data in an arbitrary axial direction of the subject.

12. 12. The information processing apparatus according to claim 11, wherein the image generating means generates the projection image for each of the spatial coordinates using a group of analysis data corresponding to the map with higher analytical performance out of the first map and the second map.

13. the image acquisition means acquires a third group of tomographic images of the subject along a third axis; the map acquisition means acquires a third map indicating analysis performance of feature amounts related to the analysis object corresponding to the third group of tomographic images; the comparison means compares the first map, the second map, and the third map for each of the spatial coordinates; the analysis data acquisition means inputs the third group of tomographic images into a third machine learning model that outputs feature quantities related to the object to be analyzed, and acquires the feature quantities output from the third machine learning model as a third group of analysis data; The integrated data acquisition means integrates the first analysis data group, the second analysis data group, and the third analysis data group to acquire the integrated data based on the comparison result.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

14. 3. The information processing apparatus according to claim 1, wherein the integrated data acquisition means adjusts the integrated data according to analytical capabilities indicated by the first map and the second map for each of the spatial coordinates.

15. The information processing device described in claim 1 or 2, characterized in that the map acquisition means weights the analytical performance of the first map using statistical values ​​obtained based on the correct answer data and output data related to the first machine learning model, and weights the analytical performance of the second map using statistical values ​​obtained based on the correct answer data and output data related to the second machine learning model.

16. The information processing device described in claim 1 or 2, characterized in that the map acquisition means acquires the first map using the output of the first machine learning model when data in which the values ​​for each spatial coordinate are predetermined values ​​is input to the first machine learning model initialized with predetermined parameters, and acquires the second map using the output of the second machine learning model when data in which the values ​​for each spatial coordinate are predetermined values ​​is input to the second machine learning model initialized with predetermined parameters.

17. an image acquisition means for acquiring a first set of tomographic images of a subject along a first axis and a second set of tomographic images of the subject along a second axis; a map acquisition means for acquiring a first map indicating analysis performance of feature quantities related to an analysis object corresponding to the first group of tomographic images, and acquiring a second map indicating analysis performance of feature quantities related to the analysis object corresponding to the second group of tomographic images; a comparison means for comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images; The first group of tomographic images is input to a first machine learning model that outputs feature quantities related to the object to be analyzed, the feature quantities output from the first machine learning model are acquired as a first group of analysis data, and the second group of tomographic images is input to a second machine learning model that outputs feature quantities related to the object to be analyzed. an analysis data acquisition means for inputting the feature values ​​into a machine learning model and acquiring the feature values ​​output from the second machine learning model as a second analysis data group; an integrated data acquisition means for adjusting the feature amounts of the first analysis data group and the feature amounts of the second analysis data group using the analysis performance of the first map and the analysis performance of the second map, and integrating the adjusted first analysis data group and the adjusted second analysis data group to acquire integrated data based on the comparison result by the comparison means; An information processing device comprising:

18. an image acquisition means for acquiring a first set of tomographic images of a subject along a first axis and a second set of tomographic images of the subject along a second axis; an analysis data acquisition means for inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to an analysis object, acquiring the feature quantities output from the first machine learning model as a first analysis data group, inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the analysis object, and acquiring the feature quantities output from the second machine learning model as a second analysis data group; an integrated data acquisition means for acquiring integrated data by integrating the first analysis data group and the second analysis data group by replacing feature amounts of the first analysis data group at spatial coordinates corresponding to a predetermined region among spatial coordinates including the first tomographic image group and the second tomographic image group with feature amounts of the second analysis data group; An information processing device comprising:

19. acquiring a first set of tomographic images of a subject along a first axis and a second set of tomographic images of the subject along a second axis; acquiring a first map indicating analysis performance of feature quantities related to an analysis object corresponding to the first group of tomographic images, and acquiring a second map indicating analysis performance of feature quantities related to the analysis object corresponding to the second group of tomographic images; comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images; inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the first machine learning model as a first group of analysis data; inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second group of analysis data; Integrating the first analysis data group and the second analysis data group to obtain integrated data based on a comparison result between the first map and the second map; An information processing method comprising:

20. acquiring a first set of tomographic images of a subject along a first axis and a second set of tomographic images of the subject along a second axis; acquiring a first map indicating analysis performance of feature quantities related to an analysis object corresponding to the first group of tomographic images, and acquiring a second map indicating analysis performance of feature quantities related to the analysis object corresponding to the second group of tomographic images; comparing the first map with the second map for each spatial coordinate including the first group of tomographic images and the second group of tomographic images; inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the first machine learning model as a first group of analysis data; inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the object to be analyzed, and acquiring the feature quantities output from the second machine learning model as a second group of analysis data; The first analysis is performed using the analysis performance of the first map and the analysis performance of the second map. adjusting the feature amounts of the data group and the feature amounts of the second analysis data group, and integrating the adjusted first analysis data group and the second analysis data group to obtain integrated data based on a comparison result between the first map and the second map; An information processing method comprising:

21. acquiring a first set of tomographic images of a subject along a first axis and a second set of tomographic images of the subject along a second axis; inputting the first group of tomographic images into a first machine learning model that outputs feature quantities related to an analysis object, and acquiring the feature quantities output from the first machine learning model as a first analysis data group; inputting the second group of tomographic images into a second machine learning model that outputs feature quantities related to the analysis object, and acquiring the feature quantities output from the second machine learning model as a second analysis data group; a step of replacing feature quantities of the first analysis data group at spatial coordinates corresponding to a predetermined region among spatial coordinates including the first tomographic image group and the second tomographic image group with feature quantities of the second analysis data group, thereby integrating the first analysis data group and the second analysis data group to obtain integrated data; An information processing method comprising:

22. A program for causing a computer to execute each step of the information processing method according to any one of claims 19 to 21.

Images