Information processing device, information processing method, and recording medium

Abstract

The aim is to provide an information processing apparatus, an information processing method, and a recording medium that can suppress the omission of lesions in medical images. An information processing apparatus is provided, comprising a data acquisition unit and a data processing unit. The data acquisition unit acquires medical image data and additional data. The additional data includes at least one of display-related data and environmental data. The display-related data defines the presentation mode of an image displayed on a display screen. The environmental data represents the surrounding environment of the display. The data processing unit calculates omission suppression data based on the medical image data and the additional data. The omission suppression data is data in the medical image data that suppresses the omission of lesions.

Description

Technical Field

[0001] This invention relates to an information processing apparatus, an information processing method, and a computer program. Background Technology

[0002] Radiologists use medical images, such as mammograms, to determine the presence or absence of lesions. This determination is performed carefully by the radiologist, but on the other hand, radiologists sometimes have to interpret multiple medical images, and there is a possibility that lesions may be overlooked. For example, Patent Document 1 discloses a technique that uses computer image analysis to calculate the risk of physical lesions, etc., in mammograms in which potential lesions cannot be observed with the naked eye.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: U.S. Patent Application Publication No. 2017 / 0249739 Summary of the Invention

[0006] (The problem the invention aims to solve)

[0007] For example, the way medical images are presented depends on factors such as the monitor used by the radiologist or the environment in which the radiologist interprets the images. Furthermore, the calculated risk also varies depending on these factors. However, the technology in Patent Document 1 does not consider these factors when calculating the aforementioned risk, therefore the calculated risk deviates from the actual situation, increasing the possibility that lesions in medical images may be overlooked.

[0008] The present invention was made in view of the following circumstances, and its object is to provide an information processing apparatus, an information processing method, and a computer program that can suppress the situation where lesions in medical images are ignored.

[0009] (Technical solution used to solve the problem)

[0010] According to the present invention, an information processing apparatus is provided, comprising a data acquisition unit and a data processing unit. The data acquisition unit acquires medical image data and additional data. The additional data includes at least one of display-related data and environmental data. The display-related data is data used to define the presentation mode of an image displayed on a display unit of a display. The environmental data is data representing the surrounding environment of the display. The data processing unit calculates neglect suppression data based on the medical image data and the additional data. The neglect suppression data is data in the medical image data that suppresses lesion neglect.

[0011] According to the present invention, the data processing unit is capable of calculating data that suppresses lesion ignoring in medical image data (ignoring suppression data). The ignoring suppression data is based on additional data including at least one of display-related data and environmental data. Therefore, the ignoring suppression data takes into account the above-mentioned factors, and as a result, it is possible to suppress the situation where lesions in medical images are ignored.

[0012] Various embodiments of the present invention are illustrated below. The embodiments shown below can be combined with each other.

[0013] Preferably, an information processing apparatus is provided, wherein the neglect suppression data includes image data representing regions within the medical image data where lesions are easily ignored.

[0014] Preferably, an information processing apparatus is provided, wherein the neglect suppression data includes fractional data on the probability that lesions in the medical image data are ignored.

[0015] Preferably, an information processing apparatus is provided, wherein the neglect suppression data includes location data, which is data on the location of a specific lesion that is easily ignored in the region of the medical image data.

[0016] Preferably, an information processing apparatus is provided, wherein the display-related data includes at least one of display setting values, display specifications, viewer setting values, and display measurement values. The display setting values ​​are settings used to define the presentation mode of an image displayed on the display unit. The display specifications represent the pre-existing characteristics of the display. The viewer setting values ​​are settings used to define the presentation mode of an image displayed on the display unit and are also settings for an application program used to display an image on the display unit. The display measurement values ​​are measurement values ​​of the brightness or chromaticity of the display unit.

[0017] Preferably, an information processing device is provided, wherein the environmental data includes at least one of an illuminance value and a distance measurement value, wherein the illuminance value represents the illuminance around the display unit, and the distance measurement value represents the distance between the display and a human body.

[0018] Preferably, an information processing apparatus is provided, wherein the data processing unit calculates the probability based on a learning model that inputs the medical image data and the additional data and outputs a probability, the probability being a value indicating whether a lesion is easily ignored, and the data processing unit generates the ignoring suppression data based on the probability.

[0019] Furthermore, according to another aspect of the embodiments of the present invention, an information processing method is provided, comprising an acquisition step and a calculation step. In the acquisition step, medical image data and additional data are acquired. The additional data includes at least one of display-related data and environmental data. The display-related data is used to define the presentation mode of an image displayed on a display screen. The environmental data is data representing the surrounding environment of the display. In the calculation step, neglect suppression data is calculated based on the medical image data and the additional data. The neglect suppression data is data in the medical image data that suppresses lesion neglect.

[0020] In another aspect of the present invention, a computer program is provided that enables a computer to execute an information processing method comprising an acquisition step and a calculation step. In the acquisition step, medical image data and additional data are acquired. The additional data includes at least one of display-related data and environmental data. The display-related data is used to define the presentation mode of an image displayed on a display screen. The environmental data is data representing the surrounding environment of the display. In the calculation step, neglect suppression data is calculated based on the medical image data and the additional data. The neglect suppression data is data in the medical image data that suppresses lesion neglect. Attached Figure Description

[0021] Figure 1 This is a functional block diagram of the first embodiment. Figure 1 This schematically illustrates the flow of various data during the application phase of the information processing system 100.

[0022] Figure 2 yes Figure 1 The diagram shows the specific functional block diagram of the data processing unit 4 in the application stage.

[0023] Figure 3 The diagram illustrates the flow of various data during the learning phase of the information processing device 1.

[0024] Figure 4 yes Figure 3 The diagram shows the specific functional block diagram of the data processing unit 4 in the learning stage.

[0025] Figure 5 This is a schematic diagram representing an example of medical image data d2.

[0026] Figure 6 This is a schematic diagram representing an example of probability graph d21.

[0027] Figure 7 This is a schematic diagram representing an example of candidate pixel image d22.

[0028] Figure 8 This is a schematic diagram representing an example of ignoring region map d23.

[0029] Figure 9 This is a schematic diagram illustrating an example of ignoring the suppressed data d10.

[0030] Figure 10 This is a variation of the information processing apparatus 1 of the first embodiment. Figure 10 The diagram illustrates the flow of various data during the application phase of the information processing system 100 in Modified Example 1.

[0031] Figure 11 middle, Figure 11 A is a schematic representation of the ignored area of ​​line L used for scoring, diagram d23. Figure 11 B represents a graph with the position of each pixel on line L as the horizontal axis and the probability P of each pixel on line L as the vertical axis.

[0032] Figure 12 This is a functional block diagram of the data processing unit 4 and the display 21 of Modified Example 4.

[0033] Figure 13 This is a functional block diagram of the second embodiment. Figure 13 This schematically illustrates the flow of various data during the operation of the display 21 (information processing device). Detailed Implementation

[0034] Embodiments of the present invention are described below with reference to the accompanying drawings. The various features shown in the embodiments described below can be combined with each other. Furthermore, each feature independently makes the invention possible.

[0035] 1. First Implementation Method

[0036] like Figure 1 As shown, the information processing system 100 of the first embodiment includes an information processing device 1 and a display 21. The information processing device 1 includes a processing unit Ct, an output unit 5, and a storage unit 6. The processing unit Ct includes a data acquisition unit 2, a preprocessing unit 3, and a data processing unit 4. Figure 2 As shown, the data processing unit 4 includes a probability calculation unit 4A, a post-processing unit 4B, and a generation unit 4C. Furthermore, the post-processing unit 4B includes a candidate pixel extraction unit 4B1 and a region forming unit 4B2.

[0037] Each of the above-mentioned components can be implemented in software or hardware. In the case of software implementation, various functions can be achieved by executing a computer program through the CPU. The program can be stored in the built-in storage or in a computer-readable, non-transitory recording medium. Furthermore, programs stored in external storage can be read, enabling implementation through what is known as cloud computing. In the case of hardware implementation, various circuits such as ASICs, FPGAs, or DRPs can be used. In the first embodiment, various information or concepts containing such information are involved, but these are displayed as a set of bits of binary numbers consisting of 0s and 1s, showing the high and low values ​​of signal values, and communication or computation can be performed through the aforementioned software or hardware methods.

[0038] The display 21 includes a display unit 22, a data acquisition unit 23, an output unit 24, a storage unit 25, and a light sensor 26. The data acquisition unit 23 acquires image data, etc., processed by the information processing device 1, and the display unit 22 displays the acquired image data. The display unit 22 may be configured as, for example, an LCD display, a CRT display, an OLED display, etc.

[0039] 1-1 Data Summary

[0040] The processing unit Ct of the information processing device 1 is configured to acquire medical image data d1 and additional data during the learning and application phases, and generate neglect suppression data d10. The neglect suppression data d10 includes regions within the medical image data that lesions are easily overlooked by radiologists (equivalent to...). Figure 9 The image data includes the neglected regions Rg1 to Rg3. In other words, the neglect suppression data d10 contains image data indicating locations where lesions are easily overlooked by radiologists when interpreting images. By using this neglect suppression data d10 to interpret images, radiologists can determine locations that require special attention and can conserve their attention during image interpretation. Furthermore, the information processing device 1 suppresses the decrease in radiologist attention during image interpretation, thereby preventing lesions in medical images from being overlooked.

[0041] It should be noted that in the first embodiment, the ignored suppression data d10 is image data that emphasizes the area of ​​the lesion that is easily ignored, but it is not limited to this. The ignored suppression data d10 can also be image data that uses an emphasis line to surround the area of ​​the lesion that is easily ignored. That is, the emphasized area does not have to be the area of ​​the lesion that is easily ignored itself, and it can be wider than the area of ​​the lesion that is easily ignored.

[0042] Here, we will explain the various data used in the information processing device 1 and the display 21.

[0043] 1-1-1 Medical image data d1, d2

[0044] Medical image data d1 may be, for example, mammography image data, ultrasound image data, MRI image data, CT image data, chest X-ray image data, and angiography image data. In the first embodiment, medical image data d1 is mammography image data (see...). Figure 5 A mammogram is a digital image composed of multiple pixels. Each pixel has a pixel value. For example... Figure 4 As shown, a mammogram typically includes the pectoralis major muscle region G and the breast region B. The pectoralis major muscle region G corresponds to the pectoralis major muscle, and the breast region B corresponds to the entire breast. Breast region B contains the mammary gland region R. The mammary gland region R is a narrower region than breast region B. Breast region R contains mammary gland pixels and fat pixels. Mammary gland pixels are pixels corresponding to the mammary gland itself, and fat pixels are pixels corresponding to fat tissue; these are the pixels outside the mammary gland pixels within breast region R. Breast region R is the region that roughly surrounds the mammary gland pixels.

[0045] In the preprocessing unit 3 of the processing unit Ct, medical image data d1 is converted into medical image data d2. Medical image data d2 is, for example, data converted from image size or window level.

[0046] 1-1-2 Additional Data

[0047] The supplementary data includes at least one of the display-related data d3 and the environmental data d4. In the first embodiment, the supplementary data includes both the display-related data d3 and the environmental data d4. During the learning phase, the information processing device 1 processes not only the medical image data d1 but also the supplementary data, enabling it to perform machine learning more effectively to better suit the radiologist's interpretation of the image environment. In other words, the information processing device 1 can more appropriately perform machine learning on locations where lesions are easily overlooked by radiologists, while taking into account the interpretation of the image environment.

[0048] It should be noted that another approach is conceivable: the information processing device, without relying on machine learning, calculates the locations of lesions that are easily overlooked based on the brightness of each pixel in the image data. However, such a method might, for example, only identify lesions that are easily overlooked based on areas of high brightness. Not all lesions that are easily overlooked are located in areas of high brightness. The information processing device 1 can learn to take into account not only overlooking factors such as the brightness of the image data, but also overlooking factors based on the radiologist's experience.

[0049] 1-1-2-1 Monitor Association Data d3

[0050] Display-related data d3 is data used to define the image presentation mode displayed on the display unit 22 of the display 21. Furthermore, display-related data d3 includes at least one of display settings, display specifications, and viewer settings. In the first embodiment, display-related data d3 includes all three: display settings, display specifications, and viewer settings.

[0051] <Monitor Settings>

[0052] The display settings are settings used to define the way the image is displayed on the display unit 22.

[0053] Monitor settings may include, for example, brightness settings, gain settings, contrast settings, contrast ratio settings, color temperature settings, hue settings, color depth settings, sharpness settings, etc. Monitor settings include at least one of these settings.

[0054] The brightness setting is a setting related to the overall brightness of the image. It should be noted that the brightness setting is not only a setting related to the overall brightness of the image, but may also include a setting related to the brightness of a specific area (region of interest) defined by the radiologist.

[0055] The gain setting is the brightness setting value for red, green, and blue respectively.

[0056] The contrast ratio setting is a value that uses a ratio to represent the difference in brightness between the white and black areas of an image. It should be noted that the contrast ratio setting can also be a value that uses a ratio to represent the difference in brightness between displayed white and displayed black.

[0057] The hue setting is a setting related to the hue of the image.

[0058] The sharpness setting is a setting related to the adjustment of the image outline.

[0059] <Display Specifications>

[0060] The display specifications indicate the features that the display 21 is designed to have.

[0061] Display specifications may include, for example, the glare characteristics and resolution of display 21. Display specifications include at least one of glare characteristics and resolution.

[0062] When the display unit 22 of the display 21 is an LCD display, the glare characteristic is a characteristic that indicates whether the display unit 22 is composed of glare-emitting liquid crystal or non-glare-emitting liquid crystal.

[0063] <Viewer Settings>

[0064] Viewer settings are settings used to define the image presentation mode displayed on display unit 22. Furthermore, viewer settings are settings for the application that displays the image on display unit 22. This application is, for example, pre-stored in information processing device 1.

[0065] Viewer settings may include, for example, settings for black-and-white inversion, masking, gamma transformation, equal scaling, pseudo-color, sharpening, and contrast enhancement. Viewer settings include at least one of these settings.

[0066] Black-white inversion processing is an image processing technique that reverses the white and black values ​​in an image.

[0067] Masking is an image processing technique that extracts only specific portions of medical image data.

[0068] Gamma transform is an image processing technique that transforms the gamma value to correct gamma characteristics.

[0069] Equal-multiplication processing is image processing that multiplies the pixels within a predetermined range.

[0070] Pseudo-color processing is an image processing technique that uses pseudo-color to color an image.

[0071] Sharpening is an image processing technique that makes a blurry image sharp.

[0072] Contrast enhancement processing is an image processing technique that corrects the brightness, gain, and gamma values ​​of an image.

[0073] It should be noted that in the first embodiment, the viewer setting value is described as a setting value used in the application of the information processing device 1, but it is not limited to this. Alternatively, the display 21 may have such an application, and the display 21 may use the application to determine the setting value.

[0074] 1-1-2-2 Environmental Data d4

[0075] Environmental data d4 represents data about the surrounding environment of the display 21.

[0076] In the first embodiment, the environmental data d4 includes illuminance values.

[0077] The illuminance value represents the illuminance around the display unit 22. That is, the illuminance value corresponds to the illuminance in the space where the display 21 is disposed. In the first embodiment, the illuminance value can be obtained using the light sensor 26 of the display 21.

[0078] 1-2 Structural Description of Information Processing Device 1

[0079] The information processing device 1 includes a processing unit Ct, an output unit 5, and a storage unit 6.

[0080] In the first embodiment, the case of using the information processing device 1 to process various data in two phases, namely the application phase and the learning phase, is described. It should be noted that in the learning phase, an information processing device with higher computing power than the information processing device used in the application phase may also be used.

[0081] 1-2-1 Processing Unit Ct

[0082] The processing unit Ct includes a data acquisition unit 2, a preprocessing unit 3, and a data processing unit 4.

[0083] 1-2-1-1 Data Acquisition Department 2

[0084] like Figure 1 As shown, the data acquisition unit 2 is configured to acquire medical image data d1 and display-related data d3 (display specifications and viewer settings) from the storage unit 6 during the application phase. The data acquisition unit 2 is also configured to acquire display-related data d3 (display settings) and environmental data d4 (illuminance values) from the display 21 during the application phase.

[0085] And, as Figure 3 As shown, the data acquisition unit 2 is configured to acquire medical image data d1, display-related data d3 (display specifications, viewer settings, and display settings) and environmental data d4 (illuminance value) during the learning phase.

[0086] 1-2-1-2 Pretreatment Section 3

[0087] The preprocessing unit 3 performs various preprocessing operations on the medical image data d1. Preprocessing is performed to bring the medical image data d1 into a state suitable for processing by the data processing unit 4. The preprocessing unit 3 converts the medical image data d1 into medical image data d2. The preprocessing unit 3 performs, for example, size matching processing, window level adjustment processing, and noise removal processing. If not required, some or all of these processes performed by the preprocessing unit 3 may be omitted.

[0088] In the size matching process, the medical image data d1 is size-matched. The resolution of the medical image data d1 varies depending on the camera device or settings. This means that the actual size of each pixel varies depending on the input image. The size matching unit adjusts each pixel to a predetermined size to eliminate the unevenness in detection accuracy caused by the different sizes of each pixel.

[0089] In the window level adjustment process, window level adjustment is performed on the medical image data d1. Window level adjustment refers to improving the contrast of a specific grayscale region in an image with a wide range of grayscale values. By performing window level adjustment, the visibility of the medical image data d1 can be improved.

[0090] Noise removal processing is performed on medical image data d1. Medical image data d1 sometimes contains noise (e.g., manually added labels) that reduces the accuracy of lesion analysis and extraction, particularly in areas easily overlooked by radiologists. Therefore, such noise is removed during noise removal processing.

[0091] 1-2-1-3 Data Processing Unit 4

[0092] The data processing unit 4 includes a probability calculation unit 4A, a post-processing unit 4B, a generation unit 4C, and an error calculation unit 4D.

[0093] <Probability Calculation Section 4A>

[0094] The probability calculation unit 4A calculates the probability P for each pixel px in the medical image data d2. Here, probability P is a value indicating whether the part (pixel) corresponding to that probability P is a lesion that is easily overlooked by radiologists. Specifically, as... Figure 6 As shown, the probability calculation unit 4A generates a probability map d21, in which a specific probability P is given for each pixel px. In the first embodiment, the probability map covers the entire area of ​​the medical image data d2. The probability P can be represented, for example, as a value in the range of 0 to 1. The higher the value of probability P, the higher the probability that the lesion is a part (pixel) that is easily overlooked by radiologists.

[0095] The probability P can be calculated using a learning model that outputs probability P based on the input medical image data d2 and additional data. In the first embodiment, the learning model (machine learning model) of the data processing unit 4 (probability calculation unit 4A) can utilize a fully convolutional network (FCN), which is a type of convolutional neural network. It should be noted that in... Figure 2 During the application phase, the learning of data processing unit 4 is completed, but on the other hand, in Figure 4 During the learning phase, the data processing unit 4 is learning. In other words, during the application phase, the weight coefficients of the neural network filter in the probability calculation unit 4A have been determined, but during the learning phase, the weight coefficients of the neural network filter in the probability calculation unit 4A have not been determined and will be updated appropriately.

[0096] <Post-processing unit 4B>

[0097] The post-processing unit 4B extracts the ignored region Rg based on probability P. For example... Figure 8As shown, the neglected region Rg represents the area of ​​the lesion that is easily overlooked by radiologists. In the first embodiment, the neglected region Rg includes three neglected regions Rg1 to Rg3. The post-processing unit 4B includes a candidate pixel extraction unit 4B1 and a region forming unit 4B2.

[0098] · Candidate pixel extraction unit 4B1

[0099] The candidate pixel extraction unit 4B1 performs thresholding processing on the probability map d21. Specifically, the candidate pixel extraction unit 4B1 extracts pixels in the probability map d21 whose probability P is greater than or equal to the threshold Th as candidate pixels, generating a sequence such as... Figure 7 The candidate pixel map d22 shown is output to the region forming unit 4B2. In the first embodiment, the threshold Th is a predetermined value. The threshold Th can be a fixed value or a value that can be appropriately changed by the user. The position of each pixel in the candidate pixel map d22 corresponds to the position of each pixel in the probability map d21. In the candidate pixel map d22, when the probability P of a pixel is greater than or equal to the threshold Th, the value assigned to that pixel is, for example, 1; when the probability P of a pixel is less than the threshold Th, the value assigned to that pixel is, for example, 0. Figure 7 In this system, a pixel is represented by a black dot when the value assigned to it is 1, and by a white dot when the value assigned to it is 0. Black dots are candidate pixels.

[0100] • Region Formation Section 4B2

[0101] The region forming unit 4B2 performs defect region filling processing on the candidate pixel image d22 to form the ignored region Rg. Specifically, as follows: Figure 7 As shown, a non-candidate pixel px1 sometimes exists in a region where candidate pixels are concentrated. If a non-candidate pixel px1 exists, the shape of the ignored region Rg becomes complex, making it difficult to specifically ignore the region. Therefore, the region forming unit 4B2 forms closed regions (ignore regions Rg1 to Rg3) to fill the gaps corresponding to the non-candidate pixel px1 (defect region). The defect region filling process can be performed, for example, by filling the start and end points of columns and rows. Thus, the region forming unit 4B2 can generate regions such as... Figure 8 The ignored region is shown in diagram d23.

[0102] <Generation Department 4C>

[0103] The generation unit 4C generates data based on medical image data d2 and the ignored region map d23. Figure 9 The data shown is the ignored suppression data d10. Specifically, the generation unit 4C can generate the ignored suppression data d10 by overlaying the ignored region Rg of the ignored region map d23 onto the medical image data d2.

[0104] <Error Calculation Unit 4D>

[0105] like Figure 4 As shown, the error calculation unit 4D compares the positive solution neglect suppression data d11 with the neglect suppression data d10 generated by the generation unit 4C. That is, the error calculation unit 4D calculates the error between the neglected region (which is the positive solution) and the calculated neglected region. Here, the positive solution neglect suppression data d11 is medical image data of easily neglected regions specified by the radiologist. In other words, the positive solution neglect suppression data d11 is medical image data that the radiologist observes and emphasizes easily neglected regions from the corresponding medical image. The error calculation unit 4D outputs the calculated error to the probability calculation unit 4A. The probability calculation unit 4A updates the weight coefficients of the filter based on this error.

[0106] 1-2-2 Output Section 5

[0107] The output unit 5 is configured to output the neglect suppression data d10 generated by the generation unit 4C to the display 21.

[0108] 1-2-3 Storage Section 6

[0109] Storage unit 6 has the function of storing various types of data. For example... Figure 1 As shown, the storage unit 6 pre-stores medical image data d1 or display-related data d3 (display specifications and viewer settings) used during the application phase. Furthermore, as... Figure 3 As shown, the storage unit 6 pre-stores medical image data d1, display-related data d3 (display settings, display specifications, and viewer settings) and environmental data d4 (illuminance values) used during the learning phase. The processing unit Ct reads the various data stored in the storage unit 6.

[0110] 1-3 Structural Description of Display 21

[0111] The display 21 includes a display unit 22, a data acquisition unit 23, an output unit 24, a storage unit 25, and a light sensor 26.

[0112] 1-3-1 Display Unit 22

[0113] Display unit 22 has the function of displaying the data acquired by data acquisition unit 23. Specifically, display unit 22 can display the ignored suppression data d10. Radiologists will Figure 9 The neglect suppression data d10 shown is displayed on the display unit 22, and the image is interpreted. Here, the neglect suppression data d10 is based on the display-related data d3 and the environmental data d4. Therefore, the neglect suppression data d10 takes into account factors such as the display used by the radiologist or the environment in which the radiologist interprets the image. As a result, the information processing device 1 of the first embodiment can suppress the situation where lesions in medical images are easily overlooked. Furthermore, the radiologist interprets the image through the display unit 22. Figure 9The ignored suppression data d10 shown can identify areas in medical image data d2 that require special attention and save attention when interpreting the image.

[0114] 1-3-2 Data Acquisition Department 23

[0115] The data acquisition unit 23 is configured to acquire the neglected suppression data d10 output by the output unit 5.

[0116] 1-3-3 Output Section 24

[0117] The output unit 24 is configured to output various data stored in the storage unit 25 to the information processing device 1.

[0118] 1-3-4 Storage Section 25

[0119] Storage unit 25, like storage unit 6, has the function of storing various types of data. Storage unit 25 stores display-related data d3 (display settings) and environmental data d4 (illuminance value) acquired by light sensor 26, etc.

[0120] 1-3-5 optical sensor 26

[0121] The light sensor 26 is configured to acquire the illuminance value (ambient data d4) of the light surrounding the display 21 (display unit 22).

[0122] 1-4 Action Description

[0123] 1-4-1 Learning Stage

[0124] Reference Figure 3 as well as Figure 4 Explain the actions of the information processing device 1 during the learning phase.

[0125] The information processing method (learning phase) of the first embodiment includes an acquisition step and a calculation step.

[0126] The operation steps include a preprocessing step, a probability map generation step (learning step), a candidate pixel generation step, an ignored region map generation step, and an ignored suppression data generation step.

[0127] In the acquisition step, the data acquisition unit 2 acquires medical image data d1, display-related data d3 (display settings, display specifications, and viewer settings), environmental data d4, and positive resolution ignoring suppression data d11.

[0128] In the preprocessing step, the preprocessing unit 3 changes the size, etc., of the medical image data d1 to generate medical image data d2. In the probability map generation step, the probability calculation unit 4A generates a probability map d21 based on the medical image data d2, the display-related data d3, and the environmental data d4. Since this is a machine learning step, the probability map generation step can also be called a learning step. In the probability map generation step (learning step), the error calculated by the error calculation unit 4D is input to the probability calculation unit 4A. This error corresponds to the difference between the ignored suppression data d10 obtained in the ignored suppression data generation step and the positive ignored suppression data d11, which will be described later. As a result, the weight coefficients of the filter of the probability calculation unit 4A are appropriately updated, improving the output accuracy of the probability calculation unit 4A. In other words, in the probability map generation step (learning step), the probability calculation unit 4A appropriately updates the weight coefficients of the filter while learning the structure of the relationship between the input (medical image data and additional data) and the output (probability). That is, the weight coefficients of the filter are gradually updated to values ​​that better reflect the experience of radiologists. In addition, the probability plot d21 and the ignored suppression data d10 will gradually approach the correct solution ignored suppression data d11.

[0129] In the candidate pixel generation step, the candidate pixel extraction unit 4B1 performs thresholding on the probability map d21 to generate a candidate pixel map d22. In the neglected region map generation step, the region forming unit 4B2 performs defect region filling processing on the candidate pixel map d22 to form a neglected region Rg, generating a neglected region map d23. In the neglected suppression data generation step, the generation unit 4C generates neglected suppression data d10 based on the medical image data d2 and the neglected region map d23.

[0130] 1-4-2 Application Stage

[0131] Reference Figure 1 as well as Figure 2 Explain the actions performed during the application phase. Focus on the actions performed during the application phase that differ from those performed during the learning phase.

[0132] The information processing method (application stage) of the first embodiment includes an acquisition step, a calculation step, and an output step.

[0133] The operation steps include a preprocessing step, a probability map generation step, a candidate pixel generation step, an ignored region map generation step, and an ignored suppression data generation step.

[0134] In the acquisition step, the data acquisition unit 2 does not acquire the correct solution and ignores the suppressed data.

[0135] During the application phase, the weighting coefficients of the filter in the probability calculation unit 4A are determined. That is, the probability calculation unit 4A does not calculate the error, therefore the probability calculation unit 4A does not obtain the error from the error calculation unit 4D.

[0136] In the output step, the neglect suppression data d10 is output to the display unit 22 of the display 21. This allows the radiologist to determine the location of the neglected region Rg. It should be noted that in the first embodiment, the neglect suppression data d10 is described as image data, but it is not limited to this and can also be audio data. For example, the location of the neglected region Rg can be roughly determined by outputting the location of the neglected region Rg through the speaker of the display 21 in the output step.

[0137] 1-5 Variations

[0138] 1-5-1 Variation Example 1: Frequency Data Generation Unit 7

[0139] like Figure 10 As shown, the information processing apparatus 1 may also include a frequency data generation unit 7. The frequency data generation unit 7 performs processing to acquire data related to the frequency components of the medical image data d2. For example, the frequency data generation unit 7 can generate image data by performing a Fourier transform on the medical image data d2. Furthermore, the frequency data generation unit 7 can also perform filtering processing to extract specific frequencies from the medical image data d2, generating image data with edge extraction. In addition to the medical image data d2, the data processing unit 4 also learns data related to the frequency components, thereby enabling the information processing apparatus 1 to generate a more suitable neglect suppression data d10. This is because the information of the frequency components of the image is considered to be related to the way the image is presented.

[0140] Furthermore, in the first embodiment, in addition to the illuminance value, the environmental data d4 may also include a distance measurement value. Specifically, as... Figure 10 As shown, the display 21 may also include a distance sensor 27. The distance sensor 27 is configured to acquire distance values. Here, the distance value represents the distance between the display 21 and the human body. In addition to data related to frequency components, the data processing unit 4 also learns the distance value, thereby enabling the information processing device 1 to generate a more appropriate neglect suppression data d10. This is because the distance between the display 21 and the human body is considered to be related to the way the image is presented.

[0141] 1-5-2 Variation Example 2: Fractions

[0142] In the first embodiment, the neglect suppression data d10 is image data emphasizing the region Rg where lesions are easily overlooked. That is, in the first embodiment, the neglect suppression data d10 is data on the location of a specific region Rg where lesions are easily overlooked. It should be noted that the neglect suppression data d10 is not limited to data on the location of a specific region Rg. The neglect suppression data d10 can also be a score (score data) representing the probability that lesions in medical image data are easily overlooked. The generation unit 4C calculates this score. Furthermore, this score can be displayed on the display unit 22 or output as audio. The higher the score, the greater the probability that a region where lesions are easily overlooked exists in the image. This score is based on display association data d3 and environmental data d4, thus taking into account factors such as the display used by the radiologist or the environment in which the radiologist interprets the image. As a result, in the modified example 2, the situation where lesions are easily overlooked in medical images can be suppressed in the same way as in the first embodiment. Furthermore, the radiologist refers to this score when interpreting the image, thus saving attention.

[0143] It should be noted that the technology in Patent Document 1 is a technique for calculating the presence of risks such as physical lesions. Therefore, because the calculated risk is low (the probability of a lesion being present is low), radiologists may become complacent and lose focus when interpreting images, potentially causing lesions in the medical images to be overlooked. In contrast, in Modification 2, since the score is independent of the actual presence of a lesion, even if the score is low, radiologists still need to carefully observe the medical image data. That is, in Modification 2, even if the score is low, it is possible to prevent radiologists from losing focus.

[0144] Method 1 for calculating fractions 1-5-2-1

[0145] The score can be calculated by dividing the area of ​​the neglected region Rg by the area of ​​the breast region R. Generally, the breast region R has high brightness and is considered an area where lesions are easily overlooked. Therefore, it is highly likely that most of the neglected region Rg is contained within the breast region R. Thus, if the score is calculated based on the ratio of the area of ​​the neglected region Rg to the area of ​​the breast region R, this score reflects the likelihood that lesions in medical imaging data are overlooked.

[0146] There is no particular limitation on the method for calculating the area of ​​the breast region R. For example, the information processing device 1 can determine whether each pixel is a breast pixel based on the brightness of each pixel, calculate the total number of breast pixels, and use this total number as the area of ​​the breast region R.

[0147] Furthermore, the area of ​​the ignored region Rg can be equal to the total number of pixels contained within the ignored region Rg.

[0148] Method 2 for calculating fractions 1-5-2-2

[0149] In the first embodiment, the neglected region Rg includes three neglected regions Rg1 to Rg3. A score can also be calculated based on the area of ​​the largest region among these regions Rg1 to Rg3. The larger the area of ​​the neglected region, the more likely the lesion is to be ignored. If the score is calculated based on the area of ​​the largest region among the neglected regions Rg, the score reflects the likelihood that a lesion in the medical image data will be ignored.

[0150] Method 3 for calculating fractions 1-5-2-3

[0151] The score can also be calculated based on the maximum width of a specific direction within the regions Rg1 to Rg3, ignoring the left and right directions. For example... Figure 11 As shown in Figure A, the line L with the largest width in the left-right direction is the line L in the ignored region Rg3. That is, the fraction can also be calculated based on the width (length) of line L.

[0152] Furthermore, scores can also be based on connections. Figure 11 The endpoints of line L in the diagram shown in Figure A are... Figure 11 The slope S2 of the line connecting pixel L2 in the graph shown in B is used for calculation. Here, pixel L2 is the pixel on line L whose probability P is greater than the threshold P2. The larger the slope S2, the more the overall shape of the graph tends to bulge upwards. Therefore, the larger the slope S2, the higher the probability P of all pixels in the ignored region Rg will be.

[0153] If a score is calculated based on the width or slope S2 mentioned above, the score reflects the likelihood that lesions in the medical image data are being overlooked.

[0154] 1-5-3 Variation Example 3: Obtaining the measured value of the display

[0155] In the first embodiment, it is explained that the display setting value can be a brightness setting value, but it is not limited to this. The display-related data d3 can also include display measurement values. More specifically, the display-related data d3 can also include at least one of the following: display setting value, display specifications, viewer setting value, and display measurement values.

[0156] The measured values ​​of the display are, for example, brightness values ​​or chromaticity values. The display 21 is equipped with a light sensor (not shown) that measures the brightness of the display unit 22, thereby enabling the information processing device 1 to use the brightness value obtained by the light sensor to replace the brightness setting value in the display settings.

[0157] 1-5-4 Variation Example 4: Visual processing performed by display 21

[0158] The first embodiment describes a method in which the information processing device 1 performs visual processing for specific neglected regions Rg1 to Rg3, but it is not limited to this. For example... Figure 12 As shown, the display 21 can also perform visual processing for specific neglected areas. Figure 12 In the variant 4 shown, the data processing unit 4 does not have a generation unit 4C, and the data processing unit 4 has a position-specific unit 4B3 instead of a region-forming unit 4B2. Additionally, the display 21 has a brightness adjustment unit 28.

[0159] The location-specific unit 4B3 generates location data (ignorance suppression data d10) for a specific ignored region. For example, the location-specific unit 4B3 performs defect region filling processing similarly to the region forming unit 4B2, generating location data (ignorance suppression data d10) that includes location data of candidate pixels specified by the candidate pixel map d22 and location data of pixels filled by the defect region filling processing. That is, the generated location data (ignorance suppression data d10) is location data for regions in the specific medical image data d2 where lesions are easily ignored. The location-specific unit 4B3 outputs the generated location data to the output unit 5.

[0160] The brightness adjustment unit 28, based on the location data and the medical image data d2, emphasizes pixels (areas) in the medical image data d2 that are easily overlooked due to lesions. Specifically, when the display 21 displays the medical image data d2, the brightness adjustment unit 28 has the function of increasing the brightness value of the pixel corresponding to the location data. That is, the brightness value of the pixel corresponding to the location data is adjusted from the brightness value of the medical image data d2 to a brightness value greater than that value. It should be noted that the brightness adjustment unit 28 may also relatively reduce the brightness value of pixels surrounding the pixel corresponding to the location data to emphasize pixels where lesions are easily overlooked.

[0161] 2. Second Implementation Method

[0162] For the second embodiment, the structures common to the first embodiment are appropriately omitted, and the description focuses on the different structures.

[0163] In the first embodiment, the information processing device 1 includes a data processing unit 4, but is not limited thereto. In the second embodiment, as... Figure 13 As shown, the display 21 includes a processing unit Ct (data processing unit 4). That is, in the second embodiment, the display 21 functions as an information processing device for calculating the ignored suppression data d10. The second embodiment also has the same effects as the first embodiment.

[0164] (Symbol Explanation)

[0165] 1: Information processing device

[0166] 2: Data Acquisition Department

[0167] 3: Pre-processing Department

[0168] 4: Data Processing Department

[0169] 4A: Probability Calculation Department

[0170] 4B: Post-processing unit

[0171] 4B1: Candidate pixel extraction unit

[0172] 4B2: Region Formation Department

[0173] 4B3: Location-Specific Part

[0174] 4C: Production Department

[0175] 4D: Error Calculation Department

[0176] 5: Output Section

[0177] 6: Storage Department

[0178] 7: Frequency Data Generation Unit

[0179] 21: Monitor

[0180] 22: Display Section

[0181] 23: Data Acquisition Department

[0182] 24: Output Department

[0183] 25: Storage Department

[0184] 26: Light sensor

[0185] 27: Distance sensor

[0186] 100: Information Processing System

[0187] Ct: Processing Unit

[0188] B: Breast area

[0189] G: Pectoralis major region

[0190] R: Breast region

[0191] Rg: Ignore region

[0192] Rg1: Ignore region

[0193] Rg2: Ignore region

[0194] Rg3: Ignore region

[0195] d1: Medical image data (before preprocessing)

[0196] d2: Medical image data (after preprocessing)

[0197] d3: Monitor-related data

[0198] d4: Environmental data

[0199] d10: Ignore suppression data

[0200] d11: The correct solution ignores suppressed data.

[0201] d21: Probability diagram

[0202] d22: Candidate pixel image

[0203] d23: Area map.

Claims

1. An information processing device comprising a data acquisition unit and a data processing unit, The data acquisition unit acquires medical image data and additional data. The additional data includes at least one of the following: display-related data and environmental data. The display association data is used to define how the image displayed on the display screen is presented. The environmental data refers to data representing the surrounding environment of the display. The data processing unit calculates the probability based on a learning model that outputs a probability if the input medical image data and the additional data are given, and generates data that ignores suppression based on the probability. The ignored suppression data refers to the data in the medical image data that suppresses lesion ignoring. The probability is a value indicating whether a lesion is easily overlooked.

2. The information processing apparatus according to claim 1, wherein, The ignored suppression data includes image data representing regions within the medical image data where lesions are easily ignored.

3. The information processing apparatus according to claim 1 or 2, wherein, The neglect suppression data includes fractional data representing the likelihood that lesions in the medical image data are ignored.

4. The information processing apparatus according to claim 1 or 2, wherein, The ignored suppression data includes location data. The location data is data used to determine the location of areas within the medical image data that are easily overlooked by lesions.

5. The information processing apparatus according to claim 1 or 2, wherein, The display-related data includes at least one of the following: display settings, display specifications, viewer settings, and display measurement values. The display settings are settings used to define the presentation mode of the image displayed on the display unit. The display specifications refer to the pre-existing characteristics of the display. The viewer settings are settings used to define the presentation mode of the image displayed on the display unit, and are also settings for the application used to display the image on the display unit. The measured values ​​of the display are the measured values ​​of the brightness or color of the display unit.

6. The information processing apparatus according to claim 1 or 2, wherein, The environmental data includes at least one of the following: illuminance value and distance measurement value. The illuminance value represents the illuminance around the display unit. The distance measurement value represents the distance between the display and the human body.

7. An information processing method, comprising an acquisition step and a calculation step, In the acquisition step, medical image data and additional data are acquired. The additional data includes at least one of the following: display-related data and environmental data. The display association data is used to define how the image displayed on the display screen is presented. The environmental data refers to data representing the surrounding environment of the display. In the aforementioned computational step, the probability is calculated based on a learning model that outputs a probability if the input medical image data and the additional data are given, and suppression-ignoring data is generated based on the probability. The ignored suppression data refers to the data in the medical image data that suppresses lesion ignoring. The probability is a value indicating whether a lesion is easily overlooked.

8. A recording medium, wherein, The recording medium stores a computer program that enables a computer to execute an information processing method comprising acquisition and calculation steps. In the acquisition step, medical image data and additional data are acquired. The additional data includes at least one of the following: display-related data and environmental data. The display association data is used to define how the image displayed on the display screen is presented. The environmental data refers to data representing the surrounding environment of the display. In the aforementioned computational step, the probability is calculated based on a learning model that outputs a probability if the input medical image data and the additional data are given, and suppression-ignoring data is generated based on the probability. The ignored suppression data refers to the data in the medical image data that suppresses lesion ignoring. The probability is a value indicating whether a lesion is easily overlooked.

Images