Information processing device, information processing method, medical image identification device, and program

Abstract

An information processing device (100) according to an embodiment comprises: an acquisition unit (110) that sequentially acquires a plurality of elements included in series data; a first calculation unit (120) that calculates, for each of the plurality of elements, an indicator indicating which of a plurality of classes the element would appropriately belong to, the calculation being carried out on the basis of two or more elements from among the plurality of elements; a second calculation unit (130) that calculates a weight indicating the degree of importance of the indicator of each of the plurality of elements; a third calculation unit (140) that integrates the indicators of the plurality of elements upon weighting each respective indicator with the corresponding weight, and calculates an integrated indicator indicating which of a plurality of classes the serial data would appropriately belong to; and a classification unit (150) that classifies the serial data as one of the classes on the basis of the integrated indicator.

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a medical image identification device, and a non-transitory computer-readable medium on which a program is stored. [Background technology]

[0002] Patent Documents 1 to 4 disclose information processing techniques using a sequential probability ratio test (SPRT). The SPRT is a type of technique for determining to which of a plurality of predetermined classes sequentially inputted sequence data belongs. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-245314 A [Patent Document 2] JP 2008-299589 A [Patent Document 3] Special Publication No. 2001-523824 [Patent Document 4] International Publication No. 2020 / 194497 Summary of the Invention [Problem to be solved by the invention]

[0004] In the mathematical formulas used in SPRT as disclosed in Patent Documents 1 to 4, the importance for discrimination is treated as being equal for all elements of the input sequence data, so sufficient accuracy may not be obtained depending on the nature of the sequence data.

[0005] The present disclosure has been made to solve such problems, and aims to provide an information processing device, an information processing method, a medical image identification device, and a non-transitory computer-readable medium storing a program, which are capable of classifying sequence data with high accuracy. [Means for solving the problem]

[0006] An information processing device according to the present disclosure includes an acquisition means for sequentially acquiring a plurality of elements included in sequence data, a first calculation means for calculating, for each of the plurality of elements, an index indicating to which of a plurality of classes it is appropriate for the element to belong, based on two or more of the plurality of elements, a second calculation means for calculating a weight indicating an importance of the index for each of the plurality of elements, a third calculation means for weighting the index of each of the plurality of elements with the corresponding weight and integrating the index, thereby calculating an integrated index indicating to which of the plurality of classes the element to belong, and a classification means for classifying the sequence data into one of the classes based on the integrated index.

[0007] The information processing method according to the present disclosure includes the steps of: sequentially acquiring a plurality of elements included in sequence data; calculating, for each of the plurality of elements, an index indicating to which of a plurality of classes the sequence data should belong, based on two or more of the plurality of elements; calculating a weight indicating the importance of the index for each of the plurality of elements; weighting the index for each of the plurality of elements with the corresponding weight and integrating the index, thereby calculating an integrated index indicating to which of the plurality of classes the sequence data should belong; and classifying the sequence data into one of the classes based on the integrated index.

[0008] A non-transitory computer-readable medium according to the present disclosure stores a program for causing a computer to execute an information processing method including the steps of: sequentially acquiring a plurality of elements included in sequence data; calculating, for each of the plurality of elements, an index indicating to which of a plurality of classes it is appropriate for the element to belong, based on two or more of the plurality of elements; calculating a weight indicating the importance of the index for each of the plurality of elements; weighting the index for each of the plurality of elements with the corresponding weight and integrating it to calculate an integrated index indicating to which of the plurality of classes the sequence data is appropriate to belong; and classifying the sequence data into one of the classes based on the integrated index. Effect of the Invention

[0009] The present disclosure makes it possible to provide an information processing device, an information processing method, a medical image identification device, and a non-transitory computer-readable medium storing a program, which are capable of classifying sequence data with high accuracy. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing an overall configuration of a sequence data classification system according to a first embodiment. [Diagram 2] FIG. 2 is a functional block diagram of an information processing device provided in the sequence data classification system according to the first embodiment. [Diagram 3] 5 is a flowchart showing an example of classification processing performed by the information processing device according to the first embodiment. [Figure 4] FIG. 11 is a functional block diagram of a medical image identification device which is an application example of an information processing device according to a second embodiment. [Diagram 5] FIG. 11 is a functional block diagram of an information processing device according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, exemplary embodiments of the present invention will be described with reference to the drawings. In the drawings, similar or corresponding elements are denoted by the same reference numerals, and descriptions thereof may be omitted or simplified.

[0012] First Embodiment The sequence data classification system according to the present embodiment is a system for classifying sequence data into one of a number of predetermined classes by sequentially acquiring and analyzing a number of elements contained in the sequence data.

[0013] Here, series data means a data string that can be decomposed into multiple elements. Series data may be time series data or non-time series data. Specific examples of time series data include video data and audio data. Specific examples of non-time series data include vegetation data sampled from multiple locations, inspection data of multiple locations on a product, and multiple biometric data for biometric authentication. If the series data is video data, the multiple elements included in the series data may be multiple images (frames) that make up the video. If the series data is inspection data of multiple locations on a product, the multiple elements included in the series data may be inspection data for each location on the product. Note that the series data and elements to which the classification process of this embodiment can be applied are not limited to these.

[0014] When the series data is inspection data of multiple locations on a product, the classes classified by the series data classification system of this embodiment may be, for example, a first class indicating that the product is a good product and a second class indicating that the product is defective. When the series data is multiple images (frames) constituting medical data, the classes classified by the series data classification system of this embodiment may be, for example, a first class indicating that the image contains a cancerous site and a second class indicating that the image does not contain a cancerous site. The number of classes may be three or more.

[0015] Fig. 1 is a schematic diagram showing the overall configuration of a sequence data classification system according to this embodiment. Fig. 1 shows the configuration of hardware included in the sequence data classification system. The sequence data classification system includes an information processing device 100, a data acquisition device 201, an input device 202, and a display device 203.

[0016] The information processing device 100 is a computer such as a mobile phone, a smartphone, a desktop PC (Personal Computer), a laptop PC, a server, etc. The information processing device 100 includes a processor 101, a memory 102, a storage 103, an input / output I / F (Interface) 104, and a communication I / F 105. The components of the information processing device 100 are connected to each other via a bus, wiring, a drive device, etc., and can transmit and receive control signals and data to and from each other.

[0017] The processor 101 is, for example, an arithmetic processing device such as a central processing unit (CPU) or a graphics processing unit (GPU). The memory 102 is, for example, a volatile or non-volatile storage medium such as a random access memory (RAM) or a read only memory (ROM). The storage 103 is, for example, a non-volatile storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a memory card.

[0018] The memory 102 or the storage 103 stores a program for implementing the information processing function of the information processing device 100. When executing the above-mentioned program, the processor 101 may read the program onto the memory 102 and then execute the program, or may execute the program without reading the program onto the memory 102.

[0019] The above-mentioned program may be stored in various types of non-transitory computer-readable media and provided to the information processing device 100. The non-transitory computer-readable media includes various types of storage media. The non-transitory computer-readable media includes, for example, a magnetic storage medium, a magneto-optical storage medium, an optical storage medium, and a semiconductor memory.

[0020] Examples of magnetic storage media include flexible disks, magnetic tapes, and hard disk drives. Examples of magneto-optical storage media include magneto-optical disks. Examples of optical storage media include CD-ROMs (Compact Disc Read Only Memory), CD-Rs (Compact Disc Recordable), and CD-R / Ws (Compact Disc Rewritable). Examples of semiconductor memories include mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs.

[0021] The program may be provided to the information processing device 100 by various kinds of temporary computer-readable media. The temporary computer-readable media include, for example, an electric signal, an optical signal, and an electromagnetic wave. The temporary computer-readable medium may provide the program to the information processing device 100 via a wired communication path such as an electric wire or an optical fiber, or a wireless communication path.

[0022] The input / output I / F 104 is a communication interface for communicating with peripheral devices based on standards such as USB (Universal Serial Bus) and DVI (Digital Visual Interface). The input / output I / F 104 can establish wired or wireless communication connections with the data acquisition device 201, the input device 202, and the display device 203. This allows the information processing device 100 to transmit and receive data and control signals to and from the data acquisition device 201, the input device 202, and the display device 203.

[0023] The communication I / F 105 is a communication interface based on standards such as Bluetooth (registered trademark), Wi-Fi (registered trademark), and 4G. The communication I / F 105 can establish a wired or wireless communication connection with an external device. This allows the information processing device 100 to transmit and receive data to and from the external device.

[0024] The data acquisition device 201 is a device for acquiring sequence data. For example, if the sequence data is product inspection data, the data acquisition device 201 may be an inspection device installed in a factory or the like. For example, if the sequence data is biometric data for biometric authentication, the data acquisition device 201 may be a device for acquiring biometric information, such as a digital camera, a microphone, or a fingerprint scanner. For example, if the sequence data is medical data for lesion detection, the data acquisition device 201 may be a device for acquiring medical information, such as an endoscope system, an MRI (Magnetic Resonance Imaging) system, or a CT (Computed Tomography) system. If the data acquisition device 201 includes a device for acquiring an analog signal, such as a sensor, the data acquisition device 201 may include an AD conversion (Analog-to-Digital Conversion) device for converting an analog signal into digital data. The sequence data acquired by the data acquisition device 201 is input to the information processing device 100.

[0025] The input device 202 is a user interface for accepting operations of the information processing device 100 by a user. Examples of the input device 202 include a keyboard, a mouse, a trackball, a touch sensor, a pen tablet, and a button. The display device 203 is a device that displays a screen based on drawing data processed by the processor 101. Examples of the display device 203 include an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube) display, and an OLED (Organic Light Emitting Diode) display. The input device 202 and the display device 203 may be integrally formed as a touch panel.

[0026] Note that the hardware configuration shown in FIG. 1 is an example, and other devices may be added, or some devices may not be provided. Some devices may be replaced with other devices having similar functions. Furthermore, some functions of this embodiment may be provided by other devices via a network, or the functions of this embodiment may be distributed and realized among multiple devices. For example, the storage 103 may be replaced with a cloud storage external to the information processing device 100. Furthermore, when acquisition of sequence data is performed in a system other than the sequence data classification system, the data acquisition device 201 may be omitted. Alternatively, the data acquisition device 201, the input device 202, or the display device 203 may be provided in the information processing device 100. In this way, the hardware configuration shown in FIG. 1 can be appropriately changed.

[0027] 2 is a functional block diagram of an information processing device 100 according to this embodiment. The information processing device 100 includes an acquisition unit 110, a first calculation unit 120, a second calculation unit 130, a third calculation unit 140, and a classification unit 150. The first calculation unit 120 includes an index calculation unit 121 and a first storage unit 122. The second calculation unit 130 includes a weight calculation unit 131 and a second storage unit 132. The third calculation unit 140 includes an integrated index calculation unit 141 and a third storage unit 142.

[0028] The processor 101 executes a program stored in the memory 102, the storage 103, or the like, to realize the functions of the acquisition unit 110, the index calculation unit 121, the weight calculation unit 131, the integrated index calculation unit 141, and the classification unit 150. The processor 101 also controls the storage 103 based on the program to realize the functions of the first storage unit 122, the second storage unit 132, and the third storage unit 142. Specific processes performed by each of these units will be described later.

[0029] Fig. 3 is a flowchart showing an example of the classification process performed by the information processing device 100 according to this embodiment. The classification process shown in Fig. 3 is a process for classifying input sequence data into one of a plurality of predetermined classes. The classification process in Fig. 3 includes a loop process (steps S101 to S107) for obtaining elements one by one from sequence data including a plurality of elements and sequentially calculating an integrated index. This loop process is repeated until a class into which the sequence data is to be classified is determined based on the integrated index.

[0030] The process in Fig. 3 may be started, for example, when a predetermined user operation is performed on the input device 202. However, the start timing of the process is not limited to this. For example, the process in Fig. 3 may be executed when sequence data is input from the data acquisition device 201. In addition, when sequence data is continuously input, such as when the data acquisition device 201 is a surveillance camera, the process in Fig. 3 may be repeatedly executed at predetermined time intervals.

[0031] In step S101, the acquiring unit 110 acquires one element of the sequence data. The acquisition process at this time may be to acquire data directly from the data acquiring device 201, or to read data that has been acquired in advance from the data acquiring device 201 and stored in the storage 103 or the like. Then, the process proceeds to step S102.

[0032] In step S102, the index calculation unit 121 judges whether past data exists by referring to the first storage unit 122. The past data includes a previously acquired element, and a processing result (an index, for example, a likelihood ratio) by the index calculation unit 121 for one or more previously acquired elements including the element.

[0033] For example, in step S102, if no past data exists (NO in step S102), the process proceeds to step S103. In step S103, the index calculation unit 121 calculates an index indicating which of the multiple classes the input element is likely to belong to, taking into account the first element of the sequence data. The first calculation unit 120 outputs the calculated index to the second calculation unit 130, and stores the processing result in the first storage unit 122 as necessary. Here, the index may be, for example, a likelihood ratio indicating the likelihood that a certain element belongs to a certain class among the multiple classes. Alternatively, the index may be a function including the likelihood ratio as a variable. In the following description, the index is assumed to be the likelihood ratio. Note that the likelihood ratio calculated in step S103 is used as an integrated score. After that, the process proceeds to step S108. The process of step S108 will be described later.

[0034] On the other hand, if past data exists in step S102 (YES in step S102), the process proceeds to step S104. In step S104, the index calculation unit 121 reads out the past data stored in the first storage unit 122. The first storage unit 122 stores the processing results and the input elements each time the index calculation unit 121 performs processing. This storage process may overwrite previously stored information with new information, or may add new information while leaving previously stored information. Thereafter, the process proceeds to step S105.

[0035] In step S105, the index calculation unit 121 calculates an index (likelihood ratio) indicating to which of multiple classes the input element is likely to belong, taking into account two or more elements among the multiple elements included in the sequence data. The two or more elements include a newly acquired element and an element that has been previously processed and is included in the past data. The first calculation unit 120 outputs the calculated index to the second calculation unit 130, and stores the processing result in the first storage unit 122 as necessary.

[0036] The index calculation unit 121 extracts features from the elements input from the sequence data. At this time, when past data does not exist, the index calculation unit 121 extracts features taking into consideration the input elements, and when past data exists, the index calculation unit 121 extracts features taking into consideration the association between the input elements and the past data. A specific method for extracting features may be, for example, a Convolutional Neural Network (CNN), but is not limited to this. Also, a specific method for storing past data and calculating the relationship with current input data may be, for example, a Long Short Term Memory (LSTM), but is not limited to this.

[0037] A specific example of the likelihood ratio will be explained. Let us assume that N elements (N is an integer equal to or greater than 1) that make up the sequence data are x1, . . . , x N and the multiple classes are denoted as C1 and C2. That is, in this example, for the sake of simplicity, it is assumed that the number of classes is two, that is, two-class classification.

[0038] Here, if there is no past data (NO in step S102), etc., the element x i (where i is an integer between 1 and N) belongs to class C1, and the probability is p(x i |C1), and the element x i The probability that belongs to class C2 is p(x i |C2). In this case, these likelihood ratios are expressed by the following equation (1).

[0039]

number

[0040] The likelihood ratio of equation (1) is the element x i The probability that x belongs to class C1 and the element x i This shows the likelihood ratio of the probability that the class belongs to class C2. For example, if the likelihood ratio is greater than 1, then p(x i |C1)>p(x i |C2), so element xi is more likely to be classified into class C1 than into class C2. In this way, the likelihood ratio of equation (1) functions as an index showing whether the input element is more likely to belong to class C1 or class C2.

[0041] In addition, when past data exists (YES in step S102), the index calculation unit 121 can calculate the index by taking into consideration the relevance between a plurality of elements, that is, the input element and the past data, as described above. In this case, for example, two elements x i ,x j The likelihood ratio calculated taking into consideration (i, j are any integers from 1 to N, where i≦j) is expressed as in the following formula (2).

[0042]

number

[0043] After the process of step S105, the process proceeds to step S106. In step S106, the weight calculation unit 131 reads out the likelihood ratio calculated in the past from the first storage unit 122. The weight calculation unit 131 uses the likelihood ratio calculated this time and the likelihood ratio calculated in the past by the index calculation unit 121 to calculate weights corresponding to the likelihood ratio calculated this time and the likelihood ratio calculated in the past. The second storage unit 132 stores the weights calculated by the weight calculation unit 131. The "weight" here is a value for adjusting the degree of influence of the likelihood ratio calculated this time and the likelihood ratio calculated in the past on the newly calculated integrated index, and is calculated according to the reliability of the newly acquired element. The "reliability" is the degree of reliability related to the calculation of the likelihood ratio, and for example, the reliability of an element for which an appropriate likelihood ratio is calculated is high, and the reliability of an element for which an inappropriate likelihood ratio is calculated is low.

[0044] Here, the weight is, for example, the weight of the element x i From element x j A vector V whose elements are the probability that the input data belongs to each class, calculated taking into account the elements up toij Among the elements that make up the sequence data, element x i From element x N A vector V whose elements are the probability that the input data belongs to each class, calculated taking into account the elements up to iN The inner product of V ij ·V iN Here, both i and j are arbitrary integers from 1 to N, where i≦j. In the following description, it is assumed that the weight is determined by the inner product.

[0045] The calculation of the weight is not limited to the case where it is obtained by the inner product. For example, an integrated index calculated in the past may be read from the third storage unit 142 and used to calculate the weight. Then, the process proceeds to step S107.

[0046] In step S107, the integrated index calculation unit 141 integrates the likelihood ratio calculated this time by the index calculation unit 121, the likelihood ratios calculated in the past, and the weight calculated this time by the weight calculation unit 131 to calculate a new integrated index.

[0047] The integrated index indicates to which of a plurality of classes the entire sequence data belongs. The past integrated index means the integrated index calculated by the integrated index calculation unit 141 for an element before the jth element when the present process is a process for the jth element of the sequence data. The past likelihood ratio means the likelihood ratio calculated by the index calculation unit 121 for an element before the jth element when the present process is a process for the jth element of the sequence data. The third storage unit 142 stores the integrated index every time the integrated index calculation unit 141 performs processing. This storage process may be a process of updating the value of the integrated index by overwriting the previously stored integrated index with a new integrated index, or a process of adding a new integrated index while leaving the previously stored integrated index.

[0048] The integrated index may be, for example, the sum of all the calculated likelihood ratios including the past likelihood ratios multiplied by a weight corresponding to each likelihood ratio. In the following description, the sum of the values ​​multiplied by the weight corresponding to each likelihood ratio is called an integrated score. Alternatively, the integrated index may be a function including the integrated score as a variable. In the following description, the integrated index is assumed to be the integrated score.

[0049] A specific example of the integrated score will be described for the case where the number of classes is two. When N elements are input at the time of calculating the integrated score, these N elements are x1, . . . , x N Here, the probability that the entire data including the i-th element of the sequence data through the N-th element belongs to class C1 is expressed as p(x i ,x N In addition, the probability that the entire data including the i-th element of the sequence data through the N-th element belongs to class C2 is denoted as p(x i ,x N |C2). Note that when 1≦i≦N and i=N, the probability that data containing only the Nth element belongs to class C1 is p(x N |C1) and the probability that data containing only the Nth element belongs to class C2 is p(x N |C2). In this case, these likelihood ratios are expressed by the following equation (3).

[0050]

number

[0051] When using only the likelihood ratio including the first element through the Nth element of the sequence data as the integrated score, as in conventional SPRT, the integrated score is expressed by the following equation (4).

[0052]

number

[0053] However, as described above, in this embodiment, the integrated score is calculated by considering the importance of each of all the calculated likelihood ratios including the past likelihood ratios. Therefore, instead of using only one likelihood ratio as the integrated score as in formula (4), the integrated score is calculated using a different formula depending on the number of all the calculated likelihood ratios including the past likelihood ratios.

[0054] For example, when up to the Nth element of sequence data are input, the integrated score can be calculated using the following formula (5). Note that i is an integer between 1 and N, and j is an integer between i and N. When i=j, the probability that data containing only the i-th element belongs to class C1 is p(x i |C1) and the probability that data containing only the i-th element belongs to class C2 is p(x i |C2). Also, w ij is a weight calculated by the weight calculation unit 131 and indicates the importance of each likelihood ratio.

[0055]

number

[0056] A specific example of weights is explained. Among the elements that make up the sequence data, the 1st to Nth elements x1,...,x N Assume that up to the element x is input, and the multiple classes are C1, C2. That is, in this example, for simplicity, it is assumed that the number of classes is two, which is a two-class classification. Here, i From element x j The probability that sequence data containing elements up to belongs to class C1 is p(x i, ... , x j |C1). Also, element x i From element x j The probability that sequence data containing elements up to belongs to class C2 is p(x i, ... , x j Here, i is an integer between 1 and N, and j is an integer between i and N. When i=j, the probability that data containing only the i-th element belongs to class C1 is p(xi |C1) and the probability that data containing only the i-th element belongs to class C2 is p(x i |C2). p(x i, ... , x j |C1) and p(x i, ... , x j |C2) is the vector V ij Then, the weight w in the above formula (5) ij is V ij and V iN The inner product of V ij ·V iN As a result, the weight corresponding to the likelihood ratio when considering the i-th element to the j-th element of the sequence data becomes larger as the classification results when considering the i-th element to the N-th element, which is the most recent element, of the sequence data are more similar to the classification results when considering the i-th element to the j-th element of the sequence data. As a result, even if previously acquired sequence data contains data inappropriate for discrimination, such as data of a different class, it is possible to reduce the influence of the likelihood ratio calculated by considering the data inappropriate for discrimination on the integrated score, thereby improving discrimination accuracy.

[0057] The method of calculating the weight is not limited to the above. For example, N Probability p(x N |C1) and p(x N |C2) is the vector V N Let p(x i, ... , x j |C1) and p(x i, ... , x j |C2) is the element of vector V ij and the above V N The inner product of this is the weight w in equation (5). ij In this case, the latest element x of the sequence data can be used as N is the most important weighting. In another example, the vector V ij and p(x i, ... , x j-1 |C1) and p(x i, ... , x j-1|C2) is the element of vector V ij-1 Difference with V ij -V ij-1 Calculate V ij -V ij-1 The absolute value of the weight w in the above formula (5) ij In this case, the weighting is such that the newly input sequence data is weighted to the data that has changed significantly compared to the sequence data from one time before. Note that the method of calculating the weighting is not limited to these.

[0058] After all the weights that can be calculated have been calculated, a process of normalizing the weights may be performed. For example, a method of using a Softmax function may be used as a normalization method.

[0059] For the likelihood ratio shown in equation (5), the likelihood ratio calculated in advance by the index calculation unit 121 in step S103 can be used.

[0060] Although formula (5) shows an example of a two-class classification in which the likelihood ratio between class C1 and class C2 is calculated, the number of classes may be three or more. For example, when the number of classes is M, an extension of formula (5) can be used so that an integrated score between the a-th class among the M classes and all classes other than the a-th class can be calculated. An example of such an extension is one that uses the maximum likelihood among all classes other than the a-th class, as shown in the following formula (6). Note that in formula (6), the m-th class (m is an arbitrary integer from 1 to M) is defined as C m where a can be any integer from 1 to M, and b can be any integer from 1 to M other than a.

[0061]

number

[0062] Another example is one that uses the sum of the likelihoods of all classes other than the a-th class, as shown in the following formula (7). In formula (7), the m-th class (m is an arbitrary integer from 1 to M) is Cm where a can be any integer from 1 to M, and b can be any integer from 1 to M other than a. Note that the method of calculating the integrated score when the number of classes is three or more is not limited to these.

number

[0063] Although equations (5) to (7) illustrate the case where all elements from the first element to the Nth element of the input sequence data are used, any number of elements may be considered. For example, a maximum number of elements P to be used may be set in advance, and when the number of elements of the input sequence data exceeds P, only the latter P elements of the sequence data may be considered. This makes it possible to prevent the calculation load from becoming too large.

[0064] The method of calculating the integrated index is not limited to the above. For example, the integrated index may be calculated by a method using LSTM or a deep neural network.

[0065] After the process of step S107, the process proceeds to step S108. In step S108, the classification unit 150 determines whether the sequence data can be classified into any class based on the integrated index calculated by the third calculation unit 140. When the integrated index is an integrated score, the classification unit 150 determines whether the class can be classified based on, for example, whether there is a class whose integrated score exceeds a predetermined threshold. If classification is not possible (NO in step S108), the process proceeds to step S101, and the acquisition unit 110 acquires the next element. If classification is possible (YES in step S108), the process proceeds to step S109.

[0066] In step S109, the classification unit 150 classifies the sequence data into one of the classes based on the integrated index. For example, if the integrated index is an integrated score, the sequence data is classified as belonging to a class whose integrated score exceeds a predetermined threshold.

[0067] The processing of steps S108 and S109 will be described in more detail with a concrete example. The classification processing in this example is assumed to be two-class classification into class C1 and class C2, and the thresholds used to determine class C1 and class C2 are T1 and T2, respectively. Also, the integrated score is L.

[0068] In this case, if L>T1, the classification unit 150 classifies the sequence data into class C1, and the process ends. If L>T2, the classification unit 150 classifies the sequence data into class C2, and the process ends. If L≦T1 and L≦T2, the classification unit 150 determines that classification is not possible, and the acquisition unit 110 acquires the next element.

[0069] When the number of classes is three or more, M thresholds are prepared as described above, and a similar classification process can be performed by determining whether each of the M integrated scores is greater than or smaller than the corresponding threshold. In this case, the classification unit 150 classifies the sequence data into the class whose integrated score exceeds a threshold first. If the integrated score does not exceed any threshold, the classification unit 150 determines that classification is not possible, and the acquisition unit 110 acquires the next element.

[0070] The above classification method is an example and is not limited to this. For example, if the number of elements input in steps S107 and S108 is greater than a predetermined value (maximum number of elements), the procedure may be modified so that the sequence data is forcibly classified into one of the classes and the process is terminated even if there is no class whose integrated score exceeds the threshold. This makes it possible to prevent the calculation time from becoming too long. In this example, it is preferable to use mutually exclusive criteria so that the data is reliably classified into one of the classes.

[0071] A specific example of mutually exclusive criteria will be described. In the case of two-class classification, if the number of elements exceeds the maximum number of elements, a method can be used in which the sequence data is classified into one of two classes depending on whether the value of the integrated score L is equal to or greater than 0. In the case of M-class classification, a method can be used in which the sequence data is classified into the class with the maximum value of the integrated likelihood ratio corresponding to each class.

[0072] As described above, according to the present embodiment, sequence data is classified using an integrated index that takes into account multiple elements of the sequence data and the importance of each element. This allows classification according to the correlation length of the sequence data that takes into account the nature of the sequence data, thereby providing an information processing device 100 that can classify sequence data with high accuracy.

[0073] <Second embodiment> In this embodiment, a medical image identification device 300 will be described as one application example of the information processing device 100 of the first embodiment. In the following, differences from the first embodiment will be mainly described, and descriptions of common parts will be omitted or simplified.

[0074] 4 is a functional block diagram of a medical image identification device 300 according to the second embodiment. The medical image identification device 300 includes a classification device 301, a medical information acquisition unit 302, and a medical information storage unit 303. The medical image identification device 300 may be configured to include a computer, similar to the information processing device 100 shown in FIG. 1. Therefore, a description of the hardware configuration of the medical image identification device 300 will be omitted.

[0075] The medical image identification device 300 is a device that detects cancer from medical information such as, for example, endoscopic images, CT images, and MRI images. The medical image identification device 300 may be equipped with a device (such as an endoscope system) for acquiring medical information and operate as a standalone device, or may acquire medical information from other devices in a cancer detection system and perform cancer detection. The medical image identification device 300 may also be composed of multiple devices that are communicatively connected to each other.

[0076] The medical image identification device 300 may be, for example, an examination device for cancer examination, or an examination device for aneurysm examination.

[0077] The medical information acquisition unit 302 is a device that acquires medical information, and may be, for example, an endoscope system capable of capturing video. In identifying medical information, features for matching may be extracted from images acquired by the medical information acquisition unit 302. This feature extraction process may be performed in the classification device 301, may be performed in the medical information acquisition unit 302 when acquiring medical information, or may be performed by other devices. In this specification, the images acquired by the medical information acquisition unit 302 and the features extracted therefrom may be collectively referred to as medical information.

[0078] The medical information storage unit 303 stores information required for processing of medical information and the like in the classification device 301. The information processing device 100 of the first embodiment is used for the classification device 301. The classification device 301 acquires sequence data having medical information as an element as the sequence data described in the first embodiment. The classification device 301 classifies the sequence data into one of a plurality of predetermined classes while referring to the information stored in the medical information storage unit 303. Here, the plurality of classes may be, for example, a class indicating the presence or absence of cancer. In other words, the plurality of classes may include, for example, a class indicating the presence of a cancerous site in the input sequence data and a class indicating the absence of a cancerous site in the input sequence data.

[0079] The medical image identification device 300 of this embodiment includes a classification device 301 that can classify sequence data with high accuracy. This provides the medical image identification device 300 that can more appropriately detect cancer.

[0080] In the medical image identification device 300 of this embodiment, an example of cancer detection will be described as one example in which the feature of the information processing device 100 of the first embodiment, that is, high classification accuracy of sequence data, is more effectively utilized. As a method for detecting cancer from medical images such as endoscope images, there is a method in which the confidence that cancer is included in each image (frame) included in the image is calculated by machine learning, the confidence corresponding to one input image or the weighted sum of the confidence corresponding to a number of images of a predetermined fixed length is used as a classification score, and if the classification score exceeds a preset threshold, it is determined that the image contains a cancerous site. In the classification device 301 of this embodiment, a time-series image partially extracted from a medical image is input as sequence data, and the brightness value of the image is used as a feature to perform class classification indicating the presence or absence of a cancerous site in the sequence data, thereby making it possible to detect cancer.

[0081] A specific example of how the classification device 301 of this embodiment actually detects cancer from an endoscopic video is shown below. In this example, data at a certain point in time in an endoscopic video is treated as the first element of sequence data, and elements from that point up to N frames ahead are considered as one sequence data, and this sequence data is classified as being a cancerous or non-cancerous area. N is an integer equal to or greater than 1.

[0082] First, in step S101, one frame included in an endoscopic video is read out from the medical information acquisition unit 302 or the medical information storage unit 303.

[0083] Next, in step S102, it is confirmed whether or not there are past elements of the sequence data.

[0084] If no past elements exist, the process proceeds to step S103, where a likelihood ratio is calculated taking into account only the elements of the input sequence data, and this is taken as an integrated score.

[0085] If past elements exist, the process proceeds to step S104, where the past elements are read, and in steps S104 to S107, the likelihood ratio taking into account the past data and the weight corresponding to the likelihood ratio are calculated, and an integrated score is calculated.

[0086] In step S108, the integrated score is compared with a threshold for classifying, and if the integrated score exceeds the threshold for distinguishing a cancerous site, the series of data is classified as an image of a cancerous site in step S109. If the integrated score exceeds the threshold for distinguishing a non-cancerous site, the series of data is classified as an image of a non-cancerous site in step S109. If the integrated score does not exceed the threshold for any class, the process returns to step S101, and one new frame included in the endoscopic image is obtained from the medical information acquisition unit 302 or the medical information storage unit 303.

[0087] In this method, the inputted sequence data includes cancerous sites that are easy to distinguish, cancerous sites that are difficult to distinguish, and non-cancerous sites, and the size of the cancer often differs for each medical image. An example of a cancerous site that is easy to distinguish is a cancerous site that is raised like a lump. An example of a cancerous site that is difficult to distinguish is a flat cancerous site that does not have any protuberance or depression. For cancerous sites that are difficult to distinguish, it is effective to use multiple images for detection. In addition, if the sequence data is long, cancerous sites and non-cancerous sites may be mixed in the sequence data. In order to prevent deterioration of detection accuracy due to the mixing of data that should be classified into different classes in the sequence data, it is effective to set weights so that the importance of the likelihood ratio calculated from sequence data that includes elements that should be classified into a class different from the currently acquired element among the elements of the sequence data is reduced, and to reduce the influence of the likelihood ratio on the integrated score for classification. Therefore, when performing classification for cancer detection, it is effective to use the classification process of the information processing device 100 of the first embodiment, which uses one or more elements of sequence data while considering the importance of each element.

[0088] The device or system described in the above embodiments can also be configured as in the following third embodiment.

[0089] <Third embodiment> FIG. 5 is a functional block diagram of an information processing device 400 according to the third embodiment. The information processing device 400 includes an acquisition unit 410, a first calculation unit 420, a second calculation unit 430, a third calculation unit 440, and a classification unit 450. The acquisition unit 410 sequentially acquires a plurality of elements included in sequence data. The first calculation unit 420 calculates an index indicating which of a plurality of classes each of the plurality of elements is appropriate to belong to, taking into consideration two or more of the plurality of elements. The second calculation unit 430 calculates a weight indicating the importance of each index of the plurality of elements. The third calculation unit 440 weights each index of the plurality of elements with the corresponding weight, and then integrates the weights to calculate an integrated index indicating which of a plurality of classes the sequence data is appropriate to belong to. The classification unit 450 classifies the sequence data into one of the classes based on the integrated index.

[0090] According to this embodiment, an information processing device 400 capable of classifying sequence data with high accuracy is provided.

[0091] <Other embodiments> The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with another embodiment, is also an embodiment of the present invention.

[0092] The scope of each embodiment also includes a processing method in which a program that operates the configuration of the embodiment to realize the functions of the above-mentioned embodiment is recorded on a storage medium, the program recorded on the storage medium is read as code, and executed on a computer. That is, a computer-readable storage medium is also included in the scope of each embodiment. In addition, not only the storage medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment. Furthermore, one or more components included in the above-mentioned embodiments may be circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays) configured to realize the functions of each component.

[0093] Examples of the storage medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD (Compact Disk)-ROM, a magnetic tape, a non-volatile memory card, and a ROM. In addition, the scope of each embodiment does not include those that execute processing by themselves recorded on the storage medium, but also those that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an expansion board.

[0094] The services realized by the functions of the above-described embodiments can also be provided to users in the form of SaaS (Software as a Service).

[0095] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0096] Furthermore, some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0097] (Appendix 1) an acquisition means for sequentially acquiring a plurality of elements included in the sequence data; a first calculation means for calculating an index indicating to which of a plurality of classes each of the plurality of elements is appropriate to belong based on two or more of the plurality of elements; A second calculation means for calculating a weight indicating the importance of the index of each of the plurality of elements; a third calculation means for weighting the indices of the plurality of elements with the corresponding weights and integrating the weights to calculate an integrated index indicating to which of the plurality of classes the sequence data is likely to belong; a classification means for classifying the sequence data into any one of classes based on the integrated index; An information processing device comprising:

[0098] (Appendix 2) The second calculation means calculates a weight indicating the importance of the index of each of the plurality of elements using one or more of the indexes including the index corresponding to the weight calculated by the first calculation means; 2. An information processing device according to claim 1.

[0099] (Appendix 3) the second calculation means calculates a weight indicating the importance of the index of each of the plurality of elements using two or more of the indexes including the index corresponding to the weight calculated by the first calculation means; 3. The information processing device according to claim 1 or 2.

[0100] (Appendix 4) the index includes a likelihood ratio indicating the likelihood that each of the plurality of elements belongs to a certain class among the plurality of classes; 4. An information processing device according to any one of claims 1 to 3.

[0101] (Appendix 5) the integrated index includes an integrated score indicating a likelihood that the sequence data belongs to a certain class among the plurality of classes. 5. An information processing device according to any one of claims 1 to 4.

[0102] (Appendix 6) the classification means classifies the sequence data into a class in which the integrated score exceeds a predetermined threshold when there is a class in which the integrated score exceeds the threshold. 6. The information processing device according to claim 5.

[0103] (Appendix 7) When there is no class whose integrated score exceeds a predetermined threshold, the classifying means does not classify the sequence data into any class, and the acquiring means further acquires elements. 7. The information processing device according to claim 5 or 6.

[0104] (Appendix 8) the classifying means classifies the sequence data into one of the classes based on the integrated score when there is no class whose integrated score exceeds a predetermined threshold and the number of elements acquired by the acquiring means is greater than a predetermined value. 8. An information processing device according to any one of claims 5 to 7.

[0105] (Appendix 9) The first calculation means is A first storage means for storing at least the elements previously acquired by the acquisition means; an index calculation means for calculating, when an element of the sequence data is newly acquired by the acquisition means, the index for the newly acquired element based on the newly acquired element and the previously acquired elements stored in the first storage means; Equipped with 9. An information processing device according to any one of claims 1 to 8.

[0106] (Appendix 10) The second calculation means is A second storage means for storing the index previously calculated by the first calculation means; a weight calculation means for calculating the weight for each of the multiple indicators used by using the indicator newly calculated by the first calculation means and the indicators previously calculated and stored in the second storage means; Equipped with 10. The information processing device according to claim 9.

[0107] (Appendix 11) The third calculation means is an integrated index calculation means for calculating the integrated index based on the index newly calculated by the first calculation means, the index previously calculated and stored in the second storage means, and weights calculated by the weight calculation means for the newly calculated index and the index previously calculated; Equipped with 11. The information processing device according to claim 10.

[0108] (Appendix 12) The sequence data is time-series data. 12. An information processing device according to any one of claims 1 to 11.

[0109] (Appendix 13) A medical information acquisition means for acquiring medical information of a subject; An information processing device according to any one of Supplementary Notes 1 to 12; Equipped with the information processing device classifies the sequence data including the medical information as an element into any one of classes. Medical image identification device.

[0110] (Appendix 14) the information processing device classifies the series of data into any of classes indicating the presence or absence of a cancerous site in the medical information; 14. A medical image identification device as described in appended claim 13.

[0111] (Appendix 15) Sequentially acquiring a plurality of elements included in the sequence data; calculating an index indicating to which of a plurality of classes each of the plurality of elements is appropriate to belong based on two or more of the plurality of elements; calculating a weight indicating the importance of the index for each of the plurality of elements; a step of weighting the indices of the plurality of elements with the corresponding weights and integrating the indices to calculate an integrated index indicating to which of the plurality of classes the sequence data is likely to belong; classifying the sequence data into any one of classes based on the integrated index; An information processing method comprising:

[0112] (Appendix 16) On the computer, Sequentially acquiring a plurality of elements included in the sequence data; calculating an index indicating to which of a plurality of classes each of the plurality of elements is appropriate to belong based on two or more of the plurality of elements; calculating a weight indicating the importance of the index for each of the plurality of elements; a step of weighting the indices of the plurality of elements with the corresponding weights and integrating the indices to calculate an integrated index indicating to which of the plurality of classes the sequence data is likely to belong; classifying the sequence data into any one of classes based on the integrated index; A non-transitory computer-readable medium having a program stored thereon for executing an information processing method comprising:

[0113] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0114] 100, 400 Information processing device 101 Processor 102 Memory 103 Storage 104 Input / Output Interface 105 Communication I / F 110, 410 Acquisition Department 120, 420 First calculation section 121 Indicator calculation section 122 1st memory section 130, 430 Second calculation section 131 Weight calculation unit 132 2nd memory section 140, 440 3rd calculation section 141 Integrated index calculation section 142 Third memory section 150, 450 classification section 201 Data Acquisition Device 202 Input Device 203 Display device 300 Medical image identification device 301 Classification device 302 Medical Information Acquisition Department 303 Medical Information Storage Unit

Claims

1. an acquisition means for sequentially acquiring a plurality of elements included in the sequence data; a first calculation means for calculating an index indicating to which of a plurality of classes each of the plurality of elements is appropriate to belong based on two or more of the plurality of elements; A second calculation means for calculating a weight indicating the importance of the index of each of the plurality of elements; a third calculation means for weighting the indices of the plurality of elements with the corresponding weights and integrating the weights to calculate an integrated index indicating to which of the plurality of classes the sequence data is appropriate to belong; a classification means for classifying the sequence data into any one of classes based on the integrated index; An information processing device comprising:

2. the second calculation means calculates a weight indicating the importance of the index of each of the plurality of elements using one or more of the indexes including the index corresponding to the weight calculated by the first calculation means; The information processing device according to claim 1 .

3. the second calculation means calculates a weight indicating the importance of the index of each of the plurality of elements using two or more of the indexes including the index corresponding to the weight calculated by the first calculation means; 3. The information processing device according to claim 1 or 2.

4. the index includes a likelihood ratio indicating the likelihood that each of the plurality of elements belongs to a certain class among the plurality of classes; The information processing device according to claim 1 .

5. the integrated index includes an integrated score indicating a likelihood that the sequence data belongs to a certain class among the plurality of classes. The information processing device according to claim 1 .

6. the classification means classifies the sequence data into a class in which the integrated score exceeds a predetermined threshold when there is a class in which the integrated score exceeds the threshold. The information processing device according to claim 5 .

7. When there is no class whose integrated score exceeds a predetermined threshold, the classifying means does not classify the sequence data into any class, and the acquiring means further acquires elements.

7. The information processing device according to claim 5 or 6.

8. the classifying means classifies the sequence data into one of the classes based on the integrated score when there is no class whose integrated score exceeds a predetermined threshold and the number of elements acquired by the acquiring means is greater than a predetermined value. The information processing device according to claim 5 .

9. The first calculation means a first storage means for storing at least the elements previously acquired by the acquisition means; an index calculation means for calculating, when an element of the sequence data is newly acquired by the acquisition means, the index for the newly acquired element based on the newly acquired element and the previously acquired elements stored in the first storage means; Equipped with The information processing device according to claim 1 .

10. The second calculation means is A second storage means for storing the index previously calculated by the first calculation means; a weight calculation means for calculating the weight for each of the multiple indicators used by using the indicator newly calculated by the first calculation means and the indicators previously calculated and stored in the second storage means; Equipped with The information processing device according to claim 9.

11. The third calculation means is an integrated index calculation means for calculating the integrated index based on the index newly calculated by the first calculation means, the index previously calculated and stored in the second storage means, and weights calculated by the weight calculation means for the newly calculated index and the index previously calculated; Equipped with The information processing device according to claim 10.

12. The sequence data is time-series data. The information processing device according to claim 1 .

13. A medical information acquisition means for acquiring medical information of a subject; An information processing device according to any one of claims 1 to 12, Equipped with the information processing device classifies the sequence data including the medical information as an element into any one of classes. Medical image identification device.

14. the information processing device classifies the series of data into any of classes indicating the presence or absence of a cancerous site in the medical information; 14. The medical image identification device according to claim 13.

15. A computer comprising: Sequentially acquiring multiple elements contained in the sequence data, calculating an index indicating to which of a plurality of classes each of the plurality of elements is appropriate to belong based on two or more of the plurality of elements; Calculating a weight indicating the importance of the index for each of the plurality of elements; weighting the indices of the plurality of elements with the corresponding weights and integrating the indices to calculate an integrated index indicating to which of the plurality of classes the sequence data belongs; classifying the sequence data into any one of classes based on the integrated index; Information processing methods.

16. A process of sequentially acquiring a plurality of elements included in sequence data; calculating an index indicating to which of a plurality of classes each of the plurality of elements is appropriate to belong based on two or more of the plurality of elements; A process of calculating a weight indicating the importance of the index for each of the plurality of elements; a process of weighting the indices of the plurality of elements with the corresponding weights and integrating the indices to calculate an integrated index indicating to which of the plurality of classes the sequence data belongs; classifying the sequence data into any one of classes based on the integrated index; A program for causing a computer to execute the following.

Images