Image processing device, image processing method, and recording medium

Abstract

An image processing device 3 is provided with a processor 33 that processes a captured image captured of the inside of a specimen. The processor 33 calculates an evaluation value of the captured image on the basis of the captured image, determines whether the evaluation value exists within a specific extraction range indicating an image of interest, extracts the captured image as the image of interest if it is determined that the evaluation value exists within the specific extraction range, and updates the specific extraction range, on the basis of a result of the determination of whether the evaluation value exists within the specific extraction range.

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and Recording medium Regarding. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there has been known an endoscope system that uses a swallowable capsule endoscope to obtain images of the inside of a subject and allows a medical professional to observe the images (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-293237 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the number of captured images taken by a capsule endoscope is enormous, and therefore, if a medical professional is required to check all of the captured images, it will take a long time to diagnose the subject. Therefore, it is conceivable to extract some captured images as images of interest from a huge number of captured images. Here, the image of interest means an image that includes a bleeding site or a lesion and requires a diagnosis by a medical professional. In other words, if the medical professional checks only the image of interest, it will not take much time to diagnose the subject.

[0005] Here, a conceivable method for extracting an image of interest is to calculate an evaluation value, which is an index for extracting the image as an image of interest, for each captured image, and to extract an image whose evaluation value exceeds a specific threshold value as an image of interest. However, this method of extracting an image of interest has a problem in that similar captured images that are close in time and have little need for confirmation are also extracted as images of interest. Therefore, there is a demand for a technique that can extract images of interest that are highly likely to be checked by medical professionals.

[0006] The present invention has been made in view of the above, and provides an image processing device, an image processing method, and a method for extracting captured images that are highly likely to be checked by medical personnel as images of interest. Recording medium The purpose of this invention is to provide [Means for solving the problem]

[0007] In order to solve the above problems and achieve the object, an image processing device according to the present invention includes a processor having hardware, The processor, Calculating an evaluation value of a captured image showing the inside of the subject; It is determined whether the evaluation value is included in a specific extraction range recorded in a memory, and when it is determined that the evaluation value is included in the extraction range, the captured image is extracted as an image of interest including a lesion, and the extraction range is updated based on the evaluation value. the extraction range is a range defined by a first reference value and a second reference value greater than the first reference value, the captured image is an image captured by an endoscope, and when the evaluation value exceeds the first reference value and also exceeds the second reference value, the processor determines that the evaluation value is included in the extraction range, determines whether or not the endoscope has reached a specific organ, and resets the second reference value to an initial value when it is determined that the endoscope has reached the specific organ. .

[0008] Further, an image processing method according to the present invention comprises: A processor in the image processing device calculates an evaluation value for evaluating the importance of a captured image showing an inside of a subject, determining whether the evaluation value is within a specific extraction range stored in a memory; The processor, When it is determined that the evaluation value is included in the extraction range, the captured image is extracted as an image of interest including a lesion; The processor, updating the extraction range based on the evaluation value; The extraction range is a range defined by a first reference value and a second reference value greater than the first reference value, the captured image is an image captured by an endoscope, and the processor determines that the evaluation value is included in the extraction range when the evaluation value exceeds the first reference value and also exceeds the second reference value, determines whether or not the endoscope has reached a specific organ, and resets the second reference value to an initial value when it is determined that the endoscope has reached the specific organ. .

[0009] The present invention also relates to Recording medium teeth, Non-transitory storage media that: calculating an evaluation value for evaluating the importance of a captured image showing the inside of a subject, determining whether or not the evaluation value is included in a specific extraction range recorded in a memory, and extracting the captured image as an image of interest including a lesion when it is determined that the evaluation value is included in the extraction range; the extraction range is a range defined by a first reference value and a second reference value greater than the first reference value, the captured image is an image captured by an endoscope, and a processor determines that the evaluation value is included in the extraction range when the evaluation value is greater than the first reference value and also greater than the second reference value, determines whether or not the endoscope has reached a specific organ, and resets the second reference value to an initial value when it is determined that the endoscope has reached the specific organ; The extraction range is updated based on the evaluation value. Effect of the Invention

[0010] According to the image processing device, the image processing method, and the image processing program of the present invention, it is possible to extract, as an image of interest, a captured image that is highly likely to be checked by a medical professional. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an endoscope system according to the first embodiment. [Diagram 2] FIG. 2 is a diagram showing a receiving device. [Diagram 3] FIG. 3 is a diagram showing a receiving device. [Figure 4] FIG. 4 is a flowchart showing the operation of the receiving device. [Diagram 5] FIG. 5 is a diagram illustrating step S2. [Figure 6] FIG. 6 is a diagram showing a specific extraction range. [Figure 7] FIG. 7 is a flowchart showing the operation of the receiving device according to the second embodiment. [Figure 8] FIG. 8 is a flowchart showing the operation of the receiving device according to the third embodiment. [Figure 9] FIG. 9 is a flowchart showing the operation of the receiving device according to the fourth embodiment. [Figure 10] FIG. 10 is a flowchart showing the operation of the receiving device according to the fifth embodiment. [Figure 11] FIG. 11 is a diagram showing a specific example of the reset operation. [Figure 12] FIG. 12 is a diagram showing a specific example of the reset operation. [Figure 13] FIG. 13 is a diagram showing a specific example of the reset operation. [Figure 14] FIG. 14 is a flowchart showing the operation of the receiving device according to the sixth embodiment. [Figure 15] FIG. 15 is a diagram showing a specific extraction range. [Figure 16] FIG. 16 is a flowchart showing the operation of the receiving device according to the seventh embodiment. [Figure 17] FIG. 17 is a flowchart showing the operation of the receiving device according to the eighth embodiment. [Figure 18] FIG. 18 is a flowchart showing the operation of the receiving device according to the ninth embodiment. [Figure 19] FIG. 19 is a flowchart showing the operation of the receiving device according to the tenth embodiment. [Figure 20] FIG. 20 is a diagram showing a modification of the first to fifth embodiments. [Figure 21] FIG. 21 is a diagram showing a modification of the sixth to tenth embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, a mode for carrying out the present invention (hereinafter, referred to as an embodiment) will be described with reference to the drawings. Note that the present invention is not limited to the embodiment described below. Furthermore, in the description of the drawings, the same parts are given the same reference numerals.

[0013] (Embodiment 1) [Overall configuration of endoscope system] FIG. 1 is a diagram showing an endoscope system 1 according to the first embodiment. The endoscope system 1 is a system that uses a swallowable capsule endoscope 2 to obtain images of the inside of a subject 100 and allows a medical professional or the like to observe the images. As shown in FIG. 1, the endoscope system 1 includes a capsule endoscope 2, a receiving device 3, and an image display device 4.

[0014] The capsule endoscope 2 is a capsule-type endoscope device formed to a size that can be introduced into the organ of the subject 100, and is introduced into the organ of the subject 100 by oral ingestion or the like, and sequentially captures images while moving inside the organ by peristaltic motion. Then, the capsule endoscope 2 sequentially transmits image data generated by capturing images.

[0015] The receiving device 3 corresponds to the image processing device according to the present invention. The receiving device 3 receives image data from the capsule endoscope 2 in the subject 100 through at least one of a plurality of receiving antennas 3a to 3f, each of which is configured using, for example, a loop antenna or a dipole antenna. In the first embodiment, the receiving device 3 is used in a state where it is carried by the subject 100, as shown in FIG. 1. The reason for adopting such a usage mode is to suppress the restriction of the behavior of the subject 100 while the capsule endoscope 2 is introduced. That is, the receiving device 3 needs to continue receiving the image data transmitted while moving within the subject 100 for several hours to several dozen hours, but keeping the subject 100 in the hospital for such a long period of time would impair the convenience of using the capsule endoscope 2. For this reason, in the first embodiment, the receiving device 3 is made small enough to be portable, thereby ensuring the freedom of movement of the subject 100 even while the capsule endoscope 2 is introduced, and reducing the burden on the subject 100.

[0016] 1, the receiving antennas 3a to 3f may be disposed on the body surface of the subject 100, or may be disposed in a jacket worn by the subject 100. The number of receiving antennas 3a to 3f may be one or more, and is not particularly limited to six. The detailed configuration of the receiving device 3 will be described later in the section "Configuration of the Receiving Device."

[0017] The image display device 4 is configured as a workstation that acquires image data from inside the subject 100 from the receiving device 3 and displays an image corresponding to the acquired image data.

[0018] [Configuration of the receiving device] Next, the detailed configuration of the receiving device 3 will be described. 2 and 3 are diagrams showing the receiving device 3. FIG. As shown in Figure 2 or Figure 3, the receiving device 3 includes a receiving unit 31 (Figure 3), an image processing unit 32 (Figure 3), a control unit 33 (Figure 3), a memory unit 34 (Figure 3), a data transmission / reception unit 35 (Figure 3), an operation unit 36 ​​(Figure 3), and a display unit 37.

[0019] The receiving unit 31 receives the image data transmitted from the capsule endoscope 2 via at least one of the multiple receiving antennas 3a to 3f. The image processing unit 32 performs various types of image processing on the image data (digital signal) received by the receiving unit 31. Examples of the image processing include optical black subtraction processing, white balance adjustment processing, digital gain processing, demosaic processing, color correction matrix processing, gamma correction processing, and YC processing that converts RGB signals into luminance signals and color difference signals (Y, Cb / Cr signals).

[0020] The control unit 33 corresponds to a processor according to the present invention. The control unit 33 is configured using, for example, a CPU (Central Processing Unit) or an FPGA (Field-Programmable Gate Array), and controls the operation of the entire receiving device 3 according to a program (including an image processing program according to the present invention) stored in the storage unit 34. The function of the control unit 33 will be described later in "Operation of the Receiving Device." The storage unit 34 stores the programs (including the image processing program according to the present invention) executed by the control unit 33 and information necessary for the processing of the control unit 33. The storage unit 34 also stores the image data sequentially transmitted from the capsule endoscope 2 and subjected to image processing by the image processing unit 32.

[0021] The data transmission / reception unit 35 is a communication interface, and transmits and receives data via wired or wireless communication with the image display device 4. For example, the data transmission / reception unit 35 transmits image data stored in the storage unit 34 to the image display device 4. The operation unit 36 ​​is configured using operation devices such as buttons and a touch panel, and receives user operations. The operation unit 36 ​​then outputs to the control unit 33 an operation signal in accordance with the user operation.

[0022] The display unit 37 is configured with a display using liquid crystal or organic EL (Electro Luminescence) or the like, and displays an image under the control of the control unit 33. In the first embodiment, the receiving device 3 has two display modes: a real-time view mode and a playback view mode. The two display modes can be switched by a user's operation on the operation unit . Specifically, in the real-time view mode, the display unit 37 sequentially displays images based on image data that is sequentially transmitted from the capsule endoscope 2 and has been subjected to image processing by the image processing unit 32. In the playback view mode, the display unit 37 displays an image of interest extracted by the control unit 33.

[0023] [Operation of the receiving device] Next, a description will be given of the operation of the above-mentioned receiving device 3. The operation of the receiving device 3 corresponds to an image processing method according to the present invention. FIG. 4 is a flowchart showing the operation of the receiving device 3. First, the receiving unit 31 receives (acquires) the Nth image data (hereinafter, referred to as the captured image) transmitted from the capsule endoscope 2 (step S1). The image processing unit 32 performs image processing on the Nth captured image received by the receiving unit 31. Then, the Nth captured image after the image processing is performed is stored in the storage unit 34.

[0024] After step S1, the control unit 33 reads out the Nth captured image stored in the storage unit 34, and extracts a feature amount of the Nth captured image (step S2). Fig. 5 is a diagram for explaining step S2. Specifically, Fig. 5 is a diagram showing the Nth captured image Pn. In Fig. 5, a black area Ar indicates a bleeding site or the like that appears in the captured image Pn. In the first embodiment, in step S2, the control unit 33 calculates a feature amount for each pixel of the Nth captured image Pn. Here, the feature amount is a feature amount indicating the features of a bleeding part, a lesion, or the like captured in the captured image. Specifically, the control unit 33 calculates R / B, which is obtained by dividing R in the pixel value (R, G, B) by B, as the feature amount for each pixel of the Nth captured image Pn. For example, when the pixel value (R, G, B) of a specific pixel PI shown in FIG. 5 is (180, 0, 10), the control unit 33 calculates R / B=18 as the feature amount for the specific pixel PI.

[0025] After step S2, the control unit 33 calculates an evaluation value of the N-th captured image Pn (step S3). Specifically, in step S3, the control unit 33 compares R / B, which is a feature amount for each pixel, with a specific reference value (e.g., 10). Then, the control unit 33 calculates the number of pixels having an R / B ratio exceeding the specific reference value among all pixels of the Nth captured image Pn as an evaluation value of the Nth captured image Pn.

[0026] After step S3, the control unit 33 determines whether or not the evaluation value calculated in step S3 exists within a specific extraction range indicating the image of interest (step S4). Here, the image of interest refers to a captured image that includes a bleeding site or a lesion and requires a diagnosis by a medical professional. The evaluation value is an index for extracting a captured image as an image of interest. FIG. 6 is a diagram showing a specific extraction range. In the first embodiment, the specific extraction range is a range having a first reference value and a second reference value (n) larger than the first reference value, and is a range that exceeds the first reference value and also exceeds the second reference value (n), as shown in Fig. 6. Note that the initial value of the second reference value is at least equal to or larger than the first reference value. Specifically, in step S4, the control unit 33 determines whether or not the evaluation value exceeds a first reference value (step S41). If it is determined that the evaluation value does not exceed the first reference value (step S41: No), the receiving device 3 proceeds to step S8.

[0027] On the other hand, when it is determined that the evaluation value exceeds the first reference value (step S41: Yes), the control unit 33 determines whether the evaluation value exceeds a second reference value (step S42). If it is determined that the evaluation value does not exceed the second reference value (step S42: No), the receiving device 3 proceeds to step S8.

[0028] On the other hand, if it is determined that the evaluation value exceeds the second reference value (step S42: Yes), the control unit 33 extracts the N-th captured image Pn as an image of interest (step S5). Specifically, in step S5, the control unit 33 associates the captured image Pn stored in the storage unit with information indicating that the captured image Pn is an image of interest (hereinafter, referred to as interest information).

[0029] After step S5, the control unit 33 notifies specific information (step S6). Specifically, in step S6, the control unit 33 causes the display unit 37 to display a message such as "Please call a medical professional" and also causes a speaker (not shown) to output a sound. The method of notifying the specific information is not limited to the above-mentioned message display and sound output, but may be a method of applying vibration to the subject 100.

[0030] After step S6, the control unit 33 updates the specific extraction range (step S7), after which the receiving device 3 proceeds to step S8. In the first embodiment, the control unit 33 updates the specific extraction range by changing the second reference value to a larger value in step S7. For example, as shown in Fig. 6, the second reference value (n) used so far is changed to a second reference value (n+1) larger than the second reference value (n).

[0031] In step S8 and after, the receiving device 3 switches to the captured image next to the N-th captured image Pn (N=N+1) and executes the processes of steps S1 to S7 again.

[0032] According to the above-described first embodiment, the following effects are achieved. In the receiving device 3 according to the first embodiment, the control unit 33 calculates an evaluation value of a captured image based on the captured image. The control unit 33 also determines whether the evaluation value exists within a specific extraction range indicating an image of interest. Specifically, when the evaluation value exceeds a first reference value and also exceeds a second reference value that is greater than the first reference value, the control unit 33 determines that the evaluation value exists within the specific extraction range. Furthermore, when the control unit 33 determines that the evaluation value exists within the specific extraction range, it extracts the captured image as an image of interest. Then, when the control unit 33 determines that the evaluation value exists within the specific extraction range, it updates the specific extraction range by changing the second reference value to a larger value. For example, assume that a captured image with an evaluation value exceeding a specific threshold is extracted as an image of interest. In this case, a similar captured image that is close in time and has little need for confirmation is also extracted as the image of interest. In contrast, in the first embodiment, an image of interest is extracted using a specific extraction range, and the specific extraction range is updated as described above. Therefore, a representative captured image including a bleeding site or the like can be extracted as the image of interest without extracting similar captured images that are close in time and have little need for confirmation as the image of interest. Therefore, the receiving device 3 according to the first embodiment can extract, as an image of interest, a captured image that is highly likely to be checked by a medical professional.

[0033] Particularly in the first embodiment, the receiving device 3 is configured as an image processing device according to the present invention. The receiving device 3 notifies specific information when a captured image is extracted as an image of interest. Therefore, the receiving device 3 performs a process of extracting the captured image as an image of interest in real time, and notifies specific information when the image is extracted as an image of interest, thereby enabling medical personnel to quickly decide on a diagnostic policy for the subject.

[0034] Furthermore, the control unit 33 calculates a feature amount of the captured image based on the pixel value (R, G, B) of each pixel in the captured image. Therefore, the feature amount can be calculated by simple processing.

[0035] (Embodiment 2) Next, a second embodiment will be described. In the following description, the same components as those in the above-described first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 7 is a flowchart showing the operation of the receiving device 3 according to the second embodiment. In the present embodiment 2, as shown in Fig. 7, the operation of the receiving device 3 is different from that of the above-mentioned embodiment 1. Specifically, in the present embodiment 2, steps S9 to S14 are added to the above-mentioned embodiment 1. Therefore, steps S9 to S14 will be mainly described below.

[0036] Step S9 is executed after step S6. Specifically, in step S9, the control unit 33 determines whether or not resetting of the second reference value to the initial value (step S11 or step S14 described later) has already been executed.

[0037] If it is determined that the second reference value has already been reset to the initial value (step S9: Yes), the receiving device 3 proceeds to step S7. On the other hand, if it is determined that the second reference value has not yet been reset to the initial value (step S9: No), the control unit 33 determines whether or not a preset time has elapsed (step S10). For example, in step S10, the control unit 33 measures the time from the time when the first image data is received, and determines whether or not the measured time has elapsed the preset time.

[0038] If it is determined that the predetermined time has not elapsed (step S10: No), the receiving device 3 proceeds to step S7. On the other hand, if it is determined that the predetermined time has elapsed (step S10: Yes), the control unit 33 resets the second reference value to the initial value (step S11). After that, the receiving device 3 proceeds to step S8.

[0039] Step S12 is executed when it is determined that the evaluation value does not exceed the second reference value (Step S42: No). Specifically, in step S12, the control unit 33 determines whether or not resetting of the second reference value to the initial value (step S11 or step S14 described later) has already been executed.

[0040] If it is determined that the second reference value has already been reset to the initial value (step S12: Yes), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that the second reference value has not yet been reset to the initial value (step S12: No), the control unit 33 determines whether a predetermined time has elapsed (step S13), similar to step S10.

[0041] If it is determined that the predetermined time has not elapsed (step S13: No), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that the predetermined time has elapsed (step S13: Yes), the control unit 33 resets the second reference value to the initial value (step S14). After that, the receiving device 3 proceeds to step S8.

[0042] According to the above-described second embodiment, in addition to the same effects as those of the first embodiment, the following effects are achieved. Incidentally, if the capsule endoscope 2 captures an image of red clothing, a red wall, or the like before the subject 100 swallows the capsule endoscope 2, a specific extraction range that should not be updated is updated in step S7. In this case, there is a possibility that a captured image that includes a bleeding site or the like that is highly required to be checked by a medical professional will not be extracted as an image of interest. In the receiving device 3 according to the second embodiment, when a predetermined time has elapsed, the second reference value is reset to an initial value. Therefore, even in the above-mentioned case, a captured image that includes a bleeding site or the like that is highly required to be checked by a medical professional can be extracted as an image of interest.

[0043] (Embodiment 3) Next, a third embodiment will be described. In the following description, the same components as those in the above-described first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 8 is a flowchart showing the operation of the receiving device 3 according to the third embodiment. In the third embodiment, as shown in Fig. 8, the operation of the receiving device 3 is different from that of the first embodiment. Specifically, the third embodiment is different from the first embodiment in that steps S15 and S16 are added. Therefore, steps S15 and S16 will be mainly described below.

[0044] Step S15 is executed when it is determined that the evaluation value does not exceed the second reference value (Step S42: No). Specifically, in step S15, the control unit 33 determines whether or not a preset time has elapsed. For example, the control unit 33 measures the time during which the evaluation value exceeds the first reference value but does not exceed the second reference value, and determines whether or not the measured time has elapsed the preset time.

[0045] If it is determined that the predetermined time has not elapsed (step S15: No), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that the predetermined time has elapsed (step S15: Yes), the control unit 33 resets the second reference value to the initial value (step S16). After that, the receiving device 3 proceeds to step S8.

[0046] According to the above-described third embodiment, in addition to the same effects as those of the first embodiment, the following effects are achieved. However, there are cases where the capsule endoscope 2 is stagnant in the subject 100, or where there are multiple bleeding sites and some of the bleeding sites are less severe than the bleeding site captured in the captured image that is initially extracted as the image of interest. In such cases, there is a possibility that the captured image in which the bleeding site that is highly necessary to be confirmed by a medical professional is captured will not be extracted as the image of interest. In the receiving device 3 according to the third embodiment, the second reference value is periodically reset to an initial value as a predetermined time elapses. Therefore, even in the above-mentioned case, a captured image that includes a bleeding site that is highly likely to be checked by a medical professional can be extracted as an image of interest.

[0047] (Embodiment 4) Next, a fourth embodiment will be described. In the following description, the same components as those in the above-described first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 9 is a flowchart showing the operation of the receiving device 3 according to the fourth embodiment. In the fourth embodiment, as shown in Fig. 9, the operation of the receiving device 3 is different from that of the first embodiment. Specifically, the fourth embodiment is different from the first embodiment in that steps S17 to S20 are added. Therefore, steps S17 to S20 will be mainly described below.

[0048] Step S17 is executed after step S6. Specifically, in step S17, the control unit 33 determines whether the capsule endoscope 2 has reached the organ of interest. The organ of interest is an organ that exists on the route taken by the capsule endoscope 2 and means at least one specific organ set in advance. For example, the control unit 33 determines whether the capsule endoscope 2 has reached the organ of interest based on the elapsed time from the time when the first image data was received, the shape or color of the subject captured in the Nth captured image Pn, and the like.

[0049] If it is determined that the radiation has not reached the organ of interest (step S17: No), the receiving device 3 proceeds to step S7. On the other hand, when it is determined that the organ of interest has been reached (step S17: Yes), the control unit 33 resets the second reference value to the initial value (step S18). After that, the receiving device 3 proceeds to step S8.

[0050] Step S19 is executed when it is determined that the evaluation value does not exceed the second reference value (Step S42: No). Specifically, in step S19, the control unit 33 determines whether or not the capsule endoscope 2 has reached the organ of interest, similarly to step S17.

[0051] If it is determined that the radiation has not reached the organ of interest (step S19: No), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that the organ of interest has been reached (step S19: Yes), the control unit 33 resets the second reference value to the initial value (step S20). After that, the receiving device 3 proceeds to step S8.

[0052] According to the fourth embodiment described above, in addition to the same effects as those of the first embodiment, the following effects are achieved. Incidentally, there are cases where it is desired to check bleeding sites and the like for each organ such as the stomach, small intestine, etc. However, for example, when steps S1 to S8 are repeatedly performed on images captured within the stomach, the second reference value is updated to a large value, and there is a possibility that an image in which a bleeding site or the like is captured in the small intestine, which is reached after the stomach, is not extracted as an image of interest. In the receiving device 3 according to the fourth embodiment, the second reference value is reset to an initial value every time the capsule endoscope 2 reaches the organ of interest. Therefore, in each organ of interest, a captured image that includes a bleeding site or the like that is highly likely to be checked by a medical professional can be extracted as an image of interest.

[0053] (Embodiment 5) Next, a fifth embodiment will be described. In the following description, the same components as those in the above-described first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 10 is a flowchart showing the operation of the receiving device 3 according to the fifth embodiment. In the fifth embodiment, as shown in Fig. 10, the operation of the receiving device 3 is different from that of the first embodiment. Specifically, the fifth embodiment is different from the first embodiment in that steps S21 to S24 are added. Therefore, steps S21 to S24 will be mainly described below.

[0054] Step S21 is executed after step S6. Specifically, in step S21, the control unit 33 determines whether or not a reset operation (user operation) has been performed on the operation unit 36 ​​by the user.

[0055] If it is determined that a reset operation has not been performed (step S21: No), the receiving device 3 proceeds to step S7. On the other hand, if it is determined that a reset operation has been performed (step S21: Yes), the control unit 33 resets the second reference value to the initial value (step S22). After that, the receiving device 3 proceeds to step S8.

[0056] Step S23 is executed when it is determined that the evaluation value does not exceed the first reference value (step S41: No) or when it is determined that the evaluation value does not exceed the second reference value (step S42: No). Specifically, in step S23, the control unit 33 determines whether or not a reset operation (user operation) has been performed on the operation unit 36 ​​by the user.

[0057] If it is determined that the reset operation has not been performed (step S23: No), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that a reset operation has been performed (step S23: Yes), the control unit 33 resets the second reference value to the initial value (step S24). After that, the receiving device 3 proceeds to step S8.

[0058] Fig. 11 to Fig. 13 are diagrams showing specific examples of the reset operation. Specifically, Fig. 11 shows the state of the receiving device 3 when step S6 is executed. Fig. 12 shows a state in which the medical staff switches the display mode of the receiving device 3 to the playback view mode after step S6 is executed. Fig. 13 shows a state in which the medical staff switches the display mode of the receiving device 3 to the real-time view mode after checking the state of Fig. 12. Note that the icon IC displayed on the display unit 37 in Figs. 11 to 13 is the icon pressed by the above-mentioned reset operation. In the real-time view mode, the display unit 37 sequentially displays captured images based on image data that is sequentially transmitted from the capsule endoscope 2 and subjected to image processing by the image processing unit 32. Here, in step S6, when the Nth captured image Pn (FIG. 11) is extracted as an image of interest, specific information is notified from the receiving device 3.

[0059] In response to the notification of the specific information from the receiving device 3, the medical worker checks the receiving device 3. Specifically, the medical worker switches the display mode of the receiving device 3 to the playback view mode by a user operation on the operation unit 36. Then, as shown in FIG. 12, the medical worker checks the captured image Pn that is based on image data received in the past and is extracted as an image of interest.

[0060] After checking the captured image Pn in the playback view mode, the medical worker switches the display mode of the receiving device 3 to the real-time view mode by a user operation on the operation unit 36. Then, the medical worker checks whether there is bleeding from the captured image Pn' based on the currently received image data, as shown in Fig. 13. If the medical worker can confirm that there is no bleeding and that the patient is in a normal state, he or she presses the icon IC.

[0061] According to the fifth embodiment described above, in addition to the same effects as those of the first embodiment, the following effects are achieved. In the receiving device 3 according to the fifth embodiment, the second reference value is reset to an initial value in response to a reset operation on the operation unit 36 ​​by the user. Therefore, if the medical staff checks the captured image Pn in the playback view mode and then checks that there is no bleeding and that the image is normal in the real-time view mode, they reset the second reference value to the initial value by performing a reset operation, which allows the next captured image that includes the bleeding site or the like to be extracted as the image of interest.

[0062] (Embodiment 6) Next, a sixth embodiment will be described. In the following description, the same components as those in the above-described first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 14 is a flowchart showing the operation of the receiving device 3 according to the sixth embodiment. In the sixth embodiment, as shown in Fig. 14, the operation of the receiving device 3 is different from that of the first embodiment. Specifically, the sixth embodiment is different from the first embodiment in that steps S2A to S4A and S7A are adopted instead of steps S2 to S4 and S7. Therefore, steps S2A to S4A and S7A will be mainly described below.

[0063] [Step S2A] The control unit 33 calculates, for each pixel of all pixels in the Nth captured image Pn, B / R, which is obtained by dividing B in the pixel value (R, G, B) by R, as a feature amount. For example, when the pixel value (R, G, B) of a specific pixel PI shown in Fig. 5 is (180, 0, 10), the control unit 33 calculates B / R = 1 / 18 as a feature amount for the specific pixel PI.

[0064] [Step S3A] The control unit 33 calculates the smallest value of B / R, which is the characteristic amount for each pixel, as the evaluation value of the Nth captured image Pn.

[0065] [Step S4A] The control unit 33 determines whether or not the evaluation value calculated in step S3A exists within a specific extraction range indicating the image of interest. FIG. 15 is a diagram showing a specific extraction range. In the sixth embodiment, the specific extraction range is a range having a third reference value and a fourth reference value (n) smaller than the third reference value, and is a range below the third reference value and below the fourth reference value (n), as shown in Fig. 15. Note that the initial value of the fourth reference value is at least equal to or smaller than the third reference value.

[0066] Specifically, in step S4A, the control unit 33 determines whether or not the evaluation value is below a third reference value (step S41A). If it is determined that the evaluation value is not below the third reference value (step S41A: No), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that the evaluation value is lower than the third reference value (step S41A: Yes), the receiving device 3 proceeds to step S5.

[0067] [Step S7A] The control unit 33 updates the specific extraction range by changing the fourth reference value to a smaller value. For example, as shown in Fig. 15, the fourth reference value (n) that has been used so far is changed to a fourth reference value (n+1) that is smaller than the fourth reference value (n).

[0068] Even when the receiving device 3 operates as in the sixth embodiment described above, the same effects as those of the first embodiment can be achieved.

[0069] (Embodiment 7) Next, a seventh embodiment will be described. In the following description, the same components as those in the sixth embodiment described above are given the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 16 is a flowchart showing the operation of the receiving device 3 according to the seventh embodiment. In the seventh embodiment, as shown in Fig. 16, the operation of the receiving device 3 is different from that in the sixth embodiment. Specifically, in the seventh embodiment, steps S9A to S14A are added to the sixth embodiment. Therefore, steps S9A to S14A will be mainly described below.

[0070] Step S9A is executed after step S6. Specifically, in step S9A, control unit 33 determines whether or not resetting of the fourth reference value to the initial value (step S11A or step S14A described later) has already been executed.

[0071] If it is determined that the fourth reference value has already been reset to the initial value (step S9A: Yes), the receiving device 3 proceeds to step S7A. On the other hand, if it is determined that the fourth reference value has not yet been reset to the initial value (step S9A: No), the control unit 33 determines whether or not a preset time has elapsed (step S10A). For example, in step S10A, the control unit 33 measures the time from the time when the first image data is received, and determines whether or not the measured time has elapsed the preset time.

[0072] If it is determined that the predetermined time has not elapsed (step S10A: No), the receiving device 3 proceeds to step S7A. On the other hand, if it is determined that the predetermined time has elapsed (step S10A: Yes), the control unit 33 resets the fourth reference value to the initial value (step S11A). After that, the receiving device 3 proceeds to step S8.

[0073] Step S12A is executed when it is determined that the evaluation value is not below the fourth reference value (Step S42A: No). Specifically, in step S12A, the control unit 33 determines whether or not resetting of the fourth reference value to the initial value (step S11A or step S14A described later) has already been executed.

[0074] If it is determined that the fourth reference value has already been reset to the initial value (step S12A: Yes), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that the fourth reference value has not yet been reset to the initial value (step S12A: No), the control unit 33 determines whether a predetermined time has elapsed (step S13A), similar to step S10A.

[0075] If it is determined that the predetermined time has not elapsed (step S13A: No), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that the predetermined time has elapsed (step S13A: Yes), the control unit 33 resets the fourth reference value to the initial value (step S14A). After that, the receiving device 3 proceeds to step S8.

[0076] Even when the receiving device 3 operates as in the seventh embodiment described above, the same effects as those in the second and sixth embodiments described above can be achieved.

[0077] (Embodiment 8) Next, an eighth embodiment will be described. In the following description, the same components as those in the sixth embodiment described above are given the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 17 is a flowchart showing the operation of the receiving device 3 according to the eighth embodiment. In the eighth embodiment, as shown in Fig. 17, the operation of the receiving device 3 is different from that in the sixth embodiment. Specifically, the eighth embodiment is different from the sixth embodiment in that steps S15A and S16A are added. Therefore, steps S15A and S16A will be mainly described below.

[0078] Step S15A is executed when it is determined that the evaluation value is not below the fourth reference value (Step S42A: No). Specifically, in step S15A, the control unit 33 determines whether or not a preset time has elapsed. For example, the control unit 33 measures the time during which the evaluation value is below the third reference value but not below the fourth reference value, and determines whether or not the measured time has elapsed the preset time.

[0079] If it is determined that the predetermined time has not elapsed (step S15A: No), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that the predetermined time has elapsed (step S15A: Yes), the control unit 33 resets the fourth reference value to the initial value (step S16A). After that, the receiving device 3 proceeds to step S8.

[0080] Even when the receiving device 3 operates as in the seventh embodiment described above, the same effects as those in the third and sixth embodiments described above can be achieved.

[0081] (Embodiment 9) Next, a ninth embodiment will be described. In the following description, the same components as those in the sixth embodiment described above are given the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 18 is a flowchart showing the operation of the receiving device 3 according to the ninth embodiment. In the ninth embodiment, as shown in Fig. 18, the operation of the receiving device 3 is different from that in the sixth embodiment. Specifically, in the ninth embodiment, steps S17A to S20A are added to the sixth embodiment. Therefore, steps S17A to S20A will be mainly described below.

[0082] Step S17A is executed after step S6. Specifically, in step S17A, the control unit 33 determines whether or not the capsule endoscope 2 has reached the organ of interest. The organ of interest is a specific organ that is present on the path taken by the capsule endoscope 2 and is set in advance. For example, the control unit 33 determines whether or not the capsule endoscope 2 has reached the organ of interest based on the elapsed time from the time when the first image data was received, the shape or color of the subject captured in the Nth captured image Pn, and the like.

[0083] If it is determined that the radiation has not reached the organ of interest (step S17A: No), the receiving device 3 proceeds to step S7A. On the other hand, when it is determined that the organ of interest has been reached (step S17A: Yes), the control unit 33 resets the fourth reference value to the initial value (step S18A). After that, the receiving device 3 proceeds to step S8.

[0084] Step S19A is executed when it is determined that the evaluation value is not below the fourth reference value (Step S42A: No). Specifically, in step S19A, the control unit 33 determines whether or not the capsule endoscope 2 has reached the organ of interest, similarly to step S17A.

[0085] If it is determined that the radiation has not reached the organ of interest (step S19A: No), the receiving device 3 proceeds to step S8. On the other hand, when it is determined that the organ of interest has been reached (step S19A: Yes), the control unit 33 resets the fourth reference value to the initial value (step S20A). After that, the receiving device 3 proceeds to step S8.

[0086] Even when the receiving device 3 operates as in the above-described ninth embodiment, the same effects as those in the fourth and sixth embodiments can be achieved.

[0087] (Embodiment 10) Next, a tenth embodiment will be described. In the following description, the same components as those in the sixth embodiment described above are given the same reference numerals, and detailed description thereof will be omitted or simplified. FIG. 19 is a flowchart showing the operation of the receiving device 3 according to the tenth embodiment. In the tenth embodiment, as shown in Fig. 19, the operation of the receiving device 3 is different from that of the sixth embodiment described above. Specifically, in the sixth embodiment, steps S21A to S24A are added to the sixth embodiment described above. Therefore, steps S21A to S24A will be mainly described below.

[0088] Step S21A is executed after step S6. Specifically, in step S21A, the control unit 33 determines whether or not a reset operation (user operation) has been performed on the operation unit 36 ​​by the user. The reset operation according to the tenth embodiment may be exemplified by the reset operation shown in FIGS. 11 to 13 in the fifth embodiment described above.

[0089] If it is determined that a reset operation has not been performed (step S21A: No), the receiving device 3 proceeds to step S7A. On the other hand, if it is determined that a reset operation has been performed (step S21A: Yes), the control unit 33 resets the fourth reference value to the initial value (step S22A). After that, the receiving device 3 proceeds to step S8.

[0090] Step S23A is executed when it is determined that the evaluation value is not below the third reference value (step S41A: No) or when it is determined that the evaluation value is not below the fourth reference value (step S42A: No). Specifically, in step S23A, the control unit 33 determines whether or not a reset operation (user operation) has been performed on the operation unit 36 ​​by the user.

[0091] If it is determined that the reset operation has not been performed (step S23A: No), the receiving device 3 proceeds to step S8. On the other hand, if it is determined that a reset operation has been performed (step S23A: Yes), the control unit 33 resets the fourth reference value to the initial value (step S24A). After that, the receiving device 3 proceeds to step S8.

[0092] Even when the receiving device 3 operates as in the tenth embodiment described above, the same effects as those of the fifth and sixth embodiments described above can be achieved.

[0093] (Other embodiments) Although the embodiments for carrying out the present invention have been described above, the present invention should not be limited to only the above-mentioned first to tenth embodiments. In the above-mentioned first to fifth embodiments, only one evaluation value (the number of pixels having R / B exceeding a specific reference value among all pixels of the Nth captured image Pn) is calculated for the Nth captured image Pn and compared with only one first and second reference value, but this is not limited thereto. It is also possible to calculate n evaluation values ​​and compare them with n first and second reference values. FIG. 20 is a diagram showing a modification of the first to fifth embodiments, and illustrates a case in which two evaluation values ​​are calculated and compared with two first and second reference values, respectively. Specifically, the first evaluation value (x) may be, for example, the number of pixels having R exceeding a specific reference value among all pixels of the Nth captured image Pn. The second evaluation value (y) may be, for example, the largest value of R / B for each pixel of the Nth captured image Pn. Furthermore, two first reference values ​​(x) and first reference values ​​(y) are provided corresponding to the two evaluation values ​​(x) and evaluation values ​​(y), and two second reference values ​​(x)(n) and second reference values ​​(y)(n). Then, in step S41, it is determined whether the two evaluation values ​​(x) and evaluation values ​​(y) are outside the area surrounded by the X-axis, the Y-axis, and the lines connecting the two first reference values ​​(x) and the first reference values ​​(y). Similarly, in step S42, it is determined whether the two evaluation values ​​(x) and (y) are outside the area surrounded by the X-axis, the Y-axis, and the line connecting the two second reference values ​​(x)(n) and the second reference value (y)(n). Here, the two second reference values ​​(x)(n+1) and the second reference value (y)(n+1) shown in Fig. 20 indicate the values ​​obtained by changing the two second reference values ​​(x)(n) and the second reference value (y)(n) in step S7. In FIG. 20, the line connecting the two first reference values ​​and the line connecting the two second reference values ​​are shown as curved lines, but the lines may also be curved lines that are part of an ellipse, curved lines that are part of a circle, straight lines, and straight lines that are part of a rectangle.

[0094] In the sixth to tenth embodiments described above, only one evaluation value for the Nth captured image Pn (the smallest value among B / R for each pixel of the Nth captured image Pn) is calculated and compared with only one third and fourth reference value, but this is not limiting. It is also possible to calculate n evaluation values ​​and compare them with n third and fourth reference values. FIG. 21 is a diagram showing a modification of the sixth to tenth embodiments, and illustrates a case in which two evaluation values ​​are calculated and compared with two third and fourth reference values, respectively. Specifically, the first evaluation value (x) may be exemplified by the smallest value of G / R for each pixel of the Nth captured image Pn. The second evaluation value (y) may be exemplified by the smallest value of B / R for each pixel of the Nth captured image Pn. Furthermore, two third reference values ​​(x) and third reference values ​​(y) are provided corresponding to the two evaluation values ​​(x) and evaluation values ​​(y), and two fourth reference values ​​(x)(n) and fourth reference values ​​(y)(n). Then, in step S41A, it is determined whether the two evaluation values ​​(x) and evaluation values ​​(y) are present within an area surrounded by the X-axis, the Y-axis, and the lines connecting the two third reference values ​​(x) and the third reference values ​​(y). Similarly, in step S42A, it is determined whether or not the two evaluation values ​​(x) and (y) are within an area surrounded by the X-axis, the Y-axis, and a line connecting the two fourth reference values ​​(x)(n) and the fourth reference value (y)(n). Note that the two fourth reference values ​​(x)(n+1) and the fourth reference value (y)(n+1) shown in Fig. 21 indicate values ​​obtained by changing the two fourth reference values ​​(x)(n) and the fourth reference value (y)(n) in step S7A. In FIG. 21, the line connecting the two third reference values ​​and the line connecting the two fourth reference values ​​are shown as curves, but the lines may also be curves that are part of an ellipse, curves that are part of a circle, straight lines, and straight lines that are part of a rectangle.

[0095] In the above-mentioned first to tenth embodiments, the following evaluation values ​​may be adopted as the evaluation value of the Nth captured image Pn. For example, in the above-mentioned first to fifth embodiments, the largest value among the B / R values ​​for each pixel in the Nth captured image Pn may be used as the evaluation value of the Nth captured image Pn. Furthermore, for example, in the above-mentioned first to tenth embodiments, a deep learning technique may be used to digitize a lesion or a bleeding site, and the digitized value may be used as the evaluation value of the N-th captured image Pn.

[0096] In the above-mentioned first to tenth embodiments, the receiving device 3 is configured as the image processing device according to the present invention, but this is not limiting, and the image display device 4 may be configured as the image processing device according to the present invention.

[0097] In the above-described first to tenth embodiments, a configuration is adopted in which processing is performed on the captured images captured by the capsule endoscope 2, but this is not limited thereto, and a configuration may be adopted in which processing is performed on other captured images as long as the captured images are acquired in time series.

[0098] Furthermore, the processing flow is not limited to the processing order in the flowcharts explained in the first to tenth embodiments, and may be changed as long as there is no inconsistency. [Explanation of symbols]

[0099] 1 Endoscope system 2. Capsule endoscope 3. Receiving device 3a~3f Receiving antennas 31 Receiving section 32 Image processing section 33 Control Unit 34 Storage section 35 Data transmission and reception unit 36 Control section 37 Display section 4 Image display device 100 subjects Ar Black areas IC Icon PI pixel Pn Nth captured image Pn´ Captured image

Claims

1. a processor having hardware; The processor, Calculating an evaluation value of a captured image showing the inside of the subject; determining whether the evaluation value is within a specific extraction range stored in a memory; When it is determined that the evaluation value is included in the extraction range, the captured image is extracted as an image of interest including a lesion; updating the extraction range based on the evaluation value; The extraction range is: a range defined by a first reference value and a second reference value greater than the first reference value, The captured image is An image captured by an endoscope, The processor, determining that the evaluation value is included in the extraction range when the evaluation value is greater than the first reference value and greater than the second reference value; an image processing device that determines whether or not the endoscope has reached a specific organ, and resets the second reference value to an initial value when it is determined that the endoscope has reached the specific organ.

2. Further comprising an alarm for notifying that the image of interest has been extracted, or a monitor for displaying the same, The processor, 2 . The image processing apparatus according to claim 1 , further comprising control for causing the alarm to notify, or for causing the monitor to display, when the captured image is extracted as the image of interest, that the image of interest has been extracted.

3. The processor, The image processing apparatus according to claim 1 , wherein the evaluation value is calculated based on a feature amount of the captured image.

4. The processor, The image processing apparatus according to claim 3 , wherein the feature amount is calculated based on a pixel value of each pixel in the captured image.

5. The processor, The image processing device according to claim 1 , wherein, when it is determined that the evaluation value is included in the extraction range, the extraction range is updated so that the evaluation value becomes the second reference value.

6. The processor, The image processing apparatus according to claim 1 , further comprising: determining whether a preset time has elapsed; and resetting the second reference value to an initial value before the update when it is determined that the preset time has elapsed.

7. Further comprising an operation unit for accepting a user operation, The processor, The image processing apparatus according to claim 1 , wherein the second reference value is reset to an initial value when the user operation is performed on the operation unit.

8. The extraction range is: a range defined by a third reference value and a fourth reference value smaller than the third reference value, The processor, The image processing device according to claim 1 , wherein, when the evaluation value is below the third reference value and the evaluation value is below the fourth reference value, it is determined that the evaluation value is included in the extraction range.

9. The processor, The image processing device according to claim 8 , wherein, when it is determined that the evaluation value is included in the extraction range, the extraction range is updated so that the evaluation value becomes the fourth reference value.

10. The processor, The image processing apparatus according to claim 8 , further comprising: determining whether a preset time has elapsed; and resetting the fourth reference value to an initial value when it is determined that the preset time has elapsed.

11. The captured image is An image captured by an endoscope, The processor, The image processing apparatus according to claim 8 , further comprising: determining whether or not the endoscope has reached a specific organ; and resetting the fourth reference value to an initial value when it is determined that the endoscope has reached the specific organ.

12. Further comprising an operation unit for accepting a user operation, The processor, The image processing device according to claim 8 , wherein the fourth reference value is reset to an initial value when the user operation is performed on the operation unit.

13. The evaluation value is The image processing device according to claim 1 , wherein the importance of the captured image is determined.

14. A processor in the image processing device calculates an evaluation value for evaluating the importance of an image showing an inside of a subject, The processor determines whether the evaluation value is included in a specific extraction range stored in a memory; When the processor determines that the evaluation value is included in the extraction range, the processor extracts the captured image as an image of interest including a lesion; The processor updates the extraction range based on the evaluation value; The extraction range is: a range defined by a first reference value and a second reference value greater than the first reference value, The captured image is An image captured by an endoscope, The processor, determining that the evaluation value is included in the extraction range when the evaluation value is greater than the first reference value and greater than the second reference value; An image processing method comprising: determining whether or not the endoscope has reached a specific organ; and resetting the second reference value to an initial value when it is determined that the endoscope has reached the specific organ.

15. The image processing method of claim 14, wherein the processor, when extracting the captured image as the image of interest, causes an alarm to notify the user that the image of interest has been extracted, or displays the information on a monitor.

16. The image processing method according to claim 14, wherein the processor calculates the evaluation value based on features of the captured image.

17. The image processing method according to claim 16, wherein the processor calculates the feature amount based on pixel values ​​of each pixel in the captured image.

18. Non-transitory recording media that: Calculating an evaluation value for evaluating the importance of the captured image showing the inside of the subject; determining whether the evaluation value is within a specific extraction range stored in a memory; When it is determined that the evaluation value is included in the extraction range, the captured image is extracted as an image of interest including a lesion; The extraction range is: a range defined by a first reference value and a second reference value greater than the first reference value, The captured image is An image captured by an endoscope, The processor determining that the evaluation value is included in the extraction range when the evaluation value is greater than the first reference value and greater than the second reference value; determining whether the endoscope has reached a specific organ, and resetting the second reference value to an initial value when it is determined that the endoscope has reached the specific organ; A recording medium that updates the extraction range based on the evaluation value.

Images