Image processing device, image processing system, control method and program

Abstract

To provide an image processing device that can detect sagging of an affected part from a captured image of a subject including a skin area and an affected area.SOLUTION: An image processing device 130 capable of capturing images of a subject including a skin area and an affected area, includes: first detection means (S402) for detecting the affected area based on a captured image; second detection means (S403) for detecting an affected-part peripheral area, which is a peripheral area of the affected part, based on the detected affected area; edge detection means (S405) for detecting an edge of the detected affected-part peripheral area; quantification means (S405) for quantifying the detected edge; and determination means (S406) for determining whether or not the affected part has sagging based on a quantified amount of edge.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing system, a control method, and a program.

Background Art

[0002] In recent years, informatization has also advanced in medical institutions such as hospitals, and the number of medical institutions introducing electronic medical records for managing patients' diagnostic information has been increasing. Electronic medical records centrally manage medical information such as patients' diagnostic histories, medication information, surgical information, affected part images, and X-ray images, enabling easy data sharing and reuse. In addition, due to the recent increase in storage capacity, it has become possible to capture a large number of digital images of affected parts and store them in electronic medical records.

[0003] In such an informatized environment, a usage mode in which a patient is imaged using a digital camera and the captured image is stored in an electronic medical record as a medical record is frequently performed. In particular, at the sites of dermatology, surgery, nursing care, etc., the affected parts of patients, such as wounds, surgical scars, and pressure ulcers (bedsores), are regularly imaged with a digital camera to observe the changes over time of the affected parts. In addition, with the recent development of communication technology, telemedicine while staying at home is also being realized. For example, the patient himself / herself, family members, nurses, etc. take pictures of the affected part and send the captured pictures to a medical institution to receive a diagnosis from a doctor. In such imaging with a digital camera, there are problems such as the affected part area being blocked by the "sagging" of the skin around the affected part, and the inability to capture the affected part in an appropriate size that reflects the condition of the affected part.

[0004] When observing the affected part over time, it is important to perform reproducible imaging with the "sagging" of the skin of the same affected part eliminated. For this reason, when imaging the affected part with a digital camera, two nurses or caregivers execute it. One nurse or caregiver pulls the sagging skin around the affected part, and the other nurse or caregiver follows the procedure of imaging the affected part.

[0005] However, there are cases where the affected area is imaged without pulling the sagging skin, or cases where, even though the skin is actually sagging, it is determined that the sagging is not significant and the affected area is imaged without pulling the sagging skin. Furthermore, in the case of imaging alone, there is also a risk of imaging the sagging skin without pulling it. In this case, problems such as the affected area being blocked and the inability to obtain an appropriate-sized image reflecting the condition of the affected area occurred, and the affected area could not be observed appropriately. There is "wrinkle" as a skin condition similar to such "sagging" of the skin, and as a method for detecting wrinkles, there is, for example, a wrinkle state analysis device disclosed in Patent Document 1. In this wrinkle state analysis device, a method is disclosed in which edge detection processing is performed on a skin image, and the wrinkle state is quantified based on the density and number of edges.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] However, the wrinkle state analysis device disclosed in Patent Document 1 cannot detect "sagging" in the affected area.

[0008] An object of the present invention is to provide an image processing apparatus, an image processing system, a control method, and a program capable of detecting sagging of an affected area from a captured image of a subject including a skin area and an affected area.

Means for Solving the Problems

[0009] In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus capable of capturing an image of a subject including a skin region and an affected area region, and includes a first detection means for detecting the affected area region based on the captured image, a second detection means for detecting an affected area peripheral region which is a peripheral region of the affected area based on the affected area region detected by the first detection means, an edge detection means for detecting an edge of the affected area peripheral region detected by the second detection means, a quantification means for quantifying the edge detected by the edge detection means, and a determination means for determining whether there is sagging in the affected area based on the edge amount quantified by the quantification means.

Effect of the Invention

[0010] According to the present invention, it is possible to detect sagging of an affected area from a captured image of a subject including a skin region and an affected area region, and an effect of preventing the affected area from being imaged in a state where the skin is sagging can be obtained.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Modes for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in the embodiments.

[0013] <<First Embodiment>> (FIG. 1: Hardware configuration diagram of an image processing system according to the first embodiment) First, with reference to FIG. 1, the hardware configuration of an image processing system 100 according to the first embodiment of the present invention will be described. As shown in FIG. 1, the image processing system 100 according to the first embodiment includes an imaging device 160 and an image processing device 130. Both are connected via a wireless network 190 so as to be able to wirelessly communicate required information. The captured image acquired by the imaging device 160 through imaging and the information output by the image processing device 130 are configured to be able to mutually transmit and receive. Note that the wireless network 190 is an example, and for example, a wired network or the like may be used to form a network.

[0014] (Imaging device 160) The imaging device 160 is a camera used by one or more users such as doctors, nurses, and caregivers (hereinafter referred to as "user U") at the medical site. Also, when "swelling" is observed in the affected area, the person who plays the role of pulling up the patient's skin is referred to as "imaging assistant V". The user U, with the assistance of the assistant V, uses the imaging device 160 to capture and store images (hereinafter referred to as "affected area images") of the affected area "W" of the skin disease of the "patient P", such as pressure sores caused by trauma, ulcers, and bedridden conditions, over a plurality of times to confirm the change over time. The user U can easily grasp the progress of the improvement, deterioration, etc. of the affected area W by displaying and arranging the affected area image group (image group of the affected area W) captured and stored in this way for observation. Therefore, the affected area image group is suitable for the purpose of diagnosing the affected area W that changes over several days to several years.

[0015] (Left side of Figure 1: Hardware configuration diagram of the imaging device 160) The imaging device 160 includes a controller 161, an image sensor 162, an image processing circuit 163, a button group 164, a touch panel 165, a display device 166, an external memory I / F 167, and an internal memory 168. The imaging device 160 further includes a lens group 169, a shutter 170, a network I / F 171, and an AF control circuit 172. The imaging device 160 is, for example, a digital still camera or a digital movie camera, but is not limited thereto. The imaging device 160 may be, for example, a mobile phone or a tablet terminal. The imaging device 160 is preferably a portable device that can be carried by the user, but is not limited thereto.

[0016] (Controller 161: Image sensor 162: Image processing circuit 163) The controller 161 summarizes the information processing within the imaging device 160 and controls the units within the imaging device 160. The image sensor 162 is a charge accumulation type solid-state image sensor such as a CCD or CMOS sensor that converts the formed optical image into image data. The image sensor 162 forms the optical image of the subject captured through the lens group 169, converts the formed optical image into an electrical signal, and outputs the image data. The image processing circuit 163 performs predetermined image processing on the image data output from the image sensor 162. Specifically, the image processing circuit 163 performs various image processing such as white balance adjustment, gamma correction, color interpolation, demosaicking, filtering, and development processing. Further, the image processing circuit 163 generates image data by performing compression processing on the image data subjected to image processing in accordance with a standard such as JPEG.

[0017] (Button group 164: Touch panel 165: Display device 166) The button group 164 and the touch panel 165 capable of touch operations receive various operations of the user U. The display device 166 displays the captured image to the user U. Further, when the "sagging determination information" is issued by the image processing device 130, the display device 166 displays the sagging determination information as it is or together with the captured image. Further, the display device 166 may also display "shooting parameters" including information such as the shooting date and time, zoom, autofocus, exposure, ISO sensitivity, and ambient light.

[0018] (External memory I / F 167) The external memory I / F 167 is an interface with a non-volatile storage medium that can be attached from outside the imaging device 160 or a non-volatile storage medium fixed inside the imaging device 160. The external memory that can be attached from outside the imaging device 160 is, for example, an SD card, a CF card, etc. The external memory I / F 167 is an interface that stores the image data processed by the image processing circuit 163, the image data received by the network I / F 171 from the image processing device 130, analysis data, etc. in the storage medium attached to the imaging device 160. Also, when reproducing the image data, the external memory I / F 167 serves as an interface for reading out the image data stored in the storage medium that can be attached outside the imaging device 160. The image data read out via the external memory I / F 167 can be configured to be displayed on the display device 166 or output outside the imaging device 160.

[0019] (Internal memory 168) The internal memory 168 is configured to include a non-volatile memory such as a ROM that records programs, and a system memory such as a RAM that the controller uses as a working memory. The internal memory 168 temporarily stores various setting information such as the focus position information at the time of image capture necessary for the operation of the imaging device 160, and the images processed by the image processing circuit 163. Also, the internal memory 168 may be configured to temporarily store the image data received by the network I / F 171 through communication with the image processing device 130, analysis data such as sag determination information, etc. The internal memory 168 is realized by a memory device such as a flash memory, an SDRAM, etc. that can rewrite and store information non-volatily.

[0020] (Lens group 169: Shutter 170: AF control circuit 172) The lens group 169 has a zoom function, and the angle of view, focus, etc. during imaging are adjusted according to the operation of the user U. Note that the lens group 169 may be a single-focus lens group without a zoom function. The shutter 170 performs exposure and light shielding by its opening and closing operation, and the incident light to the image sensor 162 is controlled by the speed control of the shutter 170. Note that the shutter 170 is not limited to a mechanical shutter, and an electronic shutter or the like may be used. The AF control circuit 172 extracts the high-frequency components of the imaging signal, and drives and controls the focus lens by searching for the position of the focus lens at which the extracted high-frequency components are maximized, thereby automatically adjusting the focus. The focus lens is included in the lens group 169. The focus control method is also referred to as TV-AF or contrast AF and is characterized by obtaining high-precision focusing. Also, the focus control method is not limited to contrast AF, and control methods such as phase difference AF and image plane phase difference AF may be adopted.

[0021] (Network I / F 171) The network I / F 171 is an interface mainly for transmitting and receiving the image generated by the image processing circuit 163 and its additional information to and from an external device (in this embodiment, the image processing device 130). When performing communication using Wi-Fi (registered trademark), the network I / F 171 may be configured as a wireless communication module.

[0022] (Right side of FIG. 1: Hardware configuration diagram of the image processing device 130) Next, the hardware configuration of the image processing device 130 according to the first embodiment of the present invention will be described with reference to FIG. 1. As shown in FIG. 1, the image processing device includes a central processing unit 110, a storage device 120, an input device 135, an output device 140, a network I / F 141, and an auxiliary arithmetic unit 150. Hereinafter, the central processing unit 110 will be referred to as the CPU 110.

[0023] (CPU 110: Arithmetic unit 111: Storage device 120: Main storage device 121: Auxiliary storage device 122) The CPU 110 includes an arithmetic unit 111. The storage device 120 includes a main storage device 121 and an auxiliary storage device 122. The main storage device 121 includes a non-volatile memory such as a ROM that stores programs, and a system memory such as a RAM that the CPU 110 uses as a work memory. The auxiliary storage device 122 is, for example, a magnetic disk device, an SSD (Solid State Drive), or the like. Note that the number of CPUs 110 and storage devices 120 included in the image processing device 130 may be one or more. That is, when one or more processing devices (CPUs) and one or more storage devices are connected and the one or more processing devices execute a program stored in the one or more storage devices, the image processing device 130 may have a configuration having the required functions in the present embodiment. Note that the processing device is not limited to a CPU, and may be another processor such as a DSP or an MPU, an FPGA, an ASIC, or the like.

[0024] (Auxiliary arithmetic unit 150) The auxiliary arithmetic unit 150 is an auxiliary arithmetic IC used under the control of the CPU 110. The auxiliary arithmetic unit 150 is realized by a GPU (Graphic Processing Unit) as an example. Since the GPU includes a plurality of multiply-accumulate units and is suitable for matrix calculations, it is generally used as a processor for performing deep learning processing. Note that the auxiliary arithmetic unit 150 may be realized by an FPGA (Field-Programmable Gate Array), an ASIC, or the like. The arithmetic unit 111 executes the "sagging determination process" described later by executing a program stored in the storage device 120.

[0025] (Input device 135: Output device 140) The input device 135 is an input device such as a mouse or a keyboard. Note that a touch panel display in which the touch panel 165 of the image processing device 130 is integrally configured with the output device 140 may be used as the input device 135. The output device 140 is a display device such as a computer display.

[0026] (Network I / F 141) The network I / F 141 is an interface for transmitting and receiving images that have been subjected to various processes by the image processing circuit 163, their additional information, etc., to and from an external device (in this embodiment, the imaging device 160). When performing communication using Wi-Fi (registered trademark), the network I / F 141 may be configured as a wireless communication module.

[0027] (Operation of the image processing system 100) (FIG. 2: Flowchart showing an operation example of the image processing system 100 according to the first embodiment) FIG. 2 is a flowchart showing an operation example of the image processing system 100 including the imaging device 160 and the image processing device 130. In the process shown in FIG. 2, on the imaging device 160 side, the controller 161 reads and executes a program stored in the internal memory 168. Also, in the image processing device 130, the CPU 110 reads and executes a program stored in the main storage device 121.

[0028] First, in step S201, in response to the power of the imaging device 160 being turned on (power on), the controller 161 makes a connection request to the image processing device 130 via the network I / F 171. Similarly, in step S202, in response to the power of the image processing device 130 being turned on (power on), the CPU 110 makes a connection request to the imaging device 160 via the network I / F 141.

[0029] Next, in step S203, when the image processing device 130 responds to the connection request in step S201, the controller 161 establishes a connection between the imaging device 160 and the image processing device 130 (connection establishment). Similarly, in step S204, when the imaging device 160 responds to the connection request in step S202, the CPU 110 establishes a connection between the imaging device 160 and the image processing device 130 (connection establishment). As a result, the imaging device 160 and the image processing device 130 can transmit and receive the necessary information including the captured image to and from each other.

[0030] Next, in step S205, the controller 161 starts live view processing (Live view start). That is, the image sensor 162 generates image data, and the image processing circuit 163 performs development processing necessary for generating image data for live view display on the generated image data. By repeating these processes, a live view image at a predetermined frame rate is displayed on the display device 166. Next, in step S206, the controller 161 executes AF processing by the AF control circuit 172 (AF processing). The AF processing is a process of automatically adjusting the focus so as to focus on the affected part W.

[0031] Next, in step S207, the controller 161 captures an image of the subject including the affected part W (Image acquisition). Note that, in the live image display state, the processes from S205 to S216 are repeatedly executed until an instruction operation of the release is performed. Next, in step S208, the controller 161 performs development and compression processing on the image data acquired in step 207 by the image processing circuit 163 to generate image data in, for example, the JPEG standard. The image processing circuit 163 executes resizing processing on the compressed image data to reduce the size of the image data (Compression / Resizing processing). Next, in step S209, the controller 161 displays the image generated in step S208 on the display device 166 (Display).

[0032] Next, in step S210, the controller 161 transmits the image data acquired by live view via the network I / F 171 to the image processing apparatus 130 (Transmission). Then, in step S211, the CPU 110 of the image processing apparatus 130 receives (acquires) the image data transmitted from the imaging apparatus 160 via the network I / F 141 (Reception).

[0033] Next, in step S212, the CPU 110 executes "sagging determination processing" on the live view image acquired in step S212 and issues "sagging determination information" as information on the presence or absence of "sagging" (sagging determination processing). Note that the "sagging determination processing" will be described later with reference to FIG. 4. Next, in step S213, the CPU 110 transmits the "sagging determination information" to the imaging device 160 via the network I / F 141 (transmission).

[0034] Next, in step S214, the controller 161 receives (acquires) the "sagging determination information" from the image processing device 130 via the network I / F 171 (reception). Next, in step S215, the CPU 110 of the image processing device 130 superimposes and displays the "sagging determination information" on the live view image displayed on the display device 166 (sagging determination result display). The user U can proceed to this imaging process after confirming whether the image of the affected part W and the "sagging determination result" are appropriate.

[0035] (FIG. 3: Explanatory diagram of an example of a sagging determination result display displayed on the display device of the imaging device) FIG. 3 is an explanatory diagram of an example of a "sagging determination result" displayed on the display device 166 of the imaging device 160. As shown in FIG. 3, an image of the affected part W is displayed in live view on the display device 166 of the imaging device 160. Sagging determination information 301 is superimposed on the live view display image by alpha blending. In this way, the sagging determination information 301 is displayed in a state that is easy to visually recognize on the display device 166.

[0036] Next, in step S216, the controller 161 determines whether the release button (164) has been pressed. If the controller 161 determines that it has been pressed (yes), the process proceeds to S217. On the other hand, if it determines that it has not been pressed (no), the process returns to S205. In this S216, the user U checks the "Slack determination information 301: Slack detection (see FIG. 3)" displayed on the display device 166 of the imaging device 160, and determines whether to press the release button (shutter button). The user checks the "Slack determination information", and by pressing the release button (164) at the timing when the "Slack determination information" is "No slack detected (see FIG. 3)", an image of the state in which the slack of the affected part W has been eliminated can be acquired. On the other hand, if the user does not press the release button (164), the process shifts to step S205 (live view start) and the live view process continues.

[0037] Next, in step S217, the same process as in step S206 is executed (AF process). Next, in step S218, the controller 161 of the imaging device 160 captures an image so that the affected part W of the subject is included (imaging). Then, in step S219, the last live view image before release is input, and the "Slack determination information" displayed in step S215 is used as metadata and written to the Exif of the captured image generated in step 218 (metadata writing).

[0038] Note that the slack determination information may be written to an individual metadata file such as an "xml file" instead of Exif. In this embodiment, the configuration is such that the "Slack determination information" is written as metadata to the live view image, but it may also be as follows. That is, the image captured in step S218 may be transmitted to the image processing device 130, the image processing device 130 may be made to perform the "Slack determination information", and the result may be written as metadata. The above-described processing is the operation of the image processing system 100 executed in the imaging device 160 and the image processing device 130.

[0039] (FIG. 4: Flowchart showing the "sagging determination process" in step S212) Next, the details of step S212 (sagging determination process) executed by the image processing apparatus 130 will be described. FIG. 4 is a flowchart for explaining the "sagging determination process" executed in the image processing apparatus 130.

[0040] The image processing apparatus 130 includes a first detection unit that detects a "lesion area" based on a captured image of a subject including a skin area and a lesion area, and a second detection unit that detects a "perilesional area", which is a peripheral area of the lesion, based on the lesion area. Further, the image processing apparatus 130 further includes an edge detection unit that detects the edges of the perilesional area, a quantification unit that quantifies the amount of the detected edges, and a determination unit that determines whether there is sagging in the lesion area based on the quantified amount of the edges. The image processing apparatus 130 executes the "sagging determination process" with these configurations. The following explanation will be given.

[0041] First, in step S401, the CPU 110 reads the input image file received by the image processing apparatus 130 from the imaging apparatus 160 into the main storage device 121 (reading of the image file).

[0042] (FIG. 5: Explanatory diagram of the input image in which the subject is imaged) FIGS. 5(a) and 5(b) are explanatory diagrams of the input image in which the subject is imaged. As shown in FIG. 5(a), the image 510 is an image in a state where the lesion W is not pulled, and wrinkles 512 are formed on the skin around the lesion of the lesion area 513 in a direction substantially perpendicular to the gravity direction 511. 514 is a wrinkle within the lesion area. On the other hand, the image 520 shown in FIG. 5(b) is an image in a state where the lesion W is pulled by the hand 521, and the wrinkles 512 around the lesion are eliminated due to the skin being pulled. Note that what pulls the lesion W is not limited to the hand 521, and taping or the like may be pasted around the lesion W and pulled.

[0043] The "flaccidity determination process" in this embodiment is a process for determining the presence or absence of flaccidity based on the amount of edges obtained by detecting and quantifying the wrinkles 512 around the affected area. In a state where flaccidity has occurred, in the affected area, grooves 514 due to flaccidity are generated, and there are injuries, depressions, etc. due to bedsores, etc. in the affected area W. Therefore, it is difficult to judge the "flaccidity" of the affected area based on the presence or absence of edges in the affected area. However, it is possible to judge the "presence or absence of flaccidity" of the affected area by detecting the wrinkles 512 around the affected area as edges.

[0044] Next, in step S402, the CPU 110 detects the affected area 513 from the input image. The process of step S402 corresponds to the "first detection means".

[0045] (Fig. 6: Explanation diagram of the affected area of the input image) Fig. 6 is an explanatory diagram of the affected area. The affected area 513 is extracted using a classifier obtained using machine learning such as deep learning. Note that 610 etc. in Fig. 6 will be described below. As an example of detecting the affected area using deep learning, the following method of classifying an image at the pixel level may be used. That is, "Semantic Segmentation", "Instance Segmentation", etc. may be used. Also, the affected area may be detected by image processing that determines color and texture based on rules.

[0046] Next, in step S403, the CPU 110 detects a "peripheral region of the affected part", which is the peripheral region of the affected part W, based on the affected part region of the input image. Step S403 corresponds to the "second detection means". The CPU 110 detects the region around the affected part region 513 as the peripheral region of the affected part 610. That is, the peripheral region of the affected part 610 is detected by the CPU 110 detecting the affected part region 513 and based on the detection result of the affected part region 513. The point obtained by extending the distance from the centroid 611 (see FIG. 6) of the affected part region 513 to the boundary of the affected part region 513 by "20% (peripheral region ratio)" is set as the outer boundary 613 (see FIG. 6) of the peripheral region of the affected part 610. By extending the distance from the centroid 611 in all directions of 360 degrees to the boundary of the affected part region 513 by "20 (%)", a closed region surrounded by the outer boundary 613 is formed (the gray part in FIG. 6).

[0047] Then, the closed boundary connecting the outer boundary 613 of the peripheral region of the affected part 610 and the belt-shaped region surrounded by the boundary of the affected part region 513 (the gray part in FIG. 6) is detected as the peripheral region of the affected part 610. The endpoints of the line segment 612 obtained by extending the distance from the centroid 611 of the affected part region 513 to the boundary of the affected part region 513 by 20% (peripheral region ratio) are the points on the outer boundary 613 of the peripheral region of the affected part. Note that the peripheral region ratio does not necessarily have to be "20%", and it is desirable to set the peripheral region ratio to a ratio that can enclose the wrinkles 512 around the affected part due to skin sagging in a sufficient region. Also, the detection of the peripheral region of the affected part 610 only needs to be a substantially belt-shaped region surrounding the periphery of the affected part region 513, and the detection method is not limited to this embodiment.

[0048] Next, in step S404, the CPU 110 detects the edge of the peripheral region of the affected part 610 (edge detection). Step S404 corresponds to the "edge detection means". The edge detection executes the process aiming to detect the edge corresponding to the wrinkle 512 of the peripheral region of the affected part 610. The edge detection executes an edge detection application process and an edge filtering process for filtering and removing edges other than the wrinkle 512 from the detected edges. The edge filtering process corresponds to the "removing means" for removing edges.

[0049] First, as an edge detection application process, an edge detection algorithm for detecting edge components in the gravitational direction and the perpendicular direction is applied. Wrinkles 512 caused by sagging around the affected area W have the property of occurring in a direction perpendicular to the gravitational direction. Therefore, by applying an edge detection filter in the direction opposite to the gravitational direction, wrinkles 512 caused by sagging can be selectively detected as edges. In the present embodiment, a horizontal Sobel filter is applied. Then, a threshold value is set such that wrinkles 512 can be detected almost without omission, and the Sobel filter is applied so that the CPU 110 acquires a binarized image.

[0050] (FIG. 7: Explanation diagram of the binarized image of the affected area of the input image) FIG. 7 is an explanatory diagram of the binarized image of the affected area for explaining the edge detection in step 404. In the binarized image of the area 610 around the affected part after the edge detection application process, as shown in FIG. 7, there are round edges 711, short edges 712, etc. as edges other than wrinkles 512 caused by skin spots and the like. In order to filter and exclude these edges other than wrinkles 512, as an edge filtering process, edges with an area of a predetermined value or less and a circularity of a predetermined value or more are removed. Here, the circularity metric is equal to "1" in the case of an ideal circle and less than "1" in the case of other shapes. As a result of this edge filtering process, round edges 711, short edges 712, etc. are removed and only wrinkles 512 are generally detected.

[0051] Note that the edge detection application process is not limited to using the Sobel filter. The Sobel filter is a first-order differential filter and has the characteristic of emphasizing thin edges. Since wrinkles may appear as thin edges on the image, a filter that emphasizes such thin edges is suitable as the edge detection application process. However, as long as it is an edge detection application process in which wrinkles are appropriately detected, it is not limited to the Sobel filter. As the edge detection filter, a Prewitt filter, Canny edge detection, etc. may be used.

[0052] Next, in step S405, the CPU 110 quantifies the edge detected in step 404 (edge quantification). Step S405 corresponds to the "quantification means". In the present embodiment, the edge amount is the area size of the edge of the binarized image or the number of pixels. Note that the edge amount may be weighted by the degree of edge shading after the edge detection application process and before the binarization process application. Furthermore, the edge amount may be the number of edges, the length, or a value calculated based on these.

[0053] Then, in step S406, based on the edge amount obtained in step S405, "sagging determination processing" is executed to issue "sagging determination information" (sagging determination processing). Step S406 corresponds to the "determination means". In the present embodiment, the CPU 110 compares a preset predetermined threshold with the edge amount, and when the edge amount exceeds the predetermined threshold, it determines that "sagging is detected", and when it is less than or equal to the predetermined threshold, it determines that "sagging is not detected". Through the above "sagging determination processing", "sagging determination information" can be obtained.

[0054] <Effect of the First Embodiment> As described above, according to the image processing system 100 according to the first embodiment, it is possible to execute "sagging determination" between the skin and the affected part W from the images of the skin and the affected part W. Furthermore, by superimposing and displaying the sagging detection result on the live view of the display device 166 of the imaging device 160, the user can confirm that the sagging has been eliminated and obtain an image in which the "sagging" has been eliminated. Also, when browsing the captured image with a dedicated viewer, in step S219 (metadata writing), by displaying the "sagging determination information" written in Exif, the presence or absence of sagging determination of the captured image is displayed. As a result, the user can use it for assisting in the diagnosis of the affected part W.

[0055] Note that the various controls described above as being performed by the controller 161 of the imaging device 160 or the CPU 110 of the image processing device 130 may be performed by a single piece of hardware (such as a CPU). On the other hand, the various controls described above may be shared by a plurality of pieces of hardware to control the imaging device 160, the image processing device 130, and the image processing system 100. Further, although preferred embodiments of the present invention have been described, the present invention is not limited to the above-described embodiments, and forms in which various modifications are made without departing from the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely shows an embodiment of the present invention, and it is also possible to appropriately combine the embodiments.

[0056] Also, in the above-described embodiments, an example in which the present invention is applied to the imaging device 160 and the image processing device 130 has been described, but the present invention is not limited to this example. Basically, the present invention is applicable to any image processing device that can acquire and display a captured image. Further, the imaging device 160 and the image processing device 130 may operate on the same hardware. For example, the present invention is applicable to electronic devices such as a PC, a PDA, a mobile terminal, a tablet terminal, a smartphone, a projection device, and a portable image viewer.

[0057] <<Second Embodiment>> Next, a second embodiment of the present invention will be described. The second embodiment is different from the first embodiment in the "slack determination process". In the first embodiment, the "slack determination" in the image processing device 130 uses a still image (one-frame image) as an input and targets the "area around the affected part" of the affected part for the edge detection process of "slack determination". In contrast, in the second embodiment, the input image for the "slack determination process" is a live view image of a plurality of frames. Further, in addition to the "area around the affected part", the "affected part area" is also targeted for the edge detection process of "slack determination" performed by the image processing device 130. Then, the controller 161 displays the "slack determination information" obtained by performing the "slack determination" on the affected part W based on the variation in the edge amount for each frame of the affected part area and the area around the affected part on the display device 166 of the imaging device 160.

[0058] Specifically, in order for the imaging assistant V to eliminate the sagging of the affected part W, the skin near the affected part is gradually pulled, and the state is imaged in a live view or a video. The image processing device 130 detects the wrinkles in the "peripheral area of the affected part" and the grooves in the "affected part area" for each frame, and determines whether the "sagging" has been eliminated based on these fluctuations.

[0059] (Fig. 8: A flowchart showing an operation example of the image processing system according to the second embodiment) Fig. 8 is a flowchart for explaining the operation of the image processing system 100 according to the second embodiment. The operation of the image processing system 100 according to the second embodiment is characterized in that the following processes are added to the processes in the first embodiment. That is, step (sagging determination flag reception process) S801, step S802 (Is the sagging determination process flag True?), step S803 (Is the sagging determination result sagging?), and step S804 (Set the sagging determination flag to False) are added. Hereinafter, these will be described in detail. Note that since there are no changes in other processes, duplicate explanations will be omitted.

[0060] In step S801, the controller 161 receives a "sagging determination process flag" that determines whether to execute the "sagging determination process" (sagging determination flag reception process). The "sagging determination process flag" can be received by operating the touch panel 165 or the button group 164 of the imaging device 160. The "sagging determination process flag" is set to "False" in the initial state, and when the controller 161 receives an instruction input for the "sagging determination process" by a user operation, the "sagging determination process flag" is set to "True". The user U captures the affected part W in a live view, and the CPU 110 sets the "sagging determination flag" by an operation in accordance with the timing when the pulling assistant V of the imaging assistant starts pulling the skin.

[0061] Next, in step S802, the controller 161 branches the process according to the value of the "sag determination process flag" (Is the sag determination process flag True?). When the CPU 110 determines that the "sag determination process flag" is not "True", that is, "False", the process proceeds to step S216 and skips the "sag determination process". On the other hand, when the controller 161 determines that the "sag determination process flag" is "True" (yes), the process proceeds to step S210, and a live view image is transmitted to the image processing device 130 for the "sag determination process" (transmission).

[0062] Also, in step S803, the controller 161 branches the process according to the value of the "sag determination result" received from the image processing device 130 (conditional branch: Is there sag in the sag determination result?). When the controller 161 determines that the "sag determination result" is "there is sag", the process proceeds while the "sag determination flag" remains "True". That is, in step S216, when the live view is continued in the "no" case, the frames following the live view image are transmitted to the image processing device 130 for the "sag determination process".

[0063] On the other hand, when the controller 161 determines that the "sag determination result" is "no sag", in step S804, the CPU 110 sets the "sag determination process flag" to "False" (Set the sag determination process flag to "False"). That is, in step S216, when the live view is continued in the "no" case, the frames following the live view image are not transmitted to the image processing device 130 for the "sag determination process", and a series of "sag determination processes" ends. The value of the "sag determination process flag" is displayed on the display device 166 of the imaging device 160. The user can visually distinguish whether the displayed "sag determination result" is real-time or from the previous determination.

[0064] In addition, the imaging device 160 may be configured to automatically capture an image at the timing when the sag determination detects "no sag". In this case, in FIG. 8, after step S804 (set the sag determination processing flag to "False"), the controller 161 proceeds to step S217 (AF processing), and imaging is executed in step S218. By performing such an operation, imaging can be performed at the timing immediately after the sag is eliminated, and the time for pulling the affected part W is shortened, so the burden on the patient can also be reduced.

[0065] (Processing of FIG. 4 in the second embodiment) Next, the "sag determination process" of the second embodiment will be described with reference to FIG. 4. Since FIG. 4 is used for the flow of the sag determination process of the first embodiment, only the parts of the processing content specific to the second embodiment will be described. The "sag determination process" of the second embodiment applies the following processing to each of a plurality of frames of images transmitted from the imaging device 160. Step S401 (loading of image file), step S402 (detection of affected part area), and step S403 (detection of area around affected part) are the same processes as those in the first embodiment.

[0066] (Differences between the first and second embodiments in the processing of FIG. 4) In step S404 (edge detection), edges within the affected part area detected in step S402 (detection of affected part area) and edges within the area around the affected part detected in step S403 (detection of area around affected part) are detected. The detection of the edges of the area around the affected part in step S403 is the same as the method described in step S404 (edge detection) of the first embodiment.

[0067] (FIG. 9: Explanation diagram of the affected part area of the input image) With reference to FIG. 9, the detection of the "affected area" will be described. FIG. 9 is an explanatory diagram showing the affected area. In this step S402, the affected area 413 is detected. The method for detecting the affected area 413 is the same as the method described in step S403 (detection of the area around the affected part). The edge detection of the affected area 413 aims to detect the edge corresponding to the wrinkle 1003 generated by the sag in the affected part W in FIG. 9. The edge detection corresponding to the wrinkle 1003, similar to step S404 (edge detection), performs an edge detection application process and an edge filtering process to filter out edges other than wrinkles from the detected edges.

[0068] The edge detection application process uses the processing method of Canny edge detection. The Canny edge is characterized in that it can detect a "one-pixel-wide" clean and relatively long edge when the edge is thick enough. Therefore, it is suitable for detecting the wrinkle 1003 generated by the "sag" in the affected part W compared with the Sobel filter suitable for detecting thin edges in detail. Two threshold values of the Canny edge are set to suitable values so that the wrinkle 1003 generated by the "sag" in the affected part W can be detected.

[0069] The edge filtering process may use the same method as in step S404 (edge detection). By step S404, the edges of the area around the affected part and the edges of the affected area are detected. Note that, as in this embodiment, the edge detection algorithms for the area around the affected part and the affected area may be different depending on the characteristics of the object to be detected.

[0070] In step S405 (edge quantification), the CPU 110 quantifies the edges of the area around the affected part and the edges of the affected area. The method of edge quantification is the same as the method described in step S405 (edge quantification) of FIG. 4. In step S406 (wrinkle determination process), the CPU 110 performs a "wrinkle determination" based on the variation of the edge amounts of multiple frames of the area around the affected part and the affected area obtained in step S405 (edge quantification) and issues "wrinkle determination information".

[0071] (Fig. 10: Diagram for explaining frame change of edge amount in the second embodiment) Fig. 10 is an explanatory diagram showing the frame change of the edge amount in the second embodiment. Referring to Fig. 10, the method of "flaccidity determination processing" in the second embodiment will be described. In Fig. 10, the horizontal axis is the frame No (No: number), and the vertical axis is the edge amount. The start of pulling of the affected part W is frame No "0". As the pulling of the affected part W becomes stronger, wrinkles and grooves in the area around the affected part and in the affected part area spread and disappear. Accordingly, it is shown that the edge amount 1101 of the area around the affected part and the edge amount 1002 of the affected part area are decreasing. Also, the state of the affected part W at that time is shown in Figs. 5(a) and 5(b). As shown in Image 510, when the affected part W is not being pulled, wrinkles 512 appear on the skin around the affected part W and grooves 514 appear in the affected part area 513.

[0072] On the other hand, as shown in Fig. 5(b), in Image 520 where the affected part W is being pulled by the hand 521, it is shown that the wrinkles 512 around the affected part and the grooves 514 in the affected part area are eliminated due to the pulling of the skin. In the affected part area 513, there are various textures such as wounds and depressions due to diseases in addition to the grooves 514 caused by "flaccidity", and these are edge-detected. However, since the grooves 514 caused by "flaccidity" decrease due to the pulling of the affected part W, it becomes possible to determine whether or not the grooves 514 in the affected part W have disappeared at the timing when the edge amount of the affected part area 513 becomes constant (bottomed out).

[0073] Then, the CPU 110 calculates the differential values of the edge amount 1001 of the area around the affected part and the edge amount 1002 of the affected part area, and distributes them according to a preset threshold value to determine whether there is a change in the edge amount or whether it is a substantially constant amount. The threshold value "n1" in Fig. 10 indicates the frame no at which the edge amount becomes a substantially constant value. The CPU 110 executes "flaccidity determination processing" based on this combination of changes in the edge amount.

[0074] (Fig. 11: Explanatory diagram of "flaccidity determination processing" in the second embodiment) FIG. 11 is an explanatory diagram of the "sagging determination process" in the second embodiment. With reference to FIG. 11, the "sagging determination process" in the second embodiment will be described. As shown in FIG. 11, there are four combinations of the variation / constancy of the edge amount 1101 in the area around the affected part and the edge amount 1102 in the affected part area, namely A, B, C, and D.

[0075] The combination of "A" is the case where the edge amounts of both are changing. Since the edge amounts are changing in both the affected part area and the area around the affected part, it is considered that both areas are in a sagged state. The combination of "B" is the case where the edge amount in the area around the affected part is changing while the edge amount in the affected part area is substantially constant. The edge amount in the area around the affected part is changing and it is sagged. On the other hand, in the affected part area, although the edge amount is substantially constant and the wrinkles are eliminated, it is considered to be in a shrunk state, that is, the affected part W is not sufficiently expanded and the area cannot be accurately measured.

[0076] The combination of "C" is the case where the edge amount in the area around the affected part is substantially constant and the edge amount in the affected part area is changing. Although the wrinkles in the area around the affected part are eliminated, it is considered that the wrinkles due to the "sagging" in the affected part area are not eliminated. The combination of "D" is the case where the edge amounts of both are substantially constant. This is considered to be a state where the wrinkles in the area around the affected part and the wrinkles in the affected part area are eliminated. In this way, "D" is the state where the "sagging" in the affected part area is eliminated, and the timing when the state of "D" is first obtained during the sequential acquisition of frames is considered to be the state where the skin is not stretched too much and the "sagging" is just eliminated.

[0077] Therefore, until the timing when the CPU 110 detects the state of "D" where the edge amount in the area around the affected part and the edge amount in the affected part area are substantially constant, the CPU 110 outputs the determination result of "sagging" as the "sagging determination information". Then, the CPU 110 determines "no sagging" at the timing when the state of "D" is detected and outputs the "sagging determination information".

[0078] <Effect of the Second Embodiment> As described above, according to the image processing system 100 according to the second embodiment, when starting to pull the skin and the imaging device 160 determines and outputs "no sag" as "sag determination information", imaging is performed to stop pulling the skin. Thereby, it is possible to obtain an image in which the "sag" of the skin is eliminated while preventing excessive pulling, and the burden on the patient can be reduced.

[0079] <<Third Embodiment>> Next, a third embodiment of the present invention will be described. In the first embodiment, the image processing system 100 has a configuration including the image processing device 130 and the imaging device 160. In contrast, the image processing system according to the third embodiment is characterized in that it has a configuration including the image processing device 130, the medical image management device 1260, and the client device 1230. In the third embodiment, the image processing device 130 executes "sag determination processing" on existing images and stores the results of the "sag determination processing" in the image DB 1266 of the medical image management device 1260. Further, an image viewer operated by the medical image management device 1260 is displayed on the display unit 1262 of the client device 1230, and "sag determination information" is displayed as additional information in the image viewer of the client device 1230.

[0080] (FIG. 12: Configuration diagram of the image processing system 1200 according to the third embodiment) Hereinafter, with reference to FIG. 12, the configuration of the image processing system 1200 according to the third embodiment of the present invention will be described. As shown in FIG. 12, the image processing system 1200 according to the third embodiment includes an image processing apparatus 130, a client apparatus 1230, and a medical image management apparatus 1260. The image processing apparatus 130, the client apparatus 1230, and the medical image management apparatus 1260 are connected to each other via a network 1280 so that required information can be transmitted and received mutually. The images managed by the medical image management apparatus 1260 are transmitted to the image processing apparatus 130 via the network 1280, and "sagging determination processing" is executed to issue "sagging determination information". Further, the medical images managed by the medical image management apparatus 1260 and the "sagging determination information" issued by the image processing apparatus 130 are transmitted to and displayed on the client apparatus 1230. Note that the network 1280 may be either a wireless network or a wired network. Also, since the image processing apparatus 130 has the same configuration as that in the first embodiment, duplicate description will be omitted.

[0081] (Medical image management apparatus 1260) The medical image management apparatus 1260 includes an AP server 1264, a WEB server 1265, and an image DB 1266. The medical image management apparatus 1260 is a WEB system responsible for managing imaging images and executing a program of a viewer for medical images. The AP server 1264 operates an application program of a medical image viewer for displaying images stored in the image DB 1266. The WEB server 1265 responds to requests from a WEB browser executed on the client apparatus 1230 and sends data such as static screens and images, which are the results processed by the AP server 1264, to the WEB browser. The image DB 1266 stores and manages medical images or imaging images including the affected part W and their additional information. The additional information is, for example, sagging determination information, patient ID, patient name, and the like.

[0082] (Client apparatus 1230) The client device 1230 is a device used by users U such as medical staff. As shown in FIG. 12, the client device 1230 includes an input unit 1261, a display unit 1262, and a display control unit 1263. The client device 1230 executes a WEB browser. The input unit 1261 receives instructions such as medical image display from medical staff and the like. The display unit 1262 has a function of displaying required information such as photographed patient information, photographed image data, and affected part information, and these information can be browsed by medical staff.

[0083] The display control unit 1263 performs display control such as causing the display unit 1262 to display the patient name list, the display order of the personal image data of the patient, etc. in an appropriate order. Note that the display control unit 1263 performs display control for causing the display unit 1262 to display the required display information. The display control unit 1263 also has a UI execution function of causing the display unit 1262 to display the screen used by the user, such as an editing screen, a search screen, etc., not only the display order.

[0084] (FIG. 13: Flowchart showing an operation example of the image processing system 1200 according to the third embodiment) FIG. 13 is a flowchart showing an operation example of the image processing system 1200 executed in the client device 1230, the medical image management device 1260, and the image processing device 130. The process shown in FIG. 13 is executed by the CPU 110 reading out and executing the program stored in the main storage device 121 on the image processing device 130 side.

[0085] On the client device 1230 and medical image management device 1260 sides, the process shown in FIG. 13 is realized by the computers (CPUs) constituting each reading out and executing the necessary data and programs from a storage device such as an HDD.

[0086] (Operation) First, in step S1301, the CPU 110 of the image processing apparatus 130 requests an image file from the medical image management apparatus 1260 to acquire one or more images (loading of the image file). This image is, for example, an image of a subject including a skin area and an affected area. Next, in step S1302, the CPU 110 of the image processing apparatus 130 executes a "flaccidity determination process" and issues "flaccidity determination information" for each image (flaccidity determination process). Note that the method of the flaccidity determination process is the same as the method of the first embodiment.

[0087] Next, in step S1303, the CPU 110 of the image processing apparatus 130 saves the result of the "flaccidity determination process" (saving of the flaccidity determination result). Specifically, the "flaccidity determination information" issued in step 1302 is used as metadata and written into the Exif of the captured image. Note that the "flaccidity determination information" may be written into an individual metadata file such as an xml file instead of Exif. Further, the CPU 110 saves the "flaccidity determination information" as additional information of the image in the image DB 126 of the medical image management apparatus 1260.

[0088] Then, in step 1304, the user operates the input unit 1261 of the client apparatus 1230 to specify a desired image by the patient name, date and time, etc. The CPU of the client apparatus 1230 that has received the specification requests an image from the image processing apparatus 130. The display control unit 1263 displays the image obtained from the image processing apparatus 130 on the display unit 1262 as an image. As a result, the result of the "flaccidity determination process" is displayed on the image viewer.

[0089] Then, in step 1304, the user operates the input unit 1261 of the client apparatus 1230 to specify a desired image by the patient name, date and time, etc. The CPU of the client apparatus 1230 that has received the specification requests the specified image from the image processing apparatus 130. The specified image is, for example, one with additional information such as flaccidity determination information, patient name, patient ID, etc. The display control unit 1263 displays the specified image obtained from the image processing apparatus 130 on the display unit 1262 as an image. As a result, the result of the "flaccidity determination process" is displayed on the image viewer.

[0090] (FIG. 14: Explanatory diagram of a screen display example of the client device according to the third embodiment) FIG. 14 is an explanatory diagram of a screen display example in an image viewer displayed on the display unit 1262 of the client device 1230. At the upper part of the image viewer 1400 displayed on the display unit 1262 of the client device 1230, a patient name 1402 associated with the patient ID 1401 is displayed. Further, the captured images 1403 of the patient, the pressure ulcer evaluation results 1404 associated with the images, and the sagging detection information 1405 are displayed in a list in chronological order.

[0091] (Effect of the Third Embodiment) As described above, according to the third embodiment, required display information can be displayed as additional information in the image viewer 1400 on the display unit 1262 of the client device 1230. As a result, the user U can observe the image displayed on the display unit 1262 while taking into account whether the image has skin sagging. As a result, the doctor can make a more accurate diagnosis.

[0092] <Addendum> The disclosure of this embodiment includes the following configurations (including systems), methods, and programs. (Configuration 1) An image processing apparatus capable of capturing a captured image of a subject including a skin region and an affected part region, a first detection means for detecting the affected part region based on the captured image, a second detection means for detecting a peripheral region of the affected part, which is a peripheral region of the affected part, based on the affected part region detected by the first detection means, an edge detection means for detecting an edge of the peripheral region of the affected part detected by the second detection means, a quantification means for quantifying the edge detected by the edge detection means, and a determination means for determining whether there is sagging in the affected part based on the edge amount quantified by the quantification means. The image processing apparatus is characterized by comprising the above. (Configuration 2) The edge detection means is An image processing apparatus of Configuration 1, characterized by detecting an edge component in a direction perpendicular to the direction of gravity. (Configuration 3) The determination means When the amount of edges quantified by the quantification means exceeds a predetermined threshold, it determines that sagging is detected, while when it is below the predetermined threshold, it makes a sagging determination that sagging is not detected. The image processing apparatus according to Configuration 1 or 2, characterized by this. (Configuration 4) The image processing apparatus according to Configuration 1 or 2, further comprising removal means for removing edges unrelated to wrinkles in the area around the affected part detected by the second detection means. (Configuration 5) The removal means The image processing apparatus of Configuration 4, characterized by removing edges whose area detected by the edge detection means is equal to or less than a predetermined value. (Configuration 6) The removal means The image processing apparatus of Configuration 4, characterized by removing edges whose circularity detected by the edge detection means is equal to or more than a predetermined value. (Configuration 7) An image processing system in which an imaging device and an image processing apparatus are connected so as to be able to communicate required information with each other The imaging device An acquisition means for acquiring a captured image of a subject including a skin area and an affected part area, A transmission means for transmitting the captured image acquired by the acquisition means to the image processing apparatus, A reception means for receiving a sagging determination result transmitted by the following second transmission means, A display means for displaying, on a display device, the sagging determination processing result received by the reception means. The imaging device is provided with The image processing apparatus A second reception means for receiving the captured image transmitted by the transmission means, A first detection means for detecting the affected part area based on the captured image received by the second reception means, A second detection means for detecting an area around the affected part, which is an area around the affected part, based on the affected part area detected by the first detection means, Edge detection means for detecting the edge of the area around the affected part detected by the second detection means; Quantification means for quantifying the edge detected by the edge detection means; Judgment means for judging whether there is sagging in the affected part based on the edge amount quantified by the quantification means; An image processing system, comprising second transmission means for transmitting the sagging judgment result by the judgment means to the imaging device. (Configuration 8) The image processing system according to Configuration 7, wherein the captured image is a live view image of a plurality of frames. (Configuration 9) The judgment means further judges whether there is sagging in the affected part based on the variation of the edge amount for each frame between the affected part area and the area around the affected part. The image processing system according to Configuration 8. (Configuration 10) The judgment means further judges that there is no sagging in the affected part when the variation of the edge amount for each frame between the affected part area and the area around the affected part becomes substantially constant. The image processing system according to Configuration 9. (Configuration 11) The acquisition means further acquires a captured image of a new subject including a new skin area and an affected part area at the timing when the judgment means determines that there is no sagging in the affected part. The image processing system according to Configuration 9 or 10. (Configuration 12) An image processing system in which an image processing device, a medical image management device, and a client device are communicably connected to each other to communicate required information, The image processing device acquisition means for requesting and acquiring an image of a subject including a skin area and an affected part area from the medical image management device; first detection means for detecting an affected part area based on the image acquired by the acquisition means; second detection means for detecting an area around the affected part, which is an area around the affected part, based on the affected part area detected by the first detection means; edge detection means for detecting the edge of the area around the affected part detected by the second detection means; Quantification means for quantifying the edges detected by the edge detection means, Determination means for determining whether there is sagging in the affected area based on the edge amount quantified by the quantification means, Storage means for storing the image with the sagging determination result determined by the determination means as additional information in the following image DB, The medical image management device includes an image DB for storing images, The client device, Second acquisition means for requesting and acquiring a specified image from the image processing device, Display means for displaying the specified image acquired by the second acquisition means on an image viewer, characterized in that the image processing system comprises the same. (Configuration 13) The second acquisition means, Requests and acquires a specified image with the sagging determination result, patient name, and patient ID as additional information from the image processing device, and the image processing system according to Configuration 12. (Method 1) A control method for an image processing device capable of capturing an image of a subject including a skin area and an affected area, A first detection step of detecting the affected area based on the captured image, A second detection step of detecting a peripheral area of the affected area, which is a peripheral area of the affected area, based on the affected area detected in the first detection step, An edge detection step of detecting an edge of the peripheral area of the affected area detected in the second detection step, A quantification step of quantifying the edge detected in the edge detection step, A determination step of determining whether there is sagging in the affected area based on the edge amount quantified in the quantification step, characterized in that the control method of the image processing device comprises the same. (Program 1) A program for causing a computer to execute a control method for an image processing device capable of capturing an image of a subject including a skin area and an affected area, The control method, A first detection step of detecting the affected area based on the captured image, A second detection step of detecting a peripheral region of the affected part, which is a peripheral region of the affected part, based on the affected part region detected in the first detection step; An edge detection step of detecting an edge of the peripheral region of the affected part detected in the second detection step; A quantification step of quantifying the edge detected in the edge detection step; A determination step of determining whether there is sagging in the affected part based on the edge amount quantified in the quantification step, characterized by a program.

[0093] As described above, the preferred embodiments of the present invention have been described. However, the present invention is not limited to the above-described embodiments, and various modifications and changes are possible within the scope of the gist. For example, the present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a recording medium, and having the processor of the computer of the system or device read and execute the program. Further, the present invention can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Explanation of symbols

[0094] 100 Image processing system 110 CPU 120 Storage device 121 Main storage device 122 Auxiliary storage device 130 Image processing device 135 Input device 140 Output device 141 Network I / F 150 Auxiliary arithmetic unit 160 Imaging device 161 Controller 162 Image sensor 163 Image processing circuit 164 Button group 165 Touch panel 166 Display device 167 External memory I / F 168 Internal memory 170 Shutter 171 Network I / F 172 AF Control Circuit 1200 Image Processing System 1230 Client Device 1260 Medical Image Management Device 1261 Input Unit 1262 Display Unit 1263 Display Control Unit 1264 AP Server 1265 Web Server 1266 Image DB

Claims

1. An image processing apparatus capable of capturing an image of a subject including a skin area and an affected area, a first detection means for detecting the affected area based on the captured image, a second detection means for detecting a peripheral area of the affected area, which is a peripheral area of the affected area, based on the affected area detected by the first detection means, an edge detection means for detecting an edge of the peripheral area of the affected area detected by the second detection means, a quantification means for quantifying the edge detected by the edge detection means, and a determination means for determining whether there is sagging in the affected area based on the edge amount quantified by the quantification means. The image processing apparatus is characterized by comprising the above.

2. The edge detection means is characterized by detecting an edge component in a direction perpendicular to the gravitational direction. The image processing apparatus according to claim 1.

3. The determination means is characterized in that when the edge amount quantified by the quantification means exceeds a predetermined threshold value, it is determined that sagging is detected, while when it is below the predetermined threshold value, it is determined that there is no sagging detected. The image processing apparatus according to claim 1 or 2.

4. The image processing apparatus according to claim 1 or 2, further comprising a removal means for removing edges that have no relation to wrinkles in the peripheral area of the affected area detected by the second detection means.

5. The removal means is characterized by removing an edge whose area detected by the edge detection means is equal to or less than a predetermined value. The image processing apparatus according to claim 4.

6. The removal means is characterized by removing an edge whose circularity detected by the edge detection means is equal to or greater than a predetermined value. The image processing apparatus according to claim 4.

7. An image processing system in which an imaging device and an image processing apparatus are communicably connected to each other with required information, wherein the imaging device comprises an acquisition means for acquiring an image of a subject including a skin area and an affected area, a transmission means for transmitting the captured image acquired by the acquisition means to the image processing apparatus, a reception means for receiving a sagging determination result transmitted by the following second transmission means, and a display means for displaying the sagging determination processing result received by the reception means on a display device, wherein the image processing apparatus comprises a second reception means for receiving the captured image transmitted by the transmission means, and a first detection means for detecting the affected area based on the captured image received by the second reception means. A second detection means for detecting a surrounding area of the affected part, which is a surrounding area of the affected part, based on the affected part area detected by the first detection means; An edge detection means for detecting an edge of the surrounding area of the affected part detected by the second detection means; A quantification means for quantifying the edge detected by the edge detection means; A determination means for determining whether there is sagging in the affected part based on the edge amount quantified by the quantification means; An image processing system comprising a second transmission means for transmitting the sagging determination result by the determination means to the imaging device.

8. The image processing system according to claim 7, wherein the captured image is a live view image of a plurality of frames.

9. The determination means further The image processing system according to claim 8, wherein it is determined whether there is sagging in the affected part based on the variation of the edge amount for each frame between the affected part area and the surrounding area of the affected part.

10. The determination means further The image processing system according to claim 9, wherein when the variation of the edge amount for each frame between the affected part area and the surrounding area of the affected part becomes substantially constant, it is determined that there is no sagging in the affected part.

11. The acquisition means further The image processing system according to claim 9 or 10, wherein an imaging image of a new subject including a new skin area and an affected part area is acquired at the timing when the determination means determines that there is no sagging in the affected part.

12. An image processing system in which an image processing device, a medical image management device, and a client device are communicably connected to each other to communicate required information, The image processing device An acquisition means for requesting and acquiring an image of a subject including a skin area and an affected part area from the medical image management device; A first detection means for detecting an affected part area based on the image acquired by the acquisition means; A second detection means for detecting a surrounding area of the affected part, which is a surrounding area of the affected part, based on the affected part area detected by the first detection means; An edge detection means for detecting an edge of the surrounding area of the affected part detected by the second detection means; A quantification means for quantifying the edge detected by the edge detection means; A determination means for determining whether there is sagging in the affected part based on the edge amount quantified by the quantification means; A storage means for storing the image with the sagging determination result determined by the determination means as additional information in the following image DB; The medical image management device Comprises an image DB for storing images The client device is provided with second acquisition means for requesting and acquiring a specified image from the image processing device, and display means for displaying the specified image acquired by the second acquisition means on an image viewer, and is characterized by an image processing system.

13. The second acquisition means requests and acquires, from the image processing device, a specified image with the sagging determination result, patient name, and patient ID as additional information, and the image processing system according to claim 12 is characterized by this.

14. A control method for an image processing device capable of capturing an imaging image of a subject including a skin region and an affected part region, comprising: a first detection step of detecting the affected part region based on the imaging image; a second detection step of detecting an affected part peripheral region, which is a peripheral region of the affected part, based on the affected part region detected in the first detection step; an edge detection step of detecting an edge of the affected part peripheral region detected in the second detection step; a quantification step of quantifying the edge detected in the edge detection step; and a determination step of determining whether there is sagging in the affected part based on the edge amount quantified in the quantification step, and is characterized by a control method for an image processing device having this.

15. A program for causing a computer to execute a control method for an image processing device capable of capturing an imaging image of a subject including a skin region and an affected part region, wherein the control method comprises: a first detection step of detecting the affected part region based on the imaging image; a second detection step of detecting an affected part peripheral region, which is a peripheral region of the affected part, based on the affected part region detected in the first detection step; an edge detection step of detecting an edge of the affected part peripheral region detected in the second detection step; a quantification step of quantifying the edge detected in the edge detection step; and a determination step of determining whether there is sagging in the affected part based on the edge amount quantified in the quantification step, and is characterized by a program having this.

Images