Image processing device and skin photographic image generation method

The image processing device and method enhance skin diagnosis by generating three-dimensional images of the skin and internal objects using infrared light distance measurement and visible light reflection, improving diagnostic accuracy.

JP2024162663AInactive Publication Date: 2024-11-21JVC KENWOOD CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023078400
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing image processing devices and methods fail to provide accurate diagnosis of the inside of the skin.

Method used

An image processing device and method that utilize infrared light to generate distance measurement data, extract in-body object regions, and combine this data with visible light images to create three-dimensional images of the skin and internal objects, allowing for a composite image generation.

Benefits of technology

Enables more accurate diagnosis of the inside of the skin by providing detailed three-dimensional images of both the skin surface and internal objects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162663000001_ABST
    Figure 2024162663000001_ABST
Patent Text Reader

Abstract

To provide an image processing device capable of contributing to diagnosing the inside of the skin more accurately.SOLUTION: An in-vivo object region extraction unit 23 extracts an in-vivo object region, which is a region where an in-vivo object in the body on an inner surface side of the skin is present, on the basis of range finding data generated on the basis of reflected light of infrared light emitted to a predetermined region of the skin surface. A stereoscopic image generation unit 24 generates a stereoscopic image of the skin and a stereoscopic image of the in-vivo object on the basis of the range finding data and the information indicating the in-vivo object region extracted by the in-vivo object region extraction unit 23. A compositing unit 26 composites a visible light image based on visible light reflected in the predetermined region, the stereoscopic image of the skin, and the stereoscopic image of the in-vivo object.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an image processing device and a method for generating a skin image. [Background technology]

[0002] Patent Document 1 describes a skin surface observation device that observes the skin using visible light and infrared light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-200404 A Summary of the Invention [Problem to be solved by the invention]

[0004] It is desirable to have an image processing device and a method for generating a skin image that contribute to a more accurate diagnosis of the inside of the skin. The present invention aims to provide an image processing device and a method for generating a skin image that can contribute to a more accurate diagnosis of the inside of the skin. [Means for solving the problem]

[0005] The present invention provides an image processing device comprising: an internal object region extraction unit that extracts an internal object region, which is a region where internal internal objects exist on the inner side of the skin, based on ranging data generated based on reflected light of infrared light irradiated to a predetermined region on the surface of the skin; a three-dimensional image generation unit that generates a three-dimensional image of the skin and a three-dimensional image of the internal object based on the ranging data and information indicating the internal object region extracted by the internal object region extraction unit; and a synthesis unit that synthesizes a visible light image based on visible light reflected at the predetermined region, the three-dimensional image of the skin, and the three-dimensional image of the internal object.

[0006] The present invention provides a skin image generation method which includes irradiating infrared light onto a predetermined area on the surface of the skin, generating distance measurement data based on reflected light of the infrared light irradiated onto the predetermined area, extracting an internal object area which is an area on the inner surface of the skin where internal objects are present based on the distance measurement data, generating a three-dimensional image of the skin and a three-dimensional image of the internal object based on the distance measurement data and information indicating the internal object area, and generating a composite image by combining a visible light image based on visible light reflected from the predetermined area, the three-dimensional image of the skin, and the three-dimensional image of the internal object. Effect of the Invention

[0007] The image processing device and the skin image generating method of the present invention can contribute to more accurate diagnosis of the inside of the skin. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a state in which a three-dimensional imaging device and an image processing device according to an embodiment are connected to each other. [Diagram 2] FIG. 2 is a block diagram showing an image processing apparatus according to an embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of a state in which the three-dimensional imaging device is imaging skin. [Figure 4] FIG. 4 is a diagram showing a suitable auxiliary tool for use when the three-dimensional imaging device images the skin. [Diagram 5] FIG. 5 is a conceptual diagram showing how visible light and infrared light irradiated to the skin are reflected when an internal object is present inside the body on the inner surface side of the skin. [Figure 6] FIG. 6 is a diagram showing an example of a region where the three-dimensional imaging device receives visible light and infrared light reflected from the skin. [Figure 7] FIG. 7 is a diagram conceptually showing distance measurement data of an internal object and its surroundings. [Figure 8] FIG. 8 is a flowchart showing the operation of the image processing device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An image processing device and a skin image generating method according to an embodiment will be described below with reference to the accompanying drawings. In Fig. 1, a three-dimensional imaging device 10 and an image processing device 20 are connected by a coaxial cable Cc that connects BNC (Bayonet Neill Concelman) connectors B1 and B2, for example. The coaxial cable Cc may be an SDI cable that complies with the SDI (Serial Digital Interface) standard. The three-dimensional imaging device 10 and the image processing device 20 may be connected by other cables such as an HDMI (High-Definition Multimedia Interface) cable. A display unit 30 is connected to the image processing device 20.

[0010] 1, a three-dimensional imaging device 10 includes a lens group 11, a vertical cavity surface emitting laser 12, a splitter 13, an imaging element 14, a visible light image data generator 15, a Time of Flight (TOF) sensor 16, a distance measurement data generator 17, and a synthesis unit 18. The vertical cavity surface emitting laser 12 is abbreviated as VCSEL (Vertical Cavity Surface Emitting Laser). The VCSEL 12 is an example of an infrared light source.

[0011] The VCSEL 12 irradiates the subject with infrared light as indicated by the dashed and dotted lines. The wavelength of the infrared light may be any wavelength between 800 nm and 1000 nm. When the VCSEL 12 irradiates the subject with infrared light, visible light as indicated by the solid lines and infrared light as indicated by the dashed and dotted lines reflected from the subject are incident on the lens group 11. The splitter 13 separates the visible light and the infrared light, for example, by reflecting the visible light and transmitting the infrared light. The splitter 13 may be formed of a prism.

[0012] The imaging element 14 is a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor, and captures an image of a subject using visible light. The visible light image data generating unit 15 generates visible light image data consisting of three primary color signals of red (R), green (G), and blue (B) based on the imaging signal from the imaging element 14. The TOF sensor 16 measures the reflection level of the reflected light a predetermined time after the time when the VCSEL 12 emits infrared light, thereby measuring the time until the light is reflected by the subject and returns. The distance measurement data generating unit 17 generates distance measurement data based on the time measured by the TOF sensor 16. In this way, the distance measurement data generating unit 17 generates distance measurement data based on the reflected infrared light.

[0013] The synthesis unit 18 synthesizes the visible light image data and the distance measurement data to generate synthetic data, and supplies the synthetic data to the image processing device 20 via the BNC connector B1, the coaxial cable Cc, and the BNC connector B2. The visible light image data and the distance measurement data may not be synthesized, but may be supplied to the image processing device 20 separately.

[0014] 2 shows a specific example of the configuration of the image processing device 20. The image processing device 20 includes a separation unit 21, a skin surface confirmation unit 22, an internal body object region extraction unit 23, a stereoscopic image generation unit 24, a coloring unit 25, and a synthesis unit 26. The image processing device 20 may be configured as a personal computer, a microcomputer, or a circuit. The operation of the image processing device 20 will be described later.

[0015] The three-dimensional imaging device 10 configured as above images human skin as a subject. As shown in FIG. 3, a patient 40 places his / her arm on a table 50. As an example, the three-dimensional imaging device 10 is placed a predetermined distance away from skin 41 of the arm to image the skin 41. As shown in FIG. 4, a cup-shaped tube 60 may be placed on the skin 41 so that the distance between the three-dimensional imaging device 10 and the skin 41 is constant, and the three-dimensional imaging device 10 placed on the tube 60 may image the skin 41. The tube 60 is an example of an auxiliary tool.

[0016] As shown in Fig. 5, it is assumed that an internal object 42 is present inside the body on the inner side of the skin 41. The internal object 42 may be an object present inside the body such as a bone, a foreign body generated inside the body that does not actually exist inside the body, or an object that does not actually exist inside the body and has entered the body from the outside. In Fig. 5, the solid line represents visible light, and the dashed dotted line represents infrared light. The visible light is reflected by the surface of the skin 41. The infrared light may be reflected by the surface of the skin 41, or may enter the body and be reflected by the internal object 42.

[0017] 6, the three-dimensional imaging device 10 is positioned directly above the intracorporeal object 42. Regardless of whether the tube 60 is used, the three-dimensional imaging device 10 irradiates infrared light onto a predetermined region 10C including the portion where the intracorporeal object 42 is located, and receives visible light and infrared light reflected from the region 10C.

[0018] The operation of the image processing device 20 shown in Fig. 2 will be described. The separation unit 21 separates the visible light image data and the distance measurement data, supplies the visible light image data to the skin surface confirmation unit 22 and the synthesis unit 26, and supplies the distance measurement data to the internal body object region extraction unit 23 and the stereoscopic image generation unit 24. It is preferable that the skin 41 is smooth in the region 10C when the three-dimensional imaging device 10 captures an image of the skin 41. The skin surface confirmation unit 22 determines that the skin 41 is in a smooth state if the variation in the luminance component for each pixel in the input visible light image data is within a predetermined range.

[0019] If the skin surface confirming section 22 confirms that the skin 41 is in a smooth state, the internal object region extracting section 23 extracts the region of the internal object 42. It is not essential to provide the skin surface confirming section 22, but it is preferable to provide it.

[0020] 7 conceptually shows distance measurement data of an internal object 42 and its surroundings. In an area where an internal object 42 is not present, part of the infrared light is reflected by the skin 41, and part of the infrared light that enters the body is absorbed at a predetermined absorption rate. On the other hand, in an area where an internal object 42 is present, part of the infrared light is reflected by the skin 41, and part of the infrared light that enters the body is reflected by the internal object 42.

[0021] Therefore, the distance measurement data in the region where the internal object 42 exists is shown as waveform D42. Waveform D42 includes the reflection level reflected by the skin 41 and the reflection level reflected by the internal object 42. In the region where the internal object 42 does not exist, the data is roughly the same as waveform D41. The average value of the distance measurement data in the entire region 10C is Dave.

[0022] The internal object region extraction unit 23 compares the input distance measurement data with the average value Dave, and extracts a region having a signal level higher than the average value Dave as an internal object region R42 where an internal object 42 exists. The internal object region R42 indicates that the internal object 42 exists in a large area. Information indicating the internal object region R42 is supplied to the stereoscopic image generation unit 24.

[0023] The stereoscopic image generating unit 24 subtracts the signal level of the waveform D41 from the signal level in the internal object region R42 based on the information indicating the internal object region R42 to generate distance measurement data reflected only by the internal object 42. The stereoscopic image generating unit 24 generates a stereoscopic image showing the internal object 42 based on the distance measurement data reflected only by the internal object 42. The stereoscopic image generating unit 24 generates a stereoscopic image showing the skin 41 based on the distance measurement data in the portion of the waveform D41. To generate a stereoscopic image of the skin 41 in the portion of the internal object region R42, the stereoscopic image generating unit 24 may interpolate the distance measurement data in the portion of the internal object region R42 based on the distance measurement data in the portion of the waveform D41.

[0024] The stereoscopic image data representing the stereoscopic image of the skin 41 generated by the stereoscopic image generating unit 24 is supplied to the synthesis unit 26, and the stereoscopic image data representing the stereoscopic image of the internal body object 42 generated by the stereoscopic image generating unit 24 is supplied to the coloring unit 25. In response to an external coloring instruction signal, the coloring unit 25 processes the stereoscopic image data so as to apply the instructed color to the stereoscopic image of the internal body object 42. The stereoscopic image data processed by the coloring unit 25 is supplied to the synthesis unit 26. Note that the stereoscopic image data is composed of point cloud data. Coloring by the coloring unit 25 is not essential.

[0025] The synthesis unit 26 synthesizes the input visible light image data, the stereoscopic image data of the skin 41, and the stereoscopic image data of the internal body object 42, and supplies synthetic image data showing a synthetic image to the display unit 30. The display unit 30 displays an image obtained by synthesizing the stereoscopic image of the skin 41 and the stereoscopic image of the internal body object 42 on a visible light image showing the skin 41 of the arm of the patient 40 shown in FIG.

[0026] The operation of the image processing device 20 will be described again with reference to the flowchart shown in FIG. 8. In FIG. 8, the skin surface confirmation unit 22 confirms that the surface of the skin 41 is smooth based on the visible light image data in step S1. The internal body object region extraction unit 23 extracts the internal body object region R42 in step S2. The stereoscopic image generation unit 24 subtracts the surrounding signal level from the signal level in the internal body object region R42 in step S3. The stereoscopic image generation unit 24 generates a stereoscopic image of the surface of the skin 41 and the internal body object 42 in step S4. The synthesis unit 26 synthesizes the visible light image and the stereoscopic image and supplies the synthesized image to the display unit 30 in step S5.

[0027] Although not shown in FIG. 8, the coloring unit 25 colors the stereoscopic image of the intracorporeal object 42 between steps S4 and S5 when a coloring instruction signal is input.

[0028] The image processing device 20 determines whether or not to end the operation by turning off the power, etc. If the operation is not to be ended (NO), the image processing device 20 repeats the processes of steps S1 to S6. If the operation is to be ended (YES), the image processing device 20 ends the process.

[0029] The present invention is not limited to the embodiment described above, and various modifications are possible without departing from the scope of the present invention. The three-dimensional imaging device 10 and the image processing device 20 may be integrally configured. In this case, the synthesis unit 18 and the separation unit 21 are not necessary. [Explanation of symbols]

[0030] 10. Three-dimensional imaging device 11 Lens group 12 Vertical-cavity surface-emitting laser 13. Splitter 14 Image sensor 15 Visible light image data generator 16 TOF Sensor 17 Distance measurement data generator 18,26 Synthesis section 20 Image processing device 21 Separation part 22 Skin surface confirmation part 23 In-body object region extraction unit 24 Stereoscopic image generation unit 25 Coloring Department 30 Display section

Claims

1. an internal object region extraction unit that extracts an internal object region, which is a region where an internal object exists inside the body on the inner surface side of the skin, based on distance measurement data generated based on reflected light of infrared light irradiated onto a predetermined region on the surface of the skin; a stereoscopic image generating unit that generates a stereoscopic image of the skin and a stereoscopic image of the internal body object based on the distance measurement data and information indicating the internal body object region extracted by the internal body object region extracting unit; a synthesis unit that synthesizes a visible light image based on the visible light reflected at the predetermined area, the three-dimensional image of the skin, and the three-dimensional image of the internal body object; An image processing device comprising:

2. The image processing device according to claim 1 , further comprising a coloring unit that colors the stereoscopic image of the internal body object.

3. a skin surface confirmation unit that confirms that the surface of the skin in the predetermined area is smooth based on the visible light image, The internal object region extraction unit extracts the internal object region when the skin surface confirmation unit confirms that the surface of the skin in the predetermined region is in a smooth state.

3. The image processing device according to claim 1 or 2.

4. Irradiating a predetermined area of ​​the skin surface with infrared light; generating distance measurement data based on reflected light of the infrared light irradiated onto the predetermined area; extracting an internal object region, which is a region where an internal object exists inside the body on the inner surface side of the skin, based on the distance measurement data; generating a stereoscopic image of the skin and a stereoscopic image of the internal body object based on the distance measurement data and information indicating the internal body object region; A composite image is generated by combining a visible light image based on the visible light reflected at the predetermined area, the three-dimensional image of the skin, and the three-dimensional image of the internal body object. A method for generating a skin image.

Citation Information

Patent Citations

  • Optical module and optical device

    JP2019200404A