Method for providing ar image for recommendation information related to laser treatment, and ar wearable device using same

Abstract

The present invention may provide an AR wearable device using an AR image for recommendation information related to laser treatment, the AR wearable device comprising: an AR image processing unit configured to receive an image including a laser treatment region of a patient, divide the laser treatment region into a plurality of mesh regions, and determine display information for each of the plurality of mesh regions; and a display unit for providing display information for the plurality of mesh regions generated through the AR image processing unit, wherein the display information for each of the plurality of mesh regions further includes recommendation information related to an irradiation order determined on the basis of laser energy information and current laser irradiation position information.

Description

Method for providing AR images for recommended information related to laser treatment and AR wearable device using the same

[0001] The present invention relates to a method for providing AR images for recommending information related to laser treatment, and to an AR wearable device utilizing the same. More specifically, the present invention relates to a method, device, and system for providing AR images that can divide a patient's laser treatment area into multiple mesh areas and display recommended information related to laser treatment through the AR image.

[0002] Today, various laser treatments and treatment techniques are being developed that irradiate the skin with laser light for therapeutic purposes, and medical laser devices for such laser treatments are also being actively researched.

[0003] Laser light (beam) refers to a "special light" produced by injecting energy into a specific substance and then going through a stimulated emission amplification stage. Laser treatment is a convenient method that is widely used because it penetrates this light deep into the skin in an extremely short period of time to selectively destroy only the target skin lesion without damaging the surrounding tissue, leaving no visible scars and allowing for simple treatment in a short period of time.

[0004] Traditionally, these laser treatment devices were manually operated by practitioners, such as doctors. This relied on the practitioner's subjective experience and senses, which sometimes reduced the precision and effectiveness of laser treatment. Furthermore, prolonged treatment periods made it difficult for practitioners to maintain focus, posing safety risks for patients.

[0005] Accordingly, new laser treatment methods and devices are needed to improve the precision and efficiency of laser treatment and ensure patient safety.

[0006] The purpose of the present invention is to provide a method and device for providing AR images that can improve the precision and accuracy of laser treatment.

[0007] In addition, the present invention aims to provide a method and device for providing patient-tailored AR images to provide an optimal laser treatment method according to the patient's skin characteristics.

[0008] In addition, the present invention aims to provide a new laser treatment method and device that can improve the precision and accuracy of laser treatment by dividing a patient's laser treatment area into a plurality of mesh areas and intuitively showing the laser irradiation amount and alarm indications, etc., through color or contrast for each mesh area through AR image.

[0009] In addition, the present invention aims to provide a laser treatment method optimized for each patient by providing AR images customized according to the patient's skin characteristics.

[0010] In addition, the present invention aims to provide a method and device that enables a practitioner to perform laser treatment more safely and systematically by providing real-time temperature information and laser irradiation amount information of each mesh area within a patient's laser treatment area through AR images.

[0011] In addition, the present invention aims to provide a laser treatment method optimized for the current skin recovery state of a patient by providing recommendation information related to the order of laser irradiation areas in conjunction with laser irradiation history information of a specific point in the past for a specific skin area.

[0012] The problems to be solved by the present invention are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0013] According to one embodiment of the present invention, an AR wearable device using an AR image for recommendation information related to laser treatment may be provided, comprising: an AR image processing unit configured to receive an image including a laser treatment area of ​​a patient, divide the laser treatment area into a plurality of mesh areas, and determine display information for each of the plurality of mesh areas; and a display unit configured to provide display information for the plurality of mesh areas generated through the AR image processing unit, wherein the display information for each of the plurality of mesh areas includes recommendation information related to an irradiation order determined based on laser energy information and current laser irradiation position information.

[0014] Here, the display information may further include color or contrast information.

[0015] Additionally, the display information may be set differently based on the skin characteristic values ​​of the patient.

[0016] Additionally, the display information may be changed based on the amount of time that has elapsed since the point at which laser irradiation was applied to the corresponding mesh area.

[0017] Additionally, the above recommendation information may be determined based on the laser irradiation history information of a specific skin area of ​​the patient at a specific point in the past.

[0018] Additionally, the laser irradiation history information may include information related to the time elapsed from the specific point in time in the past, the laser irradiation location, the number of laser irradiations, and the laser irradiation wavelength.

[0019] In addition, the method further includes a communication unit configured to receive information related to the number of laser irradiations and irradiation energy applied to the patient from the laser irradiator, and the AR image processing unit may be configured to determine recommendation information for each of the plurality of mesh areas based on the number of laser irradiations and irradiation energy associated with each of the plurality of mesh areas.

[0020] Additionally, the color or brightness information of each mesh area can be changed in real time based on the number of laser irradiations and irradiation energy applied to each of the plurality of mesh areas.

[0021] In addition, the AR image processing unit may be configured to calculate an energy accumulation value of each mesh area based on the number of laser irradiations and irradiation energy applied to each of the plurality of mesh areas, and determine whether to generate an alarm display based on whether the energy accumulation value exceeds a preset standard.

[0022] Additionally, the preset standard to be compared with the energy accumulation value may be changed based on the skin characteristic value of the patient.

[0023] Additionally, the size of each of the plurality of divided mesh regions can be changed based on the skin characteristic values ​​of the patient.

[0024] In addition, the device further includes a temperature detection sensor configured to obtain real-time temperature information on the laser treatment area of ​​the patient, and the AR image processing unit may further include display information for each of the plurality of mesh areas including skin temperature information corresponding to the corresponding mesh area obtained by the temperature detection sensor.

[0025] Additionally, the AR image processing unit may be configured to determine whether to generate an alarm based on whether the skin temperature information exceeds a preset standard.

[0026] In addition, according to another embodiment of the present invention, a method for providing an AR image for laser treatment using an AR wearable device may be provided, including: receiving an image including a laser treatment area of ​​a patient through an AR image processing unit; dividing the laser treatment area into a plurality of mesh areas through the AR image processing unit and determining display information for each of the plurality of mesh areas; and providing, through a display unit, display information for the plurality of mesh areas generated through the AR image processing unit, wherein the display information for each of the plurality of mesh areas includes recommendation information related to an irradiation order determined based on laser energy information and current laser irradiation position information.

[0027] In addition, according to another embodiment of the present invention, a computer-readable recording medium storing a computer program for performing the AR image providing method described above can be provided.

[0028] According to the present invention, a method and device for providing AR images that can improve the precision and accuracy of laser treatment can be provided.

[0029] In addition, according to the present invention, a method and device for providing patient-tailored AR images to provide an optimal laser treatment method according to the patient's skin characteristics can be provided.

[0030] In addition, according to the present invention, a new laser treatment method and device can be provided that can improve the precision and accuracy of laser treatment by dividing a patient's laser treatment area into a plurality of mesh areas and intuitively showing the laser irradiation amount and alarm indication, etc. through color or contrast for each mesh area through AR image.

[0031] In addition, according to the present invention, a laser treatment method optimized for each patient can be provided by providing AR images customized according to the patient's skin characteristics.

[0032] In addition, according to the present invention, a method and device can be provided that enable a practitioner to perform laser treatment more safely and systematically by providing real-time temperature information and laser irradiation amount information of each mesh area within a patient's laser treatment area through AR images.

[0033] In addition, according to the present invention, by providing recommendation information related to the order of laser irradiation areas in conjunction with the history of laser irradiation of a specific skin area at a specific point in the past, a laser treatment method optimized for the current skin recovery state of a patient can be provided.

[0034] The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0035] FIG. 1 is an exemplary diagram showing a state in which laser treatment is performed on a patient using an AR wearable device according to one embodiment of the present invention.

[0036] FIG. 2 is a block diagram illustrating the configuration of an AR wearable device according to one embodiment of the present invention.

[0037] FIG. 3 is a block diagram illustrating the configuration of an AR image processing unit of an AR wearable device according to one embodiment of the present invention.

[0038] FIG. 4 is an exemplary diagram showing a plurality of mesh areas that divide a treatment area according to one embodiment of the present invention.

[0039] FIGS. 5A, 5B, and 5C are exemplary diagrams showing various display methods of multiple mesh areas that divide a treatment area according to one embodiment of the present invention.

[0040] FIG. 6 is a flowchart illustrating a method for providing AR images for laser treatment using an AR wearable device according to one embodiment of the present invention.

[0041] Hereinafter, the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily practice the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein.

[0042] The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise.

[0043] As used herein, the terms “comprises” and “comprising” do not exclude the presence or addition of one or more other components, steps, operations and / or elements.

[0044] Additionally, terms containing ordinal numbers, such as "first," "second," etc., used in the present invention may be used to describe components, but the components should not be limited by these terms. Such terms are used solely to distinguish one component from another. Furthermore, when describing the present invention, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the present invention.

[0045] Furthermore, the components shown in the embodiments of the present invention are depicted independently to represent different characteristic functions, and this does not mean that each component is composed of separate hardware or a single software component. That is, each component is described by listing each component for convenience of explanation, and at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separate embodiments of each component are also included in the scope of the present invention, as long as they do not deviate from the essence of the present invention.

[0046] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. The configuration and resulting operational effects of the present invention will be clearly understood through the detailed description below.

[0047]

[0048] FIG. 1 is an exemplary diagram showing a state in which laser treatment is performed on a patient using an AR wearable device according to one embodiment of the present invention.

[0049] During laser treatment or procedure, the laser practitioner (110) may wear an augmented reality (AR) wearable device (300) according to the present invention and use a laser irradiator (200) to irradiate laser to the treatment area of ​​the patient (120). The laser practitioner (110) may be a medical professional such as a doctor or nurse.

[0050] The AR wearable device (300) has a form that can be worn by the practitioner on the body, such as the head or face, and can be configured to display the laser treatment area visible to the practitioner's (110) eyes by dividing it into a plurality of mesh areas, and to display display information including color or brightness information for each of the plurality of mesh areas together. A more specific configuration of the AR wearable device (300) will be described in detail below.

[0051] The AR wearable device (300) may also include a temperature detection sensor (330) configured to obtain real-time temperature information for a laser treatment area, and the AR wearable device (300) may provide skin temperature information corresponding to a corresponding mesh area obtained by the temperature detection sensor (330) through display information for each of a plurality of mesh areas.

[0052]

[0053] FIG. 2 is a block diagram illustrating the configuration of an AR wearable device according to one embodiment of the present invention.

[0054] The AR wearable device (300) has a form that can be worn on the head or face of a laser practitioner, and is configured to provide an augmented reality service by providing visual information for laser treatment through a display, and is configured to include a display unit (310), a camera unit (320), a temperature detection sensor (330), a communication unit (340), an AR image processing unit (350), and an audio output unit (360), but is not limited to these components.

[0055] The display unit (310) is configured to display visual information to the practitioner (110) who is the wearer of the AR wearable device (300), and may provide an augmented reality experience to the practitioner (110) by, for example, using at least one of a semi-reflective curved mirror method, a light guide method, or a waveguide method capable of optical see-through, or providing a mixed reality image obtained by adding necessary information to a line-of-sight image obtained through the camera unit (320). The display unit (310) may be configured to show, in particular, a laser treatment area generated through the AR image processing unit (350) divided into a plurality of mesh areas, and display information including laser irradiation information and skin temperature information expressed as color or brightness information for each of the plurality of mesh areas, together with reality information.

[0056] The camera unit (320) is configured to capture an image in front of the operator (110) and obtain a captured image, and may include, for example, an RGB-Depth camera capable of capturing an RGB image and depth information, or, as another example, an RGB acquisition module capable of capturing an RGB image and a depth acquisition module capable of capturing depth information. After capturing the RGB image and depth information through the camera unit (320), the three-dimensional position of at least one object included in the image can be obtained based on a camera coordinate system. The image captured through the camera unit (320) can be transmitted to the AR image processing unit (350) for AR image processing.

[0057] The temperature detection sensor (330) is configured to be mounted on the AR wearable device (300) and detect the skin temperature of the patient (120). For example, the temperature detection sensor may be a thermal imaging camera that detects temperature with infrared rays as a non-contact temperature sensor, and may measure the skin temperature through the amount of radiation emitted from the skin surface. For example, the temperature detection sensor (330) may be a sensor manufactured through a CMOS process and capable of detecting the temperature of each pixel of a camera image. The thermal image and skin temperature information obtained through the temperature detection sensor (330) may be transmitted to the AR image processing unit (350) for AR image processing.

[0058] The communication unit (340) may include one or more wired / wireless communication modules for communication with an external device, and may include, for example, a short-range communication unit and a mobile communication unit, and may be configured to transmit and receive necessary data through wired / wireless communication with the laser irradiator (200) and the patient information system (400). Here, the short-range wireless communication unit may include, for example, a Bluetooth communication unit, a BLE (Bluetooth Low Energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared (IrDA, infrared Data Association) communication unit, a WFD (Wi-Fi Direct) communication unit, an UWB (ultra wideband) communication unit, an ANT+ communication unit, etc., but is not limited thereto. In addition, the mobile communication unit can transmit and receive wireless signals with at least one of a base station, an external terminal, or a server on a mobile communication network, and the wireless signals can include various forms of data, such as a voice call signal, a video call signal, or text / multimedia message transmission and reception.

[0059] The AR wearable device (300) can be connected to the laser irradiator (200) via a communication unit (340) through wired or wireless communication to receive setting information or irradiation information of the laser irradiator (200), and can receive, for example, laser irradiation-related setting information such as the number of laser irradiations (the number of laser shot operations), irradiation energy per laser shot, and laser wavelength information in real time. In addition, the AR wearable device (300) can receive previous laser treatment information and skin characteristic value information of the corresponding patient through the patient information system (400). Here, the previous laser treatment information may include previous treatment area location information, laser treatment type information, number of laser shots irradiated, energy information, and the like, and the patient's skin characteristic value information may include skin sensitivity information, allergy information, and information such as moisture content, epidermal thickness, and dermal thickness obtained through skin measurement of the patient. In addition to the patient's skin characteristic values, information on the season considering the external environment, such as the outside temperature and humidity, the type of moisturizer or cosmetic applied to the patient's skin, and the time after applying the moisturizer or cosmetic can be additionally considered.

[0060] The AR image processing unit (350) may be configured to divide the laser treatment area captured by the camera unit (320) into a plurality of mesh areas and determine display information for each of the plurality of mesh areas. At this time, the display information may be aligned so that it appears on the corresponding mesh area using 3D position information acquired through the camera coordinate system. Here, the mesh area may be composed of points, lines, polygons, polyhedrons, etc., and may be composed of a polygon mesh divided into a plurality of mesh areas based on the similarity of vector values ​​such as, for example, a normal vector. In addition, the size of the mesh area may be changed by adjusting the sophistication by adjusting the similarity judgment standard of the vector value.

[0061] Here, the display information displayed for each mesh region can be configured to change in real time the color or brightness information of each mesh region based on the number of laser irradiations and the irradiation energy applied to each of the plurality of mesh regions. For example, the accumulated value of the laser energy applied to the corresponding mesh region can be calculated based on the product of the number of laser irradiations and the irradiation energy, and the color or brightness information of the corresponding mesh region can be changed based on the accumulated laser energy value. For example, the greater the applied energy, the darker the brightness of the corresponding mesh region can be displayed, thereby allowing the operator to intuitively understand the applied energy value.

[0062] In addition, the display information displayed for each mesh area may be set differently based on the patient's skin characteristic value measured through the camera unit (320) or acquired through the patient information system (400). For example, the AR image processing unit (350) may calculate the energy accumulation value of each mesh area based on the number of laser irradiations and irradiation energy applied to each of the plurality of mesh areas, and may determine whether to generate an alarm display based on whether the energy accumulation value exceeds a preset standard, and may display the alarm display as the display information. At this time, the preset standard compared with the energy accumulation value may be changed based on the patient's skin characteristic value. That is, when the patient's skin characteristic is sensitive or weak, the alarm display generation standard may be set more sensitively. For example, the warning level brightness display standard during laser treatment may be set differently depending on the patient's skin characteristic value, so that when the same laser energy is applied, a patient with strong skin characteristic does not reach the brightness display indicating the alarm display, but a patient with weak skin characteristic can quickly reach the brightness display indicating the alarm display. Additionally, for example, if a patient's skin moisture content is determined to be relatively high, the brightness display criteria for indicating an alarm can be adjusted to make the warning level brightness display criteria relatively less sensitive. In this way, by changing the display information in the mesh area to reflect the patient's skin characteristics, customized information optimized for each patient can be provided.

[0063] In addition, the AR image processing unit (350) may be configured to change the size of each of the plurality of divided mesh regions based on the patient's skin characteristic value. For example, when the patient's skin characteristic is sensitive or weak, the size setting of the mesh region may be set to be small, and when the patient's skin characteristic is less sensitive and strong, the size setting of the mesh region may be set to be large.

[0064] In addition, the display information such as color or brightness displayed for each mesh area can be changed based on the amount of time that has elapsed since the time when the laser was applied to the corresponding mesh area. For example, by considering the time that has elapsed since the laser was applied to each mesh area and the time it takes for each patient to produce collagen and regenerate elastin fibers according to laser treatment of a specific type or wavelength band in a specific area, the skin recovery at the current time can be simulated to correct the skin characteristic value, and the display information can be changed based on the corrected skin characteristic value. For example, the brightness of the corresponding mesh area can be displayed dark based on the number of laser irradiations applied to the corresponding mesh area and the wavelength or magnitude of the irradiation energy, and then the brightness of the corresponding mesh area can be displayed so that it becomes lighter based on the amount of time that has elapsed since the end of the laser treatment. Such display information values ​​can be updated and stored so that they can be used for the next treatment. For example, when a 1064 nm Nd:YAG laser is used, collagen production in the dermis can be induced, which can achieve a skin regeneration effect as well as the effect of improving dermal vascular lesions and pigmented lesions. Alternatively, a copper vapor laser, which uses copper vapor as a gain medium and generates two wavelengths, a green wavelength of 511 nm and a yellow wavelength of 578.2 nm, can be used to treat vascular lesions.

[0065] In addition, the AR image processing unit (350) may process the display information for each of the plurality of mesh areas to further include skin temperature information corresponding to the corresponding mesh area acquired by the temperature detection sensor (330), and for example, may display a color corresponding to the skin temperature information or display separate temperature numerical information together in each mesh area. In addition, the AR image processing unit (350) may be configured to determine whether to generate an alarm based on whether the skin temperature information exceeds a preset standard, and the alarm generation standard for such skin temperature information may be changed based on the patient's skin characteristic value.

[0066] The audio output unit (360) may be configured to perform audio output when an alarm indication is generated through the AR image processing unit (350). When the operator (110) or patient (120) reaches an alarm (warning) indication that may occur during laser treatment, a warning sound or alarm-related information may be output as audio, thereby enhancing patient safety.

[0067]

[0068] FIG. 3 is a block diagram illustrating the configuration of an AR image processing unit of an AR wearable device according to one embodiment of the present invention.

[0069] The AR image processing unit (350) may include an image receiving unit (351), a mesh area generating unit (352), a mesh area information processing unit (353), an AR image generating unit (354), and an alarm processing unit (355). The AR image processing unit (350) may include a program or program module that can be executed by one or more processors, and a necessary memory, etc., wherein the program or program modules may be configured in the form of an operating system, an application program, or a program, and may be physically stored on various types of widely used storage devices. Such a program or program module may include one or more routines, subroutines, programs, objects, components, instructions, data structures, and various forms for performing a specific task or executing a specific data type, but is not limited to these forms.

[0070] First, the image receiving unit (351) may be configured to receive an image including a laser treatment area of ​​a patient acquired through a camera unit (320) or a separate camera device. In addition, the image receiving unit (351) may be configured to receive a thermal image acquired through a temperature detection sensor (330). In addition, the image receiving unit (351) may be configured to receive or generate an image processed by translucently overlapping a thermal image acquired through the temperature detection sensor (330) and a real-world image acquired through the camera unit (320).

[0071] The mesh region generation unit (352) may be configured to divide the laser treatment area into a plurality of mesh regions and generate a plurality of segmented regions, such as polygons. The mesh region generation unit (352) may perform an image analysis task to analyze the image image acquired through the image receiving unit (351) to separate only the treatment area. For example, if the treatment area is a face by analyzing the patient's image, the mesh region generation unit (352) may separate only the face area into objects and divide only the face area into a plurality of mesh regions to generate a mesh region only for the corresponding treatment area. In addition, the mesh region generation unit (352) may be configured to change the size of each of the plurality of segmented mesh regions based on the patient's skin characteristic value when determining the size.

[0072] The mesh region information processing unit (353) may be configured to generate display information to be added as an AR image. The mesh region information processing unit (353) may determine display information for each of the generated plurality of mesh regions, and the display information may include color or contrast information, so that the color or contrast of each mesh region may be displayed differently. Here, the display information may be set differently for each individual by applying different criteria to the same numerical value based on the patient's skin characteristic value. In addition, the display information may be set to change based on the amount of time that has elapsed since the time when the laser irradiation was applied to the corresponding mesh region. In addition, after receiving information related to the number of laser irradiations and the irradiation energy applied to the patient from the laser irradiator (200) through the communication unit (340), the mesh region information processing unit (353) may be configured to calculate an accumulated value of energy applied to each mesh region based on the number of laser irradiations and the irradiation energy associated with each of the plurality of mesh regions, and determine display information for each of the plurality of mesh regions based thereon. In addition, for example, the mesh area information processing unit (353) is configured to change the color or brightness information of each mesh area in real time based on the number of laser irradiations and irradiation energy applied to each of the plurality of mesh areas, and according to such change in display information, the operator (110) can intuitively and quickly understand the cumulative laser irradiation amount applied to the corresponding mesh area and proceed with treatment more systematically.

[0073] In addition, the display information of each mesh area may include alarm display information, and the alarm display information may also be displayed in a specific color or contrast, or may appear as a separate alarm display. For example, the mesh area information processing unit (353) may be configured to calculate an energy accumulation value of each mesh area based on the number of laser irradiations and irradiation energy applied to each of the plurality of mesh areas, and determine whether to generate an alarm display based on whether the energy accumulation value exceeds a preset standard. Here, the preset standard compared with the energy accumulation value may be set to change based on the skin characteristic value of the patient, so that whether to generate an alarm display can be determined according to the skin characteristic of each patient. In addition, an alarm display may be set to generate when the energy accumulation value of each of the plurality of adjacent mesh areas exceeds a preset standard.

[0074] In addition, the mesh area information processing unit (353) may be configured to further generate recommendation information related to the irradiation order for optimal laser treatment effects based on laser energy information and current laser irradiation position information as display information, so that the practitioner (110) can conveniently identify the position and order of the optimal laser irradiation area through the AR image generated on the display unit (310). In addition, recommendation information regarding the irradiation order of the optimal laser irradiation area may be provided by reflecting the current standard skin resilience of each patient, such as causing and regenerating a mild burn in the relevant area by lasers of different wavelengths irradiated to the relevant area of ​​the patient in the past, or temporally simulating the production of collagen and elastin fibers in the dermis and subfascial layer of the relevant area. To this end, the recommendation information may be determined based on laser irradiation history information of a specific skin area of ​​each patient at a specific point in time in the past, and the laser irradiation history information may include information related to, for example, the time elapsed from a specific point in time in the past, the laser irradiation position related to the past laser treatment, the number of laser irradiations, and the laser irradiation wavelength. Additionally, the recommended information related to laser irradiation can be updated in the future by reflecting the relationship between whether treatment was performed according to the recommended laser irradiation order and the resulting patient skin prognosis information.

[0075] In addition, the display information of each mesh area may further include skin temperature information corresponding to the mesh area acquired by the temperature detection sensor (330), and the skin temperature information may be displayed in the form of numbers or colors, etc. In addition, the mesh area information processing unit (353) may be configured to determine whether to generate an alarm display based on whether the skin temperature information exceeds a standard preset for each patient.

[0076] The AR image generation unit (354) is configured to generate an AR image by adding display information for each mesh area and a plurality of mesh areas generated through the mesh area generation unit (352) and the mesh area information processing unit (353), and the AR image generated in this way can be displayed through an optical perspective method through the display unit (310), or can be displayed through a method of providing a mixed reality image obtained by adding AR image information to a line-of-sight image obtained through a camera (320).

[0077] The alarm processing unit (355) is configured to additionally provide an alarm when an alarm indication is generated by the mesh area information processing unit (353), and may be configured to output audio indicating the alarm through, for example, the audio output unit (360).

[0078]

[0079] FIG. 4 is an exemplary diagram showing a plurality of mesh areas that divide a treatment area according to one embodiment of the present invention.

[0080] Referring to FIG. 4, if the laser treatment area of ​​a patient (120) is, for example, the face and neck, the face and neck areas of the patient's captured image can be recognized and the treatment area can be divided into multiple mesh areas in the shape of polygons. The division method can be divided into multiple mesh areas based on the similarity of vector values ​​such as normal vectors, and various other methods can be used.

[0081] Here, each segmented mesh area can be displayed with different colors or contrasts, etc., and for example, the color or contrast can be determined in real time based on the accumulated energy value calculated based on the number of laser irradiations and the irradiation energy, and the color or contrast can be changed to reflect the skin's resilience based on the amount of time that has passed since the laser irradiation was applied. In addition, the color or contrast can be changed to reflect the current degree of skin recovery for each patient, such as causing a mild burn in the area and regeneration due to lasers of different wavelengths irradiated to the area in the past, or temporally simulating the production of collagen and elastin fibers in the dermis and subfascial layer of the area. In addition, when the accumulated energy value during laser treatment exceeds a preset standard, an alarm can be displayed for the mesh area in a specific color or contrast, and the judgment criteria for determining whether to display such an alarm can be applied differently depending on the patient's skin characteristic value.

[0082] Additionally, the size of each mesh region can be changed according to the patient's skin characteristic values ​​by adjusting the similarity judgment criteria of the vector values.

[0083]

[0084] FIGS. 5A, 5B, and 5C are exemplary diagrams showing various display methods of multiple mesh areas that divide a treatment area according to one embodiment of the present invention.

[0085] Referring to Fig. 5a, it can be seen that the brightness of each mesh area changes based on the accumulated energy value applied to the corresponding mesh area during laser treatment. For example, if the accumulated energy value applied to mesh area A is greater than the accumulated energy value applied to mesh area B, the brightness of mesh area A may be displayed darker than the brightness of mesh area B, and the operator (110) can quickly determine the level of energy applied to each mesh area through the displayed brightness information.

[0086] Referring to FIG. 5b, the size of each mesh region can be changed according to the patient's skin characteristic value. For example, if the patient's skin characteristic is highly sensitive or weak, the size of the mesh region can be set smaller and smaller as in the drawing on the right, thereby providing laser treatment-related display information for a more precise range.

[0087] Referring to FIG. 5c, information related to the irradiation order determined based on laser energy information and current laser irradiation position information can be provided for each mesh area. For example, in the A mesh area divided into four, the irradiation order is determined as A1, A2, A3, A4 based on the degree of proximity to the current laser irradiation position and the similarity of the size information of the laser energy to be irradiated to the corresponding mesh area, and then in the B mesh area divided into four adjacent mesh areas, the irradiation order is similarly determined as B1, B2, B3, B4 and displayed as display information, thereby helping the operator (110) to more systematically determine the irradiation position and order during laser irradiation.

[0088]

[0089] FIG. 6 is a flowchart illustrating a method for providing AR images for laser treatment using an AR wearable device according to one embodiment of the present invention.

[0090] Referring to FIG. 6, the AR wearable device (300) can obtain image data of the patient's laser treatment area through the camera unit (320). (S610)

[0091] In addition, through skin photography of the camera unit (320), or through the patient information system (400), the patient's skin characteristic values, such as skin sensitivity information, allergy information, and information such as moisture content, epidermal thickness, and dermal thickness obtained by measuring the patient's skin, can be acquired from the AR wearable device (300). (S620)

[0092] The AR wearable device (300) can divide the laser treatment area into multiple mesh areas through the AR image processing unit (350). (S630)

[0093] The AR wearable device (300) can update display information for each mesh area based on laser irradiation information for each mesh area through the AR image processing unit (350). (S640) For example, the color or brightness of each mesh area can be changed in real time based on the accumulated laser energy value calculated using the number of laser irradiations and irradiation energy for each mesh area.

[0094] The AR wearable device (300) can update display information for each mesh area based on skin temperature information through the AR image processing unit (350). (S650) Based on the skin temperature information of each mesh area obtained through the temperature detection sensor (330), the color or brightness of each mesh area can be changed in real time, or the skin temperature information can be expressed as a numerical value.

[0095] The AR wearable device (300) can generate an alarm based on laser irradiation information or skin temperature information. The criteria for determining whether to generate such an alarm can be applied differently depending on the patient's skin characteristics, and can be configured to generate an alarm if the accumulated energy value or skin temperature information from laser irradiation exceeds a predetermined judgment criterion.

[0096] The AR wearable device (300) can store abnormal information, such as whether an alarm has occurred for each mesh area (S670). Based on alarm information that occurred during laser treatment for the patient, information related to the mesh area and the cause of the alarm occurrence is stored, and such information is provided during the next laser treatment, thereby enabling safer laser treatment.

Claims

1. In an AR wearable device using AR images for recommended information related to laser treatment, An AR image processing unit configured to receive an image including a laser treatment area of ​​a patient, divide the laser treatment area into a plurality of mesh areas, and determine display information for each of the plurality of mesh areas; and A display unit for providing display information on the plurality of mesh areas generated through the AR image processing unit. Including, The display information for each of the above plurality of mesh areas includes recommendation information related to the irradiation order determined based on laser energy information and current laser irradiation position information, The above display information is set differently based on the skin characteristic values ​​of the patient, The above display information changes based on the amount of time that has elapsed since the time when the laser was irradiated on the corresponding mesh area. The skin characteristic value of the patient is corrected based on the result of simulating the skin recovery power at the current point in time using information related to the wavelength of laser irradiation applied to each mesh area of ​​the patient and the amount of time elapsed from the time of laser irradiation. The above recommendation information is determined based on the laser irradiation history information of a specific skin area of ​​the patient at a specific time in the past, An AR wearable device wherein the above laser irradiation history information includes a result of simulating skin resilience at the current point in time.

2. An AR wearable device according to claim 1, wherein the display information further includes color or contrast information.

3. An AR wearable device according to claim 1, wherein the laser irradiation history information includes information related to the time elapsed from the specific point in time in the past, the laser irradiation location, the number of laser irradiations, and the laser irradiation wavelength.

4. In the first paragraph, a communication unit is further configured to receive information related to the number of laser irradiations and irradiation energy applied to the patient from the laser irradiator, An AR wearable device, wherein the AR image processing unit is configured to determine recommendation information for each of the plurality of mesh areas based on the number of laser irradiations and irradiation energy associated with each of the plurality of mesh areas.

5. An AR wearable device in which the color or brightness information of each mesh area is changed in real time based on the number of laser irradiations and irradiation energy applied to each of the plurality of mesh areas in the first paragraph.

6. In the first paragraph, the AR image processing unit is configured to calculate an energy accumulation value of each mesh area based on the number of laser irradiations and irradiation energy applied to each of the plurality of mesh areas, and determine whether to generate an alarm display based on whether the energy accumulation value exceeds a preset standard.

7. An AR wearable device according to claim 6, wherein the preset standard compared with the energy accumulation value is changed based on the patient's skin characteristic value.

8. An AR wearable device according to claim 1, wherein the size of each of the plurality of divided mesh regions is changed based on the skin characteristic value of the patient.

9. In the first paragraph, a temperature detection sensor configured to obtain real-time temperature information on the laser treatment area of ​​the patient is further included. The AR wearable device, wherein the AR image processing unit further includes the display information for each of the plurality of mesh areas including skin temperature information corresponding to the corresponding mesh area acquired by the temperature detection sensor.

10. An AR wearable device according to claim 9, wherein the AR image processing unit is configured to determine whether to generate an alarm based on whether the skin temperature information exceeds a preset standard.

11. A method for providing AR images for laser treatment using an AR wearable device, A step of receiving an image including a patient's laser treatment area through an AR image processing unit; A step of dividing the laser treatment area into a plurality of mesh areas through the AR image processing unit and determining display information for each of the plurality of mesh areas; and A step of providing display information for the plurality of mesh areas generated through the AR image processing unit through the display unit. Including, The display information for each of the above plurality of mesh areas includes recommendation information related to the irradiation order determined based on laser energy information and current laser irradiation position information, The above display information is set differently based on the skin characteristic values ​​of the patient, The above display information changes based on the amount of time that has elapsed since the time when the laser was irradiated on the corresponding mesh area. The skin characteristic value of the patient is corrected based on the result of simulating the skin recovery power at the current point in time using information related to the wavelength of laser irradiation applied to each mesh area of ​​the patient and the amount of time elapsed from the time of laser irradiation. The above recommendation information is determined based on the laser irradiation history information of a specific skin area of ​​the patient at a specific time in the past, An AR image providing method, wherein the above laser irradiation history information includes a result of simulating skin recovery power at the current point in time.

12. A computer-readable recording medium storing a computer program for performing the method described in Article 11.

Images