Display devices for providing same level of image and control methods thereof
The display system uses a reflector to correct image distortions and synchronize display settings, enabling patients and doctors to view medical images at the same quality, thus enhancing remote medical treatment efficiency.
Patent Information
- Application Number
- PCT/KR2024/005203
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-25
- Filing Date
- 2024-04-18
- Publication Date
- 2024-11-28
AI Technical Summary
In remote medical treatments, differences in picture quality between the screens of doctors and patients lead to reduced efficiency in medical treatment, as patients struggle to understand medical judgments due to varying image quality, especially when images of body parts like X-rays or MRIs are shared.
A display system comprising two devices, one on the doctor's side and one on the patient's side, that use a reflector (color patch) to correct image distortions based on illuminance and device characteristics, allowing for synchronized display settings to ensure images are viewed at the same level by both parties.
This solution ensures that patients and doctors can view medical images at the same quality level, improving the effectiveness of remote medical treatments by minimizing distortions caused by environmental and device-related factors.
Smart Images

Figure KR2024005203_28112024_PF_FP_ABST
Abstract
Description
Display devices and control methods thereof for providing images of the same level
[0001] The present disclosure relates to display devices and control methods thereof, and more particularly, to display devices and control methods thereof for providing images of the same level.
[0002] Advances in electronic technology have led to the development of various electronic systems. In particular, recent developments in the telehealth and telemedicine fields have enabled the exchange of images and data for medical purposes with remote doctors via video calls and remote communications.
[0003] However, there is a problem that the efficiency of treatment is reduced due to the difference in image quality of the screen shared between doctors and patients located far away.
[0004] According to one embodiment of the present disclosure for achieving the above purpose, a display device includes a communication interface, a memory, a display, and at least one processor connected to the communication interface, the memory, and the display to control the display device, wherein the processor can control the communication interface to receive a photographed image including a reflector (color patch) from another display device through the communication interface, update a setting value of the display based on the photographed image and digital values for a plurality of colors included in the reflector stored in the memory, and transmit the updated setting value to the other display device.
[0005] In addition, the processor may further include a sensor, and may change the updated setting value based on the illumination around the display device detected through the sensor, and control the display to display the captured image based on the changed setting value.
[0006] And, further including a user interface, the processor can control the communication interface to change the changed setting value again when a user command to change the changed setting value again is received through the user interface and transmit the changed setting value again to the other display device.
[0007] Additionally, the plurality of colors may include at least one color obtained by photographing a preset body part of a user of the other display device.
[0008] And, the at least one color may be a color corrected based on the reflectance of the preset body part and the reflectance of the reflector.
[0009] Additionally, the settings may include at least one of the gamma, color temperature or light source profile of the display.
[0010] Meanwhile, according to one embodiment of the present disclosure, a display device includes a camera, a sensor, a communication interface, a display, and at least one processor connected to the camera, the sensor, the communication interface, and the display to control the display device, wherein the processor obtains a photographed image including a reflector (color patch) through the camera, corrects the photographed image based on the illuminance around the display device detected through the sensor and the device characteristics of the camera, and controls the communication interface to transmit the corrected photographed image to another display device, receives a setting value of another display included in the other display device from the other display device through the communication interface, and controls the display to display the corrected photographed image based on the illuminance and the received setting value.
[0011] Additionally, the processor can receive a changed setting value of the other display from the other display device through the communication interface, and control the display to display the corrected captured image based on the illuminance and the changed setting value.
[0012] And, the processor can control the communication interface to transmit the received setting value to a plurality of other display devices excluding the other display device.
[0013] Additionally, the plurality of colors included in the reflector may include at least one color obtained by photographing a preset body part of a user of the display device.
[0014] And, the at least one color may be a color corrected based on the reflectance of the preset body part and the reflectance of the reflector.
[0015] Additionally, the received settings may include at least one of the gamma, color temperature or light source profile of the other display.
[0016] Meanwhile, according to one embodiment of the present disclosure, a method for controlling a display device includes the steps of receiving a photographed image including a reflector (color patch) from another display device, updating a setting value of the display based on digital values for a plurality of colors included in the photographed image and the reflector, and transmitting the updated setting value to the other display device.
[0017] In addition, the method may further include a step of changing the updated setting value based on the illumination around the display device and a step of displaying the captured image based on the changed setting value.
[0018] And, when a user command to change the changed setting value is received, the method may further include a step of changing the changed setting value again and a step of transmitting the changed setting value to the other display device.
[0019] Additionally, the plurality of colors may include at least one color obtained by photographing a preset body part of a user of the other display device.
[0020] And, the at least one color may be a color corrected based on the reflectance of the preset body part and the reflectance of the reflector.
[0021] Additionally, the settings may include at least one of the gamma, color temperature or light source profile of the display.
[0022] Meanwhile, according to one embodiment of the present disclosure, a method for controlling a display device includes the steps of obtaining a photographed image including a reflector (color patch) through a camera included in the display device, correcting the photographed image based on illuminance around the display device and device characteristics of the camera, transmitting the corrected photographed image to another display device, receiving a setting value of another display included in the other display device from the other display device, and controlling a display included in the display device to display the corrected photographed image based on the illuminance and the received setting value.
[0023] In addition, the method may further include a step of receiving a changed setting value of the other display from the other display device and a step of controlling the display to display the corrected captured image based on the illuminance and the changed setting value.
[0024] In addition, the method may further include a step of transmitting the received setting value to a plurality of other display devices excluding the other display device.
[0025] Additionally, the plurality of colors included in the reflector may include at least one color obtained by photographing a preset body part of a user of the display device.
[0026] And, the at least one color may be a color corrected based on the reflectance of the preset body part and the reflectance of the reflector.
[0027] Additionally, the received settings may include at least one of the gamma, color temperature or light source profile of the other display.
[0028] Figures 1a to 1d are drawings for explaining remote treatment to help understand the present disclosure.
[0029] FIG. 2 is a block diagram showing the configuration of a display system according to one embodiment of the present disclosure.
[0030] FIG. 3 is a block diagram showing the configuration of a first display device according to one embodiment of the present disclosure.
[0031] FIG. 4 is a block diagram showing a detailed configuration of a first display device according to an embodiment of the present disclosure.
[0032] FIG. 5 is a block diagram showing the configuration of a second display device according to one embodiment of the present disclosure.
[0033] FIG. 6 is a drawing for explaining a method of using a reflector according to one embodiment of the present disclosure.
[0034] FIG. 7 is a drawing for explaining a method for manufacturing a reflector according to one embodiment of the present disclosure.
[0035] FIGS. 8 and 9 are drawings for explaining a method for manufacturing a reflector according to a pseudo-environment according to one embodiment of the present disclosure.
[0036] FIG. 10 is a drawing for explaining calibration according to one embodiment of the present disclosure.
[0037] FIG. 11 is a drawing for explaining the operation of a first display device and a second display device according to one embodiment of the present disclosure.
[0038] FIG. 12 is a flowchart for explaining a control method of a first display device according to one embodiment of the present disclosure.
[0039] FIG. 13 is a flowchart for explaining a control method of a first display device according to an embodiment of the present disclosure.
[0040] The purpose of the present disclosure is to provide display devices and control methods thereof that provide the same level of image to a doctor and a patient during a remote medical treatment process.
[0041] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.
[0042] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.
[0043] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.
[0044] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".
[0045] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.
[0046] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.
[0047] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).
[0048] Various embodiments of the present disclosure will be described in more detail with reference to the attached drawings below.
[0049] Figures 1a to 1d are drawings for explaining remote treatment to help understand the present disclosure.
[0050] As shown on the left side of Figure 1a, in a face-to-face treatment environment, the doctor and the patient meet directly, so the doctor can easily understand the patient's characteristics and the patient can also easily understand the doctor's opinion.
[0051] However, as shown on the right side of Figure 1a, in a remote medical treatment environment, the doctor's medical judgment or opinion may not be effectively conveyed to the patient because the patient communicates with the other party or medical data through different capture equipment and screens in different environments.
[0052] To solve this problem, as illustrated in Fig. 1b, the patient-side device can capture the patient's treatment area through a camera, correct color and brightness by utilizing the profile information of the patient-side device and the profile information of the doctor-side device to compensate for color distortion occurring in the external environment (illuminance) and the device, and transmit the corrected image to the doctor-side device.
[0053] This method corrects color distortion in images acquired in the patient's environment, but requires that the doctor's display be calibrated to have accurate expression and that the patient's image be free of distortion.
[0054] For this purpose, a reflector (color patch) made of the same material shared by the patient and the doctor can be used, and the doctor's device can store the RGB information of the measured values of the reflector and share it with the patient's device. Here, the reflector is also called a multi-color patch (MCP). The reflector is in the form of multiple colored tiles, and the doctor's device stores the digital code value when each color tile of the reflector is measured by a certain input device under a certain reference lighting environment. The digital code value can also be provided to the patient.
[0055] However, the above method is only a method for obtaining information without color distortion when acquiring medical (image) sample information from a patient from the doctor's perspective, and does not accurately reflect changes in the patient's environment (illumination) or the characteristics of the camera device. Furthermore, when a doctor changes the patient's image (to interpret the image) to provide a medical opinion, the patient does not receive the changed image. This can also be the case when a doctor makes a medical judgment based on an image acquired with hospital equipment. In particular, even if a hospital shares an image of a specific body part (such as an X-ray or MRI) with a patient in a general telemedicine environment, the patient may have difficulty understanding the doctor's judgment because the patient cannot see it with the same image quality as the doctor. In other words, as illustrated in Figure 1c, the doctor and the patient may not be able to see the same image quality. Here, the same image quality does not mean that it is completely identical, but rather that the difference is difficult to perceive with the naked eye.
[0056] Additionally, there may be cases where multiple doctors perform a collaborative examination, and even in this case, as illustrated in Fig. 1d, the multiple doctors and the patient may not be able to see the same level of images.
[0057] This can be a factor that reduces convenience in remote treatment, and below, a display system that provides the same level of image quality is described.
[0058] FIG. 2 is a block diagram showing the configuration of a display system (1000) according to one embodiment of the present disclosure.
[0059] As illustrated in FIG. 2, the display system (1000) may include a display device (100) and a display device (200).
[0060] The display device (100) may be a display device on the doctor's side, and the display device (200) may be a display device on the patient's side.
[0061] First, the display device (200) can obtain a photographed image including a reflector (color patch) through the camera of the display device (200), correct the photographed image based on the illumination around the display device (200) and the device characteristics of the camera of the display device (200), and transmit the corrected photographed image to the display device (100). Through this operation, a photographed image with distortion removed according to the surrounding environment and device characteristics can be provided to the display device (100).
[0062] When a photographed image including a reflector is received from the display device (200), the display device (100) can update the display setting values of the display device (100) based on digital values for a plurality of colors included in the photographed image and the reflector, and transmit the updated setting values to the display device (200).
[0063] The display device (100) can change the updated setting value based on the illumination around the display device (100) and display the captured image based on the changed setting value. Through this operation, it is possible to provide a captured image with distortion removed according to the surrounding environment.
[0064] When the display device (200) receives the setting values of the display included in the display device (100) from the display device (100), the display device (200) can display a corrected captured image based on the illuminance and the received setting values. That is, since the display device (200) controls the display of the display device (200) using the setting values of the display included in the display device (100), it can perform operations similar to those of the display device (100), and since illuminance is used in this process, it is possible to provide a captured image with distortion removed according to the surrounding environment.
[0065] Although two display devices are described in FIG. 2, this is not a limitation. For example, at least one of the two display devices may be implemented as both an electronic device and a display device. For example, the electronic device may communicate with the other device, provide settings and captured images, etc. to the display device, and the display device may display the captured images based on the received information.
[0066] For convenience of explanation, the display device (100) on the doctor's side is described as the first display device (100), and the display device (200) on the patient's side is described as the second display device (200).
[0067] FIG. 3 is a block diagram showing the configuration of a first display device (100) according to one embodiment of the present disclosure.
[0068] According to FIG. 3, the first display device (100) includes a communication interface (110), a memory (120), a display (130), and a processor (140).
[0069] The communication interface (110) is a configuration that performs communication with various types of external devices according to various types of communication methods. For example, the first display device (100) can perform communication with the second display device (200) through the communication interface (110).
[0070] The communication interface (110) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.
[0071] Wi-Fi and Bluetooth modules communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, connection information, such as the SSID and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received. Infrared communication modules use infrared data association (IrDA) technology, which wirelessly transmits data over short distances using infrared light, which lies between visible light and millimeter waves.
[0072] In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.
[0073] Alternatively, the communication interface (110) may include a wired communication interface such as HDMI, DP, Thunderbolt, USB, RGB, D-SUB, DVI, etc.
[0074] In addition, the communication interface (110) may include at least one of a LAN (Local Area Network) module, an Ethernet module, or a wired communication module that performs communication using a pair cable, a coaxial cable, or an optical fiber cable.
[0075] Memory (120) may refer to hardware that stores information such as data in an electrical or magnetic form so that a processor (140) or the like can access it. To this end, memory (120) may be implemented as at least one piece of hardware from among non-volatile memory, volatile memory, flash memory, hard disk drive (HDD), solid state drive (SSD), RAM, ROM, etc.
[0076] The memory (120) may store at least one instruction required for the operation of the first display device (100) or the processor (140). Here, the instruction is a unit of code that instructs the operation of the first display device (100) or the processor (140), and may be written in machine language, which is a language that a computer can understand. Alternatively, the memory (120) may store a plurality of instructions for performing specific tasks of the first display device (100) or the processor (140) as an instruction set.
[0077] The memory (120) may store data in bit or byte units that can represent characters, numbers, images, etc. For example, the memory (120) may store digital values for multiple colors included in a reflector.
[0078] The memory (120) is accessed by the processor (140), and reading / writing / modifying / deleting / updating instructions, instruction sets, or data can be performed by the processor (140).
[0079] The display (130) is a configuration that provides information and can be implemented as a variety of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, a PDP (Plasma Display Panel), etc. The display (130) may also include a driving circuit, a backlight unit, etc. that can be implemented as a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (130) may be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, etc.
[0080] The processor (140) controls the overall operation of the first display device (100). Specifically, the processor (140) is connected to each component of the first display device (100) and can control the overall operation of the first display device (100). For example, the processor (140) is connected to components such as a communication interface (110), a memory (120), a display (130), a sensor (not shown), and the like and can control the operation of the first display device (100).
[0081] The one or more processors may include one or more of a CPU, a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the first display device (100) and perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in the memory (120).
[0082] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).
[0083] One or more processors may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When one or more processors are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory, such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.
[0084] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.
[0085] In embodiments of the present disclosure, one or more processors may refer to a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, an NPU, a hardware accelerator, or a machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. However, for convenience of explanation, the operation of the first display device (100) is described below using the expression processor (140).
[0086] The processor (140) may receive a photographed image including a reflector (color patch) from the second display device (200) through the communication interface (110). For example, the processor (140) may receive a photographed image including a reflector and a user of the second display device (200) from the second display device (200) through the communication interface (110). Here, the reflector may include a plurality of colors, and the plurality of colors may include at least one color obtained by photographing a preset body part of the user of the second display device (200). The at least one color may be a color corrected based on the reflectivity of the preset body part and the reflectivity of the reflector. In addition, the plurality of colors may include a plurality of preset colors. For example, the plurality of preset colors may include at least one of R, G, B, C, M, Y, or White.
[0087] The processor (140) can update the setting value of the display (130) based on the digital values for the plurality of colors included in the captured image and the reflector stored in the memory (120). Here, the setting value can include at least one of the gamma, color temperature, or light source profile of the display (130). However, the present invention is not limited thereto, and the setting value can include any variety of information related to the display (130). For example, the setting value includes the gamma of the display (130) (ex: DICOM) and data in the form of an LUT having a plurality of seed points, which can be used to change the output gamma of the display of the second display device (200). Alternatively, the setting value includes the color gamut of the display (130) and the light source color spectrum profile (metamerism correction), which can be used to change the size and shape of the color gamut of the display of the second display device (200). Alternatively, the setting value includes the output brightness of the display (130) (e.g., the ratio of external light brightness), which can be used to change the output brightness of the display of the second display device (200). Alternatively, the setting value includes the color temperature of the display (130) (e.g., the color temperature of external light), which can be used to change the color temperature of the display of the second display device (200). Alternatively, the setting value includes a list of picture quality enhancement functions and operation constraints of the display (130), which can be used to control on / off of the picture quality enhancement function of the second display device (200).
[0088] The processor (140) can control the communication interface (110) to transmit the updated settings to the second display device (200).
[0089] The first display device (100) further includes a sensor, and the processor (140) can change an updated setting value based on the illuminance around the first display device (100) detected through the sensor, and control the display (130) to display a captured image based on the changed setting value. For example, the first display device (100) further includes an illuminance sensor, and the processor (140) can change an updated setting value based on the illuminance around the first display device (100) detected through the illuminance sensor, and control the display (130) to display a captured image based on the changed setting value. Through this operation, distortion caused by the surrounding environment can be reduced.
[0090] The first display device (100) further includes a user interface, and the processor (140) can control the communication interface (110) to change the changed settings and transmit the changed settings to the second display device (200) when a user command to change the changed settings is received through the user interface. Through this operation, even if the doctor changes the settings for diagnosis, the patient can also view the same screen and hear the doctor's diagnosis.
[0091] FIG. 4 is a block diagram showing a detailed configuration of a first display device (100) according to an embodiment of the present disclosure. The first display device (100) includes a communication interface (110), a memory (120), a display (130), and a processor (140). In addition, according to FIG. 4, the first display device (100) may further include a sensor (150), a user interface (160), a microphone (170), a speaker (180), and a camera (190). For components illustrated in FIG. 4 that overlap with those illustrated in FIG. 3, a detailed description thereof will be omitted.
[0092] The sensor (150) can generate detection data regarding external light. That is, the sensor (150) can detect external light and output the detected result as detection data. For example, the sensor (150) may be an illuminance sensor that is provided on the front of the first display device (100) to detect external light incident on the front of the first display device (100) and output the detected result as detection data.
[0093] The sensor (150) can generate sensing data by detecting at least one of the brightness and color temperature of external light. However, the present invention is not limited thereto, and the sensor (150) can sense various data related to external light. For example, the sensor (150) can also sense the incident direction of external light, etc. In this case, the sensor (150) can include a plurality of sensors provided on each surface of the first display device (100), and the incident direction of light can be determined based on the surface where the intensity of light incident on each sensor is the greatest.
[0094] The sensor (150) may be an individual optical sensor, a one-dimensional optical sensor, a two-dimensional optical sensor, or a composite optical sensor. The sensor (150) may be manufactured using a semiconductor material, and the semiconductor material may be selected based on the range of wavelengths used.
[0095] The user interface (160) may be implemented with buttons, a touch pad, a mouse, a keyboard, etc., or may be implemented with a touch screen capable of performing both display and manipulation input functions. Here, the buttons may be various types of buttons, such as mechanical buttons, touch pads, wheels, etc. formed on any area of the front, side, or back of the main body of the first display device (100).
[0096] The microphone (170) is configured to receive sound and convert it into an audio signal, which is an electrical signal. The microphone (170) is electrically connected to the processor (140) and can receive sound under the control of the processor (140).
[0097] For example, the microphone (170) may be formed on the upper side, front side, side side, etc. of the first display device (100). Alternatively, the microphone (170) may be provided in a separate device from the first display device (100), such as a remote control. In this case, the remote control, which is a separate device, may receive sound through the microphone (170) and provide an audio signal, which is an electrical signal corresponding to the received sound, to the first display device (100).
[0098] The microphone (170) may include various configurations such as a microphone that collects sound in analog form, an amplifier circuit that amplifies the collected sound, an A / D conversion circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.
[0099] Meanwhile, the microphone (170) may be implemented in the form of a sound sensor, and any method may be used as long as it has a configuration capable of collecting sound.
[0100] The speaker (180) is a component that outputs various audio data processed by the processor (140) as well as various notification sounds and voice messages.
[0101] The camera (190) is configured to capture still images or moving images. The camera (190) can capture still images at a specific point in time, but can also capture still images continuously.
[0102] The camera (190) can capture one direction of the first display device (100).
[0103] The camera (190) includes a lens, a shutter, an aperture, a solid-state image sensor, an AFE (Analog Front End), and a TG (Timing Generator). The shutter controls the time at which light reflected from a subject enters the camera (190), and the aperture mechanically increases or decreases the size of the opening through which light enters to control the amount of light incident on the lens. When the solid-state image sensor accumulates light reflected from a subject as a photocharge, the image generated by the photocharge is output as an electrical signal. The TG outputs a timing signal for reading out pixel data of the solid-state image sensor, and the AFE samples and digitizes the electrical signal output from the solid-state image sensor.
[0104] FIG. 5 is a block diagram showing the configuration of a second display device (200) according to one embodiment of the present disclosure.
[0105] According to FIG. 5, the second display device (200) includes a camera (210), a sensor (220), a communication interface (230), a display (240), and a processor (250).
[0106] The camera (210), sensor (220), communication interface (230), display (240), and processor (250) may have the same configuration as described in FIGS. 3 and 4, so redundant descriptions are omitted and only the operations are described.
[0107] The processor (250) can obtain a photographed image including a reflector (color patch) through the camera (210). For example, the processor (140) can obtain a photographed image including a reflector and a user of the second display device (200) through the camera (210). Here, the reflector includes a plurality of colors, and the plurality of colors may include at least one color obtained by photographing a preset body part of the user of the second display device (200). The at least one color may be a color corrected based on the reflectivity of the preset body part and the reflectivity of the reflector. In addition, the plurality of colors may include a plurality of preset colors. For example, the plurality of preset colors may include at least one of R, G, B, C, M, Y, or White.
[0108] The processor (250) can control the communication interface (230) to correct the captured image based on the illumination around the second display device (200) detected through the sensor (220) and the device characteristics of the camera (210), and transmit the corrected captured image to the first display device (100). Through this operation, the captured image with distortion removed due to the device characteristics of the second display device (200) and the surrounding environment can be provided to the first display device (100).
[0109] The processor (250) can receive the setting values of the display (130) included in the first display device (100) from the first display device (100) through the communication interface (230) and control the display (240) to display a corrected captured image based on the illuminance and the received setting values.
[0110] The processor (250) can receive the changed setting values of the display (130) from the first display device (100) via the communication interface (230) and control the display (240) to display a corrected captured image based on the illuminance and the changed setting values. Here, the received setting values can include at least one of the gamma, color temperature, or light source profile of the display (130). Accordingly, even if the doctor changes the display settings, the patient can view the screen at the same level.
[0111] The processor (250) can control the communication interface (230) to transmit the received setting values to a plurality of first display devices (100) excluding the first display device (100). Accordingly, even when multiple doctors collaborate, the multiple doctors and patients can view the same level of screen.
[0112] As described above, as the first display device (100) provides the settings to the second display device (200), the two devices can provide the same level of image.
[0113] Below, the operation of each device is described in more detail with reference to FIGS. 6 through 11. For convenience of explanation, FIGS. 6 through 11 describe individual embodiments. However, the individual embodiments of FIGS. 6 through 11 may be implemented in any combination.
[0114] FIG. 6 is a drawing for explaining a method of using a reflector according to one embodiment of the present disclosure.
[0115] Even if two remotely located display devices output the same image, the resulting image quality can vary significantly due to differences in the output devices, environmental conditions, and other factors. Specifically, when a doctor uses images of a patient's biological characteristics to make a medical judgment or provide a medical opinion, the images may be provided to a remote patient. Even in this case, even if digital images are shared, there can be significant differences between what the doctor sees and what the patient sees.
[0116] As a means of compensating for this, a reflector may be used, as illustrated in Fig. 6. For example, a patient may be provided with a reflector manufactured with special reflective properties by the hospital.
[0117] The reflector may contain preset colors such as RGB, CMY, Black, White, Gray, etc. Additionally, the reflector may further contain colors sampled from multiple skin tones (e.g., hair, eyes, etc.) on the patient's body when the patient visits the hospital.
[0118] Hospitals can capture multiple skin tones using a reference sensor under reference lighting, store them as reference values, and provide reflectors to patients.
[0119] A patient can calibrate the patient's second display device (200) remotely using a reflector. For example, the second display device (200) can adjust its light output based on illuminance (e.g., outdoor lighting, indoor lighting, etc.) and input devices, thereby enabling the patient to view images at the same level as the diagnostic images viewed by the doctor.
[0120] FIG. 7 is a drawing for explaining a method for manufacturing a reflector according to an embodiment of the present disclosure.
[0121] The patient may be provided with a reflector at the hospital. The reflector may include a preset color, as illustrated in Figure 7, or a color sampled from a specific body part of the patient.
[0122] Reflectors can be manufactured by taking images of a patient's body in a specific laboratory within a hospital. For example, reflectors can be manufactured by taking images of a patient's body using a reference device in a reference environment within a hospital.
[0123] Since the reflectance of a patient's body is different from the reflectance of the material of the reflector, a reflectance correction coefficient can be obtained and used to perform correction during the process of manufacturing the reflector. For example, when a patient and a reflector are photographed with a camera, the reflectance to light is different, so the color of the patient's body part photographed in the lab may be different from the color corresponding to the patient's body part photographed on the reflector. Accordingly, a reflectance correction coefficient can be calculated using a method such as skin spectral reflectance using the Kubelka-Munk model, and the reflector can be manufactured by correcting the color based on the reflectance correction coefficient. Here, the reflectance of the patient can be calculated by irradiating a reference light source and measuring the amount of reflected light, and the reflectance of the reflector can be recorded in the device as a measurement value obtained during the process or provided as a hardcopy.
[0124] FIGS. 8 and 9 are drawings for explaining a method for manufacturing a reflector according to a pseudo-environment according to one embodiment of the present disclosure.
[0125] When the environment in which the patient is photographed is different from the environment of the doctor during the manufacture of the reflector, calibration may be required. Accordingly, the first display device (100) can photograph the reflector placed in the doctor's environment to obtain an RGB image. In addition, the first display device (100) can detect the illuminance of the doctor's environment. The illuminance of the doctor's environment and the reference illuminance of the lab may be different, and in order to compensate for this, the first display device (100) can convert the RGB values (relative values depending on the camera or lighting environment) of the reflector obtained in the lab to XYZ, which can be converted by the ICC Profile of the camera used in the lab. Alternatively, if the ICC Profile is not provided, the XYZ measurement values may be provided separately from the RGB values of the reflector made from the patient's biological sample in the lab.
[0126] The measurement data converted to XYZ is data acquired under different lighting conditions and can be converted to chroma under reference lighting (Chromatic Adaptation Transform, CAT). For example, assuming that the reference lighting is D50, after XYZ-LMS conversion as in Fig. 8, Bradford Adaptation Transform (BAT) can be utilized as in Fig. 9. The data converted in this way is placed in the same space that can be compared with the data acquired in the reference environment. This XYZ data is the optical absolute value of the reflector using the patient's biological sample, and can be utilized to accurately calculate the correction coefficient of the image acquired by the input device (camera) in the patient environment in the future.
[0127] FIG. 10 is a drawing for explaining calibration according to one embodiment of the present disclosure.
[0128] The patient's second display device (200) provides various options for adjusting image quality (e.g., brightness, contrast ratio, color adjustment, etc.), so that differences in external environmental changes can be compensated for using these options.
[0129] First, the second display device (200) can obtain the illuminance excluding the influence of the second display device (200). Here, the illuminance may refer to the lighting environment information (offset) of the location where the patient is located. In addition, the second display device (200) can identify the influence on the patient environment when the display (240) is driven at maximum brightness. In this investigation process, the pattern output may be R, G, B, C, M, Y, White, etc. The second display device (200) can calculate the range (max range) that can be adjusted as the output light source of the display (240) compared to the offset through the illuminance sensor.
[0130] Thereafter, the second display device (200) can obtain a photographed image including a reflector and provide the photographed image to the first display device (100). Here, the photographed image may be continuous or may be a video.
[0131] The first display device (100) can obtain a correction coefficient by comparing the RGB values of the captured image with the RGB values of the reflector acquired at the time of manufacturing, as illustrated in FIG. 10. However, the present invention is not limited thereto, and the second display device (200) can also obtain the correction coefficient. For example, the second display device (200) can obtain the captured image and request the RGB values of the reflector acquired at the time of manufacturing from the first display device (100). The first display device (100) can provide the RGB values of the reflector acquired at the time of manufacturing to the second display device (200), and the second display device (200) can obtain a correction coefficient by comparing the RGB values of the captured image with the RGB values of the reflector acquired at the time of manufacturing.
[0132] Meanwhile, the second display device (200) can also adjust the output light of the display (240) to simulate a lighting environment most similar to a lab environment. In this case, the second display device (200) may not perform chroma correction through BAT, and has the advantage of reducing errors resulting from excessively large illumination environment conversion. For example, the second display device (200) can calculate display output light characteristics that make the diagonal element values of the A matrix in the BAT matrix formula of FIG. 9 ideally close to a cubic square matrix in which all 1s are 1s and all remaining elements are 0s. This process can be performed by repeatedly changing the display light output according to the characteristics of the A matrix. For example, the display light output can be changed within a certain range by utilizing an offset and a max range. The stop condition can be when the squared sum of the diagonal elements is less than a preset threshold. After calibrating the display (240) in the above manner, the second display device (200) can calibrate the final display (240) using the measured reflector value.
[0133] FIG. 11 is a diagram illustrating the operation of a first display device (100) and a second display device (200) according to an embodiment of the present disclosure. FIG. 11 illustrates the operation after a doctor provides a reflector to a patient.
[0134] First, the patient's second display device (200) can control the light output of the display (240) and the capture of the camera (210) based on the device characteristics and illuminance of the camera (210).
[0135] The second display device (200) can obtain a photographed image including a patient and a reflector through a camera (210), correct the photographed image based on the device characteristics and illuminance of the camera (210), and transmit the corrected photographed image to the first display device (100).
[0136] When a photographed image is received from a second display device (200), the first display device (100) can update the setting values of the display (130) based on the digital values for a plurality of colors included in the photographed image and the reflector, and transmit the updated setting values to the second display device (200). Here, the setting values can include various information about the setting values of the display (130).
[0137] Thereafter, the first display device (100) can change the updated setting value based on the illumination around the first display device (100) and display the captured image based on the changed setting value.
[0138] Additionally, when a user command to change the changed setting value is received, the first display device (100) can change the changed setting value again and transmit the changed setting value to the second display device (200).
[0139] For example, although the gamma of the display (240) of the second display device (200) is defined as 2.4, when a doctor shows a diagnostic screen to a patient and explains the basis of the diagnosis, a curve characteristic in the form of DICOM Gamma may be required. In this case, the first display device (100) may provide the diagnostic screen to the second display device (200) and provide the setting value of the display (130). The second display device (200) may parse the received information to read the gamma form as a single value (ex: DICOM) or read a LUT in a predefined DICOM form to adjust the gamma of the display (240) to a form most similar thereto.
[0140] FIG. 12 is a flowchart for explaining a control method of a first display device according to one embodiment of the present disclosure.
[0141] First, a photographed image including a reflector (color patch) is received from a second display device (S1210). Then, the display settings are updated based on the digital values for multiple colors included in the photographed image and the reflector (S1220). Then, the updated settings are transmitted to the second display device (S1230).
[0142] Additionally, the method may further include a step of changing the updated setting value based on the illumination around the first display device and a step of displaying the captured image based on the changed setting value.
[0143] And, when a user command to change the changed setting value is received, the step of changing the changed setting value again and the step of transmitting the changed setting value to the second display device may be further included.
[0144] Additionally, the plurality of colors may include at least one color obtained by photographing a preset body part of the user of the second display device.
[0145] And, at least one color may be a corrected color based on the reflectance of the preset body part and the reflectance of the reflector.
[0146] Additionally, the settings may include at least one of the display's gamma, color temperature, or light source profile.
[0147] FIG. 13 is a flowchart for explaining a control method of a first display device according to an embodiment of the present disclosure.
[0148] First, a photographed image including a reflector (color patch) is acquired through a camera included in a second display device (S1310). Then, the photographed image is corrected based on the illumination around the second display device and the device characteristics of the camera (S1320). Then, the corrected photographed image is transmitted to the first display device (S1330). Then, the setting values of the first display included in the first display device are received from the first display device (S1340). Then, the display included in the second display device is controlled to display the corrected photographed image based on the illumination and the received setting values (S1350).
[0149] Additionally, the method may further include a step of receiving a changed setting value of the first display from the first display device and a step of controlling the display to display a corrected captured image based on the illuminance and the changed setting value.
[0150] And, it may further include a step of transmitting the received setting value to a plurality of first display devices excluding the first display device.
[0151] Additionally, the plurality of colors included in the reflector may include at least one color obtained by photographing a preset body part of a user of the display device.
[0152] And, at least one color may be a corrected color based on the reflectance of the preset body part and the reflectance of the reflector.
[0153] Additionally, the received settings may include at least one of the gamma, color temperature or light source profile of the other display.
[0154] According to various embodiments of the present disclosure as described above, as the first display device provides settings to the second display device, the two devices can provide images of the same level.
[0155] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.
[0156] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.
[0157] Furthermore, according to one embodiment of the present disclosure, the various embodiments described above may be implemented in a computer-readable recording medium or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software. Each software may perform one or more functions and operations described herein.
[0158] Meanwhile, computer instructions for performing processing operations of a device according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations in the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.
[0159] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.
[0160] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.
Claims
1. In the display device, communication interface; memory; display; and comprising one or more processors connected to the communication interface, the memory and the display and controlling the display device; The above processor, Receive a photographed image including a reflector (color patch) from another display device through the above communication interface, Updating the setting value of the display based on the digital values for the plurality of colors included in the above photographed image and the reflector stored in the above memory, A display device that controls the communication interface to transmit the updated settings to the other display device.
2. In paragraph 1, including sensors; The above processor, Change the updated setting value based on the illuminance around the display device detected through the sensor, A display device that controls the display to display the photographed image based on the changed setting value.
3. In paragraph 2, further including a user interface; The above processor, When a user command to re-change the changed setting value is received through the user interface, the changed setting value is re-changed, A display device that controls the communication interface to transmit the changed setting value to the other display device.
4. In paragraph 1, The above multiple colors are, A display device comprising at least one color obtained by photographing a preset body part of a user of the above display device.
5. In paragraph 4, At least one of the above colors, A display device, the color of which is corrected based on the reflectivity of the preset body part and the reflectivity of the reflector.
6. In paragraph 1, The above settings are, A display device comprising at least one of a gamma, a color temperature or a light source profile of the display.
7. In a method for controlling a display device, A step of receiving a photographed image including a reflector (color patch) from another display device; A step of updating the setting value of the display based on the digital values for the plurality of colors included in the photographed image and the reflector; and A control method, comprising: a step of transmitting the updated setting values to the other display device.
8. In paragraph 7, A step of changing the updated setting value based on the illuminance around the display device; and A control method further comprising: a step of displaying the photographed image based on the changed setting value.
9. In paragraph 8, When a user command to re-change the above changed setting value is received, a step of re-changing the above changed setting value; and A control method further comprising a step of transmitting the changed setting value to the other display device.
10. In paragraph 7, The above multiple colors are, A control method comprising at least one color obtained by photographing a preset body part of a user of the above display device.
11. In paragraph 10, At least one of the above colors, A control method, wherein the color is corrected based on the reflectivity of the preset body part and the reflectivity of the reflector.
12. In paragraph 7, The above settings are, A control method comprising at least one of a gamma, a color temperature or a light source profile of the display.
Citation Information
Patent Citations
Color correction method
JP3763880B2
Apparatus for processing tongue image and health care service apparatus using tongue image
KR1020040059312A
User terminal and control method of thereof
KR1020150074459A
A system for controllings robots and detecting of abnomal robots
KR1020220099835A
A method of learning mentoring service that is using Social Network Service and Video Telecommunication Service
KR1020240132634A
