Display device and method for controlling same

Abstract

The present invention relates to a display device for identifying biometric information and a method for controlling same. The present invention comprises: controlling, on the basis of image information received from an external device by means of a communication unit, the detection of a feature point of a patient and an omega shape; determining whether the feature point of the patient is detected and whether the omega shape is detected; determining, in response to the detection of the feature point of the patient and the omega shape, whether a detection region for detecting biometric information about the patient is identified; and adjusting, on the basis of whether the detection region is identified, at least one display parameter among the received image information and a display unit displaying the received image information.

Description

Display device and control method thereof

[0001] The disclosed invention relates to a display device for correcting an image and identifying a predetermined area in the corrected image, and a control method thereof.

[0002] Feature points from human images are commonly used in various technical fields. In particular, features extracted from human facial images are useful in facial recognition technology, where they can be used to identify objects within the image. These features can also be utilized in the medical field.

[0003] As science and technology advance, technologies (Telehealth, Telemedicine, or Home-Care) are being developed that can diagnose a patient's health status or abnormalities by analyzing only the patient's images from a distance without the doctor having to meet the patient directly.

[0004] In order to make a diagnosis through a patient's image, it is important to accurately detect characteristic points in the image.

[0005] However, when acquiring patient images, the lighting conditions of the patient's room can lead to images of the patient being darkened, or the patient can be backlit depending on the relative position of the camera and lighting. This makes it difficult to detect feature points in patient images, and it also hinders the smooth identification of areas for vital signs detection.

[0006] Additionally, when the patient's skin tone is dark or the face is tilted, there was difficulty identifying areas in the patient's image for detecting vital signs. This made it difficult to properly diagnose and diagnose patients using the patient's image during remote consultations.

[0007] One aspect of the disclosed invention provides a display device and a control method thereof for identifying a predetermined detection area by correcting the brightness of a screen or image.

[0008] One aspect includes a display unit; a communication unit for performing communication with an external device; at least one memory for storing one or more commands; and at least one processor for executing one or more commands. One or more commands for causing a display device to operate when the at least one processor is executed include displaying an image on the display unit based on image information received through the communication unit, identifying one or more feature points and an omega shape of a patient included in the image, identifying whether the image includes a predetermined detection area related to biometric information of the patient based on the one or more feature points and the omega shape, and adjusting a display parameter of at least one of the display unit and the image based on whether the predetermined detection area is identified.

[0009] One or more instructions for causing the display device to operate when at least one processor is executed include: identifying a central axis of the image; identifying a left side and a right side of the patient in the image based on the central axis, the omega shape and the identification of one or more feature points; identifying whether a left feature point of the left side of the patient is included in the one or more feature points; identifying whether a right feature point of the right side of the patient is included in the one or more feature points; and mapping a feature point between the left feature point and the right feature point included in the one or more feature points to the opposite side of the image based on the identification that only one of the left feature point and the right feature point is included in the one or more feature points.

[0010] One or more instructions for causing the display device to operate when at least one processor is executed include controlling the communication unit to transmit environmental guidance information to an external device based on identifying that the image does not include at least one of one or more feature points and an omega shape. The environmental guidance information includes information related to adjusting the patient's posture and position.

[0011] One or more instructions for causing the display device to operate when at least one processor is executed include adjusting display parameters of an image based on illuminance information received through the communication unit and predetermined reference illuminance information.

[0012] One or more instructions for causing the display device to operate when at least one processor is executed include controlling the communication unit to transmit environmental guide information for illuminance adjustment to an external device based on illuminance information received through the communication unit and predetermined reference illuminance information.

[0013] One or more instructions for causing the display device to operate when at least one processor is executed performs image preprocessing to uniformize the brightness of the image based on the image being identified as having non-uniform brightness.

[0014] At least one memory stores the patient's baseline biometric information. One or more commands for causing the display device to operate when at least one processor is executed include controlling the communication unit to detect the patient's biometric information in a predetermined detection area based on the diagnosis request information received through the communication unit, acquire diagnostic result information based on the detected biometric information and the stored baseline biometric information, and transmit the acquired diagnostic result information to an external device.

[0015] One or more instructions for causing the display device to operate when at least one processor is executed include: identifying whether the image information is image information acquired in a backlit environment based on brightness information of the image information, based on an omega shape being identified and one or more feature points not being identified; adjusting a display parameter of the image based on the image information being identified as image information acquired in a backlit environment; and performing preprocessing of the image to uniformize the brightness of the image.

[0016] One or more instructions for causing a display device to operate when at least one processor is executed include: acquiring illuminance information from an external device through a communication unit based on an omega shape being identified and one or more feature points not being identified; identifying whether illuminance of a space in which the external device is provided is less than a first reference illuminance and greater than a second reference illuminance based on the acquired illuminance information and first and second reference illuminance information; identifying that received image information is image information acquired in a backlit environment if the illuminance of the space is less than the first reference illuminance and greater than the second reference illuminance; and transmitting environment guide information to the external device if the illuminance of the space is less than the second reference illuminance. The second reference illuminance is an illuminance lower than the first reference illuminance. The environment information includes information related to adjusting the illuminance of the space.

[0017] The one or more feature points include one or more feature points of the patient's face, and the one or more feature points of the face include the ends of the eyes, the ends of the mouth, and the bridge of the nose. The predetermined detection area includes the orbital area of ​​the face and the infraorbital area of ​​the face.

[0018] One or more instructions for causing the display device to operate when at least one processor is executed include obtaining a skin reflection coefficient of a patient based on first image information acquired in a lighting environment and second image information acquired in a non-illuminated environment received from an external device, and transmitting illuminance information corresponding to the skin reflection coefficient to the external device.

[0019] A method for controlling a display device according to another aspect includes identifying one or more characteristic points and an omega shape of a patient based on image information received from an external device through a communication unit, identifying whether the image information includes a detection area related to the patient's biometric information based on the identification of the one or more characteristic points and the identification of the omega shape, and adjusting a display parameter of at least one of the image information or a display unit of the display device based on the identification of the detection area.

[0020] A method for controlling a display device according to another aspect further includes transmitting first environment guide information for guiding position and posture adjustment of a patient to an external device based on the identification of one or more feature points and an omega shape in image information, identifying an environment of a space in which the external device is provided as a backlight environment based on the identification of the omega shape and the identification of one or more feature points in the image information, adjusting display parameters of a display unit, performing preprocessing of image information to uniformize the brightness of an image displayed on the display unit, and transmitting second environment guide information for illuminance adjustment to the external device based on illuminance information and reference illuminance information received from the external device through a communication unit.

[0021] A method for controlling a display device according to another aspect further includes identifying a central axis of image information, identifying the left side of a patient and the right side of a patient in the image information based on the central axis, identifying that a left feature point of the left side of the patient is included in one or more feature points, identifying that a right feature point of the right side of the patient is included in one or more feature points, identifying a left feature point and a right feature point included in the one or more feature points as detected feature points based on the identification of only one of the left feature point and the right feature point in the one or more feature points, identifying a left feature point and a right feature point not included in the one or more feature points as missing feature points, generating a missing feature point by mapping the detected feature points to a side corresponding to the missing feature point among the left side of the patient and the right side of the patient, and identifying a detection area based on the detected feature point and the generated feature point.

[0022] A method for controlling a display device according to another aspect further includes obtaining a skin reflection coefficient of a patient based on first image information acquired in a lighting environment and second image information acquired in a non-illuminated environment received from an external device, and transmitting illuminance information corresponding to the obtained skin reflection coefficient to the external device.

[0023] According to one or more embodiments of the disclosed invention, the present invention can easily and accurately detect body features by correcting an image acquired in a low-light environment or a backlit environment.

[0024] One or more embodiments of the present invention can easily and accurately detect feature points of a body by correcting an acquired image when the acquired image is in a state of uneven illumination.

[0025] One or more embodiments of the present invention can easily and accurately detect body features in an image obtained by adjusting the brightness of a screen or the brightness of lighting of a display device when the skin color of the person is dark and the reflectivity is low, and can easily identify a detection area of ​​a biosignal, which is a predetermined detection area.

[0026] One or more embodiments of the present invention can easily identify a detection area of ​​a biosignal by detecting an omega shape when a feature point of a body part is not detected in an image or when a person's head posture changes.

[0027] In this way, one or more embodiments of the present invention can improve the diagnostic performance and accuracy of a patient's diagnosis by identifying a detection area of ​​an identified biosignal.

[0028] One or more aspects of the embodiments of the present invention will be explained from the drawings and the detailed description of the invention.

[0029] FIG. 1 is a configuration diagram of a display system including first and second display devices according to one embodiment.

[0030] Figure 2 is a control configuration diagram of a first display device according to one embodiment.

[0031] Figure 3 is an example of detection of a feature point of a first display device according to one embodiment.

[0032] Figure 4 is an example of detection of an omega shape of a first display device according to one embodiment.

[0033] FIGS. 5A, 5B and 5C are exemplary diagrams of generation of feature points and identification of a predetermined detection area of ​​a first display device according to one embodiment.

[0034] Figure 6 is a control configuration diagram of a second display device according to one embodiment.

[0035] FIG. 7a and FIG. 7b are examples of obtaining a skin reflection coefficient of a second display device according to one embodiment.

[0036] Figure 8 is a control flowchart of a first display device according to one embodiment.

[0037] Figure 9 is a control configuration diagram of a display device according to another embodiment.

[0038] FIG. 10a and FIG. 10b are examples of detection of body feature points of a display device according to another embodiment.

[0039] It should be understood that the various embodiments and terms used in this document are not intended to limit the technical features described in this document to specific embodiments, but rather to encompass various modifications, equivalents, or alternatives of the embodiments.

[0040] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0041] The singular form of a noun corresponding to an item may include one or more items, unless the context clearly indicates otherwise.

[0042] In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof.

[0043] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not limit the components in any other respect (e.g., importance or order).

[0044] When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0045] The terms “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in this document, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0046] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0047] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0048] The term “and / or” includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0049] The operating principle and embodiments of the present invention will be described with reference to the attached drawings below.

[0050] FIG. 1 is an exemplary diagram of a display system (1) including first and second display devices according to one embodiment.

[0051] The display system (1) can be implemented in a cloud manner through an Internet network built online.

[0052] The display system (1) may include two or more display devices (100, 200) provided in different spaces.

[0053] The display system (1) may further include a user device (2) and may further include a server (3).

[0054] Hereinafter, a display system including two display devices will be described. To distinguish between the two display devices, they will be referred to as a first display device (100) and a second display device (200).

[0055] The first display device (100) may be a device installed in a medical institution, and the second display device (200) may be a device installed in a home or office.

[0056] The first and second display devices (100, 200) may include a display unit that displays an image sensor and a screen. The first and second display devices (100, 200) may further include a microphone and a speaker. The first and second display devices (100, 200) may further include lighting.

[0057] The first display device (100) may be a television or a user device.

[0058] The first display device (100) may include a communication module capable of communicating with a user device (2) or a server (3), a user interface for receiving user input or outputting information to a user, at least one processor for controlling the operation of the first display device (100), and at least one memory in which a program for controlling the operation of the first display device (100) is stored.

[0059] The second display device (200) may be a home appliance or a user device.

[0060] The second display device (200) may include a communication module capable of communicating with another home appliance (10), a user device (2), or a server (3), a user interface for receiving user input or outputting information to a user, at least one processor for controlling the operation of the second display device (200), and at least one memory storing a program for controlling the operation of the second display device (200).

[0061] Other home appliances (10) may include, but are not limited to, at least one of a refrigerator (11), a dishwasher (12), an electric range (13), an electric oven (14), an air conditioner (15), a clothes manager (16), a washing machine (17), a dryer (18), and a microwave oven (19) as illustrated, and may include various types of home appliances such as, for example, a cleaning robot, a vacuum cleaner, and a television.

[0062] In addition, the aforementioned home appliances are merely examples, and in addition to the aforementioned home appliances, other home appliances, user devices (2), or devices that are connected to a server (3) and can perform the operations described below may be included in the second display device (200) according to one embodiment.

[0063] The server (3) may include a communication module capable of communicating with another server, the first and second display devices (100, 200), or the user device (2), at least one processor capable of processing data received from another server, the first and second display devices (100, 200), or the user device (2), and at least one memory capable of storing a program for processing data and / or processed data.

[0064] These servers (3) can be implemented as various computing devices such as workstations, clouds, data drives, and data stations. The servers (3) can be implemented as one or more servers that are physically or logically separated based on function, detailed configuration of function, or data, and can transmit and receive data and process the transmitted and received data through communication between each server.

[0065] The server (3) can perform functions such as managing user accounts, registering first and second display devices (100, 200) by linking them to user accounts, and managing or controlling the registered first and second display devices (100, 200). For example, a user can access the server (3) through a user device (2) and create a user account. The user account can be identified by an ID and password set by the user. Here, the user can be a doctor linked to the first display device (100) and a patient linked to the second display device (200).

[0066] The server (3) can register the first and second display devices (100, 200) to a user account according to a set procedure. For example, the server (3) can register, manage, and control the first and second display devices (100, 200) by linking the identification information (e.g., serial number or MAC address) of the first and second display devices (100, 200) to the user account.

[0067] The user device (2) may include a communication module capable of communicating with the first display device (100), the second display device (200) or the server (3), a user interface for receiving user input or outputting information to the user, at least one processor for controlling the operation of the user device (2), and at least one memory in which a program for controlling the operation of the user device (2) is stored.

[0068] The user device (2) may be carried by the user or placed in the user's home, office, or medical institution. The user device (2) may include, but is not limited to, a personal computer, a terminal, a portable telephone, a smart phone, a handheld device, a wearable device, etc.

[0069] The memory of the user device (2) may store a program, i.e., an application, for controlling the first display device (100) or the second display device (200). The application may be sold installed on the user device (2) or downloaded and installed from an external server.

[0070] A user can access a server (3) by executing an application installed on a user device (2), create a user account, and perform communication with the server (3) based on the logged-in user account to register a first display device (100) or a second display device (200).

[0071] For example, when the first display device (100) is operated so that the first display device (100) can be connected to the server (3) according to the procedure guided by the application installed on the user device (2), the first display device (100) can be registered in the user account by registering the identification information (e.g., serial number or MAC address) of the first display device (100) in the corresponding user account on the server (3).

[0072] When the second display device (200) is operated so that the second display device (200) can be connected to the server (3) according to the procedure guided by the application installed on the user device (2), the second display device (200) can be registered in the user account by registering the identification information (e.g., serial number or MAC address) of the second display device (200) in the corresponding user account on the server (3).

[0073] A user can control the first display device (100) using an application installed on the user device (2). For example, when a user logs into a user account using an application installed on the user device (2), the first display device (100) registered to the user account appears, and when a control command for the first display device (100) is input, the control command can be transmitted to the first display device (100) via the server (3).

[0074] A user can control a second display device (200) using an application installed on the user device (2). For example, when a user logs into a user account using an application installed on the user device (2), a second display device (200) registered to the user account appears, and when a control command for the second display device (200) is input, the control command can be transmitted to the second display device (200) via the server (3).

[0075] A network can include both wired and wireless networks. Wired networks include cable networks or telephone networks, while wireless networks can include any network that transmits and receives signals via radio waves. Wired and wireless networks can be interconnected.

[0076] A network may include a wide area network (WAN) such as the Internet, a local area network (LAN) formed around an access point (AP), and a short-range wireless network that does not use an access point (AP). Short-range wireless networks may include, but are not limited to, Bluetooth (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), Z-Wave, etc.

[0077] An access point (AP) can connect the first and second display devices (100, 200) or the user device (2) to a wide area network (WAN) to which the server (3) is connected. The first and second display devices (100, 200) or the user device (2) can be connected to the server (3) via the wide area network (WAN).

[0078] The access point (AP) can communicate with the second display device (200) or the user device (2) using wireless communication such as Wi-Fi (IEEE 802.11), Bluetooth (IEEE 802.15.1), or Zigbee (IEEE 802.15.4), and can connect to a wide area network (WAN) using wired communication, but is not limited thereto.

[0079] According to various embodiments, the first and second display devices (100, 200) may be directly connected to the user device (2) or the server (3) without going through an access point (AP).

[0080] The first and second display devices (100, 200) can be connected to a user device (2) or a server (3) via a long-distance wireless network or a short-distance wireless network.

[0081] For example, the first and second display devices (100, 200) can be connected to the user device (2) via a short-range wireless network (e.g., Wi-Fi Direct).

[0082] As another example, the first and second display devices (100, 200) may be connected to a user device (2) or a server (3) via a wide area network (WAN) using a long-distance wireless network (e.g., a cellular communication module).

[0083] As another example, the first and second display devices (100, 200) can be connected to a wide area network (WAN) using wired communication and connected to a user device (2) or a server (3) through the wide area network (WAN).

[0084] If the second display device (200) can connect to a wide area network (WAN) using wired communication, it may also function as a connection relay. Accordingly, the second display device (200) can connect other home appliances (10) to the wide area network (WAN) to which the server (3) is connected.

[0085] The second display device (200) can transmit information about its operation or status to the user device (2) or the server (3) via a network. For example, the second display device (200) can transmit information about its operation or status to the user device (2) or the server (3) when a request is received from the server (3), when a specific event occurs in the second display device (200), or periodically or in real time.

[0086] When information about the operation or status is received from the first display device (100) or the second display device (200), the server (3) updates the stored information about the operation or status of the first display device (100) or the second display device (200), and transmits the updated information about the operation and status of the first display device (100) or the second display device (200) to the user device (2) via the network. Here, updating information may include various operations in which existing information is changed, such as an operation of adding new information to existing information, an operation of replacing existing information with new information, etc.

[0087] The first display device (100) or the second display device (200) can obtain various information from the user device (2) or the server (3) and provide the obtained information to the user. For example, the first display device (100) or the second display device (200) can obtain information related to the function of the first display device (100) or the second display device (200) (e.g., non-face-to-face treatment, etc.) and information on various environmental information (e.g., illuminance, etc.) from the server (3) and output the obtained information through the user interface.

[0088] The first display device (100) or the second display device (200) can operate according to a control command received from the user device (2) or the server (3). For example, if the first display device (100) or the second display device (200) has obtained prior approval from the user so that it can operate according to the control command of the server (3) even without user input, the first display device (100) or the second display device (200) can operate according to the control command received from the server (3). Here, the control command received from the server (3) may include, but is not limited to, a control command input by the user through the user device (2) or a control command based on a preset condition.

[0089] The user device (2) can transmit information about the user to the first display device (100), the second display device (200), or the server (3) via the communication module. For example, the user device (2) can transmit information about the user's location, the user's health status, the user's preferences, the user's schedule, etc. to the server (3). The user device (2) can transmit information about the user to the server (3) with the user's prior consent.

[0090] The first display device (100) or the second display device (200), the user device (2), or the server (3) can determine a control command using technology such as artificial intelligence. For example, the server (3) can receive information about the operation or status of the first display device (100) or the second display device (200) or receive information about the user of the user device (2), process the information using technology such as artificial intelligence, and transmit the processing result or control command to the first display device (100), the second display device (200), or the user device (2) based on the processing result.

[0091] Hereinafter, first and second display devices according to various embodiments will be specifically described with reference to the drawings.

[0092] FIG. 2 is a control configuration diagram of a first display device according to one embodiment, which is described with reference to FIGS. 3, 4, 5a, 5b, and 5c.

[0093] The first display device includes a first input interface (110), a first image sensor (120), a first light sensor (130), a first communication unit (140), a first processor (150), a first memory (151), a first display unit (160), and may further include a first microphone (170) and a first speaker (180), and may further include a first light (190).

[0094] The first input interface (110) receives user input and transmits the received user input to the first processor (150). Here, the user may be a medical professional, such as a doctor or nurse.

[0095] User input may further include a power on command and a power off command.

[0096] User input may include on and off commands for non-face-to-face treatment.

[0097] The user input may include a first command for collecting biometric information of a patient in a healthy state and a second command for collecting biometric information of a patient seeking treatment.

[0098] The biometric information of a patient in a normal state can be used as reference biometric information for diagnosing the patient.

[0099] The first input interface (110) can receive patient identification information, receive a patient registration command, and receive treatment information and diagnosis result information.

[0100] The first input interface (110) can receive an on command, an off command, and a sensing command of the first image sensor (120).

[0101] The sensing command of the first image sensor (120) may be either the first command or the second command.

[0102] The first input interface (110) can receive an on command and an off command of the first light (190).

[0103] The first input interface (110) may be provided physically separated from the first display unit (160).

[0104] The first input interface (110) may include hardware devices such as various buttons, switches, pedals, keyboards, mice, trackballs, various levers, handles, sticks, etc.

[0105] Additionally, the first input interface (110) may include a GUI (Graphical User Interface), i.e., a software device, such as a touch pad. The touch pad may be implemented as a touch screen panel (TSP) and may form a mutual layer structure with the display unit.

[0106] The first image sensor (120) can acquire an image of a person in the vicinity of the first display device (100), convert image information about the acquired image of the person into an electrical image signal, and transmit the converted image signal to the first processor (150). The person in the vicinity of the first display device (100) may be a doctor or a patient.

[0107] The first image sensor (120) may include a CCD or CMOS image sensor.

[0108] The first image sensor (120) may include a camera. The first image sensor (120) may also include a three-dimensional space recognition sensor such as a TOF (Time Of Flight) camera, a stereo camera, etc.

[0109] The first illumination sensor (130) can detect the illumination around the first image sensor (120) and transmit illumination information about the detected illumination to the first processor (150).

[0110] The first image sensor (120) and the first illumination sensor (130) may be physically separated from the first display unit (160).

[0111] The first communication unit (140) may include one or more components that enable communication between components within the first display device (100).

[0112] The first communication unit (140) may include one or more components that enable communication with an external device, for example, at least one of a short-range communication module, a wired communication module, and a wireless communication module. Here, the external device may include a second display device (200).

[0113] The first communication unit (140) may include both a wired network and a wireless network.

[0114] The first processor (150) oversees control related to the operation of the first display device (100). There may be one or more first processors (150). That is, there may be at least one first processor (150).

[0115] The first processor (150) performs control related to the operation of the first display device (100) using data stored in the first memory (151).

[0116] When a patient registration command is received from the first input interface (110), the first processor (150) can control the first display unit (160) to display a patient information recording screen for recording patient information.

[0117] The first processor (150) can control the first memory (151) to obtain patient information based on a user input received from the first input interface (110) while the patient information recording screen is displayed on the first display unit (160), and to store the obtained patient information.

[0118] Patient information is patient identification information for identifying the patient, and may include the patient's date of birth, patient's name, patient's address, patient's contact information, and patient's code information.

[0119] When a first command is received from a first input interface (110), the first processor (150) controls the activation of the first image sensor (120), acquires the patient's reference biometric information based on the patient's image information acquired by the activated first image sensor (120), and controls the first memory (151) to store the acquired patient's reference biometric information.

[0120] The patient image acquired in response to receiving the first command may be an image of the patient's body in a healthy state. The body may include at least one of the face, arms, legs, and torso.

[0121] Reference biometric information may include reference color information of the body and reference biometric signal information of a predetermined detection area. For example, the biometric signal may include tremors due to pulse or tremors due to convulsions.

[0122] When the first processor (150) obtains the patient's baseline biometric information, it can control the first lighting (190) based on preset environmental information.

[0123] The preset environment may include an environment in which the first light (190) in the first space where the first display device is provided is turned off, an environment in which the first light (190) is turned on, or an environment with a preset illuminance. The first space where the first display device (100) is provided may be a space in a medical institution where patients are treated.

[0124] The first processor (150) can control the brightness of the first light (190) so that the first space becomes an environment with a preset illuminance.

[0125] When the first processor (150) obtains the patient's baseline biometric information, it is also possible to control the brightness of the first display unit (160) so that the indoor space's illuminance becomes a preset illuminance.

[0126] The first processor (150) controls the first lighting (190) based on preset environmental information when a second command is received from the first input interface (110), controls the activation of the first image sensor (120), and then acquires diagnostic biometric information based on the patient's image information acquired by the activated first image sensor (120). In this case, the first processor (150) can compare the reference biometric information with the diagnostic biometric information to acquire diagnostic information and control the first display unit (160) to display the acquired diagnostic information.

[0127] When the first processor (150) receives treatment request information and patient identification information from the second display device (200), it can control the first display unit (160) to display the received treatment request information and patient identification information.

[0128] When a treatment approval command is received through the first input interface (110), the first processor (150) can transmit the received treatment approval command and guidance information for non-face-to-face treatment to the second display device (200).

[0129] When the first processor (150) receives image information of a patient from the second display device (200), it can control the first display unit (160) to display the received image information of the patient.

[0130] The first processor (150) can control the first display unit (160) to display the patient's identification information and the patient's diagnosis history information along with the received patient's image information.

[0131] The first processor (150) can control the first display unit (160) to obtain diagnostic biometric information based on the received patient image information and display the obtained diagnostic biometric information.

[0132] The first processor (150) can obtain reference biometric information corresponding to the patient's identification information received from the second display device (200) among the information stored in the first memory (151), compare the received diagnostic biometric information with the reference biometric information to obtain diagnostic result information, control the first display unit (160) to display the obtained diagnostic result information, and transmit the diagnostic result information to the second display device (200).

[0133] The first processor (150) can obtain diagnostic result information based on diagnostic criteria information stored in the first memory (151).

[0134] When displaying the user's image information through the first display unit (160), the first processor (150) can adjust the display parameters of the image displayed on the first display unit and can adjust the display parameters of the first display unit.

[0135] For example, the display parameters may include brightness, luminance, saturation, contrast, and color of the first display unit, and may include brightness, luminance, saturation, contrast, and color of the image.

[0136] The display parameters of the first display unit may be output parameters of the first display unit.

[0137] When displaying the patient's image information through the first display unit (160), the first processor (150) can identify the image information of a color patch in the patient's image information received from the second display device (200), obtain the color information of the color patch based on the image information of the identified color patch, obtain color correction information based on the color information of the acquired color patch and the color information of the pre-stored color patch, and correct the patient's image information based on the acquired color correction information.

[0138] The first processor (150) can transmit the corrected patient image information to the second display device (200). This allows the patient and the doctor to view the same patient image.

[0139] The first processor (150) determines whether the illumination is uneven based on the received patient image information, and if it is determined that the illumination within the image is uneven, it can correct the received patient image information so that the illumination within the image becomes uniform.

[0140] The first processor (150) can check the illumination information received from the second display device (200) and correct the brightness of the image information of the patient received based on the checked illumination information. The illumination information received from the second display device (200) may be illumination information of a second space where the patient exists together with the second display device (200), and may be illumination information detected by the second illumination sensor (230).

[0141] The first processor (150) can also transmit environmental guide information for guiding re-shooting to the second display device (200) based on the illumination information and the patient's image information received from the second display device (200).

[0142] The first processor (150) can perform a diagnosis based on the patient's image information. Here, the patient's image information for diagnosis may be original image information transmitted from the second display device (200) or image information whose color has been corrected using a color patch.

[0143] More specifically, when performing a diagnosis, the first processor (150) can detect body features and determine whether the body is symmetrical on both sides. Hereinafter, the patient's face will be used as an example to explain the patient's body.

[0144] The first processor (150) detects feature points for each face landmark based on a face image and performs labeling on a plurality of detected feature points. The landmarks may include eyes, nose, mouth, eyebrows, and chin.

[0145] That is, multiple feature points can include location information and labeling information.

[0146] As shown in FIG. 3, the first processor (150) can label the right end point of the right eye as 1-1, the left end point of the right eye as 1-2, the left end point of the left eye as 2-1, the right end point of the left eye as 2-2, the right end point of the mouth as 3-1, the left end point of the mouth as 3-2, and the nose, etc. as 4.

[0147] The first processor (150) can determine whether all landmark feature points on the left and right sides of the face image have been detected based on the location information and labeling information for each feature point.

[0148] For example, the first processor (150) can determine whether the right end point 1-1 of the right eye of the right face image, the left end point 1-2 of the right eye, and the right end point 3-1 of the mouth have all been detected, and can determine whether the left end point 2-1 of the left eye of the right face image, the right end point 2-2 of the left eye, and the left end point 3-2 of the mouth have all been detected.

[0149] The first processor (150) can determine whether all left feature points of the left face image have been detected based on labeling information of left feature points of the pre-stored left face image, labeling information of right feature points of the pre-stored right face image, and labeling information of a plurality of detected feature points, and can determine whether all right feature points of the right face image have been detected.

[0150] As illustrated in FIG. 3, if the first processor (150) determines that all feature points of the left and right facial images have been detected in the patient's image, it can identify the detection area (vs1, vs2) of the biosignal based on the feature points of the left and right facial images in the image.

[0151] The detection area of ​​the biosignal (vs1, vs2) may be a predetermined detection area.

[0152] The first processor (150) determines whether the illuminance of the second space is equal to or greater than the first reference illuminance based on the illuminance information received from the second display device (200) and the first reference illuminance information, and if it is determined that the illuminance of the second space is equal to or greater than the first reference illuminance and if it is determined that all feature points of the left and right facial images have been detected in the image received from the second display device (200), it is also possible to identify a detection area of ​​a biosignal based on the feature points of the left and right facial images in the image.

[0153] The second space is a space where a second display device (200) is provided, and may be a space where a patient lives.

[0154] If the first processor (150) determines that at least one feature point among the feature points of the left and right facial images is not detected, it can determine whether or not an omega shape (Ω-Shape) is detected based on the facial image.

[0155] The first processor (150) can also detect feature points of a facial image and determine whether an omega shape (Ω-Shape) is detected based on the facial image.

[0156] The first processor (150) can determine whether the facial image is symmetrical left and right based on the detected omega shape.

[0157] If the first processor (150) determines that the left and right sides of the facial image are not symmetrical, it can transmit environmental guide information that guides the patient's posture and position adjustment to the second display device (200).

[0158] Determining that the left and right sides of the facial image are not symmetrical may include determining that the patient is not looking at the second image sensor (220).

[0159] Determining that the patient is not looking at the second image sensor (220) may include determining whether the second image sensor and the patient's face are facing each other.

[0160] For example, the first processor (150) may obtain a difference value between the size of the left face image and the size of the right face image based on the central axis, and if the obtained size difference value is greater than a certain value, it may be determined that the patient is not looking at the second image sensor (220).

[0161] The first processor (150) can determine whether an omega shape has been detected, and if it is determined that an omega shape has not been detected, it can transmit environmental guide information that guides the patient's posture and position adjustment to the second display device (200).

[0162] As shown in Fig. 4, detecting the Ω-Shape is to detect the outline from the tip of the patient's head (i.e., crown) to the shoulder line in the patient's face image.

[0163] The first processor (150) can set or identify a point such as the nose at the center of the patient's head (crown) recognized in the image as a center pole, and can divide the face image into a left face image and a right face image based on the set center axis.

[0164] The first processor (150) can also set the center axis (center pole) from the center of the patient's head (i.e., crown) to the center of the jaw.

[0165] The center of the patient's head can be the center of the omega shape, the point where the derivative becomes zero.

[0166] As illustrated in FIG. 5A, if the first processor (150) determines that an omega shape has been detected, it divides the face into left and right, and determines whether all feature points of the divided left and right face images exist. At this time, the first processor (150) can determine whether at least one feature point of either the left feature point of the left face image or the right feature point of the right face image has not been detected.

[0167] The first processor (150) can check the location information of the feature points of the face image of the side on which all feature points are detected among the left and right faces, and map the feature points to the face image of the side on which at least one feature point is not detected (i.e., the side with the missing feature point) based on the location information of the checked feature points and the central axis.

[0168] As illustrated in FIG. 5b, if the first processor (150) determines that at least one feature point among the left feature points of the left face image is not detected, the first processor (150) can identify the right feature point of the right face image corresponding to the landmark of the missing feature point, that is, the feature point that is not detected (i.e., the detected feature point), and map the right feature points of the right face image to the left face based on the set central axis. That is, the first processor (150) can map the identified right feature point of the right face image to the left face image based on the central axis.

[0169] As illustrated in FIG. 5c, the first processor (150) generates feature points of the left and right facial images in response to the detection of the omega shape and the mapping of feature points of the left and right facial images, and can identify a detection area (vs1, vs2) of a biosignal based on the feature points of the generated left and right facial images.

[0170] Generating feature points of the left and right facial images in the image is to generate feature points that are not detected through the image received from the second display device.

[0171] The first processor (150) can check the biosignal in the detection area of ​​the identified biosignal and determine the patient's health status based on the checked biosignal.

[0172] The first processor (150) can control the first display unit to diagnose the patient's condition based on the confirmed bio-signal and display the diagnosed diagnosis result information, and can transmit the diagnosis result information to the second display device.

[0173] The detection area of ​​the biosignal may be an area selected by the user (i.e., the doctor).

[0174] The detection area of ​​the biosignal may be the affected area.

[0175] When the body is a face, the detection area of ​​the biosignal may include the orbital area and the infraorbital area under the eyes of the left and right facial images.

[0176] If the body is a face, the detection area of ​​the biosignal may include the pupil.

[0177] The first processor (150) can also determine the patient's condition, i.e., the improved condition and the worsened condition, based on the diagnosis result information and the diagnosis history information.

[0178] The first processor (150) can also determine whether the environment of the second space is a low-illuminance environment or an extremely low-illuminance environment based on the illuminance information received from the second display device and the first and second reference illuminance information.

[0179] That is, the first processor (150) determines whether the illuminance of the second space is less than the first reference illuminance based on the illuminance information received from the second display device (200), and if the illuminance of the second space is determined to be less than the first reference illuminance, the environment of the second space is determined to be a low-illuminance environment.

[0180] The first processor (150) determines that the illuminance of the second space is less than the first reference illuminance and greater than the second reference illuminance based on the illuminance information received from the second display device (200), and if the illuminance of the second space is determined to be less than the first reference illuminance and greater than the second reference illuminance, the environment of the second space can be determined to be a low-illuminance environment.

[0181] The first processor (150) determines whether the illuminance of the second space is less than the second reference illuminance based on the illuminance information received from the second display device (200), and if the illuminance of the second space is determined to be less than the second reference illuminance, determines that the environment of the second space is an extremely low illuminance environment, and transmits environment guide information that guides the brightness adjustment of the second lighting to the second display device (200). The second reference illuminance may be an illuminance lower than the first reference illuminance.

[0182] The first processor (150) can obtain brightness information of an image based on image information received from the second display device (200) and can also determine whether the environment of the second space is a low-light environment based on the brightness information of the obtained image.

[0183] The first processor (150) can obtain brightness information between the body area and the background area from the image received from the second display device (200) and determine whether the image was obtained in a backlit environment based on the obtained brightness information.

[0184] If the first processor (150) determines that the patient's image was acquired in a backlit or low-light environment, it can perform preprocessing to improve the brightness of the face area in the image using an uneven illumination image enhancement technology or a low-light image enhancement technology.

[0185] If the first processor (150) determines that the environment of the second space is a low-light environment, it can transmit environment guide information that guides brightness adjustment of the second display unit or brightness adjustment of the second light (190) to the second display device (200).

[0186] If the environment of the second space is determined to be a low-light environment or a backlit environment, the first processor (150) can control brightness adjustment of the first display unit or perform image preprocessing for uneven lighting.

[0187] To adjust the brightness of the first display unit, the first processor (150) can control the current or voltage of the first display unit.

[0188] If the first processor (150) determines that the environment of the second space is a low-light and backlit environment, it can transmit environment guide information that guides the position adjustment of the second light (290) or the position adjustment of the second image sensor (220) to the second display device (200).

[0189] If the first processor (150) determines that the omega shape is not detected and that the feature points of the left and right facial images are not detected, it may determine that patient recognition has failed and transmit environmental guide information that guides the patient's position and posture adjustment to the second display device (200).

[0190] If the first processor (150) determines that the environment of the second space is a low-light environment, that an omega shape is detected, and that feature points of the left and right facial images are not detected, it can determine that the environment is a low-light backlight environment and transmit environment guide information that guides brightness adjustment of the second lighting of the second space to the second display device (200).

[0191] If the first processor (150) determines that the environment of the second space is a low-light environment, an omega shape is detected, and only the feature points of the face image of one side (the first side) among the feature points of the left and right face images are all detected, the first processor corrects the brightness of the face image, and generates feature points of the face image of the second side based on the corrected image information, the feature points of the face image of the first side, and the information on the central axis of the omega shape, and then identifies a detection area of ​​a biosignal based on the feature points of the left and right face images.

[0192] If the first processor (150) determines that the illuminance of the second space is higher than the first reference illuminance, that an omega shape is detected, and that feature points of the left and right facial images are not detected, it determines that it is a backlight environment and corrects the brightness of the facial image, and based on the corrected image information, it re-detects feature points of the left and right facial images, and based on the re-detected feature points of the left and right facial images, it can identify a detection area of ​​a biosignal.

[0193] That is, if the first processor (150) determines that the illuminance of the second space is higher than the first reference illuminance and that it is a backlight environment, it can perform preprocessing to improve the brightness of the face area in the image using an illuminance unevenness image enhancement technology or a low-light image enhancement technology.

[0194] If the first processor (150) determines that the illuminance of the second space is equal to or greater than the first reference illuminance, that an omega shape is detected, and that only the feature points of the facial image of one side (the first side) among the feature points of the left and right facial images are all detected, the first processor (150) generates feature points of the facial image of the second side based on the feature points of the facial image of the first side and the information of the central axis of the omega shape, and then identifies a detection area of ​​a biosignal based on the feature points of the left and right facial images.

[0195] The first processor (150) can obtain a skin reflection coefficient of a patient based on first image information in a lit environment and second image information in a non-lit environment from a second display device, and transmit necessary illuminance information corresponding to the obtained skin reflection coefficient to the second display device.

[0196] The lighting environment includes an environment in which at least one of the second lighting and the second display unit of the second space is turned on, and the non-lighting environment includes an environment in which the second lighting and the second display unit of the second space are turned off.

[0197] The necessary illumination information transmitted to the second display device may include illumination information of the second display unit and illumination information of the second display unit and the second light source.

[0198] The first processor (150) can transmit voice information received through the first microphone (170) to the second display device (200) and control the first speaker (180) to output the voice information received from the second display device (200).

[0199] The first memory (151) can store first and second reference illuminance information.

[0200] The first memory (151) can store the patient's identification information, store the patient's baseline biometric information, and store the patient's treatment information and diagnosis result information.

[0201] The patient's diagnosis result information can be stored in the first memory (151) as the patient's treatment history information by date.

[0202] The first memory (151) can further store diagnostic criteria information for each type of disease for diagnosis.

[0203] Diagnostic criteria information may include color information, pulse information, and convulsion information by disease type, and may include location information and curvature information of characteristic points by body part.

[0204] The first memory (151) and the first processor (150) may be implemented as separate chips. Alternatively, the first memory (151) and the first processor (150) may be implemented as a single chip.

[0205] The first display unit (160) can output information corresponding to the control command of the first processor (150) as a video.

[0206] The first display unit (160) can display an image with display parameters corresponding to the control command of the first processor (150).

[0207] The first display unit (160) can display a patient image and patient diagnosis information. The patient image can include an image of the patient's body. The patient image can include an image of the patient's face.

[0208] The first display unit (160) can display user input in response to a control command of the first processor (150).

[0209] The first display unit (160) can also display the user's image. The user's image may include a doctor's image.

[0210] The first display unit (160) can adjust the brightness of the patient's image and display a correction image for the patient's image with uneven illumination.

[0211] The first display unit (160) can display a detection area of ​​a biosignal in a patient's image and can display diagnosis result information.

[0212] The first display unit (160) can display an image corresponding to a user command. For example, the image corresponding to a user command may include a document image containing medical information, a photographic image, or a video.

[0213] The first display unit (160) may be provided as a cathode ray tube (CRT), a digital light processing (DLP) panel, a plasma display panel, a liquid crystal display (LCD) panel, an electroluminescence (EL) panel, an electrophoretic display (EPD) panel, an electrochromic display (ECD) panel, a light emitting diode (LED) panel, or an organic light emitting diode (OLED) panel, but is not limited thereto.

[0214] The first display unit (160) may be composed of a touch screen panel (TSP) that forms a mutual layer structure with a touch pad.

[0215] The first microphone (170) can receive the user's voice and transmit voice information about the received voice to the first processor (150).

[0216] The first microphone (170) and the first image sensor (120) may be included in the first input interface (110).

[0217] The first speaker (180) can output sound information according to the control command of the first processor (150). The sound information can include the patient's voice information.

[0218] The first light (190) may be provided around the first image sensor (120).

[0219] The first light (190) can emit light in response to a control command from the first processor (150).

[0220] The first light (190) can emit light with a brightness corresponding to the control command of the first processor (150).

[0221] The first light (190) can be provided physically separated from the first display unit (160).

[0222] The first light (190) may be a backlight unit provided in the first display unit (160).

[0223] At least one component may be added or deleted to correspond to the performance of the components of the first display device illustrated in FIG. 2. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be altered to correspond to the performance or structure of the first display device.

[0224] Meanwhile, each component illustrated in FIG. 2 refers to software and / or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0225] Figure 6 is a control configuration diagram of a second display device according to one embodiment.

[0226] The second display device (200) includes a second input interface (210), a second image sensor (220), a second light sensor (230), a second communication unit (240), a second processor (250), a second memory (251), a second display unit (260), and may further include a second microphone (270) and a second speaker (280), and may further include a second light (290).

[0227] The second input interface (210) receives user input and transmits the received user input to the second processor (250).

[0228] User input may further include a power on command and a power off command.

[0229] User input may include on and off commands for non-face-to-face treatment.

[0230] The second input interface (210) can receive patient identification information.

[0231] The second input interface (210) can receive an on command, an off command, and a sensing command of the second image sensor (220).

[0232] The sensing command of the second image sensor (220) may be a patient's shooting command.

[0233] The second input interface (210) can receive on and off commands of the second light (290).

[0234] The second input interface (210) may be provided physically separated from the second display unit (260).

[0235] The second input interface (210) may include hardware devices such as various buttons, switches, pedals, keyboards, mice, trackballs, various levers, handles, sticks, etc.

[0236] Additionally, the second input interface (210) may include a GUI (Graphical User Interface), i.e., a software device, such as a touch pad. The touch pad may be implemented as a touch screen panel (TSP) and may form a mutual layer structure with the display unit.

[0237] The second image sensor (220) can acquire an image of a person in the vicinity of the second display device (200), convert image information about the acquired image of the person into an electrical image signal, and transmit the converted image signal to the second processor (250).

[0238] When the second image sensor (220) acquires an image of a person, it can also acquire an image of a color patch provided by a medical institution.

[0239] That is, when taking an image of the body using the second image sensor (220), the patient can take an image of the body while placing color patches around the patient.

[0240] The second image sensor (220) may include a CCD or CMOS image sensor.

[0241] The second image sensor (220) may include a camera. The second image sensor (220) may also include a three-dimensional space recognition sensor such as a TOF (Time Of Flight) camera, a stereo camera, etc.

[0242] The second illumination sensor (230) can detect the illumination around the second image sensor (220) and transmit illumination information about the detected illumination to the second processor (250).

[0243] The second image sensor (220) and the second contrast sensor (230) may be physically separated from the second display unit (260).

[0244] The second communication unit (240) may include one or more components that enable communication between components within the second display device (200).

[0245] The second communication unit (240) may include one or more components that enable communication with an external device, for example, at least one of a short-range communication module, a wired communication module, and a wireless communication module. Here, the external device may include the first display device (100).

[0246] The second communication unit (240) may include both wired networks and wireless networks.

[0247] The second processor (250) oversees control related to the operation of the second display device (200). There may be one or more second processors (250). That is, there may be at least one second processor (250).

[0248] The second processor (250) performs control related to the operation of the second display device (200) using data stored in the second memory (251).

[0249] When treatment request information is received through the second input interface (210), the second processor (250) can transmit the received treatment request information to the first display device (100).

[0250] When the second processor (250) receives the identification information of the medical institution and the identification information of the doctor through the second input interface (210), it can transmit the treatment request information based on the received identification information of the medical institution and the identification information of the doctor.

[0251] When a treatment approval command and guidance information for non-face-to-face treatment are received from the first display device (100), the second processor (250) can control the second display unit (260) to display the received guidance information. The second processor (250) can also control the second speaker to output the guidance information.

[0252] The second processor (250) can transmit image information received from the second image sensor (220) to the first display device (100).

[0253] The second processor (250) can transmit the illumination information received from the second illumination sensor (230) to the first display device (100).

[0254] The second processor (250) can control the brightness of the second light (290) in response to a control command received from the first display device (100).

[0255] The second processor (250) can control the second display unit (260) to display a corrected image in response to a control command received from the first display device (100).

[0256] The corrected image may be an image in which brightness, illuminance unevenness, or color of the image acquired by the second image sensor (220) are corrected.

[0257] The second processor (250) can also control the brightness of the second display unit (260) in response to a control command received from the first display device (100).

[0258] As illustrated in Fig. 7a, the second processor (250) controls the second display unit (260) to turn on, and controls the operation of the second image sensor to acquire a first image of the patient while the second display unit (260) is turned on. The second processor (250) can acquire first illuminance information of the second space detected by the second illuminance sensor (230) when acquiring the first image. In this case, the second processor (250) can acquire the first image in an environment of external illuminance and the lighting of the second display unit.

[0259] Turning on the second display unit (260) means turning on the backlight unit provided in the second display unit.

[0260] As illustrated in FIG. 7b, the second processor (250) can control the turning off of the second display unit (260) and control the operation of the second image sensor (220) to acquire a second image of the patient while the second display unit (260) is turned off. The second processor (250) can acquire second illuminance information of the second space detected by the second illuminance sensor (230) when acquiring the second image. In this case, the second processor (250) can acquire the second image in an environment where only external illuminance exists.

[0261] Controlling the second display unit (260) to turn off the backlight unit provided in the second display unit.

[0262] When a second light exists in the second space, the second processor (250) can control the second display unit and the second light to be turned on, and control the operation of the second image sensor to acquire a first image of the patient while the second display unit and the second light are turned on, and control the operation of the second image sensor (220) to control the second display unit and the second light to be turned off, and control the operation of the second image sensor to acquire a second image of the patient while the second display unit and the second light are turned off.

[0263] Turning on the second display and the second light means turning on the second display and the second light. And turning off the second display and the second light means turning off the second display and the second light.

[0264] The second processor (250) can obtain brightness information of the first image and obtain brightness information of the second image.

[0265] The second processor (250) can obtain average brightness information of a face area in the first image and obtain average brightness information of a face area in the second image.

[0266] The second processor (250) obtains the patient's skin reflection coefficient based on the average brightness information (F1) of the first image, the average brightness information (F2) of the second image, the first illuminance information (Lux1) and the second illuminance information (Lux2), obtains the necessary illuminance required by the second display unit based on the obtained patient's skin reflection coefficient, and controls the brightness of the second display unit based on the obtained necessary illuminance, or can control the brightness of the second lighting.

[0267] The second processor (250) can control the brightness of the second display unit if the acquired skin reflection coefficient of the patient is less than the reference skin reflection coefficient.

[0268] The second processor (250) can also control the brightness of the second display unit based on the difference between the acquired skin reflection coefficient of the patient and the reference skin reflection coefficient if the acquired skin reflection coefficient of the patient is less than the reference skin reflection coefficient.

[0269] The required amount of illumination required by the second display device may include the amount of illumination of the second display unit.

[0270] It is possible to obtain average brightness information of a face area in a first image and average brightness information of a face area in a second image.

[0271] The second processor (250) can obtain the average brightness change amount (F1-F2) of the face area in the image, obtain the change amount of the light sensor value (Lux1-Lux2), and obtain the illuminance change ratio (Lux1 / Lux2) and the average brightness change ratio of the face area (F1 / F2).

[0272] The second processor (250) can determine that the skin tone is darker when the average brightness change ratio (F1 / F2) of the facial area is significantly smaller than the illuminance change ratio (Lux1 / Lux2).

[0273] The second processor (250) can obtain a skin reflection coefficient (R=F1 / F2), obtain a required illuminance based on the obtained skin reflection coefficient, and control the brightness of the second display unit (260) based on the obtained required illuminance.

[0274] Required illuminance (L_a-L_c) = ((Max-Min) × R) × α, (maximum compensation illuminance when α = 1.0)

[0275] Current illuminance: L_c

[0276] Operating illuminance of the second display unit of the second display device: L_a

[0277] Max: Maximum compensation intensity

[0278] Min: Minimum compensation intensity

[0279] The second processor (250) can obtain image information on a color patch provided by a medical institution and transmit the image information of the obtained color patch to the first display device (100).

[0280] The image acquired by the second image sensor may include both an image of the patient's body and an image of the color patch.

[0281] The color patch provided by the medical institution may be a printed piece with multiple colors or an image file transmitted to the patient's user device.

[0282] The second processor (250) can control the second display unit (260) to display the corrected image received from the first display device (100).

[0283] The second processor (250) can control the second display unit (260) to display environmental guide information received from the first display device (100).

[0284] The second processor (250) can also control the second display unit based on display parameters received from the first display device (100).

[0285] The second processor (250) can transmit the image information re-acquired by the second image sensor (220) after displaying the environmental guide information to the first display device (100). In this case, the second processor (250) can also transmit the illumination information detected by the second illumination sensor to the first display device (100).

[0286] The second processor (250) can display diagnostic result information received from the first display device (100).

[0287] The second processor (250) can transmit voice information received through the second microphone (270) to the first display device (100) and control the second speaker (280) to output the voice information received from the first display device (100).

[0288] The second memory (251) can store user (i.e. patient) identification information.

[0289] The second memory (251) can store identification information of a medical institution and identification information of a doctor, and can store identification information of the first display device.

[0290] The identification information of the first display device may be the identification information of the first display device provided at a medical institution registered for face-to-face treatment.

[0291] The second memory (251) can store brightness information of the second display unit for acquiring an image of the patient in an optimal environment and can store brightness information of the second light.

[0292] Brightness information may include illuminance information.

[0293] The second memory (251) can store diagnostic result information.

[0294] The second memory (251) and the second processor (250) may be implemented as separate chips. Alternatively, the second memory (251) and the second processor (250) may be implemented as a single chip.

[0295] The second display unit (260) can output information corresponding to the control command of the second processor (250) as a video.

[0296] The second display unit (260) can display an image of the patient. The image of the patient may include an image of the patient's body. The image of the patient may include an image of the patient's face.

[0297] The second display unit (260) can also display an image of a user (i.e., a doctor) using the first display device (100).

[0298] The second display unit (260) can display user input in response to a control command of the second processor (250).

[0299] The second display unit (260) can display an image according to display parameters corresponding to the control command of the second processor (250).

[0300] The second display unit (260) may be provided as a cathode ray tube (CRT), a digital light processing (DLP) panel, a plasma display panel, a liquid crystal display (LCD) panel, an electroluminescence (EL) panel, an electrophoretic display (EPD) panel, an electrochromic display (ECD) panel, a light emitting diode (LED) panel, or an organic light emitting diode (OLED) panel, but is not limited thereto.

[0301] The second display unit (260) may be composed of a touch screen panel (TSP) that forms a mutual layer structure with the touch pad.

[0302] The second microphone (270) can receive the user's voice and transmit voice information about the received voice to the second processor (250).

[0303] The second microphone (270) and the second image sensor (220) may be included in the second input interface (210).

[0304] The second speaker (280) can output sound information according to the control command of the second processor (250). The sound information can include the doctor's voice information.

[0305] The second light (290) may be provided around the second image sensor (220).

[0306] The second light (290) can emit light in response to a control command from the second processor (250). The second light (290) can adjust the brightness of the light emitted in response to the control command from the second processor (250).

[0307] The second light (290) can be provided physically separated from the second display unit (260).

[0308] The second light (290) may include a backlight unit provided in the second display unit (260).

[0309] At least one component may be added or deleted to correspond to the performance of the components of the second display device illustrated in FIG. 6. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be altered to correspond to the performance or structure of the second display device.

[0310] Meanwhile, each component illustrated in FIG. 6 refers to software and / or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0311] Figure 8 is a control flowchart of a first display device according to one embodiment.

[0312] When the first display device receives treatment request information and patient identification information from the second display device (200), the first display device can display the received treatment request information and patient identification information through the first display unit (160).

[0313] When a treatment approval command is received through the first input interface (110), the first display device can transmit the received treatment approval command and guidance information for non-face-to-face treatment to the second display device (200).

[0314] When the first display device receives image information of a patient from the second display device (200), the first display device can display the received image information of the patient through the first display unit (160).

[0315] When the first display device displays the patient's image information through the first display unit (160), the first display device can identify the image information of the color patch in the patient's image information received from the second display device (200), acquire the color information of the color patch based on the image information of the identified color patch, acquire color correction information based on the color information of the acquired color patch and the color information of the pre-stored color patch, correct the patient's image information based on the acquired color correction information, and display the corrected image information.

[0316] The first display device can transmit the corrected image information to the second display device (200). In this case, the second display device (200) can display the corrected image information through the second display unit.

[0317] The first display device acquires (401) the illuminance information transmitted from the second display device (200), and determines (402) whether the illuminance of the second space is equal to or greater than the first reference illuminance based on the acquired illuminance information and the first reference illuminance information. Here, the second space may be a space where a patient is located and where remote, non-face-to-face treatment is performed.

[0318] If the first display device determines that the illuminance of the second space is lower than the first reference illuminance, the first display device can transmit environmental guide information guiding the brightness adjustment of the second light (290) to the second display device (403). That is, the first display device can transmit environmental guide information guiding the lighting of the second light (290) or environmental guide information guiding the brightness adjustment of the second light so that the brightness of the second light increases to the second display device.

[0319] The first display device can control the first display unit so that the brightness of the first display unit becomes brighter.

[0320] If the first display device determines that the illuminance of the second space is lower than the first reference illuminance, it is also possible to transmit environmental guide information that guides brightness adjustment of the second display unit (260) to the second display device (200).

[0321] The first display device can recognize a detection area of ​​a biosignal based on the re-received facial image when the facial image is re-received from the second display device in response to the transmission of environmental guide information. In this case, the first display device can also obtain the illumination information transmitted from the second display device (200) and determine whether the illumination of the second space is equal to or greater than the first reference illumination based on the obtained illumination information and the first reference illumination information.

[0322] The first display device can acquire a face image by recognizing the face based on image information when it is determined that the illuminance of the two spaces is greater than the first reference illuminance (404).

[0323] The first display device can detect an omega shape based on the acquired facial image. At this time, the first display device can determine whether an omega shape has been detected in the facial image (405).

[0324] If the first display device determines that an omega shape has not been detected, the first display device may transmit environmental guidance information to the second display device, guiding the patient's posture and position adjustment (406). In this case, the second display device may display environmental guidance information guiding the patient to look at the second image sensor and take a picture of the body or face.

[0325] The first display device performs detection of feature points of the facial image when it is determined that an omega shape has been detected (407).

[0326] When the first display device determines that feature points of a facial image have been detected, it labels the detected feature points and can obtain a bridge of the nose point based on the labeled information.

[0327] The first display device identifies the crown in the omega shape and connects the identified crown and the bridge of the nose to establish the central axis.

[0328] The first display device can distinguish between a left face image and a right face image based on a set central axis, and determine whether all feature points of the left face image and the right face image have been detected based on labeling information of feature points of the left face image stored in advance, labeling information of feature points of the right face image stored in advance, labeling information of feature points of the detected left face image, and labeling information of feature points of the detected right face image (408).

[0329] If the first display device determines that neither the left face image nor the right face image feature points are detected, it checks the face on which neither the left face image feature points nor the right face image feature points are detected.

[0330] If the first display device determines that neither the feature points of the left face image nor the feature points of the right face image are detected, it determines that the environment of the second space is a backlit environment, adjusts the brightness of the first display unit, and preprocesses the face image to improve uneven lighting (410).

[0331] The first display device can re-detect feature points of the facial image based on the image-processed facial image.

[0332] The first display device may determine that the face in the face image is asymmetrical if it determines that at least one of the feature points of the face image on the first side among the left and right face images is not detected. In this case, the first display device may generate all feature points of the face image on the first side by mapping (411) the feature points of the face image on the second side (i.e., the side with the detected feature point) to the face image on the first side (the side corresponding to the missing feature point) based on the central axis of the omega shape.

[0333] The first display device can identify a predetermined detection area based on feature points of the left and right facial images (412). Here, the predetermined detection area may be an area for detecting a biosignal. In other words, the first display device can identify a detection area for a biosignal in the left and right facial images.

[0334] The first display device can recognize biosignal information in a biosignal detection area and diagnose the patient's health status based on the recognized biosignal information and reference biosignal information (413).

[0335] The first display device can generate diagnostic result information for the diagnostic result, display the generated diagnostic result through the first display unit, and transmit it to the second display device (414).

[0336] The first display device can also identify a detection area for a biosignal based on a facial image received when the illuminance of the second space is below the first reference illuminance. The control sequence of the first display device in this case is briefly described.

[0337] The first display device determines that the environment of the second space is a low-light environment if the illuminance of the second space is lower than the first reference illuminance.

[0338] The first display device can obtain a face image by recognizing the face based on image information received from the second display device.

[0339] The first display device can detect an omega shape based on the acquired facial image and detect feature points of the facial image. At this time, the first display device can determine whether an omega shape has been detected in the facial image.

[0340] If the first display device determines that the omega shape is not detected and that no feature points of the facial image are detected, the first display device determines that the environment of the second space is a low-light, backlit environment, and transmits environment guide information guiding the position adjustment of the second light (290) or the position adjustment of the second image sensor (220) to the second display device (200). At this time, the second display device can display the environment guide information guiding the position adjustment of the second light (290) or the position adjustment of the second image sensor (220) through the second display unit.

[0341] When the first display device determines that an omega shape is detected and that all feature points of the first-side face image among the feature points of the left and right face images are detected, the first display device corrects the brightness of the face image, and generates feature points of the second-side face image based on the corrected image information, the feature points of the first-side face image, and the information on the central axis of the omega shape, and then identifies a detection area of ​​a biosignal of the left and right face images based on the feature points of the first-side face image and the feature points of the second-side face image.

[0342] When the first display device determines that the omega shape is detected in a state where the illuminance of the second space is less than the first reference illuminance and that all feature points of the left and right facial images are detected, the first display device can identify the detection area of ​​the biosignal of the left and right facial images based on the feature points of the left and right facial images.

[0343] The first display device determines whether the illuminance of the second space is less than the second reference illuminance if the illuminance of the second space is less than the first reference illuminance, and if the illuminance of the second space is determined to be less than the second reference illuminance, the first display device determines that the environment of the second space is an extremely low illuminance environment, and transmits environment guide information that guides brightness adjustment of the second lighting to the second display device (200). The second reference illuminance may be an illuminance lower than the first reference illuminance.

[0344] Figure 9 is a control configuration diagram of a display device according to another embodiment.

[0345] A display device according to another embodiment may be a user device.

[0346] User devices may be carried by the user or placed in the user's home or office. User devices may include, but are not limited to, personal computers, terminals, laptops, tablet PCs, portable telephones, smart phones, handheld devices, wearable devices, and the like.

[0347] The display device can run an application for a medical diagnostic platform that provides services for checking, diagnosing, and managing the user's health status. The application herein may be an application for providing medical diagnostic services.

[0348] The display device can control the downloading, installation (setup) and execution of an application for providing medical diagnosis services, and can provide a screen corresponding to the execution of the application to the user.

[0349] The display device (300) can communicate with a server for medical diagnosis and management, a web server for a website, an application server providing medical diagnosis services, and a database server.

[0350] The display device (300) can communicate with at least one of a home appliance, a massage device, a massage device, and a home medical device.

[0351] A display device (300) according to another embodiment includes an input interface (310), an image sensor (320), a light sensor (330), a communication unit (340), a processor (350), a memory (351), a display unit (360), and may further include a microphone (370) and a speaker (380), and may further include lighting (390).

[0352] The input interface (310) receives user input and transmits the received user input to the processor (350). Here, the user may be a patient.

[0353] The user input may include a first command for collecting biometric information of a user in a healthy state and a second command for collecting biometric information of a user seeking diagnosis.

[0354] The input interface (310) can receive user identification information and receive user registration commands.

[0355] The input interface (310) can receive a sensing command from the image sensor (120).

[0356] The sensing command of the image sensor (320) may be either the first command or the second command.

[0357] The input interface (310) can receive on and off commands for the lighting (390).

[0358] The image sensor (320) can acquire an image of a user of the display device (300), convert image information about the acquired image of the user into an electrical image signal, and transmit the converted image signal to the processor (350).

[0359] The light sensor (330) can detect the light intensity around the image sensor (320) and transmit light intensity information about the detected light intensity to the processor (350).

[0360] The communication unit (340) may include one or more components that enable communication between components within the display device (300).

[0361] The communication unit (340) may include one or more components that enable communication with an external device, and may include, for example, at least one of a short-range communication module, a wired communication module, and a wireless communication module. Here, the external device may include a server of a medical institution or a medical information server.

[0362] The communication unit (340) may include both wired networks and wireless networks.

[0363] The processor (350) oversees control related to the operation of the display device (300). There may be one or more processors (350). That is, there may be at least one processor (350).

[0364] The processor (350) performs control related to the operation of the display device (300) using data stored in the memory (351).

[0365] The processor (350) can control the execution or termination of an application based on user input received through the input interface (310).

[0366] When the processor (350) receives a user registration command and user information from the input interface (310), it can register the user information for treatment and diagnosis based on the received user information.

[0367] User information is user identification information for identifying the user, and may include the user's date of birth, user's name, user's address, user's contact information, and user's code information.

[0368] When a first command is received from the input interface (310), the processor (350) controls the activation of the image sensor (320), acquires the user's reference biometric information based on the user's image information acquired by the activated image sensor (320), and controls the memory (351) to store the acquired user's reference biometric information.

[0369] The first command may be a command to obtain baseline biometric information of a user in a healthy state.

[0370] The user image acquired in response to receiving the first command may be an image of the user's body in a healthy state. The body may include at least one of the face, arms, legs, and torso.

[0371] Reference biometric information may include reference color information of the body and reference biometric signal information of a predetermined area. For example, the biometric signal may include tremors due to pulse or tremors due to convulsions.

[0372] When the processor (350) obtains the user's reference biometric information, it can control the lighting (390) based on preset environmental information.

[0373] The preset environment may include an environment in which the lighting (390) in the space where the display device (300) is provided is turned off, an environment in which the lighting (390) is turned on, or an environment with a preset illuminance.

[0374] The processor (350) can control the brightness of the light (390) so that the space becomes an environment with a preset illuminance.

[0375] When the processor (350) obtains the user's reference biometric information, it is also possible to control the brightness of the display unit (360) so that the illuminance of the space becomes a preset illuminance.

[0376] When a second command is received from the input interface (310), the processor (350) controls the lighting (390) based on preset environmental information, controls the activation of the image sensor (320), and then acquires diagnostic biometric information based on the user's image information acquired by the activated image sensor (320) and controls the display unit (360) to display the acquired diagnostic biometric information.

[0377] The processor (350) can control the display unit (360) to display the user's identification information and the user's diagnostic history information along with the user's image information.

[0378] The processor (350) can obtain reference biometric information stored in the memory (351), compare the diagnostic biometric information with the reference biometric information to obtain diagnostic result information, and control the display unit (360) to display the obtained diagnostic result information.

[0379] The processor (350) can obtain diagnostic result information based on diagnostic criteria information stored in a server (not shown) or memory (351).

[0380] The processor (350) can obtain diagnostic result information based on diagnostic criteria information and diagnostic biometric information stored in the memory (351).

[0381] For example, the processor (350) can detect tremor information in a predetermined detection area and diagnose the user's health based on the detected tremor information and the reference tremor information corresponding to the diagnostic reference information.

[0382] When displaying the user's image information through the display unit (360), the processor (350) can adjust at least one display parameter of the image and the display unit.

[0383] For example, display parameters may include brightness, luminance, saturation, contrast, and color of the display unit, and may include brightness, luminance, saturation, contrast, and color of the image.

[0384] The processor (350) can identify image information of a color patch from the user's image information, obtain color information of the color patch based on the image information of the identified color patch, obtain color correction information based on the color information of the acquired color patch and the color information of the pre-stored color patch, and correct the user's image information based on the acquired color correction information.

[0385] The processor (350) can also change the illumination of the space based on the user's skin tone before acquiring the user's image information for obtaining reference biometric information or before acquiring the user's image information for diagnosis.

[0386] More specifically, the processor (350) controls the display unit (360) to turn on, controls the operation of the image sensor (320) to acquire the user's first image while the display unit (360) is turned on, and can acquire the first illuminance information of the space detected by the illuminance sensor (330) when acquiring the first image.

[0387] The processor (350) controls the display unit (360) to be turned off, controls the operation of the image sensor (320) to acquire a second image of the user while the display unit (360) is turned off, and can acquire second illumination information of the space detected by the illumination sensor (330) when acquiring the second image.

[0388] The processor (350) can obtain average brightness information of a face area in a first image and obtain average brightness information of a face area in a second image.

[0389] The processor (350) obtains the user's skin reflection coefficient based on the average brightness information (F1) of the first image, the average brightness information (F2) of the second image, the first illuminance information (Lux1) and the second illuminance information (Lux2), obtains the necessary illuminance required for diagnosis based on the obtained user's skin reflection coefficient, and controls at least one of the illuminance of the display unit (360) and the illuminance of the lighting (390) based on the obtained necessary illuminance.

[0390] The processor (350) determines whether the illumination within the image is uneven based on the user's image information, and if it is determined that the illumination within the image is uneven, it can correct the received user's image information to obtain an image with uniform illumination.

[0391] The processor (350) can check the illumination information detected by the illumination sensor (330) and correct the brightness of the received user image information based on the checked illumination information.

[0392] The configuration of image correction in low-light environments, extremely low-light environments, and backlit environments and the output configuration of environmental guide information are the same as those of the first processor, so their description is omitted.

[0393] The processor (350) can detect body feature points based on the user's image information and perform a diagnosis based on the detected body feature points.

[0394] The processor (350) can detect body feature points and omega shapes based on the user's image information, and identify a detection area of ​​a biosignal based on whether the body feature points are detected, whether the omega shape is detected, location information of the detected feature points, and the central axis of the omega shape. The user's image information may be two-dimensional image information or three-dimensional image information.

[0395] The processor (350) can recognize a face based on the user's image information, detect feature points and an omega shape from the recognized face, and obtain biometric information within the face based on whether feature points of the face image are detected, whether an omega shape is detected, location information of feature points of the detected face image, and the central axis of the omega shape.

[0396] The processor (360) can identify a detection area of ​​a biosignal using biometric information. The identification configuration of the detection area of ​​a biosignal within the face is the same as that of the first processor, and thus, a description thereof will be omitted.

[0397] When the aesthetic mode is received through the input interface (310), the processor (350) recognizes the face based on the user's image information, detects feature points and an omega shape from the recognized face, distinguishes the left and right sides of the face image based on the detected omega shape, determines the necessity of a cosmetic procedure or plastic surgery based on the color information of the left and right facial images, the position information of the feature points of the left and right facial images, and the distribution information of the feature points of the left and right facial images, and controls the display unit (360) to display suggestion information when it is determined that a procedure or plastic surgery is necessary.

[0398] The processor (350) can display left and right facial images with matched analysis information, and control the display unit (360) to display suggested information for treatment or plastic surgery for each area.

[0399] When the aesthetic mode is received through the input interface (310), the processor (350) obtains reference biometric information of the facial image stored in the memory (351), compares the reference biometric information of the obtained facial image with the biometric information of the currently detected facial image, obtains change information of the facial image corresponding to time changes, and can also determine the necessity of cosmetic treatment or plastic surgery based on the change information of the facial image.

[0400] The processor (350) may recognize a face based on the user's image information when a makeup mode is received through the input interface (310), detect feature points and an omega shape from the recognized face, distinguish the left and right sides of the face image based on the detected omega shape, and control the display unit (360) to display makeup suggestion information based on color information of the left and right face images, location information of feature points of the left and right face images, and distribution information of feature points of the left and right face images.

[0401] When a body shape correction mode is received through the input interface (310), the processor (350) can control the display unit (360) to display guidance information that guides the user on the posture or movement to be taken to analyze the body shape, and can control the speaker (380) to output the guidance information in voice.

[0402] As shown in FIG. 10a and FIG. 10b, the processor (350) can guide to acquire an image after assuming a standing or sitting posture, and can guide to acquire an image after assuming a walking posture.

[0403] As illustrated in FIG. 10, the processor (350) recognizes the entire body based on the user's image information, detects feature points and an omega shape from the recognized body, sets a central axis based on the detected omega shape, and determines whether the body is balanced based on the set central axis and the feature points of the body.

[0404] When the processor (350) determines that the user's body is unbalanced, the processor (350) can obtain front-back position information, left-right position information, and up-down position information of each body part based on the position information of the characteristic points of each body part, the distribution information of the characteristic points, and the inclination information of the body distinguished by the omega shape, and can control the display unit (360) to check the type of exercise and method of exercise required for the user based on the obtained front-back position information, left-right position information, and up-down position information of each body part, and to display the checked type of exercise and method of exercise.

[0405] For example, the processor (350) can guide a shoulder movement method in response to left-right asymmetry of the shoulder or front-back asymmetry of the shoulder based on the central axis of the omega shape and the shoulder line, and can guide a pelvic movement method in response to left-right asymmetry of the pelvis based on the central axis of the omega shape and the pelvic line.

[0406] The body features may include features corresponding to the skeleton.

[0407] When a massage mode is received through the input interface (310), the processor (350) can control the display unit (360) to obtain a massage mode required by the user based on the acquired front-back position information, left-right position information, and up-down position information of each body part, and to display the acquired massage mode.

[0408] If the processor (350) determines that communication with a massage device, massage machine, or home medical device is possible, it is also possible to transmit information about the acquired massage mode to the massage device, massage machine, or home medical device.

[0409] The processor (350) can control the display unit to obtain the next diagnosis time based on the diagnosis history information and the diagnosis result information and to display information about the next acquired diagnosis time.

[0410] The processor (350) can recognize changes in the user's health status based on the diagnosis history information and the diagnosis result information and control the display unit (360) to display the recognized changes in the user's health status.

[0411] The information that is analyzed for the face or body shape in aesthetic mode, makeup mode, and body correction mode, or suggested to the user according to each mode, can be acquired and updated through artificial intelligence and learning.

[0412] The memory (351) can store first and second reference illuminance information.

[0413] The memory (351) can store the user's identification information, store the user's reference biometric information, and store the user's medical information and diagnosis result information.

[0414] The memory (351) can further store diagnostic criteria information for each type of disease for diagnosis.

[0415] Diagnostic criteria information may include color information, pulse information, and convulsion information by disease type, and may include location information and curvature information of characteristic points by body part.

[0416] The memory (351) and the processor (350) may be implemented as separate chips. Alternatively, the memory (351) and the processor (350) may be implemented as a single chip.

[0417] The display unit (360) can output information corresponding to the control command of the processor (350) as a video.

[0418] The display unit (360) can display the user's image and the user's diagnostic history information. The user's image can include an image of the user's body. The user's image can include an image of the user's face.

[0419] The display unit (360) can display user input in response to a control command of the processor (350).

[0420] The display unit (360) can also display the user's image. The user's image may include a doctor's image.

[0421] The display unit (360) can adjust the brightness of the user's image and display a correction image for the user's image with uneven illumination.

[0422] The display unit (360) can display a detection area of ​​a biosignal in the user's image and can display diagnosis result information.

[0423] The display unit (360) can display an image corresponding to a user command. For example, the image corresponding to a user command may include a document image containing medical information, a photographic image, or a video.

[0424] The display unit (360) may be composed of a touch screen panel (TSP) that forms a mutual layer structure with a touch pad.

[0425] The microphone (370) can receive the user's voice and transmit voice information about the received voice to the processor (350).

[0426] The speaker (380) can output sound information according to the control command of the processor (350). The speaker (380) can output environmental guide information in voice.

[0427] Lighting (390) may be provided around the image sensor (320).

[0428] The light (390) can emit light in response to a control command from the processor (350).

[0429] The light (390) can emit light with a brightness corresponding to the control command of the processor (350).

[0430] The lighting (390) may be a backlight unit provided in the display unit (360).

[0431] The artificial intelligence-related functions according to the present disclosure are operated through a processor (350) and a memory (351). The processor (350) may be composed of one or more processors. In this case, the one or more processors (350) may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU.

[0432] One or more processors (350) are controlled to process input data according to predefined operating rules or artificial intelligence models stored in memory (351). Alternatively, if one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0433] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is trained using a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0434] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0435] At least one component may be added or deleted to correspond to the performance of the components of the display device illustrated in FIG. 9. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be altered to correspond to the performance or structure of the display device.

[0436] Meanwhile, each component illustrated in FIG. 9 represents software and / or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0437] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0438] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0439] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present invention can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. Display section; A communication unit that performs communication with external devices; At least one memory storing one or more instructions; and comprising at least one processor executing one or more of the above instructions; The one or more instructions for causing the display device to operate when the at least one processor is executed are: A display device comprising: displaying an image on the display unit based on image information received through the communication unit; identifying one or more characteristic points and an omega shape of a patient included in the image; identifying whether the image includes a predetermined detection area related to the patient's biometric information based on the one or more characteristic points and the omega shape; and adjusting a display parameter of at least one of the display unit and the image based on whether the predetermined detection area is identified.

2. In paragraph 1, The one or more instructions for causing the display device to operate when the at least one processor is executed are: A display device comprising: identifying a central axis of the image; identifying the left and right sides of the patient in the image based on the identification of the central axis, the omega shape and the one or more feature points; identifying whether a left feature point of the left side of the patient is included in the one or more feature points; identifying whether a right feature point of the right side of the patient is included in the one or more feature points; and mapping a feature point between the left feature point and the right specific point included in the one or more feature points to the opposite side of the image based on the identification that only one of the left feature point and the right feature point is included in the one or more feature points.

3. In paragraph 1, The one or more instructions for causing the display device to operate when the at least one processor is executed are: Controlling the communication unit to transmit environmental guide information to the external device based on the identification that the image does not include at least one of the one or more feature points and the omega shape; The above environmental guide information is a display device that includes information related to the patient's posture and adjustment of the patient's position.

4. In paragraph 1, The one or more instructions for causing the display device to operate when the at least one processor is executed are: A display device comprising adjusting display parameters of the image based on illumination information received through the communication unit and predetermined reference illumination information.

5. In paragraph 1, The one or more instructions for causing the display device to operate when the at least one processor is executed are: A display device comprising: controlling the communication unit to transmit environmental guide information for illuminance adjustment to the external device based on illuminance information received through the communication unit and predetermined reference illuminance information.

6. In paragraph 1, The one or more instructions for causing the display device to operate when the at least one processor is executed are: A display device comprising: performing image preprocessing of an image to make the brightness of the image uniform based on identification that the brightness in the image is non-uniform.

7. In paragraph 1, At least one of said memories stores the patient's baseline biometric information, The one or more instructions for causing the display device to operate when the at least one processor is executed are: A display device comprising: a communication unit configured to detect the patient's biometric information in the predetermined detection area based on treatment request information received through the communication unit; acquire diagnosis result information based on the detected biometric information and the stored reference biometric information; and control the communication unit to transmit the acquired diagnosis result information to the external device.

8. In paragraph 1, The one or more instructions for causing the display device to operate when the at least one processor is executed are: A display device comprising: identifying whether the image information is image information acquired in a backlit environment based on brightness information of the image information, based on the identification of the omega shape and the identification of one or more of the feature points not being identified; adjusting display parameters of the image based on the identification of the image information as image information acquired in the backlit environment; and performing preprocessing of the image to uniformize the brightness of the image.

9. In paragraph 8, The one or more instructions for causing the display device to operate when the at least one processor is executed are: Based on the identification of the above omega shape and the fact that one or more of the feature points are not identified, acquiring illumination information from the external device through the communication unit, and identifying whether the illumination of a space in which the external device is provided is less than a first reference illumination and more than a second reference illumination based on the acquired illumination information and the first and second reference illumination information, and identifying that the received image information is image information acquired in the backlight environment if the illumination of the space is less than the first reference illumination and more than the second reference illumination, and transmitting environment guide information to the external device if the illumination of the space is less than the second reference illumination, The above second standard illuminance is lower than the above first standard illuminance, A display device including the above environmental information including information related to adjusting the illuminance of a space.

10. In paragraph 1, wherein said one or more feature points comprise one or more feature points of said patient's face, One or more of the above facial features include the points at both ends of the eyes, the points at both ends of the mouth, and the bridge of the nose, A display device wherein the above-determined detection area includes the orbital area of ​​the face and the infraorbital area of ​​the face.

11. In paragraph 1, The one or more instructions for causing the display device to operate when the at least one processor is executed are: A display device that acquires a skin reflection coefficient of the patient based on first image information acquired in a lighting environment and second image information acquired in a no-light environment received from the external device, and transmits illuminance information corresponding to the skin reflection coefficient to the external device.

12. Identify one or more characteristic points and omega shapes of the patient based on image information received from an external device through the communication unit, Based on the identification of one or more of the above features and the identification of the omega shape, identifying whether the image information includes a detection area related to the patient's biometric information, A control method of a display device for adjusting at least one display parameter of the image information or a display section of the display device based on the identification of the detection area.

13. In paragraph 12, Transmitting first environmental guide information for guiding position and posture adjustment of the patient to the external device based on the fact that the one or more feature points and the omega shape are not identified in the image information, Based on the identification of the omega shape in the image information and the identification of one or more of the characteristic points, the environment of the space in which the external device is provided is identified as a backlight environment, and the display parameters of the display unit are adjusted, and the preprocessing of the image information is performed to make the brightness of the image displayed on the display unit uniform. A method for controlling a display device further comprising transmitting second environmental guide information for illuminance adjustment to the external device based on illuminance information and reference illuminance information received from the external device through the communication unit.

14. In paragraph 12, Identify the central axis of the above image information, Identifying the left side of the patient and the right side of the patient in the image information based on the central axis, Identifying that the left feature point of the left side of the patient is included in the one or more of the above features, and identifying that the right feature point of the right side of the patient is included in the one or more of the above features, Based on the identification of only one of the left feature point and the right feature point among the one or more feature points, Identify the left feature point and the right feature point included in the one or more feature points as detected feature points, and identify the left feature point and the right feature point not included in the one or more feature points as missing feature points. By mapping the detected feature points to the side corresponding to the missing feature point among the left side and the right side of the patient, a missing feature point is generated, A control method of a display device further comprising identifying the detection area based on the detected feature points and the generated feature points.

15. In paragraph 12, The patient's skin reflection coefficient is acquired based on the first image information acquired in a lighting environment received from the external device and the second image information acquired in a non-illuminated environment, A control method of a display device further comprising transmitting illuminance information corresponding to the acquired skin reflection coefficient to the external device.

Images