Biological Information Output Mirror Device

Abstract

This invention provides a biological information output mirror device that acquires biological information and outputs biological-related information. [Means for Solving the Problem] The bio-related information output mirror device comprises a housing and a mirror surface placed on the surface of the housing, and a small-diameter camera placed on a part of the mirror surface to capture the face of a person reflected in the mirror surface. The bio-related information output mirror device also has a face recognition unit that determines whether the small-diameter camera is capturing a person's face at a predetermined angle of view, and a bio-related information acquisition unit that acquires bio-related information based on the captured face if the face recognition unit determines that the person's face is capturing a person's face at the predetermined angle of view. Furthermore, the bio-related information output mirror device has a bio-related information output unit that outputs bio-related information based on the acquired bio-related information.

Description

Technical Field

[0001] The present invention relates to a biological information output mirror device that outputs biological information.

Background Art

[0002] Conventionally, smart mirrors have been known. For example, in Patent Document 1, (1) insurance simulation, (2) entertainment such as quizzes and fortune-telling, (3) temperature measurement, (4) gymnastics challenges, etc. are displayed as menu images on the display of a mirror-type display device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The mirror-type display device described in Patent Document 1 mainly provides information about insurance simulation, and regarding biological information, it is only a device that can obtain "temperature measurement".

[0005] That is, the mirror-type display device described in Patent Document 1 is not a device that acquires biological information such as pulse, oxygen saturation, and blood pressure, and displays biological information and biological-related information.

Means for Solving the Problems

[0006] Therefore, the present invention provides a bio-related information output mirror device having the following configuration, which outputs biological information and bio-related information. The bio-related information output mirror device comprises a housing, a mirror surface arranged on the surface of the housing, and a small-diameter camera arranged on a part of the mirror surface to photograph the face of a person reflected in the mirror surface. The bio-related information output mirror device also has a face recognition unit that determines whether the small-diameter camera is photographing a person's face at a predetermined angle of view, and a bio-related information acquisition unit that acquires biological information based on the photographed face when the face recognition unit determines that the person's face is photographed at the predetermined angle of view. Furthermore, the present invention provides a bio-related information output mirror device having a bio-related information output unit that outputs bio-related information based on the acquired biological information.

[0007] As the first invention, a bio-related information output mirror device is provided, comprising: a mirror surface disposed on the surface of the housing; a small-diameter camera disposed on a part of the mirror surface for photographing a person's face reflected in the mirror surface; a face recognition unit for determining whether the small-diameter camera is photographing a person's face at a predetermined angle of view; a bio-related information acquisition unit for acquiring bio-related information based on the photographed person's face when the face recognition unit determines that the person's face is photographed at a predetermined angle of view; and a bio-related information output unit for outputting bio-related information based on the acquired bio-related information.

[0008] As a second invention, building upon the first invention, a biological information output mirror device is provided that further includes an image acquisition range display unit that indicates within the mirror surface the range of the image used to acquire biological information from the reflection of a person on the mirror surface during biological information acquisition.

[0009] As a third invention, based on the first or second invention, the present invention provides a bio-related information output mirror device having a surface temperature measuring unit that measures the surface temperature of a person's face when the face recognition unit determines that it has captured a person's face at a predetermined angle of view, and a surface temperature acquisition means that acquires the measured surface temperature as bio-information.

[0010] As a fourth invention, based on the first or second invention, the present invention provides a bio-related information output mirror device further comprising: an illuminance measuring unit that measures the illuminance of a person's face when the face recognition unit determines that a person's face is being photographed at a predetermined angle of view; an illuminance determination unit that determines whether the measured illuminance is within a predetermined range; an illumination unit that improves the illuminance of the photographed person's face when the determination result from the illuminance determination unit is determined to be below a predetermined range; and an illumination control unit that controls the illumination unit.

[0011] As a fifth invention, based on the fourth invention, the present invention provides a bio-related information output mirror device in which the lighting unit has light sources at multiple locations, and the lighting control unit has a light source-specific illuminance control means for changing the illuminance for each of the multiple light sources.

[0012] As a sixth invention, the present invention provides a method to be executed by the CPU of a bio-related information output mirror device, which is a computer, comprising: a housing; a mirror surface disposed on the surface of the housing; a small-diameter camera disposed on a part of the mirror surface for photographing a person's face reflected in the mirror surface; a face recognition step for determining whether the small-diameter camera has photographed a person's face at a predetermined angle of view; a bio-related information acquisition step for acquiring bio-related information based on the photographed person's face, if the face recognition step determines that the person's face has been photographed at a predetermined angle of view; and a bio-related information output step for outputting bio-related information based on the acquired bio-related information.

[0013] As the seventh invention, based on the sixth invention, the present invention provides a method executed by the CPU of a biological information output mirror device, which is a computer, further comprising an image acquisition range display step that indicates within the mirror surface the range of the image used to acquire biological information from the image of a person reflected on the mirror surface during biological information acquisition.

[0014] As the eighth invention, based on the sixth or seventh invention, the present invention provides a method executed by the CPU of a biological information output mirror device, which is a computer, wherein the method includes a surface temperature measurement step for measuring the surface temperature of a person's face when the face recognition step determines that a person's face has been photographed at a predetermined angle of view, and a surface temperature acquisition substep for acquiring the measured surface temperature as biological information.

[0015] As the ninth invention, based on the sixth or seventh invention, the present invention provides a method executed by the CPU of a bio-related information output mirror device, which is a computer, comprising: an illuminance measurement step for measuring the illuminance of a person's face when the face recognition step determines that a person's face is being photographed at a predetermined angle of view; an illuminance determination step for determining whether the measured illuminance is within a predetermined range; an illumination step for improving the illuminance of the photographed person's face when the determination result in the illuminance determination step is determined to be below the predetermined range; and an illumination control step for controlling the illumination step.

[0016] As the tenth invention, based on the ninth invention, the present invention provides a method executed by the CPU of a biological information output mirror device, which is a computer, wherein the illumination step has light sources at multiple locations, and the illumination control step has a sub-step for controlling the illuminance for each of the multiple light sources.

[0017] As the eleventh invention, the present invention provides an operation program for a bio-related information output mirror device, which is a computer having a housing, a mirror surface disposed on the surface of the housing, a small-diameter camera disposed on a part of the mirror surface for photographing a person's face reflected in the mirror surface, a face recognition step for determining whether the small-diameter camera is photographing a person's face at a predetermined angle of view, a bio-related information acquisition step for acquiring bio-related information based on the photographed person's face when the face recognition step determines that the person's face is photographed at a predetermined angle of view, and a bio-related information output step for outputting bio-related information based on the acquired bio-related information.

[0018] As the twelfth invention, building upon the eleventh invention, we provide an operating program for a bio-related information output mirror device, which is a computer that further has an image acquisition range display step that indicates within the mirror surface the range of the image used to acquire bio-related information from the reflection of a person on the mirror surface during bio-related information acquisition. The program is written in a readable and executable manner for the bio-related information output mirror device.

[0019] As the thirteenth invention, based on the eleventh or twelfth invention, the invention provides an operation program for a bio-related information output mirror device, which is a computer, written in a readable and executable manner, which includes a surface temperature measurement step for measuring the surface temperature of a person's face when the face recognition step determines that a person's face has been photographed at a predetermined angle of view, and a surface temperature acquisition substep for acquiring the measured surface temperature as bio-related information.

[0020] As the fourteenth invention, based on the eleventh or twelfth invention, when it is determined in the face recognition step that a person's face is being photographed at a predetermined angle of view, an illuminance measurement step is provided to measure the illuminance of the person's face, An illumination determination step for determining whether the measured illumination is within a predetermined range, an illumination step for improving the illumination of the face of the person being photographed when the determination result in the illumination determination step is a determination result that it is below a predetermined range, and an illumination control step for controlling the illumination step. A biological information output mirror device having a computer further having a biological information output mirror device operation program described so as to be readable and executable is provided.

[0021] As a fifteenth invention, based on the fourteenth invention, the illumination step has light sources at a plurality of locations, and the illumination control step has a per-light-source illumination control sub-step for changing the illumination for each of the light sources at a plurality of locations. A biological information output mirror device having a computer further having a biological information output mirror device operation program described so as to be readable and executable is provided.

Effects of the Invention

[0022] According to the invention having the above configuration, it is possible to provide a biological information output mirror device that acquires biological information from a face image or the like and outputs biological information and biological-related information such as pulse, oxygen saturation, blood pressure, etc.

Brief Description of the Drawings

[0023] [Figure 1] A diagram showing an example of the face display screen of the biological information output mirror device according to Embodiment 1 [Figure 2] A diagram showing an example of the stand screen of the biological information output mirror device according to Embodiment 1 [Figure 3] A diagram showing an example of the full-body display screen of the biological information output mirror device according to Embodiment 1 [Figure 4] A diagram showing an example of the display of the biological information output unit of the biological information output mirror device according to Embodiment 1 [Figure 5] A diagram showing an example of the network configuration of the biological information output mirror device according to Embodiment 1 [Figure 6] A functional block diagram showing an example of the functions of the biological information output mirror device according to Embodiment 1 [Figure 7] Conceptual diagram showing stress index and emotional expression representing joy, anger, sadness, and pleasure, illustrating an example of bio-related information according to Embodiment 1. [Figure 8] This figure shows an example of the hardware configuration of the bio-related information output mirror device according to Embodiment 1. [Figure 9] Flowchart showing an example of the processing flow of the biological information output mirror device according to Embodiment 1. [Figure 10] This figure shows an example of a face display screen for a bio-related information output mirror device according to Embodiment 2. [Figure 11] Functional block diagram showing an example of the functions of the biological information output mirror device according to Embodiment 2. [Figure 12] This figure shows an example of the hardware configuration of the bio-related information output mirror device according to Embodiment 2. [Figure 13] Flowchart showing an example of the processing flow of the bio-related information output mirror device according to Embodiment 2. [Figure 14] This figure shows an example of a face display screen for a bio-related information output mirror device according to Embodiment 3. [Figure 15] Functional block diagram showing an example of the functions of the biological information output mirror device according to Embodiment 3. [Figure 16] This figure shows an example of the hardware configuration of the bio-related information output mirror device according to Embodiment 3. [Figure 17] Flowchart showing an example of the processing flow of the bio-related information output mirror device according to Embodiment 3. [Figure 18] This figure shows an example of a face display screen for a bio-related information output mirror device according to Embodiment 4. [Figure 19] Functional block diagram showing an example of the functions of the biological information output mirror device according to Embodiment 4. [Figure 20] This figure shows an example of the hardware configuration of the bio-related information output mirror device according to Embodiment 4. [Figure 21] Flowchart showing an example of the processing flow of the bio-related information output mirror device according to Embodiment 4. [Figure 22]This figure shows an example of a face display screen for a bio-related information output mirror device according to Embodiment 5. [Figure 23] Functional block diagram showing an example of the functions of the biological information output mirror device according to Embodiment 5. [Figure 24] This figure shows an example of the hardware configuration of the bio-related information output mirror device according to Embodiment 5. [Figure 25] A flowchart showing an example of the processing flow of the biological information output mirror device according to Embodiment 5. [Figure 26] Sequence diagram showing an example of processing between a biological information output mirror device and a server device via a network. [Figure 27] Sequence diagram showing an example of processing between a biological information output mirror device and a server device via a network. [Figure 28] Sequence diagram showing an example of processing between a biological information output mirror device and a server device via a network. [Figure 29] Diagram showing the hardware configuration applicable to the present invention. [Modes for carrying out the invention]

[0024] Figure 29 is a diagram showing the hardware configuration applied to the present invention. This invention, in principle, utilizes an electronic computer, but can be realized through software, hardware, and the collaboration of software and hardware. Hardware that realizes all or part of the constituent elements of this invention consists of the basic components of a computer, such as a CPU, memory, bus, input / output devices, various peripheral devices, and a user interface. Various peripheral devices include storage devices, internet interfaces, internet devices, displays, keyboards, mice, speakers, cameras, video cameras, televisions, and various sensor devices. Furthermore, this system does not necessarily have to be composed of a single enclosure; it may be composed of multiple enclosures connected by communication. The communication may be a LAN, WAN, Wi-Fi, Bluetooth®, infrared communication, or ultrasonic communication, and furthermore, some of the systems may be installed across national borders. Moreover, each of the multiple enclosures may be operated by a different entity, or they may be operated by a single entity. The operating entity of the system of this invention may be singular or plural. Furthermore, the invention can also be considered as a system that includes terminals used by third parties, and even terminals used by other third parties. Furthermore, these terminals may be installed across national borders. In addition to this system and the aforementioned terminals, devices may be provided for registering relevant information of third parties and related individuals, as well as devices used for databases to record registration details. These may be stored within this system, or they may be stored outside this system and configured to make this information available.

[0025] As shown in this diagram, the computer consists of a chipset, CPU, non-volatile memory, main memory, various buses, BIOS, various interfaces such as USB, HDMI®, and LAN, and a real-time clock, all configured on a motherboard. These work together with the operating system, device drivers (for various interfaces such as USB and HDMI®, and for embedding various devices such as cameras, microphones, speakers or headphones, and displays), and various programs. The various programs and data constituting the present invention are configured to efficiently utilize these hardware resources to perform various processes.

[0026] ≪Chipset≫ A "chipset" is a set of large-scale integrated circuits (LSIs) mounted on a computer's motherboard that integrates the functionality of bridging—that is, the communication function between the CPU's external bus and the standard bus connecting memory and peripheral devices. There are two chipset configurations and one chipset configurations. The northbridge is located closer to the CPU and main memory, while the southbridge is located further away and serves as the interface for relatively slower external I / O.

[0027] (Northbridge) The northbridge includes the CPU interface, memory controller, and graphics interface. The CPU can handle most of the functions traditionally performed by the northbridge. The northbridge connects to the main memory slots via a memory bus and to the graphics card slots via a high-speed graphics bus (AGP, PCI Express).

[0028] (Southbridge) The southbridge connects to the PCI interface (PCI slot) via the PCI bus and handles I / O functions and sound functions for ATA (SATA) interface, USB interface, Ethernet interface, etc. Circuits supporting PS / 2 ports, floppy disk drives, serial ports, parallel ports, and ISA buses, which do not require or are impossible to operate at high speeds, can be separated from the southbridge chip and handled by a separate LSI called a super I / O chip, as this would hinder the speed of the chipset itself. Buses are used to connect the CPU (MPU) to peripherals and various control units. These buses are linked by the chipset. For the memory bus used to connect to main memory, a channel structure may be adopted instead to achieve higher speeds. Either a serial bus or a parallel bus can be used. While a serial bus transfers data one bit at a time, a parallel bus transmits the original data itself or multiple bits extracted from the original data as a single unit, simultaneously over multiple communication channels. A dedicated line for the clock signal runs parallel to the data lines to synchronize data demodulation at the receiving end. It is also used as a bus to connect the CPU (chipset) to external devices, and examples include GPIB, IDE / (parallel)ATA, SCSI, and PCI. Due to limitations in speed increases, in improved versions of PCI such as PCI Express and improved versions of parallel ATA such as Serial ATA, the data lines can be a serial bus.

[0029] ≪CPU≫ The CPU sequentially reads, interprets, and executes instruction sequences called programs located in main memory, outputting signal-based information back to the main memory. The CPU functions as the central hub for performing calculations within the computer. The CPU consists of a CPU core, which is the core of the calculations, and its peripheral parts. Inside the CPU are registers, cache memory, an internal bus connecting the cache memory and the CPU core, a DMA controller, timers, and an interface to the bus connecting to the northbridge. A single CPU (chip) may have multiple CPU cores. In addition to the CPU, processing may also be performed by a graphics interface (GPU) or FPU. The description in this embodiment is for a 2-core type, but it is not limited to this. Furthermore, the CPU can also have a program embedded within it.

[0030] Non-volatile memory The basic structure of a hard disk drive (HDD) consists of a magnetic disk, a magnetic head, and an arm that houses the magnetic head. The external interface can be SATA (formerly ATA). A high-performance controller, such as SCSI, supports communication between hard disk drives. For example, when copying a file to another hard disk drive, the controller can read sectors, transfer them to the other hard disk drive, and write them. This process does not access the host CPU's memory, thus avoiding increased CPU load.

[0031] Main Memory The CPU directly accesses and executes various programs in main memory. Main memory is volatile memory, and DRAM is used. Programs in main memory are loaded from non-volatile memory into main memory upon receiving a program execution command. Subsequently, the CPU executes the program according to various execution commands and procedures within the program.

[0032] Operating System (OS) An operating system is used to manage the resources on a computer for applications to use, manage various device drivers, and manage the computer hardware itself. In small computers, firmware may be used as the operating system.

[0033] ≪BIOS≫ The BIOS is the component that instructs the CPU to start the computer hardware and run the operating system. Most typically, it is the first piece of hardware the CPU reads when it receives a computer startup command. The BIOS contains the addresses of the operating system stored on the disk (non-volatile memory), and the BIOS, deployed by the CPU, sequentially loads the operating system into main memory, bringing it into operation. The BIOS also has a check function that checks for the presence of various devices connected to the bus. The results of the check are saved in main memory and made available to the operating system as appropriate. The BIOS may also be configured to check for external devices. The above applies to all embodiments.

[0034] As shown in the figure, the present invention can basically be composed of a general-purpose computer program and various devices. The operation of the computer basically involves loading a program stored in non-volatile memory into main memory, and then executing processing between the main memory, the CPU, and various devices. Communication with devices is performed via an interface connected to a bus line. Possible interfaces include display interfaces, keyboards, and communication buffers. Embodiments of the present invention will be described below with reference to the illustrations.

[0035] <Satisfaction of the applicability of natural laws in this invention> This invention functions through the collaboration of a computer, communication equipment, and software. Specifically, it relates to a centralized health and medical information management system for accumulating and centrally managing health and medical information, which is information related to an individual's health and medical care. Various information and data are exchanged between third-party terminals and administrator terminals via a network using hardware resources. Therefore, from this perspective, judging from the matters described in the claims and specification and the common technical knowledge related to those matters, this invention as a whole utilizes natural laws and falls under the category of a computer software-related invention.

[0036] <The significance of utilizing natural laws as required by patent law> The use of natural laws required under patent law is based on the purpose of the law, and is required to ensure that an invention is industrially useful, from the perspective that the invention must have industrial applicability and contribute to the development of industry. In other words, it requires that the invention be industrially useful, that is, that the effects of the invention declared in the application can be reproduced with a certain degree of certainty by implementing the invention. From this perspective, the applicability of natural laws is interpreted as meaning that the functions exhibited by each of the inventive features (constituent elements of the invention), which are the components of the invention that exert the effects of the invention, are exerted by utilizing natural laws. Furthermore, the effect of the invention only needs to have the potential to provide a certain level of usefulness to the user who uses the invention, and should not be viewed from the perspective of how the user feels or thinks about that usefulness. Therefore, even if the effect that the user obtains from this system is a psychological effect, the effect itself is an event outside the scope of the required applicability of natural laws.

[0037] Embodiments of the present invention will be described below with reference to the figures. This is not limited in any way, and in any way that does not deviate from its essence, it may be implemented in various forms. It can be done. <Embodiment 1> Embodiment 1 mainly relates to claim 1, etc.

[0038] <Overview of Embodiment 1> The bio-related information output mirror device of Embodiment 1 comprises a housing and a mirror surface positioned on the surface of the housing. It also includes a small-diameter camera positioned on a part of the mirror surface to capture a person's face reflected in the mirror. Furthermore, it is configured to include a face recognition unit that determines whether a person's face is being captured at a predetermined angle of view by the small-diameter camera, a bio-related information acquisition unit that acquires bio-related information when the face recognition unit determines that a person's face is being captured at the predetermined angle of view, and a bio-related information output unit that outputs bio-related information based on the acquired bio-related information. The biological information output mirror device 0100 comprises a housing, a small-diameter camera, a face recognition unit, a biological information acquisition unit, and a biological information output unit. Figure 1 shows an example of the display screen of the bio-related information output mirror device 0100 according to Embodiment 1. The bio-related information output mirror device 0100 acquires the user's bio-information and outputs the bio-information and bio-related information to the bio-related information output unit 0105. The bio-related information output mirror device 0100 can acquire bio-related information in a short time and output it in nursing homes, hospitals, construction sites, the transportation industry, etc.

[0039] <Description of the configuration of Embodiment 1> (Embodiment 1 Configuration: Enclosure) The housing 0110 is the outer box portion that constitutes the biological information output mirror device 0100. The shape of the housing 0110 is not particularly limited. Nor is there any particular limitation on its size, and it can be designed as appropriate according to requirements. The housing 0110 is preferably shaped like a vertically elongated rectangle, as it will also be used as a mirror. In this embodiment 1, it is also shaped like a vertically elongated rectangular parallelepiped. The housing 0110 may be fitted into a stand and installed in a bathroom, kitchen, living room, examination room, testing room, waiting room, etc. (see Figure 2). When the housing 0110 is installed on a stand, it is preferable that the height be adjustable. Alternatively, the housing 0110 may be directly embedded in the wall of a bathroom, kitchen, living room, examination room, testing room, waiting room, etc. In the case of embedding it in the wall, the wall surface may become part of the housing. The size of the enclosure 0110 is not particularly limited, but it is between 20 and 40 inches. For displaying the user's face or upper body, the enclosure 0110 should be around 30 inches. For displaying the user's entire body, a larger, vertically oriented size of 40 inches or more is required (see Figure 2). It can be embedded in the wall of a bathroom or bedroom, or placed on a stand in a living room.

[0040] (Embodiment 1 Configuration: Mirror surface) The mirror surface is a component of the bio-related information output mirror device that has both mirror and display functions. The mirror surface is a normal mirror, no different from those found in bathrooms, dressing rooms, makeup rooms, or entryways. Such a mirror with two functions is sometimes referred to as a smart mirror. The mirror surface is configured to perform a display function through switching control. This principle will be explained below. (1) Display transparency The display uses translucent mirrored glass. When the display is off or in mirror mode, it functions as a normal mirror because it does not transmit light but reflects it. (2) Video display When the display is turned on, light from behind shines through, and images and information are displayed on the screen. In this state, the display function takes precedence over the mirror function, and it primarily displays information necessary for display purposes. (3) Switching mechanism Software or sensors could be used to automatically switch the display mode; for example, the display could be set to turn on when a person stands in front of the mirror and turn off when they move away. In terms of operating modes, in mirror-only mode, the display is completely off, and it functions only as a mirror. In hybrid mode, it functions as a mirror while also being able to display necessary information (such as the time and weather forecast). In display-only mode, it does not function as a mirror, but displays images and information so that they can be read. Many smart mirrors allow you to easily switch modes from a state that looks like a normal mirror using touch operation or voice commands. This allows users to use it as a normal mirror while also obtaining information as needed. The mirror surface may consist of a mirror, an input device (touch panel), and a display device (display). Alternatively, the entire mirror surface may function as a touch panel. The touch panel will be discussed later.

[0041] (Embodiment 1 Configuration: Small-diameter camera) A small-diameter camera is, without any particular limitations, a camera similar to those used in smartphones and tablets, equipped with wide-angle fixed-focus lenses, etc. While the pixel count of a small-diameter camera is not particularly limited, 10 million pixels or more is desirable. The image sensor size used in small-diameter cameras is typically 1 / 3-inch or 1 / 2.3-inch. The small-diameter camera may change its sensitivity depending on the ambient brightness when taking images. That is, during the day or when the illumination from lighting is sufficient and it is bright, it will take images at the normal sensitivity. On the other hand, at night or when the illumination from lighting is insufficient and it is dim, it will take images at high sensitivity. Increasing the sensitivity will generate noise, but the camera may be equipped with a mechanism to reduce noise in the image sensor and optimize the image. Furthermore, it is preferable that the small-diameter camera has an autofocus function. In particular, since it is necessary to measure the pulsation appearing on the surface of a person's face, the degree of relaxation of facial muscles, the position of relaxation, and the surface temperature, it is preferable that the camera be configured so that the condition of the skin on a person's face can be observed precisely with the autofocus function. In addition, it is preferable that the camera has the following functions. (1) Exposure compensation Exposure compensation is a function that adjusts the brightness of a photograph. Automatic exposure compensation (AE) measures the brightness of the scene and automatically selects the optimal exposure setting. (2) White balance White balance is a function that adjusts the color temperature of a photograph to reproduce natural colors. Automatic white balance (AWB) automatically adjusts the color temperature according to the light source in the scene. (3) Motion blur correction Image stabilization (IS) is a function that corrects for vibrations and movements of the subject, enabling the capture of sharp, blur-free images. Both optical and electronic IS systems exist. (4) Automatic ISO setting Automatic ISO sensitivity setting is a function that automatically adjusts the ISO sensitivity according to the amount of light in the shooting environment. This allows you to capture clear images with reduced noise even in low-light scenes. (5) Scene Modes and Presets The camera comes with preset settings optimized for specific shooting scenes, such as landscapes, portraits, nightscapes, and sports. This allows users to take photos in a relaxed state. (6) High-speed continuous shooting The high-speed continuous shooting function allows you to take many photos in quick succession. It's useful when capturing fast-moving subjects. (7) HDR (High Dynamic Range) HDR is a function that combines multiple photos taken at different exposures to capture both dark and bright areas simultaneously. It can express details even in scenes with high contrast. The combination of these functions allows for the capture of high-quality portraits. The small-diameter camera may be a camera equipped with multiple lenses, such as those found in smartphones. It is preferable that the camera has 3D camera functionality, as acquiring 3D information of the facial surface allows for more accurate biometric information. Furthermore, since using multiple lenses can cancel out extraneous information such as the subject's movement, it is also preferable to use a camera for such purposes.

[0042] (Embodiment 1: Configuration of Face Recognition Unit) The face recognition unit determines whether the small-diameter camera is capturing a person's face at a predetermined angle of view. The need to determine whether the camera is capturing at a predetermined angle of view is necessary because biometric information cannot be obtained from the image of the face if it is not captured at that angle. Therefore, if it is determined that the camera is not capturing at a predetermined angle of view, the system can be configured to display instructions via voice or text on the display, such as moving the face closer to the mirror, tilting the face left or right, or facing the camera directly in front of the face.

[0043] (Embodiment 1 Configuration: Face recognition unit, activation based on recognition, etc.) Furthermore, the determination result of whether or not a photo is being taken at a predetermined angle of view may be used for control purposes such as activating the biometric information acquisition unit. In other words, under normal conditions, this biometric information output mirror device is used as a mirror, but if a person approaches and it is determined that their face is being photographed at a predetermined angle of view, the biometric information acquisition unit can be configured to activate automatically. In this case, the face recognition unit is basically always operating, and the recognition result from the face recognition unit is used for the aforementioned control. Note that this configuration is not limited to this, and these functions can also be configured to be activated by receiving input from the user to activate the face recognition unit, biometric information acquisition unit, and biometric information output unit in some way. Furthermore, the facial recognition unit is always active, and if the facial recognition unit determines that it has captured the user's face at a predetermined angle of view, a portion of the mirror surface can be activated as a display, and the user's facial image captured by the small-diameter camera can be output to that display.

[0044] (Embodiment 1 Configuration: Face Recognition Unit, Automatic Control of Small Diameter Camera) Furthermore, the face recognition unit can act as a trigger to advise the user to adjust the position of their face according to the field of view of the user's face being captured, and it may also automatically control the magnification of the small-diameter camera to adjust the face so that it is captured at a predetermined field of view. In this way, a startup control unit that activates the biometric information acquisition unit and the biometric information output unit according to the face recognition result of the face recognition unit, a small-diameter camera control unit that adjusts the magnification of the small-diameter camera, and in some cases the aperture, shutter speed, etc., and a display startup unit that activates a display for outputting the user's face image captured by the small-diameter camera on the mirror surface can be configured to operate in conjunction with the face recognition unit. The face recognition unit recognizes the user's face captured by a small-diameter camera, cuts off all or part of the mirror surface, switches to a display, and displays the face on that display. When displaying the face on the display, it is desirable to display it in a horizontally reversed state, just like in a mirror. When the bio-related information output mirror device is powered on, part or all of the mirror surface may be configured to switch to a display, and a small-diameter camera or the like may be activated. When the face recognition unit of the bio-related information output mirror device determines that a face has been recognized, it displays the image captured by the small-diameter camera on the display, which is used as a display device.

[0045] (Embodiment 1 Configuration: Face Recognition Unit, Person Authentication) Alternatively, authentication can be performed using facial images or videos captured by a small-diameter camera, voice recognition, or by entering a password via a display. Furthermore, the user's ID information and facial information are linked and stored together. This allows the system to record the final output biometric information in association with user identification information, which identifies the authenticated person. This record may be associated with the time the biometric information was acquired and may be configured to perform statistical processing for each user. As will be explained later, biometric information such as pulse rate, oxygen saturation, blood pressure, body temperature, stress index, and emotional expressions may be calculated over time, and the average value and standard deviation may be calculated. Based on these, the system may be configured to calculate and output the standard scores of the biometric information for each user. Furthermore, if family members are also using this device, the system may be configured to calculate comparison results with family members. For example, the system may be configured to output the results of a family comparison of biometric information, such as who in the family has a relatively low oxygen saturation level, from the display. In addition, information from various sensors, cameras, and health equipment installed in living spaces, lifestyle spaces, and activity spaces can be acquired from, for example, home servers, business servers (installed in workplaces, health management facilities, hospitals, etc.), and this information can be used together with biometric information to calculate and output health-related information for each individual user. Examples of such information include: (1) Sleep patterns (smart beds and wearable devices analyze sleep quality and patterns to detect signs of sleep disorders, providing advice on how to improve sleep quality), (2) Activity levels (wearable devices measure daily exercise levels and distance traveled to detect signs of inactivity or excessive exercise), (3) Stress levels (stress sensors and biofeedback devices measure skin electrical responses and heart rate variability to assess stress levels), (4) Stool composition (the amount of nutrients and substances contained in stool, e.g., dietary fiber, protein). Examples of the information that can be obtained include: (5) information on the detection of microorganisms from stool (detection of beneficial or harmful microorganisms (bacteria and viruses) contained in stool); (6) information on the evaluation of the intestinal environment (analysis of the gut flora (community of microorganisms present in the intestines) to evaluate the health status of the intestines); (7) information on indicators of inflammation (detection of inflammatory markers (such as CRP) from stool to confirm the presence of enteritis and other inflammatory diseases); (8) tumor markers (detection of specific tumor markers to aim for early detection of tumors such as colorectal cancer); and (9) information on the detection of drugs and toxins (confirmation of the presence of drugs and toxins from stool). By combining this information with the information from this device, more precise and in-depth biological information can be obtained.

[0046] The face recognition unit determines whether a person's face is being photographed with a predetermined angle of view using a small-diameter camera. The angle of view is the range that is actually captured when a camera takes a picture, expressed in degrees. A wider angle of view means a wider range is captured by the lens, while a narrower angle of view allows for capturing objects at a greater distance. A shorter focal length widens the angle of view, and a longer focal length narrows the angle of view. In this embodiment, the predetermined field of view is the field of view from which biological information can be acquired. For example, since it is assumed that the user is imaged at an appropriate distance (20cm to 80cm) from the biological information output mirror device, the field of view is such that the face is displayed at an appropriate size.

[0047] The face recognition unit employs various face recognition algorithms, including principal component analysis for eigenface recognition, linear discriminant analysis, elastic bunch graph matching, hidden Markov models, and dynamic link matching using neuronal motivation. For example, the face recognition unit reads the user's face from videos and images, analyzes the positions of characteristic features such as eyes, nose, and mouth, and the size of the face region, and determines whether the image is captured within a predetermined field of view. The types of algorithms include the following. While the choice of which algorithm to use is a design consideration, principal component analysis is effective in this invention. (1) Haar Cascade Classifier (HCC): This algorithm is commonly used to detect faces using features. Features represent patterns of specific parts of a face (eyes, nose, mouth, etc.). HCC uses these features to detect faces in an image. (2) Convolutional Neural Networks (CNN): CNNs are very effective algorithms in image recognition. Deep learning models such as ResNet and VGG are often used for face recognition. These models learn using large amounts of data and extract and identify facial features. (3) Local Binary Patterns (LBP): LBP is a technique for recognizing fine patterns in faces. It divides an image into small regions and creates a pattern by comparing the brightness of pixels in each region. Faces are identified using this pattern. (4) Eigenfaces: Eigenfaces is a method for representing faces using principal component analysis (PCA). It converts facial images into principal components and uses those principal components to identify faces. These algorithms each take a different approach and are selected based on their intended use and the characteristics of the data.

[0048] (Embodiment 1 Configuration: Biometric Information Acquisition Unit) The biological information acquisition unit acquires biological information from images (videos) acquired by a small-diameter camera or the like. Biological information refers to various physiological and anatomical information emitted by living organisms. In this invention, the biological information acquired includes, for example, pulse rate, pulse wave signals associated with the contraction and dilation of facial blood vessels, oxygen saturation, blood pressure, body surface temperature, drowsiness, facial expression analysis results, degree of cheek elevation, depth of nasolabial folds, degree of eyelid drooping, color of the whites of the eyes, and shape of the lips.

[0049] (Embodiment 1 Configuration: Basic Biological Information Output Unit) The bio-related information output unit outputs bio-related information based on the acquired bio-information. The bio-related information output in this invention may include bio-information, and is information calculated using a predetermined calculation method based on bio-information. Specifically, this includes pulse rate, oxygen saturation, blood pressure, body temperature, stress index, emotional expressions such as joy, anger, sadness, and pleasure. The bio-related information output unit may also output the bio-related information to a display. In addition to bio-related information, the bio-related information output unit may also display short comments comparing the values ​​to normal values ​​that can be easily understood by the user. For example, if the oxygen saturation is 98%, instead of displaying "98%", or in addition to displaying "This is a normal value", the output unit may also display "This is a normal value". If the systolic blood pressure is 140 mmHg, in addition to displaying 140 mmHg, or in addition to displaying "This is a little high", the output unit may also display "This is a little high". The bio-related information output unit may be enlarged by operating a touch panel.

[0050] (Embodiment 1 Configuration: Biological Information Output Unit, External Output) The biometric information output unit may output biometric information to a display with a switchable mirror surface provided in the device, or it may be configured to output biometric information to an external device wirelessly or via wired connection based on user operation. For example, it can be configured to output biometric information via the internet to a health management server for the purpose of managing the health of individual users. It is also conceivable to output biometric information to a terminal used by the user, such as a personal computer or smartphone. Furthermore, if this device is placed on the desk of an employee working in a company, it can be configured to output to the company's employee health management server via the intranet, thereby assisting in the health management of each employee. In addition, the biometric information output in this manner may be sent to a work schedule server that plans employee work schedules, so that a more suitable work schedule is planned based on the biometric information. This planning may be done by a predetermined program or by artificial intelligence. The device placed on an employee's desk may be, for example, a personal computer and its display (with a switchable mirror surface). Furthermore, when biometric information is output to the employee health management server in this manner, the appropriateness of management may be analyzed by analyzing the biometric information of employees for each department. For example, in a high-stress workplace, the system might output advice to managers suggesting they adopt a milder management style. Furthermore, the system could be configured to manage job attribute information on a server, infer the correlation between biometric information and job attribute information, and use this information for management purposes.

[0051] (Embodiment 1 Configuration: Biological Information Output Unit; Calculation of Biological Information from Biological Information) One method for calculating bio-related information from biological data is the rPPG (Remote Photoplethysmography) method. The rPPG method is a technique that estimates biological information such as pulse rate, oxygen saturation, blood pressure, and surface temperature by analyzing changes in skin color due to blood flow by measuring changes in skin color caused by vascular vibration through acquired images. The rPPG method is a technique that detects heart rate waveforms by visualizing changes in blood volume on the surface of the face. This method is non-contact, and a camera is used in particular to analyze images of the face. (1) Analysis of the entire face: An image of the entire face is acquired, and color changes are tracked pixel by pixel. These color changes are caused by fluctuations in blood volume associated with the heartbeat. (2) Emphasis on specific areas: Specific areas where changes in blood volume are particularly noticeable, such as the forehead, cheeks, and chin, are emphasized, and the heart rate waveform is extracted from these areas. (3) Image processing technology: Using advanced image processing algorithms, noise is removed and heart rate and respiratory rate are measured with high accuracy. The advantages of the rPPG method are that it does not require direct contact with the body, making it comfortable and hygienic (non-contact); it can measure heart rate and respiration in real time from live video (real-time analysis); and it is used in a wide range of fields, including medicine, fitness, and psychological research (broad range of applications).

[0052] For example, regarding pulse rate, since hemoglobin in the blood absorbs green light, the pulse wave signal is extracted by image analysis of the reflected light from the skin surface accompanying the constriction and dilation of blood vessels, and the pulse rate is calculated from the periodic component of the measured pulse wave. For oxygen saturation, the mean and standard deviation of each color are calculated to compare the absorption rates of red and blue light, and the value is calculated using an analytical formula. Furthermore, blood pressure is calculated by applying deep learning based on a one-dimensional convolutional neural network. Additionally, body temperature is calculated by analyzing images at 30 frames / second and using an extended Kalman filter and sigmoid function on the pulse rate. This method models the fluctuations in body temperature based on pulse data and calculates body temperature based on that. The sigmoid equation is suitable for representing the shape of pulses, and the extended Kalman filter excels at predicting and updating time-series data. This approach is particularly promising for applications in the medical field, such as real-time body temperature monitoring and use as part of telemedicine. The extended Kalman filter is a powerful tool used for state estimation of nonlinear systems. The Extended Kalman Filter (EKF) can be used to remove noise from time-series data (e.g., pulse rate data) and estimate hidden variables (such as body temperature). The basic steps of EKF are: (1) Prediction step: Predict future states and update the error covariance matrix. (2) Update step: Correct the prediction using actual observation data to improve estimation accuracy. The sigmoid equation is a mathematical function commonly used to model nonlinear data, drawing an S-shaped curve where the output value changes smoothly in response to the input value. The specific procedure is as follows: (1) Data acquisition: Pulse rate data is collected in real time. Wearable devices and other sensors are used for this purpose. (2) Initialization: The initial state of the EKF (initial estimate of body temperature) and the error covariance matrix are set. (3) Modeling: The relationship between pulse rate and body temperature is modeled using the sigmoid equation. For example, pulse rate data is applied to the sigmoid function to predict body temperature fluctuations. (4) Application of EKF: Noise is removed from the time series data using EKF, and the estimated body temperature is updated in real time. The prediction and update steps are repeated to improve the estimation accuracy. This method allows for the estimation of body temperature fluctuations from pulse rate data, enabling real-time body temperature monitoring. The stress index and emotional expressions showing joy, anger, sadness, and pleasure are as follows: Based on the pulse, if the proportion of the N RRIs (RR Intervals: the time interval between one R wave indicating ventricular contraction in an electrocardiogram) where the difference in intervals between adjacent R waves exceeds a predetermined threshold is large, it is considered that the person is calm (Figure 7(a)). This is called pNN. For example, in this invention, a pNN of 50 indicates that the proportion of consecutive adjacent pulse intervals where the difference exceeds 50 ms represents the dominance of parasympathetic nervous system activity, i.e., calmness (stress index). Specifically, pNN can be calculated using the formula Conut(|R1-R2|≧threshold). Furthermore, by plotting the pNN on the horizontal axis and the results of the facial expression analysis of the captured facial image on the vertical axis, it is possible to calculate emotional expressions of joy, anger, sadness, and pleasure (Figure 7(b)). These emotional expressions are classified into nine categories: 1. Joy, 2. Satisfaction, 3. Relief, 4. Tension, 5. Neutral, 6. Calm, 7. Anger, 8. Dissatisfaction, and 9. Sadness.

[0053] The rPPG method allows for the measurement of biometric information simply by having the user stand in front of a small camera for about 10 seconds, eliminating the need to attach measuring devices such as heart rate sensors to the body. Furthermore, it is expected to reduce skin irritation caused by contact with measuring devices and stress during measurement.

[0054] The calculation of biological information and related biological information may be performed solely by the biological information output mirror device, or it may be performed via a network on an analysis server located on a cloud server. In cases of poor or unavailable communication environments, the bio-related information output mirror device may perform processing independently. When the calculation of bio-related information is performed independently by the bio-related information output mirror device, the bio-related information acquisition unit acquires bio-related information from images captured by the camera, the bio-related information output unit calculates bio-related information from the bio-related information, and outputs it to the bio-related information output mirror device. For example, the surface temperature, which is bio-related information, is calculated from images captured by the camera, and the body temperature, which is bio-related information, is output from the surface temperature. In addition, the device may output how much the current pulse rate or blood pressure differs from values ​​one month ago. If the calculation of biometric information and bio-related information from images is performed on an analysis server located on a cloud server via a network, then deep learning may be used to calculate the biometric information and bio-related information based on machine learning-prepared data.

[0055] (Embodiment 1: Acceptable Configuration - Body Temperature Sensor) The body temperature sensor 0101 is a non-contact, two-dimensional temperature measurement sensor, although it is not particularly limited. The body temperature sensor 0101 is equipped with a sensor that detects infrared radiation emitted from substances, including the human body. The intensity of the infrared radiation increases with higher temperatures and decreases with lower temperatures. The biological information acquisition unit acquires images or videos that reflect the detected intensity of the infrared radiation. The biological information output unit 0105 calculates the body temperature from the acquired images or videos. While surface temperature can also be acquired using the small-diameter camera 0103, measurements taken by the body temperature sensor 0101 are given priority and used as biometric information.

[0056] (Embodiment 1: Acceptable Configuration - Illuminance Sensor) The illuminance sensor 0103 is a sensor that senses the ambient brightness. The illuminance sensor 0103 primarily measures the illuminance of the user's face positioned in front of the mirror surface 0120 of the bio-related information output mirror device 0100 and the touch panel 0104. This is because, for analyzing the face image, it is desirable to acquire the user's face image at an illuminance within a predetermined range. Furthermore, even if the illuminance of the bio-related information output mirror device 0100 changes due to the surrounding environment or if the installation location of the bio-related information output mirror device 0100 is changed, it is desirable to acquire the face image at an illuminance within a predetermined range. If the illuminance sensor 0103 determines that the facial illumination is insufficient when acquiring biometric information, the lighting will be automatically turned on to maintain a predetermined illumination level. Illuminance will be described later in Embodiments 4 and 5.

[0057] (Embodiment 1: Permissible Configuration - Touch Panel) The touch panel 0104 is positioned on all or part of the mirror to display the user's face, etc. In other words, the touch panel 0104 acts as a display device. For example, the touch panel 0104 displays the user's face, captured by the camera 0102, on the touch panel 0104 in a horizontally inverted state, just like in a mirror. This is to maintain integration with the mirror surrounding the touch panel. The resolution of the touch panel 0104 is not particularly limited, but a resolution of 1920 x 1080 (Full HD) is preferable. A resolution of 3840 x 2160 (4K) is even more preferable. The touch panel 0104 technology is not particularly limited, but a capacitive touchscreen, which is widely used in smartphones and tablets, is preferable. The user operates the touch panel 0104 to start the measurement and acquire biometric information. Additionally, the user may operate the touch panel to display weather, time, traffic information, etc. The user may operate the bio-related information output mirror device 0100 using voice recognition. In other words, the user may operate it using voice commands such as "Start measurement."

[0058] <Description of the Network Configuration Diagram for Embodiment 1> (Embodiment 1 Network Configuration Diagram Overview) Figure 5 shows an example of a network configuration 0500 for a bio-related information output mirror device. The network configuration 0500 consists of a bio-related information output mirror device 0501, a communication network 0510, and a cloud server 0520. The cloud server 0520 includes at least an authentication server 0521, a database server 0522, and an AI analysis server 0523. The communication network 0510 is not particularly limited, but refers to a network that connects, for example, a wireless LAN such as Wi-Fi in a home or a mobile phone system such as 5G to an external public network. The authentication server 0521 is a server that verifies whether a user is a legitimate user using a password, fingerprint, facial recognition, etc. The database server 0522 is a server that stores the user's personal information, biometric information, and biometric-related information. The AI ​​analysis server 0523 is a server that calculates biometric-related information using a predetermined algorithm based on biometric information acquired from the biometric-related information output mirror device.

[0059] There are multiple methods for calculating biological information and related information using server devices. Figure 26 is a sequence diagram showing an example of processing between a biometric information output mirror device and a server device via a network. The biometric information output mirror device recognizes the user's face using a camera and acquires biometric information from the image information using a biometric information acquisition unit. Subsequently, the biometric information output mirror device transmits the acquired biometric information to the server. The server performs processing to analyze and calculate biometric information from the acquired biometric information. The server transmits the biometric information to the biometric information output mirror device that received the biometric information. The biometric information output mirror device outputs the biometric information calculated by the server. Next, Figure 27 shows an example of a sequence diagram where processing is performed solely by the server device. The bio-related information output mirror device recognizes the user's face using a camera and acquires an image including the face. Subsequently, the bio-related information output mirror device sends the acquired image to the server. The server performs processing to analyze and calculate biometric information and bio-related information from the acquired image. The server sends the bio-related information to the bio-related information output mirror device from which the image was sent. The bio-related information output mirror device outputs the bio-related information calculated by the server. Finally, Figure 28 is a sequence diagram showing an example of processing between a biometric information output mirror device and a server device via a network. The biometric information output mirror device recognizes the user's face and acquires an image containing the face. The biometric information output mirror device then sends the image to the server. The server performs processing to analyze and calculate biometric information from the image. The server sends the biometric information back to the biometric information output mirror device from which the image was sent. The biometric information output mirror device performs processing to analyze and calculate biometric information from the biometric information and outputs the biometric information.

[0060] <Description of the Functional Block Diagram of Embodiment 1> (Embodiment 1 Functional Block Diagram Overview) Figure 6 is a functional block diagram of the bio-related information output mirror device 0600 according to Embodiment 1. The bio-related information output mirror device 0600 comprises a mirror surface 0601, a face recognition unit 0602, an imaging means 0603, a bio-related information acquisition unit 0604, and a bio-related information output unit 0605.

[0061] <Description of the Hardware Configuration Diagram for Embodiment 1> (Embodiment 1 Hardware Configuration Diagram Overview) Figure 8 shows an example of the hardware configuration of the biological information output mirror device 0800. The biological information output mirror device 0800 includes a CPU 0801, a non-volatile memory 0802, a main memory 0803, an I / O controller 0804, a communication unit 0805, etc.

[0062] (Embodiment 1 Hardware Configuration Diagram CPU) The CPU (Central Processing Unit) 0801 sequentially reads, interprets, and executes a program, which is a sequence of instructions located in the non-volatile memory 0802, and outputs information consisting of signals to the main memory 0803. The CPU 0801 functions as the central hub for performing calculations within the computer. In this invention, the CPU 0801 provides overall control over the biological information output mirror device 0800.

[0063] In Embodiment 1, the CPU 0801 executes an image processing program deployed on the main memory 0803 and issues a command to perform image processing. Specifically, the CPU 0801 may instruct the cloud server to send the acquired image via the communication unit 0804, undergo AI analysis, and acquire biometric information. Alternatively, the image program may instruct the system to recognize the facial information portion in the acquired image and acquire biometric information within the biometric information output mirror device 0800.

[0064] (Embodiment 1 Hardware Configuration Diagram: Non-volatile memory) The non-volatile memory 0802 stores user information, including authentication information for facial patterns used in facial recognition. Authentication is performed based on the authentication information for facial patterns stored in the non-volatile memory 0802. The non-volatile memory 0802 also stores the authentication program and a predetermined number of authentication failures that are permitted, as described above. This user information is read from the non-volatile memory 0802 and loaded onto the main memory 0803, for example, triggered by power-on. Biometric data measured in the past may be stored, or it may not be stored from the standpoint of protecting personal information.

[0065] (Embodiment 1 Hardware Configuration Diagram Main Memory) Main memory 0803 is read by the CPU 0801 to execute programs that perform various processes, and at the same time, it provides a work area that serves as the working area for those programs. Main memory 0803 is volatile memory and uses Dynamic Random Access Memory (DRAM). Programs on main memory 0803 are loaded from non-volatile memory 0802 to main memory 0803, for example, after receiving a program startup instruction. The CPU 0801 directly accesses and executes the various programs on main memory 0803. Subsequently, the CPU 0801 executes the program according to various execution instructions and execution procedures within the program.

[0066] (Embodiment 1 Hardware Configuration Diagram Input / Output Interface Serial Bus) The input / output interface 0804 connects to an external device and transmits received signals to the CPU 0801 via the serial bus 0805.

[0067] (Embodiment 1 Hardware Configuration Diagram: Communication Unit) The communication unit 0805 exchanges information with the outside world of the bio-related information output mirror device 0800. For example, the communication unit 0805 transmits image information and other data acquired by the bio-related information output mirror device 0800 to a dedicated cloud server. The communication unit 0805 also receives bio-related information and other data analyzed by AI on the dedicated cloud server.

[0068] In this embodiment, the non-volatile memory 0802 may store a face recognition program, a biometric information acquisition program, and a biometric information output program.

[0069] The face recognition program is read from the non-volatile memory 0802 and loaded onto the main memory 0803, triggered by an operation on the touch panel after the biometric information output mirror device 0800 is powered on. The face recognition program recognizes the user's face within a predetermined time. Furthermore, the facial information of the user to be recognized can be registered in advance along with the user's information.

[0070] The biometric information acquisition program is read from the non-volatile memory 0802 and loaded onto the main memory 0803 after the biometric information output mirror device 0800 is powered on and triggered by an operation on the touch panel. The biometric information acquisition program recognizes the user's face within a predetermined time and then acquires facial image information within a predetermined time. By processing this facial image within the biometric information output mirror device 0800 and / or on the cloud server, biometric information can be calculated.

[0071] The biological information output program is read from the non-volatile memory 0802 and loaded onto the main memory 0803 after the biological information output mirror device 0800 is powered on and triggered by an operation on the touch panel. After acquiring biological information with the biological information output mirror device 0800, the biological information output program performs AI analysis using a predetermined algorithm and outputs the biological information.

[0072] <Description of the processing flow in Embodiment 1> (Embodiment 1: Overview of the Processing Flow) Figure 9 is a flowchart showing an example of the processing flow of the bio-related information output mirror device 0900. Embodiment 1 includes at least a face recognition step (S0901), a bio-related information acquisition step (S0902), and a bio-related information output step (S0903).

[0073] When the power to the biometric information output mirror device is turned on, the face recognition program, biometric information acquisition program, and biometric information output program are activated, and processing becomes possible. The face recognition step (S0901) is the process of recognizing the user's face. In principle, since faces viewed from the front are registered, face recognition is performed by acquiring a face viewed from the front. This means that faces viewed from an angle, such as a profile, are less likely to be recognized.

[0074] The biometric information acquisition step (S0902) acquires biometric information from facial images, facial video, infrared temperature information, etc., after the facial recognition step.

[0075] The biological information output step (S0903) outputs biological information calculated based on biological information.

[0076] <Effects of Embodiment 1> This embodiment provides a biometric information output mirror device that acquires facial images and other data, and displays biometric information and related biometric information. Furthermore, the biometric information and related biometric information output by the biometric information output mirror device can be used for user health management, employee attendance management, and workplace accident prevention.

[0077] <Embodiment 2> Embodiment 2 mainly relates to claim 2, etc.

[0078] <Overview of Embodiment 2> Embodiment 2 is based on Embodiment 1. The bio-related information output mirror device of the present invention further includes an image acquisition range display unit that indicates within the mirror surface the range of the image used to acquire bio-information from the human figure reflected on the mirror surface during bio-information acquisition.

[0079] (Description of Embodiment 2) Embodiment 2 will be described with reference to Figure 10. Embodiment 2 has, in principle, the same configuration as Embodiment 1.

[0080] (Embodiment 2 Structure: Corner frame) Figure 10 shows the rectangular frame 1006 displayed within the touch panel 1004 by the image acquisition range display unit. The image acquisition range display unit displays a rectangular frame 1006 within the touch panel 1004. This limits the detection of human faces to a predetermined range, allowing for an earlier transition to face recognition processing. The user adjusts their standing position and left-right balance so that their face fits within this rectangular frame 1006.

[0081] (Embodiment 2 Functional Block Diagram: Image Acquisition Range Display Unit) Figure 11 is a functional block diagram showing an example of the functions of the bio-related information output mirror device 1100 according to Embodiment 2. The image acquisition range display unit 1106 specifies the range (corner frame) to be recognized when recognizing the user's face information displayed on the mirror surface 1101 and the touch panel. In other words, the corner frame is the range of the face image for acquiring the user's bio-information.

[0082] (Embodiment 2 Hardware Configuration Diagram Image Acquisition Range Display Program) Figure 12 is a hardware configuration diagram showing an example of the hardware configuration of the bio-related information output mirror device 1200 according to Embodiment 2. The image acquisition range display program is read from the non-volatile memory 1202 and loaded onto the main memory 1203, triggered by an operation on the touch panel after the power-up of the bio-related information output mirror device 1200. The image acquisition range display program displays a rectangular frame on the touch panel for detecting the user's face within a predetermined time.

[0083] (Embodiment 2: Processing Flow - Image Acquisition Range Display Unit) Figure 13 is a flowchart showing an example of the processing flow of the bio-related information output mirror device 1300. Embodiment 2 includes a face recognition step (S1302), a bio-related information acquisition step (S1303), and a bio-related information output step (S1304), as well as an image acquisition range display step (S1301). The image acquisition range display step (S1301) is performed prior to the face recognition step, by displaying the area (corner frame) for face recognition on the touch panel.

[0084] <Effects of Embodiment 2> This embodiment provides a biometric information output mirror device that acquires a user's biometric information by capturing facial images or videos with an appropriate field of view, and displays the biometric information and related biometric information.

[0085] <Embodiment 3> Embodiment 3 mainly relates to claim 3, etc.

[0086] <Overview of Embodiment 3> Embodiment 3 is based on Embodiment 1 or Embodiment 2. The bio-related information output mirror device of the present invention includes a surface temperature measurement unit that measures the surface temperature of a person's face when the face recognition unit determines that a person's face is being photographed at a predetermined angle of view, and a bio-information acquisition unit that includes a surface temperature acquisition means for acquiring the measured surface temperature as bio-information.

[0087] (Description of the configuration of Embodiment 3) Figure 14 illustrates Embodiment 3. Embodiment 3, in principle, has the same configuration as Embodiment 1 or Embodiment 2.

[0088] (Embodiment 3 Structure: Surface temperature measurement unit) Figure 14 shows the surface temperature measurement range 1406, where the area where the surface temperature of the face is measured by the surface temperature measurement unit is displayed on the touch panel. A person's body temperature varies depending on whether it's at the surface or inside the body. Furthermore, it also varies depending on the location within the body. The internal body temperature is called the core temperature, and the temperature near the body surface is called the shell temperature. The core temperature is high and stable to maintain the function of vital organs such as the brain and heart. In a normal environment (around 25°C), the core temperature is around 37°C. The shell temperature is lower than the core temperature because heat is dissipated from the body surface. The temperature of the skin, in particular, which is in contact with the outside environment, is low, around 32-33°C. There are also significant individual differences, and even within the same person, temperatures can vary greatly depending on the body part. The surface temperature measurement range 1406 obtains reliable biometric information by consistently measuring the same location on a person's face, such as the forehead or cheeks.

[0089] (Embodiment 3 Functional Block Diagram: Image Acquisition Range Display Unit) Figure 15 is a functional block diagram showing an example of the functions of the bio-related information output mirror device 1500 according to Embodiment 3. The surface temperature measuring unit 1506 recognizes the areas of the user's face information displayed on the mirror surface 1501 and the touch panel, and measures the surface temperature of these areas. For example, the forehead and cheeks. The forehead and cheeks have a relatively large surface area and are easy to recognize, and a rise in surface temperature can also be observed if there is a high fever.

[0090] The surface temperature measuring means 1507 measures the surface temperature of the forehead and cheek areas of the face. The surface temperature measuring means 1507 is not particularly limited, but in this embodiment it is a non-contact type body temperature sensor 1501.

[0091] (Embodiment 3 Hardware Configuration Diagram Image Acquisition Range Display Program) Figure 16 is a hardware configuration diagram showing an example of the hardware configuration of the bio-related information output mirror device 1600 according to Embodiment 3. The surface temperature measurement program and surface temperature acquisition program are read from the non-volatile memory 1602 and loaded onto the main memory 1603, triggered by an operation on the touch panel after the power is turned on of the bio-related information output mirror device 1600. The surface temperature measurement program and surface temperature acquisition program detect the surface temperature of the user's forehead and cheeks within a predetermined time after face recognition.

[0092] (Embodiment 3: Processing Flow - Image Acquisition Range Display Unit) Figure 17 is a flowchart showing an example of the processing flow of the bio-related information output mirror device 1700. Embodiment 3 includes a face recognition step (S1701), a bio-related information acquisition step (S1704), and a bio-related information output step (S1705), as well as a surface temperature measurement step (S1702) and a surface temperature acquisition substep (S1705). The surface temperature measurement step (S1702) measures the surface temperature of the forehead and cheeks of the face, and the surface temperature acquisition substep (S1703) acquires the surface temperature of the forehead and cheeks of the face.

[0093] <Embodiment 3 Effects> This embodiment provides a biological information output mirror device that acquires biological information, including the surface temperature of the forehead and cheeks of the face, and displays the biological information and related biological information.

[0094] <Embodiment 4> Embodiment 4 mainly relates to claim 4, etc.

[0095] <Embodiment 4 Overview> Embodiment 4 is based on Embodiments 1 to 3. The bio-related information output mirror device of the present invention further includes an illuminance measuring unit that measures the illuminance of a person's face when the face recognition unit determines that a person's face is being photographed at a predetermined angle of view, an illuminance determination unit that determines whether the measured illuminance is within a predetermined range, a lighting unit 1707 that improves the illuminance of the person's face being photographed when the determination result from the illuminance determination unit is determined to be below a predetermined range, and a lighting control unit that controls the lighting unit.

[0096] (Description of Embodiment 4) Figure 18 illustrates Embodiment 4. Embodiment 4, in principle, has the same configuration as Embodiments 1 to 3.

[0097] (Overview of Embodiment 4) Figure 18 illustrates how the illuminance was measured by the lighting measurement unit, and because the left side of the face was dark, the lighting on the left side was turned on.

[0098] (Embodiment 4 Structure: Lighting Unit) The lighting unit 1807 is located circumferentially on the outside of the touch panel 1804 and on the inside of the housing 1810. The lighting unit 1807 is not particularly limited, but in this embodiment it is an LED light embedded in the mirror surface 1820. For example, three LED lights can be evenly arranged at predetermined intervals in the horizontal frames on the top and bottom, and five in the vertical frames on the left and right.

[0099] (Embodiment 4 Functional Block Diagram: Illuminance Measurement Unit) Figure 19 is a functional block diagram showing an example of the functions of the bio-related information output mirror device 1900 according to Embodiment 4. In addition to the normal functions, the bio-related information output mirror device 1900 includes an illuminance measuring unit 1906, an illuminance determination unit 1907, an illumination unit 1908, and an illumination control unit 1909. The illuminance measuring unit 1906 measures the illuminance of the user's face. The illuminance measuring unit 1906 quantifies the brightness of the face illuminated by light such as sunlight or indoor lighting, and measures the luminous flux per unit area. For example, the illuminance of the face is acquired by the illuminance sensor 1901.

[0100] (Embodiment 4 Functional Block Diagram: Illuminance Determination Unit) The illuminance determination unit 1907 determines whether the illuminance measured by the illumination measurement unit 1906 is within a predetermined range. The predetermined range is 300 lux to 700 lux, with 400 lux to 600 lux being more desirable.

[0101] (Embodiment 4 Functional Block Diagram: Lighting Unit) The lighting unit 1908 is not particularly limited, but is an LED light. It is desirable that the lighting unit 1908 is not arranged in a single location, but rather that multiple light sources are evenly distributed. For example, three LED lights can be evenly distributed at predetermined intervals in the horizontal frames at the top and bottom, and five LED lights can be evenly distributed in the vertical frames on the left and right.

[0102] (Embodiment 4 Functional Block Diagram: Illuminance Control Unit) The lighting control unit 1909 measures the illuminance using the illuminance measuring unit 1906, and the illuminance determination unit 1907 determines whether or not the illuminance has reached a predetermined standard. If the predetermined standard has not been reached, the lighting unit 1908 is automatically turned on. For example, the illuminance control unit 1909 may turn on all of the LEDs mounted on the housing (top, bottom, left, and right), or it may turn on only specific LEDs. Subsequently, the illuminance measuring unit 1906 measures the illuminance again, and the illuminance judgment unit 1907 determines whether the illuminance has reached a predetermined standard. If the predetermined standard has not been reached, the other lighting units 1908 are automatically turned on. For example, as shown in Figure 18, if the left side of the face is dark and the illuminance is insufficient, the lighting control unit 1909 turns on the three LED lights on the left side.

[0103] (Embodiment 4 Hardware Configuration Diagram Illuminance Measurement Program) Figure 20 is a hardware configuration diagram showing an example of the hardware configuration of the bio-related information output mirror device 2000 according to Embodiment 4. The illuminance measurement program is read from the non-volatile memory 2002 and loaded onto the main memory 2003, triggered by an operation on the touch panel after the bio-related information output mirror device 2000 is powered on. The illuminance measurement program measures the illuminance of the user's face within a predetermined time after face recognition.

[0104] (Embodiment 4 Hardware Configuration Diagram Illuminance Judgment Program) The illuminance judgment program is read from the non-volatile memory 2002 and loaded onto the main memory 2003, triggered by an operation on the touch panel after the power is turned on to the bio-related information output mirror device 2000. The illuminance judgment program determines whether the illuminance measured by the illuminance measurement program has reached a predetermined standard.

[0105] (Embodiment 4 Hardware Configuration Diagram Lighting Program) The lighting program is read from the non-volatile memory 2002 and loaded onto the main memory 2003, triggered by an operation on the touch panel after the power-up of the bio-related information output mirror device 2000. The lighting program then turns on the lights.

[0106] (Embodiment 4 Hardware Configuration Diagram Lighting Control Program) The lighting control program is read from the non-volatile memory 2002 and loaded onto the main memory 2003, triggered by an operation on the touch panel after the power-up of the bio-related information output mirror device 2000. The lighting control program controls which lights to turn on if it determines that the illuminance does not meet the standard.

[0107] (Embodiment 4: Overview of the Processing Flow) Figure 21 is a flowchart showing an example of the processing flow of the bio-related information output mirror device 2100. Embodiment 4 includes a face recognition step (S2101), a bio-related information acquisition step (S2106), and a bio-related information output step (S2107), as well as an illuminance measurement step (S2102), an illuminance determination step (S2103), an illumination step (S2104), and an illumination control step (S2105).

[0108] The illuminance measurement step (S2102) measures the illuminance on the face, the illuminance determination step (S2103) determines whether the illuminance measured in the illuminance measurement step has reached a predetermined standard, the lighting step (S2104) turns on the lights if it is determined that the illuminance does not meet the standard, and the lighting control step (S2105) controls which lights to turn on if it is determined that the illuminance does not meet the standard.

[0109] <Embodiment 4 Effects> This embodiment provides a bio-related information output mirror device that can measure biological information on a face within a predetermined brightness range by measuring the illuminance of the face and controlling the lighting.

[0110] <Embodiment 5> Embodiment 5 mainly relates to claim 5, etc.

[0111] <Embodiment 5 Overview> Embodiment 5 is based on Embodiments 1 to 4. The bio-related information output mirror device of the present invention has a lighting unit which has light sources at multiple locations, and a lighting control unit which has a light source illuminance control means which changes the illuminance for each of the multiple light sources.

[0112] (Description of Embodiment 5) Figure 22 illustrates Embodiment 5. Embodiment 5, in principle, has the same configuration as Embodiments 1 to 4.

[0113] (Overview of Embodiment 5) Figure 22 illustrates how the illumination measurement unit measures illuminance, showing that when the left side of the face is dark, the lower side tends to be darker. Therefore, the illuminance of the lower left lighting is increased compared to the upper left lighting.

[0114] (Embodiment 5 Structure) This is the same as in Embodiment 4. The lighting unit 2207 has multiple levels of illuminance. Based on the illuminance determined by the illuminance determination unit, the illuminance control unit (illuminance control means for each light source) turns on the LED lighting with the appropriate illuminance.

[0115] (Embodiment 5 Functional Block Diagram: Illuminance Control Means for Each Light Source) Figure 23 is a functional block diagram showing an example of the functions of the bio-related information output mirror device 2300 according to Embodiment 5. In addition to the normal functions, the bio-related information output mirror device 2300 includes an illuminance measurement unit 2306, an illuminance determination unit 2307, an illumination unit 3208, and an illumination control unit 2309, as well as an illuminance control unit 2310 for each light source. The light source illuminance control means 2310 controls the illuminance of each light source based on the illuminance determined by the illuminance determination unit 2306, and turns on the lighting with the appropriate illuminance. The lighting can be changed in multiple stages. For example, if there are 3 stages, they are low, medium, and high, and if there are 5 stages, they are 1, 2, 3, 4, and 5 from lowest to highest. In Figure 22, the second LED light from the bottom left has high illumination, the middle LED light has normal illumination, and the second LED light from the top left has low illumination.

[0116] (Embodiment 5 Hardware Configuration Diagram: Illuminance Control Program for Each Light Source) Figure 24 is a hardware configuration diagram showing an example of the hardware configuration of the bio-related information output mirror device 2400 according to Embodiment 5. The light source illuminance control program is activated after the power is turned on to the bio-related information output mirror device 2400. The data is read from the non-volatile memory 2302 in response to an operation on the touch panel and loaded onto the main memory 2403. If the illuminance measured by the illuminance measurement unit does not reach a predetermined standard, the illuminance control program for each light source controls which light source to turn on and at what illuminance level.

[0117] (Embodiment 5: Overview of the Processing Flow) Figure 25 is a flowchart showing an example of the processing flow of the bio-related information output mirror device 2500. Embodiment 5 includes a face recognition step (S2501), a bio-related information acquisition step (S2507), a bio-related information output step (S2508), an illuminance measurement step (S2502), an illuminance determination step (S2503), an illumination step (S2504), an illumination control step (S2505), and an illuminance control step for each light source (S2506). The light source illuminance control step (S2506) controls which light source to turn on and at what illuminance level if the illuminance measured by the illuminance measuring unit does not reach a predetermined standard.

[0118] <Embodiment 5: Effects> This embodiment provides a bio-related information output mirror device that can measure biological information on a face within a predetermined brightness range by measuring the illuminance of the face and controlling it for each lighting source. [Explanation of Symbols]

[0119] 0100 Biological Information Output Mirror Device 0101 Body temperature sensor 0102 Small Diameter Camera 0103 Illuminance Sensor 0104 Touch Panel 0105 Biological Information Output Unit 0110 Casing 0120 Mirror surface 0405 Biological Information Output Unit 0406 Remeasurement display 0500 Network system for a biological information output mirror device 0501 Biological Information Output Mirror Device 0510 Communication Network 0520 Cloud Server 0521 Authentication Server 0522 Database Server 0523 AI Analysis Server 0603 Imaging means 0801 CPU 0802 Non-volatile memory 0803 Main Memory 0804 I / O Controller 1006 Display of image acquisition range 1106 Image acquisition range display section 1406 Surface temperature measurement range 1506 Surface temperature measurement section 1507 Surface temperature acquisition means 1807 Lighting Department 1906 Illuminance measurement section 1907 Illuminance judgment section 1909 Lighting Control Unit 2310 Illumination level control means for each lighting fixture

Claims

1. The casing and A mirror surface is placed on the surface of the housing, A small camera is positioned on a part of the mirror surface to photograph the face of a person reflected in the mirror surface, A face recognition unit that determines whether a person's face is being captured by the aforementioned small-diameter camera at a predetermined angle of view, The aforementioned facial recognition unit recognizes a person A biometric information acquisition unit, which determines that a person's face has been photographed at a predetermined angle of view, acquires biometric information based on the photographed person's face. A biological information output unit that outputs biological information based on acquired biological information, A biological information output mirror device having the following features.

2. The bio-related information output mirror device according to claim 1, further comprising an image acquisition range display unit that indicates within the mirror surface the range of the image used to acquire bio-related information from the reflection of a person on the mirror surface while bio-related information is being acquired.

3. When the face recognition unit determines that it has captured a person's face at a predetermined angle of view, a surface temperature measuring unit measures the surface temperature of the person's face. The biological information acquisition unit is a surface temperature acquisition means for acquiring the measured surface temperature as biological information, as described in claim 1 or claim 2 of the biological information output mirror device.

4. When the face recognition unit determines that it has captured a person's face at a predetermined angle of view, the illuminance measuring unit measures the illuminance of the person's face, An illuminance determination unit that determines whether the measured illuminance is within a predetermined range, If the illuminance determination unit determines that the result is below a predetermined range, the lighting unit improves the illuminance of the face of the person being photographed. A lighting control unit that controls the aforementioned lighting unit, A biological information output mirror device according to claim 1 or claim 2, further comprising:

5. The aforementioned lighting unit has light sources in multiple locations, The bio-related information output mirror device according to claim 4, wherein the lighting control unit has a light source illuminance control means for changing the illuminance for each of the multiple light sources.

6. A method executed by the CPU in a computer-based bio-related information output mirror device, The casing and A mirror surface is placed on the surface of the housing, A small camera is positioned on a part of the mirror surface to photograph the face of a person reflected in the mirror surface, A face recognition step in which the small-diameter camera determines whether a person's face is being captured at a predetermined angle of view, If the face recognition step determines that a person's face has been captured at a predetermined angle of view, the biometric information acquisition step involves acquiring biometric information based on the captured person's face. A bio-related information output step that outputs bio-related information based on acquired bio-information, A method of having.

7. A method executed by the CPU in a computer-based bio-related information output mirror device, The method according to claim 6, further comprising an image acquisition range display step that indicates within the mirror surface the range of the image used to acquire biometric information from the reflection of a person on the mirror surface during biometric information acquisition.

8. A method executed by the CPU in a computer-based bio-related information output mirror device, If the face recognition step determines that a person's face has been captured at a predetermined angle of view, the surface temperature measurement step measures the surface temperature of the person's face. The method according to claim 6 or 7, wherein the biological information acquisition step is further comprising a surface temperature acquisition substep of acquiring the measured surface temperature as biological information.

9. A method executed by the CPU in a computer-based bio-related information output mirror device, If the face recognition step determines that a person's face has been photographed at a predetermined angle of view, the illuminance measurement step measures the illumination of the person's face. An illuminance determination step to determine whether the measured illuminance is within a predetermined range, If the result of the illuminance determination step is determined to be below a predetermined range, a lighting step is performed to improve the illuminance of the face of the person being photographed. A lighting control step that controls the aforementioned lighting step, The method according to claim 6 or claim 7, further comprising:

10. A method executed by the CPU in a computer-based bio-related information output mirror device, The aforementioned lighting step has light sources in multiple locations, The method according to claim 9, wherein the lighting control step has a sub-step for controlling the illuminance for each of the multiple light sources.

11. The casing and A mirror surface is placed on the surface of the housing, A small camera is positioned on a part of the mirror surface to photograph the face of a person reflected in the mirror surface, A face recognition step in which the small-diameter camera determines whether a person's face is being captured at a predetermined angle of view, If the face recognition step determines that a person's face has been captured at a predetermined angle of view, the biometric information acquisition step involves acquiring biometric information based on the captured person's face. A bio-related information output step that outputs bio-related information based on acquired bio-information, An operating program for a bio-related information output mirror device, which is a computer having a bio-related information output mirror device, written in a readable and executable format.

12. An operation program for a bio-related information output mirror device described in claim 11, which is a computer, further comprising an image acquisition range display step that indicates within the mirror surface the range of the image used to acquire bio-related information from the reflection of a person on the mirror surface during bio-related information acquisition.

13. If the face recognition step determines that a person's face has been captured at a predetermined angle of view, the surface temperature measurement step measures the surface temperature of the person's face. The biological information acquisition step is an operation program for a biological information output mirror device described in a readable and executable manner, which is a computer according to claim 11 or claim 12, having a surface temperature acquisition substep for acquiring the measured surface temperature as biological information.

14. If the face recognition step determines that a person's face has been photographed at a predetermined angle of view, the illuminance measurement step measures the illumination of the person's face. An illuminance determination step to determine whether the measured illuminance is within a predetermined range, If the result of the illuminance determination step is determined to be below a predetermined range, a lighting step is performed to improve the illuminance of the face of the person being photographed. A lighting control step that controls the aforementioned lighting step, An operation program for a bio-related information output mirror device described in a readable and executable manner by the bio-related information output mirror device, which is a computer according to claim 11 or claim 12, further comprising:

15. The aforementioned lighting step has light sources in multiple locations, The aforementioned lighting control step is an operation program for a bio-related information output mirror device described in a readable and executable manner, which is a computer according to claim 14, having a substep for controlling the illuminance for each of the multiple light sources.

Images