Display device and display method

Abstract

This technology displays the presence of a camera to prevent unauthorized recording. [Solution] The display device comprises a display element, a camera positioned on the back side of the display element, and a control unit that displays an indicator showing the presence of the camera at a position corresponding to the camera within the display area of ​​the display element.

Description

Technical Field

[0001] The present invention relates to a display device and a display method.

Background Art

[0002] In smartphones and tablet terminals, cameras are used for video chat, user face authentication, and the like. Also, in devices such as display devices mounted on vehicles, cameras are used for recording when an accident occurs in the vehicle and detecting the state of a user such as an occupant. In recent years, in order to reduce the size and narrow the bezel of these display devices, a camera is sometimes arranged on the back side of the display element of the display device.

[0003] Patent Document 1 discloses a display device including a transmissive liquid crystal panel, a backlight unit arranged on the back side of the liquid crystal panel, and a camera arranged on the back side of the backlight unit.

Prior Art Literature

Patent Literature

[0004]

Patent Literature 1

Summary of the Invention

Problem to be Solved by the Invention

[0005] When a camera is arranged on the back side of a display element, there is a possibility that the user's face or the like is photographed before the user notices the existence of the camera. This may cause the user to feel uncomfortable from the viewpoint of privacy and the like.

[0006] An object of the technology of the present disclosure is to provide a technology that displays the existence of a camera and suppresses photographing performed without the user's notice.

Means for Solving the Problem

[0007] To solve the above problems, the display device of this disclosure is Display element and A camera positioned on the back side of the display element, The system includes a control unit that causes an indicator to be displayed at a position corresponding to the camera within the display area of ​​the display element, indicating the presence of the camera. [Effects of the Invention]

[0008] The technology disclosed herein can provide a method to display the presence of a camera and prevent unauthorized recording from occurring without the user's knowledge. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a front view showing the configuration of a display device according to the first embodiment. [Figure 2] Figure 2 is a partial longitudinal cross-sectional view of a display device. [Figure 3] Figure 3 shows the configuration of the control device. [Figure 4] Figure 4 shows an indicator that shows the presence of a camera. [Figure 5] Figure 5 shows an example of an indicator. [Figure 6] Figure 6 shows the flow of the process for determining whether or not to consent to taking photos with the indicators hidden, in the first embodiment. [Figure 7] Figure 7 shows an example of a selection screen to be displayed on a display element. [Figure 8] Figure 8 shows the flow of the display method executed by the display device controller according to the application program (display control program). [Figure 9] Figure 9 shows the flow of the process in the second embodiment where the user agrees to take a photo with the indicator hidden and sets whether or not to register facial information. [Figure 10] Figure 10 is a diagram showing the flow of a display method executed by the controller of the display device according to the display control program in the second embodiment. [Figure 11] Figure 11 shows an example of a registration screen to be displayed on the display element. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings. The configurations of the following embodiments are illustrative, and the present invention is not limited to the configurations of these embodiments.

[0011] <First Embodiment> <Display device configuration> Figure 1 is a front view showing the configuration of the display device according to this embodiment, and Figure 2 is a partial longitudinal cross-sectional view of the display device. The display device 100 of this embodiment includes a display element 20 and a camera 30 installed on the back side of the display element 20 inside the housing 10. The display device 100 also has a control device 40 that controls the display of the display element 20 and the operation of the camera 30. In the following description, the side of the display device 100 that displays images (display surface side) will be referred to as the front side, and the opposite side will be referred to as the back side. In the following description, the display device 100 will mainly be shown as an example of a CID (Center Information Display) mounted on a vehicle that displays map information, images of the vehicle's surroundings, air conditioning information, fuel efficiency information, etc., but the display device 100 is not limited to this. For example, the display device 100 may be a navigation device, a drive recorder, or audio equipment. Furthermore, the display device 100 is not limited to an in-vehicle device, but may be a smartphone, a tablet terminal, a notebook PC (personal computer), etc.

[0012] The display element 20 is, for example, a liquid crystal display element, and as shown in Figure 2, TSTN (Triple Super A Twisted Nematic display panel (liquid crystal panel) 21 is configured to be sandwiched between a front polarizing plate 22 arranged on the front side and a rear polarizing plate 23 arranged on the rear side. Furthermore, a backlight 25 is provided on the rear side of the rear polarizing plate 23. It should be noted that the display element 20 is not limited to a liquid crystal display element, and may be a display panel using OLED (Organic Light Emitting Diode), that is, a so-called organic EL panel. In this case, the backlight 25 may be omitted. At least a part of the display panel 21 is formed to be transparent so that a subject on the front side can be photographed by a camera 30 provided on the rear side. In addition, a touch panel 27 for detecting a touch operation may be provided on the display surface side of the display element 20.

[0013] The backlight 25 is configured to emit light from a cold cathode fluorescent tube serving as a light source, an organic electroluminescent sheet, an LED (Light Emitting Diode), or the like toward the front side to illuminate the display panel. In addition, the backlight 25 is provided with a hole 26 for passing light from a subject transmitted through the display panel 21 at a position corresponding to the camera 30 provided on the rear side (near the upper center in the example of FIG. 1). The lens 32 of the camera 30 is arranged at the position of this hole 26.

[0014] In addition, the display element 20 includes a drive circuit that drives each pixel of the display panel 21 in accordance with an input image signal (video signal). When the display element 20 acquires display image data from an externally connected computer or the like (not shown), it drives the display panel 21 to display an image corresponding to the display image data.

[0015] The camera 30 includes a solid-state imaging element 31 such as a CCD (Charge Coupled Device) element or a CMOS (Complementary Metal Oxide Semiconductor) element, and light incident from the front side comprising a lens 32 that forms an image of this on the imaging surface of the solid-state imaging device. The lens 32 is not particularly limited as long as it has a configuration capable of forming an image of a subject on the imaging surface of the solid-state imaging device 31. For example, the lens 32 may be a single-focus lens or a zoom lens. The camera 30 also includes an image processing circuit that outputs a signal captured by the solid-state imaging device as imaging data (captured image). The camera 30 may be configured to obtain still images or moving images. Furthermore, the camera 30 may be a so-called ToF (Time of Flight) sensor or a three-dimensional scanner as long as it captures an image of a subject.

[0016] FIG. 3 is a diagram showing the configuration of the control device 40. The control device 40 is an information processing device (computer) including a controller (control unit) 401, a storage unit 402, and an input / output interface (IF) 403 mutually connected via a connection bus 410. For example, the control device 40 may be an MCU (Micro Controller Unit) provided in the display device 100.

[0017] The controller 401 controls the entire display device, and is constituted by, for example, a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a main storage device (main memory). The controller 401 is not limited to a configuration including a single processor, and may have a multiprocessor configuration. The controller 401 may have a configuration in which a single processor connected via a single socket has multiple cores. The main storage device is used as a work area for the controller 401, a storage area for programs and data, and a buffer area for communication data. The main storage device includes, for example, Random Access Memory (RAM), or a combination of RAM and Read Only Memory (ROM).

[0018] The memory unit 402 is an auxiliary storage device that stores programs executed by the controller 401, as well as operation setting information. The memory unit 402 is not limited to an internal storage device built into the display device 100, but may also be an external storage device such as an external storage device or a NAS (Network Attached Storage). The memory unit 402 may be, for example, an HDD (Hard-disk Drive) or an SS These include D (Solid State Drive), EPROM (Erasable Programmable ROM), flash memory, USB memory, memory cards, etc.

[0019] The input / output IF403 is an interface for inputting and outputting data with peripheral devices. The input / output IF403 may, for example, input and output data with devices such as disk drives that read data from storage media such as CDs and DVDs, readers / writers that read and write data to storage media such as memory cards, microphones, speakers, CCUs (Communication Control Units), touch panels 27, display elements 20, and cameras 30. The CCU communicates with other devices via a communication line. For example, the CCU connects to a network such as the Internet using wireless communication and communicates with external data servers and other devices. The CCU also communicates content via a network such as a CAN (Controller Area Network). It communicates with playback devices, navigation systems, ECUs (Electronic Control Units), etc. cormorant.

[0020] Figure 4 shows an indicator representing the presence of camera 30. In this embodiment, camera 30 is located near the upper center of the display area 28 when viewed from the front, as shown by the dotted line in Figure 1. Therefore, the indicator 35 displayed at the position corresponding to camera 30 is displayed near the upper center of the display area 28, as shown in Figure 4. Here, the position corresponding to camera 30 is a position where the presence of the camera can be indicated by the display of indicator 35, and is, for example, a position where camera 30 overlaps when viewing the display element 20 from the front (display surface). Furthermore, the position corresponding to camera 30 may include not only the position overlapping with camera 30 but also its periphery. Note that the periphery may be, for example, a predetermined distance from the outer edge of camera 30 on the display surface of the display element 20. This predetermined distance may be a specific value such as 1 mm to 10 mm. It may be defined as a percentage of the screen size, such as 1% to 10% of the screen (display area) size.

[0021] The indicator 35 may be a letter indicating camera 30, an icon resembling camera 30 or lens 32, a symbol indicating the location of camera 30, or a frame surrounding the installation location of camera 30. For example, the indicator 35 in Figure 4 includes a letter 351 indicating camera 30, an icon 352 resembling lens 32, and a frame 353 surrounding the installation location of camera 30. However, the indicator 35 may be any one of the letters, icons, symbols, or frames, or any combination of two or more of them.

[0022] Figure 5 shows examples of indicator 35. Figures 5(a) to (d) show examples of text such as "Camera," "CAM," "Shooting," and "DMS Activated." These texts may indicate the camera, as in (a) and (b), or they may indicate functions that use the camera, as in (c) and (d). Also, Figures 5(e) to (g) are icons that indicate the camera, and Figure 5(h) is a symbol that indicates the camera's position.

[0023] <Display device operation> Figure 6 shows the flow of the process for setting whether or not to consent to taking a photograph with the indicator 35 hidden in this embodiment. The controller 401 of the display device 100 starts the process shown in Figure 6 when the user touches the indicator, performs a predetermined gesture in front of the camera 30, or performs an operation to start the setting.

[0024] In step S10, the controller 401 outputs an option to the user: whether or not to take a picture with the indicator 35 hidden. Figure 7 shows an example of a selection screen to be displayed on the display element 20. In the example in Figure 7, a message 61 prompting a selection, a button 62 to agree to taking a picture with the indicator hidden, and a button 63 to disagree with taking a picture are displayed. The user then selects either button 62 or 63, or inputs "agree" or "disagree" by voice. Note that message 61 may also be output by voice.

[0025] In step S20, the controller 401 determines whether the user consented in step S10. If the controller 401 determines that the user consented, it proceeds to step S30; otherwise, it proceeds to step S40.

[0026] In step S30, the controller 401 stores a flag indicating agreement in the memory unit 402. Also in step S40, the controller 401 stores a flag indicating disagreement in the memory unit 402.

[0027] Figure 8 shows the flow of the display method executed by the controller 401 of the display device 100 according to the application program (display control program). The controller 401 starts the process shown in Figure 8 when power is supplied to the display device 100, such as when the vehicle's accessory power is turned on, when a function using the camera (hereinafter also referred to as a predetermined function) is executed, or when a user performs an operation to start the process.

[0028] In step S110, the controller 401 obtains a flag from the memory unit 402 indicating whether or not to consent to taking images with the indicator 35 hidden.

[0029] In step S120, the controller 401 determines whether the flag obtained in step S110 indicates consent, that is, whether the user has consented to recording without being seen. If the controller 401 determines that the user has consented, it proceeds to step S130. If disagreement is determined, the process proceeds to step S140.

[0030] In step S130, the controller 401 hides indicator 35. Then, in step S140, the controller 401 displays indicator 35.

[0031] <Effects of the Embodiment> (1) As described above, the display device 100 of this embodiment includes a display element 20, a camera 30 positioned on the back side of the display element 20, and a controller 401 that displays an indicator 35 indicating the presence of the camera 30 at a position corresponding to the camera 30 in the display area of ​​the display element 20. As a result, the display device 100 of this embodiment can prevent the capture of faces and other objects from being performed without the user being aware of the presence of the camera 30 positioned on the back side of the display element 20.

[0032] (2) Furthermore, the display device 100 of this embodiment hides the indicator 35 when it obtains information that the user consents to taking pictures with the indicator 35 hidden. This allows the display device 100 of this embodiment to hide the presence of the camera 30 and the indicator 35, thereby achieving a clean design, and also prevents the taking of pictures of faces, etc., without the user being aware of the presence of the camera 30.

[0033] <Second Embodiment> Figure 9 shows the flow of the process in the second embodiment for deciding whether or not to consent to taking a photograph with the indicator 35 hidden and to register facial information, and Figure 10 shows the flow of the display method executed by the controller 401 of the display device 100 according to the display control program in the second embodiment. This embodiment differs from the first embodiment described above in that it recognizes the user's face and determines whether or not the user has consented to taking a photograph with the indicator 35 hidden, but the other configurations are the same. For this reason, the same elements are denoted by the same reference numerals and their repeated explanations are omitted.

[0034] In step S310, the controller 401 agrees to take a picture with the indicator 35 hidden and outputs a UI (User Interface) to the user for registering the face image (face information). Figure 11 shows an example of the registration screen displayed on the display element 20. In the example in Figure 11, a message 64 prompting the registration of face information, a display 65 of the user's face being captured by the camera 30, a registration button (agreement button) 66 for registering face information, and a non-registration button (disagreement button) 67 for not registering face information are displayed. The user then selects either button 66 or 67, or inputs "Register" or "Do not register" by voice. Note that message 64 may also be output by voice.

[0035] In step S320, the controller 401 determines whether the user selected registration in step S310. If the controller 401 determines that the user has registered, it proceeds to step S330; otherwise, it terminates the process shown in Figure 9.

[0036] In step S330, the controller 401 stores the user's face information captured when the registration button 66 is selected in the storage unit 402. Note that the face information is not limited to an image, but may also be feature information extracted from the captured image. Furthermore, if the display device 100 has a ToF sensor, the controller 401 may use the ToF sensor to acquire three-dimensional information of the user's face and store it as face information in the storage unit 402.

[0037] The controller 401 starts the process shown in Figure 10 when power is supplied to the display device 100, such as when the vehicle's accessory power is turned on, when a function using the camera (hereinafter also referred to as a predetermined function) is executed, or when a user performs an operation to start the process.

[0038] In step S410, the controller 401 recognizes the user's face from the image captured by the camera 30 and acquires face information. If the captured image contains multiple users, face information is acquired for each user.

[0039] In step S420, the controller 401 determines whether the face information acquired in step S410 is stored in the memory unit 402. That is, if the face information acquired in step S410 is stored in the memory unit 402, the controller 401 makes an affirmative determination, assuming it belongs to a user who consented to being photographed without their face visible. On the other hand, if the face information acquired in step S410 is not stored in the memory unit 402, the controller 401 makes a negative determination, assuming it belongs to a user who did not consent to being photographed without their face visible. If multiple users are recognized in step S410, the controller 401 makes an affirmative determination if face information is registered in the memory unit 402 for all users, and a negative determination if even one user's face information is not registered in the memory unit 402. If the determination is affirmative, the controller 401 proceeds to step S430; if the determination is negative, it proceeds to step S440.

[0040] In step S430, the controller 401 hides the indicator 35. Then, in step S440, the controller 401 displays the indicator 35. In this way, the display device 100 of this embodiment can switch the display and hiding of the indicator 35 according to the user's face information. If multiple users are detected from the captured image, and even one of these users is an unconsenting user who has not given consent to be photographed, then in step S420, a negative determination is made, and the indicator 35 is displayed. If all users do not give consent, then in step S420, a positive determination is not made, and the indicator 35 is displayed until information indicating consent is obtained from all unconsenting users. For this reason, even if there are passengers other than the driver, it is possible to prevent the photographing of faces, etc., from being taken without the other passengers being aware of the presence of the camera 30.

[0041] <Effects of the Embodiment> (3) In the display device 100 of this embodiment, the controller 401 performs face recognition of the user using the image captured by the camera 30, and hides the indicator 35 if the recognized face belongs to a user who has consented to being photographed. In the case where the user may differ each time the display device 100 of this embodiment is used, the controller 401 decides whether or not to display the indicator 35 depending on the user. As a result, the display device 100 of this embodiment can appropriately switch the display of the indicator 35 depending on the user.

[0042] (4) In the display device 100 of this embodiment, the controller 401 performs face recognition of the user using the image captured by the camera 30, and displays the indicator 35 if the recognized face belongs to a user who has not consented to being photographed. As a result, the display device 100 of this embodiment can appropriately switch the display of the indicator 35 according to the user.

[0043] (5) In the display device 100 of this embodiment, the controller 401 recognizes multiple users from the image captured by the camera 30, and if the multiple users include users who have not given their consent to be photographed, the controller displays the indicator 35 until it obtains information that the unconsented users have given their consent. As a result, the display device 100 of this embodiment can prevent all users from having their faces or other images taken without realizing the presence of the camera 30 when there are multiple users.

[0044] (6) In this embodiment, the display device 100 has an indicator 35 which is a character indicating camera 30, an icon that resembles camera 30 or lens 32, a symbol indicating the location of camera 30, or a frame surrounding the installation location of camera 30. This makes it possible for the user to easily understand the presence of camera 30. [Explanation of symbols]

[0045] 10: Cabinet 20: Display elements 21: Display Panel 22: Front polarizing plate 23: Back polarizing plate 25: Backlight 26: Hole 27: Touch panel 28: Display area 30: Camera 31: Solid-state image sensor 32: Lens 35: Indicators 40: Control device 61: Message 62: Agree button 63: Disagree button 64: Message 65:Display 66: Register button (agreement button) 67: Do not register button (Disagree button) 100:Display device 351: Character 352: Icon 353: Frame 401: Controller 402: Storage section 410: Connecting bus 403: Input / Output Interface

Claims

1. Display element and A camera positioned on the back side of the display element, A control unit that displays an indicator showing the presence of the camera at a position corresponding to the camera within the display area of ​​the display element, A display device equipped with the following features.

2. The display device according to claim 1, wherein the control unit hides the indicator when it obtains information that the user consents to taking a photograph with the indicator hidden.

3. The display device according to claim 2, wherein the control unit performs face recognition of the user using the image captured by the camera, and hides the indicator when the recognized face belongs to a user who has consented to the capture.

4. The display device according to claim 3, wherein the control unit performs face recognition of a user using an image captured by the camera, and displays the indicator when the recognized face belongs to a user who has not consented to the photo being taken.

5. The display device according to any one of claims 1 to 4, wherein the control unit recognizes multiple users from images captured by the camera, and if the multiple users include unconsented users who have not given their consent to be photographed, the display indicator is shown until information is obtained indicating that the unconsented users have given their consent.

6. The display device according to any one of claims 1 to 4, wherein the indicator is a character representing the camera, an icon resembling a camera or lens, a symbol indicating the position of the camera, or a frame surrounding the installation position of the camera.

7. A display method performed by a display device comprising a display element and a camera positioned on the back side of the display element, When taking pictures with the aforementioned camera, The system determines whether the user has consented to taking the photos without displaying an indicator showing the presence of the camera. If the user does not consent, the indicator will be displayed in the display area of ​​the display element at a position corresponding to the camera; if the user consents, the indicator will not be displayed. Display method.

Images