Terminal device and control method
The terminal device automatically conceals confidential information during online conferences by using motion detection and processing to generate a concealment video, addressing the challenge of pre-setting secret information in existing technologies.
Patent Information
- Application Number
- PCT/JP2024/019383
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-03-13
- Filing Date
- 2024-05-27
- Publication Date
- 2024-12-05
AI Technical Summary
Existing screen-sharing technologies during online conferences require pre-setting of secret information, making it difficult to immediately conceal confidential information if it is inadvertently shared.
A terminal device equipped with a display section, motion information acquisition, concealed motion detection, and secret area processing units that detect and generate a concealment video based on the presenter's motion to automatically obscure confidential information from view.
Enables easy and immediate concealment of confidential information during online meetings, preventing unintended disclosure by analyzing and masking sensitive areas in real-time.
Smart Images

Figure JP2024019383_05122024_PF_FP_ABST
Abstract
Description
Terminal device and control method
[0001] The present disclosure relates to a terminal device and a control method.
[0002] 2. Description of the Related Art There is known a technique for concealing confidential information contained on a screen when the screen is shared with others in an online conference or the like (see, for example, Patent Document 1).
[0003] JP 2015-191487 A
[0004] However, with the technology disclosed in Patent Document 1, it is necessary to set the information to be kept secret in advance. Therefore, with the technology disclosed in Patent Document 1, if the information to be kept secret is included in the shared screen and the information needs to be kept secret immediately, there is a possibility that the information cannot be kept secret appropriately.
[0005] The present disclosure aims to provide a terminal device and a control method that can easily conceal confidential information contained in a shared screen or the like during an online conference.
[0006] The terminal device of the present disclosure includes a display unit that displays information, a motion information acquisition unit that acquires motion information indicating a person's motion on the display unit that displays the information, a concealed motion detection unit that detects whether the person's motion on the display unit is a concealed motion based on the motion information, and a concealed area processing unit that, when the concealed motion detection unit detects the concealed motion, determines a concealed area based on the motion information acquired by the motion information acquisition unit and generates a concealed video based on the concealed area.
[0007] The control method of the present disclosure includes the steps of acquiring motion information indicating a person's motion on a display unit that displays information, detecting whether the person's motion on the display unit is a concealed motion based on the motion information, and, when the concealed motion is detected, determining a concealed area based on the motion information and generating a concealed video based on the concealed area.
[0008] According to the present disclosure, confidential information contained on a shared screen during an online conference can be easily concealed.
[0009] FIG. 1 is a diagram illustrating a presentation system according to an embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of a terminal device according to an embodiment. FIG. 3 is a diagram illustrating a concealment operation and concealment processing according to an embodiment. FIG. 4 is a flowchart illustrating the flow of the concealment processing according to an embodiment. FIG. 5 is a diagram illustrating an example of concealment area information according to an embodiment. FIG. 6 is a diagram illustrating the coordinates of a concealment area according to an embodiment. FIG. 7 is a diagram illustrating a concealment video displayed on another terminal device according to an embodiment. FIG. 8 is a diagram illustrating a method for updating a concealment area according to an embodiment.
[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that the present disclosure is not limited to these embodiments, and in the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted. [Embodiment]
[0011] (Presentation System) A presentation system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining the presentation system according to an embodiment.
[0012] As shown in FIG. 1 , the presentation system 1 includes a terminal device 10-1 and a terminal device 10-2. The terminal device 10-1 and the terminal device 10-2 are communicatively connected via a network N. The presentation system 1 is a system capable of conducting an online conference using the terminal device 10-1 and the terminal device 10-2. When there is no need to distinguish between the terminal device 10-1 and the terminal device 10-2, they may be collectively referred to as the terminal device 10. While the presentation system 1 includes two terminal devices 10, the present disclosure is not limited to this. The number of terminal devices 10 included in the presentation system 1 may be three or more. Furthermore, although the following examples are described as a presentation system, the present system may also be an online conference system that shares information without a presentation, or a system that displays content displayed on the display screen of the terminal device 10-1 on the display screen of the terminal device 10-2. In this case, the terminal device 10-2 may be an external monitor or projector.
[0013] For example, when a document is shared and a presentation is being made during an online conference, if the shared document contains confidential information such as a password, the presenter generally stops sharing the screen. In the presentation system 1, if the shared document contains confidential information, the presenter can quickly take action such as covering the display screen of the terminal device 10 with their hand to prevent the confidential information from being displayed on the display screen of another terminal device 10.
[0014] (Terminal Device) A configuration example of a terminal device according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing a configuration example of a terminal device according to the embodiment.
[0015] 2, the terminal device 10 includes an input unit 20, an audio input unit 22, a display unit 24, an audio output unit 26, a storage unit 28, a communication unit 30, a sensor unit 32, and a control unit 34. The terminal device 10 may be, for example, a personal computer, a tablet terminal, a smartphone, a head-mounted display (HMD), a head-up display (HUD), or the like, but is not limited to these.
[0016] The input unit 20 accepts various operations for the terminal device 10. The input unit 20 is realized by various input devices such as a switch, a button, or a touch panel.
[0017] The voice input unit 22 detects the voice of the user using the terminal device 10. The voice input unit 22 converts the detected voice into a voice signal. The voice input unit 22 is realized by a microphone.
[0018] The display unit 24 has a display surface that displays various types of image information. The display unit 24 displays, for example, materials shared in an online conference. The display unit 24 is realized, for example, by an LED (Light Emitting Diode) display or an organic EL (Electro-Luminescence) display. When the input unit 20 is a touch panel, the input unit 20 and the display unit 24 are integrally configured. Here, when the terminal device 10 is a device that displays image information by projecting or projecting it, such as a HUD, the display unit 24 does not need to have a display surface.
[0019] The audio output unit 26 is a speaker that outputs various types of audio, for example, audio of users of other terminal devices 10 participating in the online conference.
[0020] The storage unit 28 stores various types of information, such as the contents of calculations performed by the control unit 34 and information such as programs. The storage unit 28 includes at least one of a main storage device such as a random access memory (RAM) and a read-only memory (ROM), and an external storage device such as a hard disk drive (HDD).
[0021] The communication unit 30 is a communication interface that performs communication between the terminal device 10 and an external device. The communication unit 30 performs communication between the terminal device 10 and an external sensor, for example. The communication unit 30 performs communication between the terminal device 10-1 and the terminal device 10-2, for example. The communication unit 30 is configured to be able to communicate via, for example, a wireless LAN (Local Area Network), a wired LAN, Wi-Fi (registered trademark), or the like.
[0022] The sensor unit 32 is a sensor that detects the motion of the user using the terminal device 10. The sensor unit 32 detects, for example, the motion of the user relative to the display unit 24. More specifically, the sensor unit 32 detects the motion of a part of the user's body relative to the display surface of the display unit 24. The sensor unit 32 detects, for example, the motion of the user's hand relative to the display surface of the display unit 24. The sensor unit 32 may be configured integrally with the terminal device 10 or may be configured separately. The sensor unit 32 outputs motion information indicating the detected motion to the motion information acquisition unit 40.
[0023] The sensor unit 32 may be, for example, a camera integrated with the terminal device 10. In this case, the sensor unit 32 detects a video of the hand movements of the user using the terminal device 10 as the user's movements relative to the display unit 24. Here, it is desirable that the camera serving as the sensor unit 32 is installed so as to have an angle of view that includes the display surface of the display unit 24 and the user's hands.
[0024] The sensor unit 32 may be, for example, a distance sensor integrated with the terminal device 10. In this case, the sensor unit 32 detects the distance between the hand of a user using the terminal device 10 and the display surface of the display unit 24 as the user's movement with respect to the display unit 24. The distance sensor serving as the sensor unit 32 may be, for example, disposed near the display surface of the display unit 24, but is not limited to this. If the sensor unit 32 is a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) or the like that can obtain distance information distributed in a matrix, the information on the distances between the user's hand and the display surface of the display unit 24, which is distributed in a matrix, may be detected as the user's movement with respect to the display unit 24.
[0025] The sensor unit 32 may be, for example, a touch sensor including a pressure sensor that is configured integrally with the terminal device 10. In this case, the sensor unit 32 is configured integrally with the display unit 24, and detects information about the position on the display surface of the display unit 24 that the user's hand touches as the user's action with respect to the display unit 24. If the sensor unit 32 is an optical touch sensor that reacts even when the hand does not touch the display unit 24, information about whether the hand has approached the display unit 24 may be detected as the user's action with respect to the display unit 24. If the sensor unit 32 is a capacitive touch sensor that can detect touches at multiple points on the display surface of the display unit 24, information about the multiple positions on the display surface of the display unit 24 that the user's hand touches may be detected as the user's action with respect to the display unit 24.
[0026] The sensor unit 32 may be, for example, a motion capture device. The motion capture device serving as the sensor unit 32 may be a method in which one or more markers (not shown) are placed and capture is performed based on the positions of the markers, or a method in which no markers are used. For example, multiple markers are placed on the user's hand. The sensor unit 32 detects motion capture data, such as three-dimensional position information, as the user's movements relative to the display unit 24.
[0027] The sensor unit 32 may be, for example, a camera configured separately from the terminal device 10. In this case, the camera serving as the sensor unit 32 may be, for example, a smart glass or headset worn by a user of the terminal device 10, and may be arranged to capture an image of the display surface of the display unit 24 of the terminal device 10 from near the user's viewpoint, such as in the user's line of sight or near the user's eyes. By using a camera arranged near the user's eyes as the sensor unit 32, an image similar to the user's field of view can be acquired, thereby accurately detecting the user's hand movements on the display surface of the display unit 24, and easily concealing confidential information. Furthermore, when the sensor unit 32 is a camera configured separately from the terminal device 10, the camera serving as the sensor unit 32 may be attached to a ceiling or wall in a room so as to capture an image of the display surface of the display unit 24 of the terminal device 10. In these cases, the terminal device 10 may acquire the image captured by the sensor unit 32 via the communication unit 30.
[0028] The sensor unit 32 may be configured by a combination of two or more of the sensors mentioned above and other sensors. For example, if the sensor unit 32 is configured by a combination of a camera provided in smart glasses worn by the user and a distance sensor integrated with the terminal device 10, the movement of the user's hand relative to the display surface of the display unit 24 can be detected with high accuracy, and confidential information can be easily concealed.
[0029] The control unit 34 controls each unit of the terminal device 10. The control unit 34 includes, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM or ROM. The control unit 34 executes a program that controls the operation of the terminal device 10 according to the present disclosure. The control unit 34 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 34 may be implemented by a combination of hardware and software.
[0030] The control unit 34 includes a motion information acquisition unit 40 , a concealed motion detection unit 42 , and a concealed area processing unit 44 .
[0031] The motion information acquisition unit 40 controls the sensor unit 32 to detect the motion of a presenter who presents information using the terminal device 10. The motion information acquisition unit 40 acquires, from the sensor unit 32, motion information indicating the motion of the presenter detected by the sensor unit 32. Specifically, the motion information acquisition unit 40 acquires, from the sensor unit 32, motion information indicating the motion of the presenter relative to the display unit 24 detected by the sensor unit 32. For example, the motion information acquired by the motion information acquisition unit 40 includes information indicating the motion of the presenter's hand relative to the display unit 24.
[0032] More specifically, if the sensor unit 32 is a camera, the motion information acquired by the motion information acquisition unit 40 is a video capturing the motion of the user's hand. If the sensor unit 32 is a distance sensor, the motion information acquired by the motion information acquisition unit 40 is information on one or more distances between the user's hand and the display surface of the display unit 24. If the sensor unit 32 is a touch sensor, the motion information acquired by the motion information acquisition unit 40 is information on the position on the display surface of the display unit 24 where the user's hand touches. If the sensor unit 32 is a motion capture sensor, the motion information acquired by the motion information acquisition unit 40 is three-dimensional position information.
[0033] Here, the motion information acquired by the motion information acquisition unit 40 does not have to be that of the presenter, and may instead be that of a person using the terminal device 10-2 viewing a screen shared by the presenter using the terminal device 10-1. In other words, the motion information acquisition unit 40 acquires motion information indicating the motion of a person relative to the display unit 24.
[0034] The concealment movement detection unit 42 detects whether or not the presenter's movement on the display unit 24 is a concealment movement based on the movement information acquired by the movement information acquisition unit 40. For example, the concealment movement detection unit 42 detects whether or not the movement is a concealment movement based on the position of the presenter's hand on the display unit 24. A concealment movement is a movement for concealing confidential information included on the shared screen. In this embodiment, the concealment movement is a predetermined movement. The concealment movement will be described later.
[0035] Here, if the display unit 24 of the terminal device 10 does not have a display surface but is a device that projects or displays image information by projecting, such as a HUD, the concealment movement detection unit 42 may detect whether the presenter's movement on the projection surface, i.e., the display range that the presenter is looking at, is a concealment movement. In this case, the sensor unit 32 is, for example, a camera provided in smart glasses worn by the user, and the concealment movement detection unit 42 detects a concealment movement based on the relationship between the projection surface and the user's hand movement.
[0036] Furthermore, when the motion information acquisition unit 40 acquires motion information of a person other than the presenter, the concealed motion detection unit 42 detects whether or not a motion on the display unit 24 of a person about whom the motion information acquisition unit 40 has acquired motion information is a concealed motion. When the presenter belongs to an organization, the concealed motion detection unit 42 may detect whether or not a concealed motion is made based on motion information of a person who is the presenter's superior. The concealed motion detection unit 42 may set in advance whether or not to detect a concealed motion for all people who use the terminal device 10.
[0037] Specifically, when the motion information is a video of a user's hand motion, the concealed motion detection unit 42 detects whether the presenter's motion is a concealed motion based on the video. Here, when the sensor unit 32 is a camera placed near the user's eyes, the concealed motion detection unit 42 analyzes the video and detects that the presenter's motion is a concealed motion based on whether at least a portion of the display unit 24 is superimposed by the user's hand and cannot be seen. On the other hand, when the sensor unit 32 is a camera placed other than near the user's eyes, the concealed motion detection unit 42 may detect a concealed motion by performing image recognition after correcting the image taking into account the installation position of the camera. The concealed motion detection unit 42 may detect a concealed motion by taking the size of the user's hand into account, for example, when the area of the user's hand shown in the video is larger than a predetermined threshold, it determines that the hand is at a predetermined distance or more from the display surface and does not constitute a concealed motion.
[0038] The concealed movement detection unit 42 detects that the presenter's movement is a concealed movement based on the distance information when the movement information is information about one or more distances between the user's hand and the display surface of the display unit 24. For example, the concealed movement detection unit 42 detects that the presenter's movement is a concealed movement based on whether or not the information about the distance between the user's hand and the display surface of the display unit 24 is smaller than a preset threshold.
[0039] The concealed movement detection unit 42 detects that the presenter's movement is a concealed movement based on the position information when the movement information is information about the position touched by the user's hand on the display surface of the display unit 24. For example, the concealed movement detection unit 42 detects that the presenter's movement is a concealed movement based on whether or not the information about the position touched by the user's hand on the display surface of the display unit 24 is equal to or greater than a preset threshold.
[0040] When the movement information is three-dimensional position information, the concealed movement detection unit 42 detects that the movement of the presenter is a concealed movement based on the three-dimensional position information. For example, the concealed movement detection unit 42 detects that the movement of the presenter is a concealed movement based on whether or not the distance between the three-dimensional position information of the display unit 24 and the three-dimensional position information of the user's hand is smaller than a preset threshold.
[0041] When the concealed movement detection unit 42 detects a concealed movement, the concealed region processing unit 44 determines a concealed region based on the movement information acquired by the movement information acquisition unit 40. The concealed region processing unit 44 performs concealment processing on the concealed region and generates a concealed video.
[0042] Specifically, when the motion information is a video of the user's hand motion, the hidden area processing unit 44 determines the hidden area based on the video. Here, when the sensor unit 32 is a camera placed near the user's eye, the hidden area processing unit 44 analyzes the video and determines the hidden area based on an area of the display unit 24 that cannot be seen because at least a part of the display unit 24 is superimposed by the user's hand.
[0043] The hidden area processing unit 44 determines the hidden area based on the distance information when the action information is information about one or more distances between the user's hand and the display surface of the display unit 24. For example, the hidden area processing unit 44 determines the hidden area based on the coordinates of the image displayed by the display unit 24, where the information about the distance between the user's hand and the display surface of the display unit 24 is smaller than a preset threshold.
[0044] The hidden area processing unit 44 determines a hidden area based on the position information when the action information is information about a position touched by the user's hand on the display surface of the display unit 24. For example, the hidden area processing unit 44 determines a hidden area based on the coordinates of the image displayed by the display unit 24, which correspond to the information about the position touched by the user's hand on the display surface of the display unit 24.
[0045] When the motion information is three-dimensional position information, the secret area processing unit 44 determines the secret area based on the three-dimensional position information. For example, the secret area processing unit 44 determines the secret area based on the coordinates of the image displayed by the display unit 24, which corresponds to the three-dimensional position information of the display unit 24 that is close to the three-dimensional position information of the user's hand.
[0046] (Encryption Operation and Encryption Processing) The encryption operation and the encryption processing according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram for explaining the encryption operation and the encryption processing according to the embodiment.
[0047] In the embodiment, when a user is sharing a screen and making a presentation and the screen contains confidential information that needs to be kept secret from other users, the user can quickly conceal the confidential information by performing a concealment operation. Specifically, when the user performs the concealment operation, the terminal device 10 executes a concealment process to conceal the confidential information. As shown in FIG. 3 , in the embodiment, there are three possible cases for the concealment operation and the concealment process. Note that the concealment operation and the concealment process according to the present disclosure are not limited to the following three cases.
[0048] The first case is when the presenter conceals only the portion covered by the hand. In this case, the concealing movement detection unit 42 detects the presenter's movement as a concealing movement when, for example, the presenter performs a predetermined movement with either the right or left hand H1 on the display surface of the display unit 24. The hand H1 is, for example, the palm or back of the presenter's hand. The concealing movement detection unit 42 detects the presenter's movement as a concealing movement when, for example, the hand H1 is in an open position with the fingers open, as shown in FIG. 3 . However, the state of the hand H1 when detecting a concealing movement is not limited to this. It may also be in a state where the fingers are extended and aligned, as when raising a hand, or in a state where only certain fingers are extended and the other fingers are clenched, as when pointing. Specifically, the concealing movement detection unit 42 detects the presenter's movement as a concealing movement when one hand covers at least a portion of the display unit 24 within a predetermined vertical distance from the display surface of the display unit 24 and stops at the same position on the display unit 24 for one second or more. Covering the display unit 24 may or may not mean that the presenter's hands are touching the display surface of the display unit 24, as long as the display unit 24 is covered when viewed from the presenter's line of sight.
[0049] Specifically, the concealed movement detection unit 42 analyzes the video and determines that one hand is covering at least a portion of the display unit 24 within a predetermined distance in the vertical direction from the display surface when at least a portion of the display unit 24 is superimposed by the user's hand H1 and cannot be seen. The concealed movement detection unit 42 may determine that one hand is covering at least a portion of the display unit 24 within a predetermined distance in the vertical direction from the display surface when information on the distance between the hand H1 and the display surface of the display unit 24 is smaller than a preset threshold. The concealed movement detection unit 42 may determine that one hand is covering at least a portion of the display unit 24 within a predetermined distance in the vertical direction from the display surface when information on the position where the hand H1 is touching the display surface of the display unit 24 is available. The concealed movement detection unit 42 may determine that one hand is covering at least a portion of the display unit 24 within a predetermined distance in the vertical direction from the display surface when the distance between three-dimensional position information of the display unit 24 and three-dimensional position information of the hand H1 is smaller than a preset threshold.
[0050] Furthermore, the time spent stopping in the same position, which is one of the conditions for the concealed movement detection unit 42 to detect a concealed movement, does not have to be one second, but may be a predetermined time. Furthermore, the condition for detecting a concealed movement, that is, whether or not the presenter stops in the same position for a predetermined time or longer, may be replaced with the condition of whether or not any movement is continued for a predetermined time or longer. For example, if the concealed movement detection unit 42 determines that the presenter is waving their hand, that is, waving their hand in any direction within a certain range, for a predetermined time or longer, the presenter's movement may be detected as a concealed movement.
[0051] When the concealed movement detection unit 42 detects a concealed movement by the hand H, the concealed region processing unit 44 determines the outer circumferential region of the hand H as the concealed region 51. Here, the outer circumferential region of the hand H is an area larger than the area covered by the hand H on the display surface of the display unit 24, and is, for example, a rectangular area. The concealed region processing unit 44 performs concealment processing such as masking on the concealed region 51 to generate a concealed video. In this case, the concealed region processing unit 44 generates a concealed video in which the concealed region 51 is masked and a password 241 consisting of the character string "ABCDEFG," which is a type of secret information, is concealed. Furthermore, when the concealed movement detection unit 42 determines that a hand is waving for a predetermined period of time or more, the concealed region processing unit 44 may be able to set a wider range in the hand waving direction as the concealed region 51 compared to when determining whether the hand is stopped in the same position.
[0052] Specifically, the concealed area processing unit 44 analyzes the video and determines the outer periphery of the area of the display unit 24 that cannot be seen because it is superimposed by the hand H as the concealed area 51. The concealed area processing unit 44 may determine the outer periphery of the coordinates of the image displayed by the display unit 24, where information about the distance between the hand H and the display surface of the display unit 24 is smaller than a preset threshold, as the concealed area 51. The concealed area processing unit 44 may determine the outer periphery of the coordinates of the image displayed by the display unit 24, which corresponds to information about the position where the hand H touched the display surface of the display unit 24, as the concealed area 51. The concealed movement detection unit 42 may determine the outer periphery of the coordinates of the image displayed by the display unit 24, which corresponds to three-dimensional position information of the display unit 24 that is close to the three-dimensional position information of the hand H, as the concealed area 51.
[0053] The second case involves concealing the segment covered by the presenter's hand. In this case, the display screen of the display unit 24 is pre-divided into a first segment 61 in the upper region and a second segment 62 in the lower region. In the second case, in addition to the first case, the concealment area processing unit 44 determines the first segment 61 to be a concealment area when the overlapping portion between the concealment area 51 and the first segment 61 is, for example, 80% or more of the concealment area 51. The concealment area processing unit 44 performs a concealment process, such as a masking process, on the first segment 61 to generate a concealed video. In this case, the concealment area processing unit 44 generates a concealed video in which the first segment 61 is masked and a password 241, which is a type of secret information, is concealed. However, when comparing the segment area and the concealment area 51, if the concealment area 51 has a larger area, the concealment process may be performed on the concealment area 51, i.e., the concealment process in the first case may be performed.
[0054] The third case is when the entire display screen of the display unit 24 is concealed. In this case, the concealment movement detection unit 42 detects that the presenter's movement is a concealment movement, for example, when the presenter makes a movement with both hands H1 and H2 on the display unit 24. Specifically, the concealment movement detection unit 42 detects that the presenter's movement is a concealment movement when both hands stop in the same position on the display unit 24 for one second or more. When the concealment movement detection unit 42 detects a concealment movement with both hands, the concealment area processing unit 44 determines that the entire screen is a concealment area. The concealment area processing unit 44 performs a concealment process, such as a masking process, on the entire screen to generate a concealed video. In this case, the entire display page of the shared material is masked. In other words, the concealment area processing unit 44 determines the concealment area based on the number of palms or backs of hands covering the display unit 24.
[0055] 3, the concealment action has been described as a hand movement on the display unit 24, but the present disclosure is not limited to this. The concealment action may be, for example, an action of covering a part of the display unit 24 with the user's finger. In this case, for example, the concealment area processing unit 44 may determine the area around the periphery of the finger as the concealment area and perform masking processing.
[0056] The concealment action may be, for example, an action of covering the display unit 24 with an object such as a notebook that is around the user of the terminal device 10. Note that a palm or back of the hand action is preferable from the viewpoint of being able to conceal confidential information with a quick action and preventing erroneous actions. Furthermore, the user may be able to arbitrarily set which action among the palm action, the back of the hand action, the finger action, and the action of an object such as a notebook to be the concealment action. For example, the user may be able to set at least one of the palm action, the back of the hand action, the finger action, and the action of an object such as a notebook as the concealment action.
[0057] Furthermore, in the present disclosure, the hidden area is described as being rectangular, but the present disclosure is not limited to this. The hidden area may be, for example, circular, elliptical, or a polygonal shape other than a rectangle.
[0058] (Encryption Processing) The encryption processing according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of the encryption processing according to the embodiment.
[0059] The process shown in FIG. 4 shows the flow of processing performed by the terminal device 10 used by the presenter in an online conference.
[0060] The control unit 34 starts screen sharing among the terminal devices 10 of the users participating in the online conference (step S10). When the control unit 34 starts screen sharing among the multiple terminal devices 10, the same material is shared among the terminal devices 10. Then, the process proceeds to step S12.
[0061] The motion information acquisition unit 40 controls the sensor unit 32 to detect the motion of the user using the terminal device 10 (step S12). For example, the motion information acquisition unit 40 may cause the sensor unit 32 to constantly detect the user's motion while an online conference is being held. In the following description, the sensor unit 32 is a camera provided in smart glasses worn by the user, and the motion information acquisition unit 40 captures the user's hand motion relative to the display unit 24 by capturing an image of the display unit 24. The motion information acquisition unit 40 acquires the image captured by the sensor unit 32 as motion information. Then, the process proceeds to step S14.
[0062] The concealed movement detection unit 42 determines whether or not a concealed movement has been detected based on the movement information acquired by the movement information acquisition unit 40 (step S14). If a concealed movement has been detected (step S14; Yes), the process proceeds to step S16. If a concealed movement has not been detected (step S14; No), the process proceeds to step S24, where a process of determining whether or not the concealed area has been updated is executed in step S24, which will be described later.
[0063] If the determination in step S14 is Yes, the secret area processing unit 44 determines a secret area based on the motion information acquired by the motion information acquisition unit 40 (step S16). Specifically, the secret area processing unit 44 identifies an area where the user of the terminal device 10 covers the display unit 24 with their hand based on the video acquired by the motion information acquisition unit 40, and sets the identified area as a secret area. Then, the process proceeds to step S18.
[0064] The secret area processing unit 44 stores secret area information indicating the identified secret area in the storage unit 28 (step S18). FIG. 5 is a diagram showing an example of secret area information according to the embodiment. As shown in FIG. 5, table TB1 is a table for managing secret area information. Table TB1 includes fields such as "document type," "document name," "page," "position," and "status." In the case where the segment covered by the presenter's hand as described above is to be concealed, the secret area information may include a "segment" field indicating which segment it is.
[0065] "Document type" indicates the name of the application that created the material to be shared. "Document type" is conceptually shown as "A01," for example, but actually indicates the specific name of the application. "Document type" may also be a web browser.
[0066] The "document name" indicates the file name of the material to be shared. The "document name" is conceptually shown as "AAA," for example, but actually indicates a specific file name, extension, and URL (Uniform Resource Locator).
[0067] "Page" includes the number of pages in the provided material that contain confidential areas. For example, "Page" indicates a name according to the application that created the material to be shared, such as "Page 1," "Slide 1," and "Sheet 1." If the material to be shared is a web page, "Page" may indicate, for example, a subdirectory of the URL.
[0068] "Position" indicates the position of the secret area. For example, "Position" indicates the coordinates of the secret area in the shared document. FIG. 6 is a diagram for explaining the coordinates of the secret area according to the embodiment. As shown in FIG. 6, the position of the rectangular secret area 51 is specified by the position coordinates of two points. For example, the position of the secret area 51 is specified by the coordinates of two vertices, the upper left vertex P1 and the lower right vertex P2. For example, the "Position" of the secret area of the document with the "Document Name" "AAA" shown in FIG. 5 is shown as "(Xa1, Ya1; Xa2, Ya2)." In this case, the front coordinates (Xa1, Ya1) indicate the coordinates of the vertex P1 shown in FIG. 6, and the back coordinates (Xa2, Ya2) indicate the coordinates of the vertex P2 shown in FIG. 6.
[0069] The "status" indicates the current status of the hidden area. For example, "unvisible" indicates that the hidden area is not visible to other terminal devices 10. If the hidden area is visible to other terminal devices 10, the "status" may be written as "visible."
[0070] Referring again to Fig. 4, the concealed area processing unit 44 performs concealment processing on the identified concealed area to generate a concealed video (step S20), and then the process proceeds to step S22.
[0071] The secret area processing unit 44 transmits the secret video to the other terminal device 10 via the communication unit 30 (step S22). FIG. 7 is a diagram illustrating the secret video displayed on the other terminal device 10 according to the embodiment. As shown in FIG. 7, the secret area 51 containing the secret information is masked in the other terminal device 10. Therefore, the user of the other terminal device 10 cannot view the secret information, such as a password. Furthermore, the secret area information indicating the secret area is stored in the storage unit 28. Therefore, even if the page of the shared document is changed and the page containing the secret information is returned to, the secret area 51 remains masked. Therefore, the embodiment can more appropriately conceal the secret information. Then, the process proceeds to step S24.
[0072] The hidden area processing unit 44 determines whether the hidden area has been updated (step S24). FIG. 8 is a diagram illustrating a method for updating a hidden area according to an embodiment. FIG. 8 shows a display screen displayed on the display unit 24 of the presenter's terminal device 10. As shown in FIG. 8, the presenter's terminal device 10 displays the password 241 included in the hidden area 51 by superimposing an image layer with a transparency of, for example, 50% so that the character string is visible. This allows the user to easily identify the hidden area on the display screen. The user can widen or narrow the hidden area 51 by, for example, operating a mouse or the like to align the pointer 71 with the edge of the hidden area 51 and dragging it. The user can also change the position of the hidden area 51 by, for example, operating a mouse or the like to align the pointer 71 with a predetermined area of the hidden area 51 and dragging it. The hidden area processing unit 44 determines that the hidden area has been updated when the size or position of the hidden area 51 has been changed. Furthermore, the hidden area 51 may be configured so that it cannot be updated unless the pointer 71 is positioned on a predetermined area of the hidden area 51 by operating a mouse or the like and pressed and held down, i.e., the hidden area 51 may be activated by pressing and holding down the hidden area 51. If it is determined that the hidden area has been updated (step S24; Yes), the process proceeds to step S26. If it is not determined that the hidden area has been updated (step S24; No), the process proceeds to step S28.
[0073] If the determination in step S24 is Yes, the secret area processing unit 44 updates the secret video (step S26). Specifically, the secret area processing unit 44 updates the secret video by changing the secret area in response to a user operation. Then, the process proceeds to step S22. In other words, by updating the secret video, the secret video displayed on the other terminal devices 10 is also updated.
[0074] If step S24 returns No, the hidden area processing unit 44 determines whether the hidden area has been unlocked (step S28). As shown in FIG. 7, a button 72 is provided at the top right corner of the hidden area 51. The user can unlock the hidden area 51 by, for example, operating a mouse or the like to position the pointer 71 on the button 72 and clicking it. The hidden area processing unit 44 determines that the hidden area has been unlocked when the user unlocks the hidden area 51. If it is determined that the hidden area has been unlocked (step S28; Yes), the process proceeds to step S30. If it is not determined that the hidden area has been unlocked (step S28; No), the process proceeds to step S34.
[0075] If the determination in step S28 is Yes, the hidden area processing unit 44 releases the hidden area (step S30), and then the process proceeds to step S32.
[0076] The hidden area processing unit 44 transmits the shared video, from which the hidden area has been removed, to the other terminal device 10 (step S32). As a result, the other terminal device 10 displays the shared video, from which the hidden area has been removed, on the display unit 24. Then, the process proceeds to step S34.
[0077] The control unit 34 determines whether to end the process (step S34). For example, the control unit 34 determines to end the process when the online conference has ended. If it is determined to end the process (step S34; Yes), the control unit 34 ends the process of FIG. 4. If it is not determined to end the process (step S34; No), the control unit 34 returns to step S12 and repeats the process.
[0078] As described above, in this embodiment, when confidential information is included in materials to be shared in an online conference, the presenter can conceal the confidential information with a simple action that can be performed on the spot. As a result, this embodiment can prevent confidential information from being leaked with a simple action.
[0079] The components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of the devices can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Note that this distribution and integration configuration may also be performed dynamically.
[0080] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.
[0081] The terminal device and control method of the present disclosure can be used in, for example, an information processing device such as a computer used to participate in an online conference.
[0082] REFERENCE SIGNS LIST 1 Presentation system 10 Terminal device 20 Input unit 22 Audio input unit 24 Display unit 26 Audio output unit 28 Storage unit 30 Communication unit 32 Sensor unit 34 Control unit 40 Motion information acquisition unit 42 Concealed motion detection unit 44 Concealed area processing unit
Claims
1. A terminal device comprising: a display unit that displays information; a motion information acquisition unit that acquires motion information indicating a motion of a person on the display unit displaying the information; a concealed motion detection unit that detects whether the motion of the person on the display unit is a concealed motion based on the motion information; and a concealed area processing unit that, when the concealed motion detection unit detects the concealed motion, determines a concealed area based on the motion information acquired by the motion information acquisition unit and generates a concealed video based on the concealed area.
2. A terminal device as described in claim 1, wherein the motion information acquired by the motion information acquisition unit includes information indicating the motion of the person's hand relative to the display unit, and the concealed motion detection unit detects the concealed motion based on the position of the person's hand relative to the display unit.
3. The terminal device according to claim 2, wherein the concealed movement detection unit detects that the concealed movement is occurring when, in the movement information acquired by the movement information acquisition unit, at least a portion of the display unit is covered by the person's hand for a predetermined period of time or more.
4. The terminal device according to claim 2 or 3, wherein the concealment area processing unit determines the concealment area based on the number of hands covering the display unit or the positions of the hands in the motion information acquired by the motion information acquisition unit.
5. The terminal device according to claim 1, wherein the motion information acquisition unit acquires the motion information from a camera that captures an image of the display surface of the display unit from a position near the user's viewpoint.
6. A control method comprising the steps of: acquiring motion information indicating a motion of a person on a display unit that displays information; detecting whether the motion of the person on the display unit is a concealed motion based on the motion information; and, when the concealed motion is detected, determining a concealed area based on the motion information and generating a concealed video based on the concealed area.
Citation Information
Patent Citations
Presentation server and mobile terminal
JP2015191487A
Digital device and method for performing video call
KR1020140041645A
