Information processing apparatus, method, and non-transitory computer-readable medium

Abstract

The present disclosure relates to an information processing apparatus, method, and non-transitory computer-readable medium. A technique for improving fatigue reduction in a video conference. A control section of an information processing apparatus 10 adjusts a virtual distance Lv2 in such a manner that a total value of an actual distance Lr1 from a display 20 disposed in an actual space to a user U1 in the actual space and the virtual distance Lv2 from a predetermined reference position in a virtual space to a target object O2 in the virtual space corresponding to a conversation partner U2 of the user U1 becomes a predetermined value.

Description

Technical Field

[0001] This disclosure relates to information processing apparatus, methods, and procedures. Background Technology

[0002] Previously, technologies for implementing video conferencing using communication lines were known. For example, Patent Document 1 discloses a technology in which the user distance L1 between the user and a television communication device is transmitted to another television communication device, and a virtual image of the user is generated at the position where the user distance L2 to the other television communication device is added to L1. The distance information control unit then instructs the display image transformation unit to transform the image.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2004-282431 Summary of the Invention

[0006] There is room for improvement in techniques to reduce fatigue during video conferencing.

[0007] The purpose of this disclosure, made in view of the above circumstances, is to improve the technology for reducing fatigue during video conferencing.

[0008] An information processing apparatus according to one embodiment of this disclosure includes a control unit that adjusts the virtual distance in such a way that the sum of the actual distance and the virtual distance becomes a predetermined value. The actual distance is the distance from a display disposed in a physical space to a user in the physical space, and the virtual distance is the distance from a predetermined reference position in the virtual space to a target object in the virtual space that corresponds to the user's dialogue partner.

[0009] One embodiment of this disclosure relates to a method executed by an information processing apparatus, comprising:

[0010] The virtual distance is adjusted in such a way that the sum of the actual distance and the virtual distance becomes a predetermined value. The actual distance is the distance from the display located in the actual space to the user in the actual space, and the virtual distance is the distance from a predetermined reference position in the virtual space to a target object in the virtual space that corresponds to the user's dialogue partner.

[0011] One embodiment of this disclosure relates to a program that causes a computer to perform the following steps:

[0012] The virtual distance is adjusted in such a way that the sum of the actual distance and the virtual distance becomes a predetermined value. The actual distance is the distance from the display located in the actual space to the user in the actual space, and the virtual distance is the distance from a predetermined reference position in the virtual space to a target object in the virtual space that corresponds to the user's dialogue partner.

[0013] According to one embodiment of this disclosure, a technique is available to improve fatigue reduction during video conferencing. Attached Figure Description

[0014] Figure 1 This is a block diagram illustrating the schematic structure of a system according to one embodiment of the present disclosure.

[0015] Figure 2 This is a block diagram illustrating the schematic structure of an information processing apparatus according to one embodiment of the present disclosure.

[0016] Figure 3 This is a flowchart illustrating the operation of an information processing apparatus according to one embodiment of the present disclosure.

[0017] Symbol Explanation

[0018] 1: System; 10: Information processing device; 11: Control unit; 12: Communication unit; 13: Storage unit; 20: First display; 30: First camera; 40: Second display; 50: Second camera; 60: Network. Detailed Implementation

[0019] The following describes the implementation of this disclosure.

[0020] Reference Figure 1 This section provides an overview of System 1, which describes an embodiment of the present disclosure.

[0021] System 1 includes an information processing device 10, a first display 20, a first camera 30, a second display 40, and a second camera 50. The information processing device 10, the first display 20, and the second display 40 are communicatively connected to a network 60.

[0022] Information processing device 10 is a computer such as a server belonging to a cloud computing system or other computing system.

[0023] The first display 20 and the second display 40 are, for example, light field displays capable of reproducing the light field emitted by any object, but are not limited thereto. The first display 20 and the second display 40 may also be conventional types of displays such as LCD (liquid crystal display) or organic EL (electroluminescence).

[0024] The first camera 30 and the second camera 50 are, for example, RGB-D cameras capable of acquiring both color and distance images, but are not limited to this. In this embodiment, the first camera 30 is positioned at the lower center of the screen of the first display 20, but is not limited to this. Furthermore, in this embodiment, the second camera 50 is positioned at the lower center of the screen of the second display 40, but is not limited to this.

[0025] Network 60 includes the Internet, at least one WAN (wide area network), at least one MAN (metropolitan area network), or any combination thereof. Network 60 may also include at least one wireless network, at least one optical network, or any combination thereof. Wireless networks are, for example, ad hoc networks, cellular networks, wireless LANs (local area networks), satellite communication networks, or terrestrial microwave networks.

[0026] Reference Figure 1 This section provides an overview of the implementation method.

[0027] The control unit 11 of the information processing device 10 adjusts the second virtual distance Lv2 in a predetermined manner, such that the sum of the first actual distance Lr1 from the first display 20 disposed in the first actual space to the first user U1 in the first actual space and the second virtual distance Lv2 from a predetermined reference position in the second virtual space to the second target object O2 in the second virtual space corresponding to the dialogue counterpart of the first user U1 is a predetermined value. Similarly, the control unit 11 of the information processing device 10 adjusts the first virtual distance Lv1 in a predetermined manner, such that the sum of the second actual distance Lr2 from the second display 40 disposed in the second actual space to the second user U2 in the second actual space and the first virtual distance Lv1 from a predetermined reference position in the first virtual space to the first target object O1 in the first virtual space corresponding to the dialogue counterpart of the second user U2 is a predetermined value.

[0028] Thus, according to this embodiment, for example, in the case of a video conference between a first user U1 with a personal space of 1m and a second user U2 with a personal space of 2m, the second target object O2 is displayed as being 1m in front of the first user U1. On the other hand, the first target object O1 is displayed as being 2m in front of the second user U2. Therefore, the personal spaces of the first user U1 and the second user U2 are each protected. Consequently, a technique for reducing fatigue during video conferences can be improved.

[0029] Reference Figure 2The structure of the information processing device 10 in this embodiment will be explained.

[0030] The information processing device 10 includes a control unit 11, a communication unit 12, and a storage unit 13.

[0031] The control unit 11 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU (central processing unit) or a GPU (graphics processing unit), or a dedicated processor for specific processing. The programmable circuit is, for example, a field-programmable gate array (FPGA). The dedicated circuit is, for example, an application-specific integrated circuit (ASIC). The control unit 11 controls the various parts of the information processing device 10 while performing processing related to the operation of the information processing device 10. In this embodiment, the control unit 11 controls the operation of a first virtual camera capable of capturing images of any target object disposed in a first virtual space. Similarly, the control unit 11 controls the operation of a second virtual camera capable of capturing images of any target object disposed in a second virtual space.

[0032] The communication unit 12 includes at least one communication interface. The communication interface may correspond to, for example, a mobile communication standard, a wired LAN standard, or a wireless LAN standard, but is not limited to these; it may correspond to any communication standard. The communication unit 12 receives data for the operation of the information processing device 10. Additionally, the communication unit 12 transmits data obtained through the operation of the information processing device 10.

[0033] The storage unit 13 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, RAM (random access memory) or ROM (read-only memory). RAM is, for example, SRAM (static random access memory) or DRAM (dynamic random access memory). ROM is, for example, EEPROM (electrically erasable programmable read-only memory). The storage unit 13 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 13 stores data for the operation of the information processing device 10 and data obtained through the operation of the information processing device 10. In this embodiment, the data for the operation of the information processing device 10 includes system programs, application programs, databases, and texture images, etc.

[0034] Reference Figure 3 This describes the operation of the information processing apparatus 10 in this embodiment. This operation corresponds to a method in one embodiment of this disclosure.

[0035] In this example, a first user U1 in a first physical space and a second user U2 in a second physical space are assumed to be conducting a video conference via network 60. Therefore, the first user U1 is equivalent to the dialogue partner of the second user U2. Similarly, the second user U2 is equivalent to the dialogue partner of the first user U1. Furthermore, in the coordinate system of the first physical space, the origin is taken at the center of the screen of the first display 20, the x-axis is taken to the right of the screen, the y-axis is taken to the top of the screen, and the z-axis is taken to the front of the screen. Similarly, in the coordinate system of the second physical space, the origin is taken at the center of the screen of the second display 40, the x-axis is taken to the right of the screen, the y-axis is taken to the top of the screen, and the z-axis is taken to the front of the screen. However, this disclosure is not limited to these examples.

[0036] Step S100: The control unit 11 of the information processing device 10 acquires a first actual distance Lr1 from the first display 20 disposed in the first actual space to the first user U1 in the first actual space. Similarly, the control unit 11 acquires a second actual distance Lr2 from the second display 40 disposed in the second actual space to the second user U2 in the second actual space.

[0037] Specifically, the control unit 11 of the information processing device 10 uses the first camera 30 to determine, for example, the center position (x1, y1, z1) of the head of the first user U1. Then, the control unit 11 sets the length of the vector (x1, y1, z1) as a first actual distance Lr1. Similarly, the control unit 11 uses the second camera 50 to determine, for example, the center position (x2, y2, z2) of the head of the second user U2. Then, the control unit 11 sets the length of the vector (x2, y1, z2) as a second actual distance Lr2.

[0038] Step S110: The control unit 11 of the information processing device 10 generates a second target object O2 corresponding to the second user U2. Similarly, the control unit 11 generates a first target object O1 corresponding to the first user U1.

[0039] Specifically, the control unit 11 of the information processing device 10 generates, for example, a second target object O2 composed of polygons that project a texture image using texture mapping technology, based on the image of the second user U2 acquired by the second camera 50. Similarly, the control unit 11 generates, for example, a first target object O1 composed of polygons that project a texture image using texture mapping technology, based on the image of the first user U1 acquired by the first camera 30.

[0040] Step S120: The control unit 11 of the information processing device 10 adjusts the second virtual distance Lv2 in such a way that the sum of the first actual distance Lr1 obtained in step S100 and the second virtual distance Lv2 from the predetermined reference position in the second virtual space to the second target object O2 generated in step S110 becomes a predetermined value. Similarly, the control unit 11 adjusts the first virtual distance Lv1 in such a way that the sum of the second actual distance Lr2 obtained in step S100 and the first virtual distance Lv1 from the predetermined reference position in the first virtual space to the first target object O1 generated in step S110 becomes a predetermined value.

[0041] Specifically, the control unit 11 of the information processing device 10 calculates a predetermined value based on the distance that the first user U1 wants to ensure between the first user U1 and the second user U2, which is pre-stored in the storage unit 13. Then, the control unit 11 adjusts the second virtual distance Lv2 in such a way that the sum of the first actual distance Lr1 and the second virtual distance Lv2 becomes the calculated predetermined value. Similarly, the control unit 11 calculates a predetermined value based on the distance that the second user U2 wants to ensure between the first user U1 and the second user U2. Then, the control unit 11 adjusts the first virtual distance Lv1 in such a way that the sum of the second actual distance Lr2 and the first virtual distance Lv1 becomes the calculated predetermined value. Furthermore, this distance can also be a person-to-person distance (personal space) that can be estimated by monitoring the first user U1 and the second user U2 separately by any means, and the predetermined value can also be equal to this person-to-person distance.

[0042] Alternatively, the control unit 11 may calculate a predetermined value based on the amount of speech from the first user U1 to the second user U2, which is pre-stored in the storage unit 13. For example, the more speech, the smaller the predetermined value is set, and the less speech, the larger the predetermined value is set. Then, the control unit 11 may adjust the second virtual distance Lv2 so that the sum of the first actual distance Lr1 and the second virtual distance Lv2 is the calculated predetermined value. Similarly, the control unit 11 may calculate a predetermined value based on the amount of speech from the second user U2 to the first user U1. Then, the control unit 11 may adjust the first virtual distance Lv1 so that the sum of the second actual distance Lr2 and the first virtual distance Lv1 is the calculated predetermined value. Furthermore, this speech volume may be the amount of speech that can be analyzed by monitoring the video conference attended by the first user U1 and the second user U2 using any method.

[0043] Alternatively, the control unit 11 may calculate a score representing the intimacy level between the first user U1 and the second user U2 based on the aforementioned distance or the amount of speech. Then, the control unit 11 may set a lower predetermined value for a higher intimacy score between the first user U1 and the second user U2, and a higher predetermined value for a lower intimacy score between the first user U1 and the second user U2. Similarly, the control unit 11 may calculate a score representing the intimacy level between the second user U2 and the first user U1 based on the aforementioned distance or the amount of speech. Then, the control unit 11 may set a lower predetermined value for a higher intimacy score between the second user U2 and the first user U1, and a higher predetermined value for a lower intimacy score between the second user U2 and the first user U1. Furthermore, the intimacy score can be a stepped indicator such as "high," "medium," or "low," or a continuous indicator such as 0% to 100%.

[0044] Step S130: The control unit 11 of the information processing device 10 positions the second target object O2 generated in step S110 at a location in the second virtual space that is farther from a predetermined reference position than the second virtual distance Lv2 adjusted in step S120. Similarly, the control unit 11 positions the first target object O1 generated in step S110 at a location in the first virtual space that is farther from a predetermined reference position than the first virtual distance Lv1 adjusted in step S120.

[0045] Here, regarding the second target object O2, the predetermined reference position can be a three-dimensional coordinate position in the second virtual space corresponding to the center position (0, 0, 0) of the screen of the first display 20. Regarding the first target object O1, the predetermined reference position can be a three-dimensional coordinate position in the first virtual space corresponding to the center position (0, 0, 0) of the screen of the second display 40. Furthermore, the coordinate system of the first actual space is mapped to the coordinate system of the second virtual space using a coordinate transformation predetermined by an arbitrary method. Additionally, the coordinate system of the second actual space is mapped to the coordinate system of the first virtual space using a coordinate transformation predetermined by an arbitrary method.

[0046] Step S140: The control unit 11 of the information processing device 10 uses a second virtual camera configured in the second virtual space to generate an image including the photographed image of the second target object O2 configured in step S130. Additionally, the control unit 11 uses a first virtual camera configured in the first virtual space to generate an image including the photographed image of the first target object O1 configured in step S130.

[0047] Specifically, the control unit 11 of the information processing device 10 configures a second virtual camera at a three-dimensional coordinate position (X1, Y1, Z1) in a second virtual space corresponding to, for example, the center position (x1, y1, z1) of the head of the first user U1 determined in step S100. Then, the control unit 11 generates (renders) an image including the captured image of the second target object O2 existing within the field of view (viewing range) of the second virtual camera. Similarly, the control unit 11 configures a first virtual camera at a three-dimensional coordinate position (X2, Y2, Z2) in a first virtual space corresponding to, for example, the center position (x2, y2, z2) of the head of the second user U2 determined in step S100. Then, the control unit 11 generates (renders) an image including the captured image of the first target object O1 existing within the field of view (viewing range) of the first virtual camera.

[0048] Step S150: The control unit 11 of the information processing device 10 outputs an image including the photographed image of the second target object O2 generated in step S140 to the first display 20 disposed in the first physical space. Additionally, the control unit 11 outputs an image including the photographed image of the first target object O1 generated in step S140 to the second display 40 disposed in the second physical space.

[0049] Specifically, the control unit 11 of the information processing device 10 outputs an image including the photographed image of the second target object O2 generated in step S140 to the first display 20 disposed in the first physical space via the communication unit 12. Additionally, the control unit 11 outputs an image including the photographed image of the first target object O1 generated in step S140 to the second display 40 disposed in the second physical space via the communication unit 12. Furthermore, the transmission method of the images from the information processing device 10 to the first display 20 and the second display 40 can also be a serial transmission method using HDMI (registered trademark).

[0050] As described above, the control unit 11 of the information processing apparatus 10 in this embodiment adjusts the second virtual distance Lv2 such that the sum of the first actual distance Lr1 from the first display 20 disposed in the first actual space to the first user U1 in the first actual space and the second virtual distance Lv2 from a predetermined reference position in the second virtual space to the second target object O2 in the second virtual space corresponding to the dialogue partner of the first user U1 is a predetermined value. Furthermore, the control unit 11 of the information processing apparatus 10 adjusts the first virtual distance Lv1 such that the sum of the second actual distance Lr2 from the second display 40 disposed in the second actual space to the second user U2 in the second actual space and the first virtual distance Lv1 from a predetermined reference position in the first virtual space to the first target object O1 in the first virtual space corresponding to the dialogue partner of the second user U2 is a predetermined value.

[0051] According to the above structure, for example, in a video conference between a first user U1 with a personal space of 1m and a second user U2 with a personal space of 2m, the second target object O2 is displayed 1m in front of the first user U1. On the other hand, the first target object O1 is displayed 2m in front of the second user U2. Therefore, the personal spaces of the first user U1 and the second user U2 are protected respectively. Thus, a technology that improves fatigue reduction during video conferences can be developed.

[0052] This disclosure has been described with reference to the accompanying drawings and embodiments. However, it should be noted that various modifications and alterations can be made based on this disclosure by those skilled in the art. Therefore, it should be understood that such modifications and alterations are included within the scope of this disclosure. For example, the functions included in each structural component or step can be reconfigured in a logically consistent manner, and multiple structural components or steps can be combined into one or divided.

[0053] As a variation, the structure and operation of the information processing device 10 can also be distributed among multiple computers capable of communicating with each other. For example, the processing on the first user U1 side and the processing on the second user U2 side in the above embodiments can also be distributed among two computers capable of communicating with each other for execution.

[0054] Furthermore, as a variation, the control unit 11 of the information processing device 10 can also perform control to prevent the dialogue counterpart from recognizing that the virtual distance has been adjusted. Specifically, the control unit 11 can also be configured not to notify the first user U1 in step S150 that the distance between the second user U2 and the first target object O1 (=Lr2+Lv1) recognized by the second user U2 is different from the distance between the first user U1 and the second target object O2 (=Lr1+Lv2) recognized by the first user U1. Similarly, the control unit 11 can also be configured not to notify the second user U2 in step S150 that the distance between the first user U1 and the second target object O2 (=Lr1+Lv2) recognized by the first user U1 is different from the distance between the second user U2 and the first target object O1 (=Lr2+Lv1) recognized by the second user U2.

[0055] Furthermore, as a variation, the control unit 11 of the information processing device 10 can also adjust the layout of the images displayed on the first display 20 and the second display 40 respectively based on the intimacy score representing the intimacy level in the above embodiment. For example, the lower the intimacy score of the first user U1 towards the second user U2, the closer the window displaying the image of the photographed object including the second target object O2 will be to the edge of the screen on the first display 20. Similarly, the lower the intimacy score of the second user U2 towards the first user U1, the closer the window displaying the image of the photographed object including the first target object O1 will be to the edge of the screen on the second display 40.

[0056] In addition, as a variation, the control unit 11 of the information processing device 10 may adjust the virtual distance for either the first user U1 or the second user U2 in the same way as in step S120, but not for the other user.

[0057] Furthermore, as a variation, an implementation can also be adopted in which a general-purpose computer functions as the information processing apparatus 10 described in the above embodiments. Specifically, a program describing the processing content for implementing the functions of the information processing apparatus 10 described in the above embodiments is stored in the memory of a general-purpose computer, and the program is read and executed by a processor. Therefore, this disclosure can also be implemented as a program executable by a processor or as a non-temporary computer-readable medium storing the program.

Claims

1. An information processing apparatus comprising a control unit, the control unit being configured as follows: The virtual distance of the first target object in the first virtual space is adjusted from the first distance to the second distance, such that the sum of the second distance and the actual distance from the second display configured in the second real space to the second user in the second real space corresponds to a first predetermined value. The first distance corresponds to the virtual distance from the predetermined reference position in the first virtual space to the first target object in the first virtual space. The first target object corresponds to the first user who is the dialogue opponent of the second user. The virtual distance to the second target object in the second virtual space is adjusted from a third distance to a fourth distance, such that the sum of the fourth distance and the actual distance from the first display configured in the first physical space to the first user in the first physical space corresponds to a second predetermined value. The third distance corresponds to the virtual distance from a predetermined reference position in the second virtual space to the second target object in the second virtual space, and the second target object corresponds to the second user, who is the dialogue opponent of the first user. The control unit calculates the second predetermined value based on the amount of speech given by the first user to the dialogue opponent, and calculates the first predetermined value based on the amount of speech given by the second user to the dialogue opponent.

2. The information processing apparatus according to claim 1, wherein, The control unit performs control to prevent the dialogue counterpart from recognizing that the virtual distance has been adjusted.

3. A method executed by an information processing device, wherein, include: The virtual distance of the first target object in the first virtual space is adjusted from the first distance to the second distance, such that the sum of the second distance and the actual distance from the second display configured in the second real space to the second user in the second real space corresponds to a first predetermined value. The first distance corresponds to the virtual distance from the predetermined reference position in the first virtual space to the first target object in the first virtual space. The first target object corresponds to the first user who is the dialogue opponent of the second user. The virtual distance to the second target object in the second virtual space is adjusted from a third distance to a fourth distance, such that the sum of the fourth distance and the actual distance from the first display configured in the first physical space to the first user in the first physical space corresponds to a second predetermined value. The third distance corresponds to the virtual distance from a predetermined reference position in the second virtual space to the second target object in the second virtual space, and the second target object corresponds to the second user, who is the dialogue opponent of the first user. Specifically, the second predetermined value is calculated based on the amount of speech given by the first user to the dialogue opponent, and the first predetermined value is calculated based on the amount of speech given by the second user to the dialogue opponent.

4. A non-transitory computer-readable medium storing a program that causes a computer to perform the following steps: The virtual distance of the first target object in the first virtual space is adjusted from the first distance to the second distance, such that the sum of the second distance and the actual distance from the second display configured in the second real space to the second user in the second real space corresponds to a first predetermined value. The first distance corresponds to the virtual distance from the predetermined reference position in the first virtual space to the first target object in the first virtual space. The first target object corresponds to the first user who is the dialogue opponent of the second user. The virtual distance to the second target object in the second virtual space is adjusted from a third distance to a fourth distance, such that the sum of the fourth distance and the actual distance from the first display configured in the first physical space to the first user in the first physical space corresponds to a second predetermined value. The third distance corresponds to the virtual distance from a predetermined reference position in the second virtual space to the second target object in the second virtual space, and the second target object corresponds to the second user, who is the dialogue opponent of the first user. in, The second predetermined value is calculated based on the amount of speech given by the first user to the dialogue opponent, and the first predetermined value is calculated based on the amount of speech given by the second user to the dialogue opponent.

Images