Image processing device, image processing method, and program

Abstract

To suppress discomfort of a user.SOLUTION: An image processing device 12 includes: an acquisition unit 60 for acquiring a discomfort index indicating a degree of discomfort felt by a target user existing in a predetermined region of a virtual pace used by the target user, for an avatar of a user other than the target user; a setting unit 62 for setting a boundary region which is a region where the target user cannot visually recognize the other avatars, around the avatar of the target user, on the basis of the discomfort index; and an image processing unit 64 for generating image data for a virtual space, on the basis of the boundary region.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Information devices have evolved significantly in recent years. In addition to smartphones, which are a representative example of such devices, so-called wearable devices have also become widespread. A known example of a wearable device is a glasses-type HMD (head-mounted display) that directly stimulates the visual field. Using such an HMD, a virtual space can be provided to a user U by displaying an image aligned with the user's line of sight. In recent years, users have increasingly been communicating with each other through avatars in a virtual space. For example, Patent Document 1 describes changing the display mode of an image depending on the positional relationship between the avatar and the image in the virtual space. Patent Document 2 describes dividing a virtual space into blocks and managing the behavioral history of an avatar based on the block the avatar entered. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-271618 [Patent Document 2] Japanese Patent Application Laid-Open No. 2009-140197 Summary of the Invention [Problem to be solved by the invention]

[0004] However, even though it is a virtual space, there are some users who feel uncomfortable in interpersonal relationships with other avatars in the virtual space.

[0005] In view of the above-described problems, the present disclosure aims to provide an image processing device, an image processing method, and a program that can suppress discomfort felt by the user. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the objectives, the image processing device of the present disclosure includes an acquisition unit that acquires a discomfort index that indicates the degree to which a target user feels uncomfortable with avatars of users other than the target user that exist within a specified area of ​​a virtual space used by the target user, a setting unit that sets a boundary area around the target user's avatar based on the discomfort index, the boundary area being an area in which the target user cannot see other avatars, and an image processing unit that generates image data for the virtual space based on the boundary area.

[0007] In order to solve the above-mentioned problems and achieve the objectives, the image processing device of the present disclosure includes an acquisition unit that acquires a discomfort index that indicates the degree to which the target user feels uncomfortable with avatars of users other than the target user that exist within a specified area of ​​a virtual space used by the target user; a setting unit that sets a boundary area around the target user's avatar based on the discomfort index, the boundary area being an area in which the target user cannot see other avatars; and an operation restriction unit that restricts operation of the avatar so that the avatar cannot enter the boundary area based on the boundary area and the operation content on the avatars of users other than the target user.

[0008] In order to solve the above-mentioned problems and achieve the objectives, the image processing method of the present disclosure includes the steps of: acquiring an discomfort index indicating the degree to which the target user feels uncomfortable with avatars of users other than the target user that exist within a specified area of ​​a virtual space used by the target user; setting a boundary area around the target user's avatar based on the discomfort index, the boundary area being an area in which the target user cannot see other avatars; and generating image data for the virtual space based on the boundary area.

[0009] In order to solve the above-mentioned problems and achieve the objectives, the program of the present disclosure causes a computer to execute the following steps: acquiring an discomfort index indicating the degree to which the target user feels uncomfortable with avatars of users other than the target user that exist within a specified area of ​​a virtual space used by the target user; setting a boundary area around the target user's avatar based on the discomfort index, which is an area in which the target user cannot see other avatars; and generating image data for the virtual space based on the boundary area. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to suppress feelings of discomfort in interpersonal relationships with other avatars. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a real space and a virtual space. [Figure 2] FIG. 2 is a schematic block diagram of the display device according to this embodiment. [Figure 3] FIG. 3 is a schematic block diagram of the image processing device according to this embodiment. [Figure 4] FIG. 4 is a schematic diagram showing an example of a predetermined area in the virtual space in the first display mode. [Figure 5] FIG. 5 is a schematic diagram showing an example of a predetermined area in the virtual space in the second display mode. [Figure 6] FIG. 6 is a flowchart illustrating the process of generating image data for a virtual space. [Figure 7] FIG. 7 is a flowchart illustrating the operation restriction process. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. However, the present disclosure is not limited to the embodiments described below.

[0013] (Image Processing System) FIG. 1 is a schematic diagram illustrating an example of a real space and a virtual space. An image processing system 100 according to this embodiment is a system that provides a virtual space to a user U and includes a display device 10 and an image processing device 12. As shown in FIG. 1, the display device 10 is a so-called HMD (Head Mount Display) worn on the head of the user U. The display device 10 displays an image to provide the virtual space to the user U. As shown in FIG. 1, the real space in which the user U actually exists is referred to as the real space SR, and the virtual space provided to the user U by the display device 10 is referred to as the virtual space SV. In this case, the display device 10 displays an image for the virtual space SV according to the user U's movement (line of sight) in the real space SR. That is, the image for the virtual space SV is displayed to simulate the user U moving as an avatar in the virtual space SV. Therefore, the user U can recognize that he or she is present in the virtual space SV. The virtual space SV may be a space that reproduces a real location away from the user U's location, i.e., MR (Mixed Reality), or a virtual space that does not actually exist, i.e., VR (Virtual Reality).

[0014] In this embodiment, the image processing system 100 provides a common virtual space SV to multiple users U. Each user U uses a display device 10, and the user U can communicate with the other users U via an avatar in the virtual space SV provided by the user's display device 10. In the example of FIG. 1, user A wears display device 10A, user B wears display device 10B, and user C wears display device 10C. The avatars of users A, B, and C in the common virtual space SV are designated as avatars Aa, Ba, and Ca, respectively. The display device 10A provides user A with an image in the virtual space SV within the field of view V of avatar Aa as an image for the virtual space SV. Similarly, the display device 10B provides user B with an image in the virtual space SV within the field of view V of avatar Ba as an image for the virtual space SV. Similarly, the display device 10C provides user C with an image in the virtual space SV within the field of view V of avatar Ca as an image for the virtual space SV. The number of avatars in the virtual space SV (i.e., the number of users U to whom a common virtual space SV is provided) is not limited to 3 and may be any number. Also, in Fig. 1 and the following figures, the virtual space SV is shown as a two-dimensional plane viewed from above in the vertical direction, but the virtual space SV may be a three-dimensional space.

[0015] (display device) FIG. 2 is a schematic block diagram of a display device according to this embodiment. The display device 10 can be considered a computer, and as shown in FIG. 2, includes an input unit 20, a display unit 22, a storage unit 24, a communication unit 26, a position detection unit 28, and a control unit 30. The input unit 20 is a mechanism for receiving operations by a user U, and may be, for example, a controller or a microphone provided in an HMD. The display unit 22 is a display that displays images. The display unit 22 provides a virtual space SV to the user U by outputting images. In this embodiment, the display unit 22 is an HMD worn by the user U, but is not limited to being worn by the user U and may be any display capable of displaying images. In addition to the display unit 22, the display device 10 may also include a device for outputting information, such as a speaker that outputs audio.

[0016] The storage unit 24 is a memory that stores various information such as the calculation contents and programs of the control unit 30, and includes at least one of a main storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive). The program for the control unit 30 stored in the storage unit 24 may be stored in a recording medium that can be read by the display device 10.

[0017] The communication unit 26 is a communication module, such as an antenna, that communicates with an external device such as the image processing device 12. The display device 10 communicates with the external device via wireless communication, but wired communication may also be used, and any communication method may be used.

[0018] The position detection unit 28 is a sensor that detects the position (coordinates) of the user U (display device 10) in the real space SR and the orientation of the user U (display device 10) in the real space SR (for example, the direction in which the user U's face or gaze is facing). The position detection unit 28 may be any sensor that can detect the position and orientation of the user U, and may be, for example, a gyro sensor or a camera. The position detection unit 28 is not limited to being one unit and may be multiple units, and may have, for example, a sensor that detects the position of the user U and a sensor that detects the orientation of the user U.

[0019] The control unit 30 is a computing device and includes a computing circuit such as a CPU (Central Processing Unit). The control unit 30 includes a position information acquisition unit 40, an image data acquisition unit 42, and a display control unit 44. The control unit 30 implements the position information acquisition unit 40, the image data acquisition unit 42, and the display control unit 44 and executes their processes by reading and executing a program (software) from the storage unit 24. The control unit 30 may execute these processes using a single CPU, or may be provided with multiple CPUs and execute the processes using the multiple CPUs. At least a portion of the processes performed by the position information acquisition unit 40, the image data acquisition unit 42, and the display control unit 44 may be implemented using hardware circuits.

[0020] The position information acquisition unit 40 acquires information about the position of the user U in the real space SR and information about the orientation of the user U in the real space SR as position information of the user U. The position information acquisition unit 40 controls the position detection unit 28 to cause the position detection unit 28 to detect the position and orientation of the user U (display device 10), and acquires the detection result as position information of the user U.

[0021] The image data acquisition unit 42 acquires image data for the virtual space SV to be displayed on the display unit 22. In this embodiment, the image data acquisition unit 42 acquires the image data for the virtual space SV from the image processing device 12 via the communication unit 26. Details of the image data for the virtual space SV will be described later.

[0022] The display control unit 44 causes the display unit 22 to display the image for the virtual space SV acquired by the image data acquisition unit 42.

[0023] (Image processing device) The image processing device 12 is a device that performs image processing of images for the virtual space SV. The image processing device 12 can be said to be a server that transmits and receives information to and from the display device 10 for each user U and performs image processing of images for the virtual space SV.

[0024] 3 is a schematic block diagram of an image processing device according to this embodiment. The image processing device 12 can also be considered a computer, and as shown in FIG. 3, includes a storage unit 50, a communication unit 52, and a control unit 54.

[0025] The storage unit 50 is a memory that stores various information such as the calculation contents and programs of the control unit 54, and includes at least one of a RAM, a main storage device such as a ROM, and an external storage device such as a HDD. The programs for the control unit 54 stored in the storage unit 50 may be stored in a recording medium that can be read by the image processing device 12.

[0026] The communication unit 52 is a communication module, such as an antenna, that communicates with an external device such as the display device 10. The image processing device 12 communicates with the external device via wireless communication, but wired communication may also be used, and any communication method may be used.

[0027] The control unit 54 is a calculation device and includes a calculation circuit such as a CPU. The control unit 54 includes an acquisition unit 60, a setting unit 62, an image processing unit 64, and an operation restriction unit 66. The control unit 54 implements the acquisition unit 60, the setting unit 62, the image processing unit 64, and the operation restriction unit 66 and executes their processes by reading and executing a program (software) from the storage unit 50. The control unit 54 may execute these processes using a single CPU, or may be provided with multiple CPUs and execute the processes using the multiple CPUs. At least a portion of the processes performed by the acquisition unit 60, the setting unit 62, the image processing unit 64, and the operation restriction unit 66 may be implemented using hardware circuits.

[0028] In this embodiment, the image processing device 12 has the functions of the acquisition unit 60, the setting unit 62, the image processing unit 64, and the operation restriction unit 66, but the display device 10 may have these functions. That is, the display device 10 may function as the image processing device 12, or may include at least one of the acquisition unit 60, the setting unit 62, the image processing unit 64, and the operation restriction unit 66 and perform these processes.

[0029] (Image processing device processing) The processing performed by the image processing device 12 will be described below.

[0030] (Generation of image data for virtual space) First, a method for generating image data for the virtual space SV by the image processing device 12 will be described. The acquisition unit 60 of the image processing device 12 acquires position information of an avatar of each user U in the virtual space SV, i.e., information on the position and orientation of the avatar in the coordinate system of the virtual space SV. Specifically, the acquisition unit 60 acquires position information of the user U (information on the position and orientation of the user U in the coordinate system of the real space SR) from each display device 10 via the communication unit 52. The acquisition unit 60 converts the position and orientation of the user U in the coordinate system of the real space SR into the coordinate system of the virtual space SV, and calculates the position and orientation of the avatar in the coordinate system of the virtual space SV. Note that any method can be used for converting the coordinate system of the real space SR to the coordinate system of the virtual space SV. For example, a correspondence relationship between the coordinate system of the real space SR and the coordinate system of the virtual space SV may be set in advance, and the coordinate system may be converted to the coordinate system of the virtual space SV based on that correspondence.

[0031] The image processing unit 64 of the image processing device 12 generates image data for the virtual space SV. The image processing unit 64 identifies the field of view V of the avatar from the position information of the avatar set by the acquisition unit 60, and extracts virtual objects and other avatars within the field of view V in the virtual space SV. The image processing unit 64 then generates image data for the virtual space SV so that the images of the extracted objects and other avatars are positioned, sized, and oriented as viewed from the position and orientation of the avatar. Note that image data representing the appearance of the avatar is set by the user U of the avatar via the display device 10, and the image processing unit 64 acquires the image data representing the appearance of the avatar from the display device 10. However, the method of acquiring the image data representing the appearance of the avatar is arbitrary, and may be set automatically by the image processing unit 64, for example.

[0032] The image processing unit 64 transmits the generated image data for the virtual space SV to the display device 10 via the communication unit 52. The display device 10 displays an image for the virtual space SV using the image data for the virtual space SV transmitted from the image processing unit 64 (image processing device 12) via the display control unit 44. That is, in this example, the image processing unit 64 generates image data for the virtual space SV as seen from the field of view V of the avatar Aa and transmits it to the display device 10A. The display device 10A displays an image for the virtual space SV as seen from the field of view V of the avatar Aa on the display unit 22 based on the image data received from the image processing unit 64. Similarly, the image processing unit 64 generates image data for the virtual space SV as seen from the field of view V of the avatar Ba and transmits it to the display device 10B. The display device 10B displays an image for the virtual space SV as seen from the field of view V of the avatar Ba on the display unit 22 based on the image data received from the image processing unit 64. Similarly, the image processing unit 64 generates image data for the virtual space SV as seen from the field of view V of the avatar Ca and transmits it to the display device 10C. The display device 10C displays, on the display unit 22, an image for the virtual space SV as seen from the field of view V of the avatar Ca, based on the image data received from the image processing unit 64. This allows the users A, B, and C to recognize that they are in the virtual space SV as the avatars Aa, Ba, and Ca.

[0033] Here, some users may feel uncomfortable in interpersonal relationships with other avatars in the virtual space SV. For example, some users may feel uncomfortable if an avatar they do not feel close to touches or approaches their avatar within close range. Furthermore, some users may feel uncomfortable if an avatar they feel close to cannot approach their avatar within close range. In response to this, the image processing device 12 according to this embodiment sets a boundary area around the user's avatar that prohibits other avatars from entering, based on an discomfort index indicating the degree of discomfort the user feels toward the avatar of the other user. This prevents the user from feeling uncomfortable.

[0034] Details of this process will be explained below. In the following process, the user of the avatar for which the boundary area is to be set will be referred to as the target user, and the avatar of the target user will be referred to as the target avatar. Also, avatars of users other than the target user that exist within a predetermined area AR of the virtual space SV used by the target user will be referred to as other avatars, and users of the other avatars will be referred to as other users. Also, the predetermined area AR is an area (space) within the virtual space SV, and may be set arbitrarily. For example, the predetermined area AR may be a partial area within the virtual space SV, or may be the entire virtual space SV.

[0035] (Obtaining discomfort index) The acquisition unit 60 of the image processing device 12 acquires a discomfort index of a target user who uses a predetermined area AR of the virtual space SV. The discomfort index is an index indicating the degree to which the target user feels discomfort with other avatars (avatars of other users) present in the predetermined area AR.

[0036] (Calculation of discomfort index based on favorable users) The acquisition unit 60 may acquire the discomfort index using any method. In this embodiment, however, the acquisition unit 60 calculates the number of favored users, who are other users determined to have a favorable impression by the target user under predetermined conditions based on information about other users, and calculates the discomfort index based on the number of favored users. Specifically, the acquisition unit 60 determines, from the information about other users, whether the target user has a favorable impression of the other users (other avatars), in other words, whether the other users are favored users. The acquisition unit 60 performs a similar determination for each other avatar (each other user) present within the predetermined area AR. The acquisition unit 60 calculates a lower discomfort index as the number of favored users among the users of the other avatars (other users) within the predetermined area AR increases. By calculating the discomfort index from the number of favored users in this way, it is possible to appropriately predict whether the target user will perceive the predetermined area AR as an intimate place, thereby preventing the target user from feeling uncomfortable.

[0037] The information of other users used to calculate the discomfort index may be, for example, friend information indicating the friend relationship between the other user and the other user. A friend relationship means that the user and the other user have a friendly relationship. The friend information includes various information about users who are friends of the user, such as their identification information and names. In this case, the acquisition unit 60 acquires the friend information of the other users. If the friend information indicates that the target user has a friend relationship with another user who has a friend relationship with the other user, the other user is considered to be a favored user. The friend relationship may be established when the input unit 20 accepts a user's operation. For example, when the display device 10 of the target user receives a friend request (a request to establish a friendly relationship) from a certain other user via the communication unit 26 and the target user approves the friend request via the input unit 20, the display device 10 transmits information indicating that the friend request has been approved to the image processing device 12 via the communication unit 26. Next, the image processing device 12 transmits information indicating that the received friend request has been approved to the display device 10 of the approved other user via the communication unit 52. Furthermore, the fact that the users are friends is stored in at least one of the storage unit 50 of the image processing device 12 or the storage unit 24 of the display device 10 of the target user and the other user. By calculating the discomfort index from the number of other users who are friends in this way, it is possible to appropriately predict whether the target user will perceive the predetermined area AR as an intimate place, and to prevent the target user from feeling uncomfortable.

[0038] Furthermore, the acquisition unit 60 may determine whether the other user is a favored user based on the friend relationships between the other user and users other than the target user in the friend information of the other user. In this case, the acquisition unit 60 extracts users who are also friends of the target user (i.e., mutual friends) from among users who are friends of the other user, and if the number of extracted users is equal to or greater than a predetermined number, the other user is determined to be a favored user. For example, if the other user is friends with users D, E, and F and the target user is friends with users D and E, the number of extracted users (the number of mutual friends) is two. That is, in this case, even if the other user and the target user are not directly friends, if the number of mutual friends is equal to or greater than a predetermined number, the other user is determined to be a favored user. Note that the predetermined number here may be set arbitrarily. By calculating the discomfort index from the number of mutual friends in this way, it is possible to appropriately predict whether the target user will perceive the predetermined area AR as an intimate place, and to prevent the target user from feeling uncomfortable.

[0039] Furthermore, the past behavioral history of the other avatar (the avatar of the other user) may be used as information about the other user for calculating the discomfort index. The acquisition unit 60 acquires the past behavioral history of the other avatar, extracts behaviors that satisfy a predetermined condition from the behaviors included in the behavioral history, and if the number of extracted behaviors is equal to or greater than a predetermined number, the other user (the user of the other avatar) is designated as a favored user. Here, the behavior of the avatar may refer to at least one (preferably both) of the avatar's statements in the virtual space SV (text or voice output from the avatar) and the avatar's actions in the virtual space SV. Furthermore, the behavior that satisfies the predetermined condition may be arbitrarily set by the target user. For example, examples of behavior that satisfies the predetermined condition include predetermined actions and predetermined statements by the avatar (for example, actions and statements preferred by the target user). For example, the acquisition unit 60 performs image recognition on the movements of the other avatar, detects whether the movements match a pattern that constitutes a predetermined movement, and counts the number of movements that match the pattern that constitutes a predetermined movement. Similarly, the acquisition unit 60 performs text analysis on the sentences and voices output by the other avatars and counts the number of times that words or sentences that become keywords for the predetermined utterances are uttered. If the number of counted predetermined actions or predetermined utterances is equal to or greater than a predetermined number, the acquisition unit 60 determines that the other avatar is a favored user. In this way, by calculating the discomfort index from the number of users who have previously performed behavior that meets the condition, it is possible to appropriately predict whether the target user will perceive the predetermined area AR as an intimate place, and to prevent the target user from feeling uncomfortable.

[0040] The acquisition unit 60 may also extract actions common to actions previously performed by the target avatar from actions included in the action history of the other avatars, and if the number of extracted actions is equal to or greater than a predetermined number, designate the other users (users of the other avatars) as favored users. The acquisition unit 60 may also extract actions common to actions previously performed by avatars of users who are friends with the target user from actions included in the action history of the other avatars, and if the number of extracted actions is equal to or greater than a predetermined number, designate the other users (users of the other avatars) as favored users. For example, the acquisition unit 60 may perform image recognition on the movements of the target user's avatar, detect whether the movements match a predetermined movement pattern, and count the movements that match the predetermined movement pattern. Next, the acquisition unit 60 may perform image recognition on the movements of the other avatars, detect whether the movements match a predetermined movement pattern, and count the movements that match the predetermined movement pattern. Similarly, the acquisition unit 60 performs text analysis on sentences and voices output by the target avatar and accumulates the number of times keywords or sentences are uttered. Next, the acquisition unit 60 performs text analysis on the sentences and voices output by the other avatars and counts the number of times that words or sentences that become keywords for the predetermined utterances are uttered. If the counted number of predetermined actions or predetermined utterances of both the target avatar and the other avatars is equal to or greater than a predetermined number, the acquisition unit 60 determines that the other avatar is a favored user. In this way, by calculating the discomfort index from the number of users who have previously performed actions that are common to the target avatar or users with a friend relationship, it is possible to appropriately predict whether the target user will perceive the predetermined area AR as an intimate place, and to prevent the target user from feeling uncomfortable.

[0041] Furthermore, the acquisition unit 60 may determine whether another avatar existed in the same area of ​​the virtual space SV as the target avatar during the same time period, based on the behavior history of the other avatar. The acquisition unit 60 may then determine the other user (the user of the other avatar) as a favored user if the number of times the other avatar existed in the same area of ​​the virtual space SV during the same time period as the target avatar is equal to or greater than a predetermined number. The acquisition unit 60 may also determine the other user (the user of the other avatar) as a favored user if the number of times the other avatar existed in the same area of ​​the virtual space SV during the same time period as the avatar of a user who is a friend of the target user is equal to or greater than a predetermined number. In this way, by calculating the discomfort index from the number of users who have previously accessed a space shared with the target user or a user who is a friend of the target user, it is possible to appropriately predict whether the target user will perceive the predetermined area AR as an intimate place, thereby preventing the target user from feeling uncomfortable.

[0042] (Calculation of discomfort index based on favorable and unfavorable users) The acquisition unit 60 may calculate the discomfort index based on the number of favored users and the number of unfavored users among the users of other avatars (other users) within the predetermined area AR. Unfavored users refer to other users whom the target user does not regard favorably. In this case, the acquisition unit 60 calculates a lower discomfort index as the number of favored users increases, and calculates a higher discomfort index as the number of unfavored users increases. By calculating the discomfort index from both the number of favored users and the number of unfavored users in this manner, it is possible to more appropriately predict whether the target user will perceive the predetermined area AR as an intimate space, thereby preventing the target user from feeling uncomfortable. Note that in this embodiment, the degree of influence that the number of favored users has on the discomfort index is set to the same degree as the degree of influence that the number of favored users has on the discomfort index. That is, for example, if the initial value of the discomfort index is 0, the number of favored users is 3, and the number of unfavored users is 2, the discomfort index is decremented by -3 and +2 from the initial value, respectively, to become -1. However, the present invention is not limited to this, and the degree of influence that the number of favored users has on the discomfort index may be different from the degree of influence that the number of unfavored users has on the discomfort index.

[0043] A non-favored user may be a user among other users who has not been set as a favored user. In this embodiment, however, the acquisition unit 60 extracts non-favored users from other users based on the past behavioral history of the other avatars. The acquisition unit 60 acquires the past behavioral history of the other avatars, extracts behaviors that satisfy a predetermined condition from the behaviors included in the behavioral history, and if the number of extracted behaviors is equal to or exceeds a predetermined number, the other users (users of the other avatars) are designated as non-favored users. Here, the behavior of the avatar may refer to at least one (preferably both) of the avatar's statements in the virtual space SV (text or voice output from the avatar) and the avatar's actions in the virtual space SV. Furthermore, the behavior that satisfies the predetermined condition may be arbitrarily set or may be set by the target user. For example, examples of behavior that satisfies the predetermined condition include a predetermined action or a predetermined statement by the avatar (e.g., a behavior or statement that the target user dislikes). For example, the acquisition unit 60 performs image recognition of the movements of the other avatars, detects whether the movements match a predetermined pattern, and counts the number of movements that match the predetermined pattern. Similarly, the acquisition unit 60 performs text analysis on sentences and voices output by other avatars to detect and count the number of times that words or sentences that become keywords for predetermined statements are uttered. If the number of counted predetermined actions or predetermined statements is equal to or greater than a predetermined number, the acquisition unit 60 determines that the other avatar is a non-favored user.

[0044] The acquisition unit 60 may acquire information about other users (friend relationships and past behavioral history) by any method, but in this embodiment, the information about other users is stored in the storage unit 50 and reads it out. However, the acquisition unit 60 is not limited to this, and may acquire information about other users via the communication unit 52 from, for example, the display device 10 used by the other users.

[0045] (Calculation of discomfort index based on area information) The acquisition unit 60 is not limited to calculating the discomfort index using information about other users. For example, the acquisition unit 60 may calculate the discomfort index based on the past usage history of the predetermined area AR. Here, the past usage history of the predetermined area AR is information indicating users who have accessed the predetermined area AR in the past. In this case, for example, the acquisition unit 60 calculates a lower discomfort index the greater the number of users who are friends with the target user in the past usage history of the predetermined area AR. By calculating the discomfort index from the past usage history of the predetermined area AR in this way, it is possible to appropriately predict whether the target user will perceive the predetermined area AR as an intimate place, and to prevent the target user from feeling uncomfortable.

[0046] The acquisition unit 60 may acquire the past usage history of the predetermined area AR by any method, but in this embodiment, the usage history is stored in the storage unit 50 and is read out. However, the acquisition unit 60 is not limited to this, and may acquire information of other users from other devices via the communication unit 52, for example.

[0047] (Setting the boundary area) The setting unit 62 sets a boundary area R around the target avatar (the avatar of the target user) based on the discomfort index acquired by the acquisition unit 60. The boundary area R is an area (space) within the virtual space SV where the target user cannot see other avatars. In other words, the boundary area R is an area (space) where other avatars are not displayed on the display device 10 of the target user. By setting the boundary area R based on the discomfort index, it is possible to set the distance between the target avatar and other visible avatars to a distance that the target user feels comfortable with, thereby preventing the target user from feeling uncomfortable. Note that since the boundary area R is an area around the target avatar, the position of the boundary area R within the predetermined area AR also moves as the position of the target avatar moves.

[0048] The setting unit 62 sets the boundary region R so that the size of the boundary region R increases as the discomfort index increases. In other words, the setting unit 62 sets the boundary region R so that the distance from the target avatar to the periphery of the boundary region R increases as the discomfort index increases. By making the boundary region R larger as the discomfort index increases, it is possible to prevent avatars that the target user does not feel close to from being viewed or coming into contact with, and to prevent the target user from feeling uncomfortable. It is also possible to view or come into contact with avatars that the target user feels close to or wants to be close to, and this allows the target user to communicate appropriately with other close users.

[0049] The setting unit 62 may also set the size of the boundary area R based on the purpose of the predetermined area AR (virtual space SV). That is, the setting unit 62 may vary the size of the boundary area R depending on the purpose of the predetermined area AR (virtual space SV). The purpose here refers to the purpose for which the user uses the virtual space SV or the predetermined area AR. Possible purposes include a meeting, a concert, a virtual office, a meeting with an acquaintance, shopping, and many others.

[0050] (Display mode settings) In this embodiment, it is preferable that the setting unit 62 also sets the display mode of the other avatars in the predetermined area AR. The setting unit 62 sets the display mode of the other avatars to either a first display mode or a second display mode.

[0051] The first display mode is a display mode in which no other avatars are allowed to exist in the boundary region R. That is, in the first display mode, no other avatars are permitted to enter the boundary region R, and the other avatars can only be located outside the boundary region R within the target region AR. Therefore, in the first display mode, no other avatars are visible to the target user within the boundary region R; in other words, no other avatars are displayed on the display device 10 of the target user within the boundary region R. In the first display mode, no other avatars exist within the boundary region R, so that actions or comments that the target user finds unpleasant can be suppressed.

[0052] The second display mode is a display mode in which other avatars are present in the boundary region R, but are displayed transparently. That is, in the second display mode, other avatars are permitted to enter the boundary region R, but are displayed transparently. Therefore, in the second display mode, other avatars cannot be seen by the target user within the boundary region R; in other words, other avatars are not displayed (displayed transparently) on the display device 10 of the target user within the boundary region R. In this way, even in the second display mode, other avatars are not seen within the boundary region R, and actions or comments that the target user finds unpleasant can be suppressed.

[0053] The setting unit 62 may set the display mode of the avatar in any manner. For example, the setting unit 62 may set the display mode of the avatar based on the purpose of the predetermined area AR (virtual space SV). In this case, for example, the display mode when the purpose is a concert or a meeting may be set as the second display mode, and the display mode when the purpose is a meeting with an acquaintance may be set as the first display mode.

[0054] However, the process of setting the display mode is not essential. That is, whether the first display mode or the second display mode is to be used may be determined in advance, and the image processing unit 64 may generate image data for the virtual space SV in the determined display mode.

[0055] (Image data generation) The image processing unit 64 generates image data for the virtual space SV based on the boundary region R. That is, the image processing unit 64 generates image data for the virtual space SV so that other avatars are not displayed on the display device 10 of the target user in the boundary region R. The image processing unit 64 may also perform similar processing on the display device 10 of other users (users of other avatars). That is, in this case, the target avatar is not displayed on the display device 10 of the other users. Note that the boundary region AR may be displayed so as to be visible, or may be displayed so as to be invisible, such as transparent. In the example of this embodiment, the boundary region R is a circular region (a cylindrical or hemispherical space in the three-dimensional case) centered on the target avatar, but the shape of the boundary region R is not limited thereto and may be a region (or space) of any shape. Note that the boundary region R is a region surrounding the target avatar, and therefore the position of the boundary region R within the target region AR also moves as the position of the target avatar moves within the target region AR.

[0056] In this embodiment, the image processing unit 64 generates image data for the virtual space SV based on the boundary area R and the display mode set by the setting unit 62 so that other avatars are not displayed on the target user's display device 10 in the boundary area R.

[0057] The image processing unit 64 transmits the generated (updated) image data for the virtual space SV to the display device 10 via the communication unit 52. The display device 10 displays an image for the virtual space SV using the image data for the virtual space SV transmitted from the image processing device 12 by the display control unit 44.

[0058] The setting unit 62 may update the boundary area R based on the discomfort index. That is, after the target avatar accesses the predetermined area AR, the discomfort index may be recalculated based on information about other users present in the predetermined area AR, and the boundary area R may be re-set based on the re-calculated discomfort index. In this case, the image processing unit 64 transmits image data for the virtual space SV updated based on the re-set boundary area R to the display device 10.

[0059] 4 is a schematic diagram showing an example of a predetermined area of ​​the virtual space in the first display mode. In the first display mode, the image processing unit 64 generates image data for the virtual space SV so that other avatars do not enter the boundary area R. FIG. 4 shows an example in which user A is the target user and users B and C are other users. As shown in FIG. 4, a boundary area R is set around user A's avatar Aa (target avatar), and other avatars, avatars Ba and Ca, cannot enter the boundary area R around avatar Aa.

[0060] Fig. 5 is a schematic diagram showing an example of a predetermined area of ​​the virtual space in the second display mode. In the second display mode, the image processing unit 64 generates image data for the virtual space SV so that other avatars are transparent within the boundary area R. Fig. 5 shows an example in which user A is the target user and users B and C are other users. As shown in Fig. 5, a boundary area R is set around user A's avatar Aa (target avatar), and avatar Ba, which is another avatar, is displayed transparently when it enters the boundary area R. On the other hand, avatar Ca outside the boundary area R is not made transparent. The transparent display process may be performed on the display devices of other avatars. That is, within the boundary region R, the avatar of the target user is displayed transparently on the display devices of other users.

[0061] (Operation restriction settings) The method for setting the display mode of other avatars has been described above, but the image processing device 12 may also restrict the operations of other users so that other avatars cannot enter the boundary region R of the target avatar.

[0062] When the operation restriction unit 66 of the image processing device 12 receives a predetermined operation of another avatar by another user, it determines whether or not a boundary area R has been set around the target avatar. If the operation restriction unit 66 determines that a boundary area R has been set for the target avatar, it acquires the coordinate position of the target avatar in the virtual space SV (predetermined area AR) and the range of the boundary area R. Next, when the operation restriction unit 66 detects that the other avatar is entering the boundary area R due to the operation of the other avatar by the other user, it does not accept the predetermined operation. This makes it possible to prohibit the other avatar from entering the boundary area R of the target avatar.

[0063] The predetermined operation here refers to an operation such as moving the other avatar in the direction of the coordinate position of the target avatar or extending a hand. The operation limiting unit 66 may acquire the content of the operation accepted by the input unit 20 of the display device 10 of the other user as the content of the predetermined operation of the other avatar via the communication unit 52. That is, the operation limiting unit 66 determines whether the other avatar will enter the boundary region R when the predetermined operation of the other avatar is applied based on the acquired content of the predetermined operation of the other avatar and the range of the boundary region R of the target avatar. If it determines that the other avatar will enter the boundary region R, it does not accept the predetermined operation. If it determines that the other avatar will not enter the boundary region R, it accepts the predetermined operation. For example, the operation limiting unit 66 calculates whether the other avatar will be located within the boundary region R when the other avatar is moved (or extended) from its current position by a movement amount corresponding to the predetermined operation. If it is located, it determines that the other avatar will enter the boundary region R and does not accept the predetermined operation.

[0064] In this embodiment, the operation limiting unit 66 limits the operation of the other avatar based on the boundary region R and the content of the received operation of the other avatar so that the other avatar cannot enter the boundary region R of the target avatar. By limiting the operation of the other avatar in this way, the other avatar can be forcibly prevented from entering the boundary region R, thereby suppressing actions and comments that the target user finds unpleasant.

[0065] Furthermore, when the operation of other avatars is restricted in this manner, the other avatars will not enter the boundary area R, and therefore the image processing unit 64 will generate image data for the virtual space SV so that the other avatars do not enter the boundary area R, as in the first display mode described above.

[0066] (Processing flow) The processing flow for generating image data of the virtual space SV described above will now be described. FIG. 6 is a flowchart illustrating the process for generating image data of the virtual space. As shown in FIG. 6, the image processing device 12 acquires information of other users using the acquisition unit 60 (step S10), and calculates a discomfort index for the target user based on the information of the other users (step S12). Then, the setting unit 62 of the image processing device 12 sets a boundary region R for the target user based on the discomfort index (step S14), and the image processing unit 64 of the image processing device 12 generates image data of the virtual space SV based on the boundary region R (step S16).

[0067] The processing flow for generating the operation restriction described above will now be described. FIG. 7 is a flowchart illustrating the operation restriction processing. As shown in FIG. 7, the image processing device 12 acquires information about other users using the acquisition unit 60 (step S20) and calculates a discomfort index for the target user based on the information about the other users (step S22). Then, the setting unit 62 of the image processing device 12 sets a boundary region R for the target user based on the discomfort index (step S24), and the operation restriction unit 66 of the image processing device 12 acquires details of a predetermined operation of the other avatar by the other user (step S26). The operation restriction unit 66 acquires details of the predetermined operation accepted by the input unit 20 of the other user and determines whether the other avatar will enter the boundary region R (step S28). If it is determined that the other avatar will enter the boundary region R (step S28; Yes), the operation restriction unit 66 restricts the predetermined operation of the other avatar by the other user (i.e., does not accept the predetermined operation) (step S30). On the other hand, if the operation restriction unit 66 determines through the acquisition unit 60 that another avatar will not enter the boundary area R (step S28; No), the operation restriction unit 66 does not restrict the specified operation (i.e., accepts the specified operation) (step S32).

[0068] The image processing device 12 may perform either the image data generation process or the operation restriction process, or may perform both. That is, the image processing device 12 may include either the image processing unit 64 or the operation restriction unit 66, or may include both.

[0069] (effect) As described above, the image processing device 12 of the present disclosure includes at least one of an acquisition unit 60 that acquires a discomfort index indicating the degree to which the target user feels uncomfortable with avatars (other avatars) of users other than the target user (other users) who are present within a predetermined area AR of the virtual space SV used by the target user, a setting unit 62 that sets a boundary area R around the avatar of the target user (target avatar) in which the target user cannot view other avatars based on the discomfort index, an image processing unit 64 that generates image data for the virtual space SV based on the boundary area R, and an operation restriction unit 66 that restricts operations of the other avatars so that the other avatars cannot enter the boundary area R based on the boundary area R and the content of the operations of the other avatars. According to the present disclosure, by setting the boundary area R based on the discomfort index, it is possible to set the distance between the target avatar and the visible other avatars to a distance that the target user feels comfortable with, thereby reducing the target user's discomfort.

[0070] The acquisition unit 60 calculates a lower discomfort index the greater the number of users (other users) other than the target user whose avatars are present within the predetermined area AR who are favored by the target user, and the setting unit 62 sets a wider boundary area R the higher the discomfort index. By making the boundary area R larger the higher the discomfort index, it is possible to prevent avatars that the target user does not feel close to from being viewed or coming into contact with, and to prevent the target user from feeling uncomfortable. It is also possible to view or come into contact with avatars that the target user feels close to or wants to be close to, allowing the target user to communicate appropriately with other close users.

[0071] The image processing unit 64 generates image data for the virtual space SV so that avatars of users other than the target user (other avatars) cannot enter the boundary area R, or so that avatars of users other than the target user (other avatars) are displayed transparently within the boundary area R. This makes it impossible for other avatars to be seen within the boundary area R, preventing the target user from feeling uncomfortable.

[0072] Although the embodiments of the present disclosure have been described above, the embodiments are not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0073] 10 Display device 12 Image processing device 60 Acquisition Department 62 Setting section 64 Image processing section AR predetermined area SV Virtual Space R boundary area

Claims

1. an acquisition unit that acquires a discomfort index indicating a degree to which the target user feels discomfort with an avatar of a user other than the target user that exists within a predetermined area of ​​a virtual space used by the target user; a setting unit that sets a boundary area around the avatar of the target user based on the discomfort index, the boundary area being an area in which the target user cannot view other avatars; an image processing unit that generates image data for the virtual space based on the boundary area; Including, Image processing device.

2. the obtaining unit calculates the discomfort index to be lower as the number of favored users who are determined to have a favorable impression of the target user under predetermined conditions among users other than the target user whose avatars are present within the predetermined area increases, and the setting unit sets the boundary region to be wider as the discomfort index is higher, The image processing device according to claim 1 .

3. the image processing unit generates image data for the virtual space so that avatars of users other than the target user cannot enter the boundary area or so that avatars of users other than the target user are displayed transparently within the boundary area.

3. The image processing device according to claim 1.

4. an acquisition unit that acquires a discomfort index indicating a degree to which the target user feels discomfort with an avatar of a user other than the target user that exists within a predetermined area of ​​a virtual space used by the target user; a setting unit that sets a boundary area around the avatar of the target user based on the discomfort index, the boundary area being an area in which the target user cannot view other avatars; an operation limiting unit that limits an operation of an avatar so that the avatar cannot enter the boundary area based on the boundary area and operations on the avatar of users other than the target user; Including, Image processing device.

5. acquiring a discomfort index indicating a degree to which the target user feels discomfort with respect to an avatar of a user other than the target user that exists within a predetermined area of ​​a virtual space used by the target user; setting a boundary area around the avatar of the target user based on the discomfort index, the boundary area being an area in which the target user cannot view other avatars; generating image data for the virtual space based on the boundary region; Including, Image processing methods.

6. acquiring a discomfort index indicating a degree to which the target user feels discomfort with respect to an avatar of a user other than the target user that exists within a predetermined area of ​​a virtual space used by the target user; setting a boundary area around the avatar of the target user based on the discomfort index, the boundary area being an area in which the target user cannot view other avatars; generating image data for the virtual space based on the boundary region; to the computer, program.

Images