Image processing device, control method thereof, and program

Abstract

To provide an image processing device capable of determining whether or not to display an avatar of other user based on a current state of drawing processing capability or a current state of a user, a control method of the image processing device, and a program.SOLUTION: An image processing device 101 for allowing an avatar of a user to participate in a virtual space together with avatars of other users includes: an image processing device information acquisition unit 204 for acquiring image processing device information 210; an other user avatar information acquisition unit 201 for acquiring other user avatar information 207; and another user avatar display determination unit 206 for determining whether or not to display the avatar of another user based on the image processing device information 210 and the other user avatar information 207.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, a control method for an image processing device, and a program. [Background technology]

[0002] XR (Extended Reality) technology is a technology that allows users to perceive a sensation different from reality by applying image processing to an image (captured image) captured of a subject existing in real space. A typical device for displaying images to which XR technology is applied is an HMD (Head Mounted Display) that is worn on the user's head. Furthermore, as one application of XR technology, a concept called the metaverse, in which people live in a virtual space, is attracting attention. Users from all over the world can participate in the metaverse. Users set their avatars as their alter egos and act in the metaverse.

[0003] By the way, an increase in users in the metaverse means that the number of avatars in the metaverse also increases, which leads to an increase in data communication volume and drawing load. As a result, problems such as a decrease in frame rate may occur in devices such as HMDs, which may impair the quality of the user experience. In this regard, Patent Document 1 discloses a technology for reducing the communication rate by reducing behavioral information for avatars that are far away in a virtual space. Patent Document 2 discloses a technology for calculating the amount of memory required to draw the texture of an avatar, and for users whose avatars exceed a set threshold, encouraging them to move. In this way, the technology of Patent Document 1 and the technology of Patent Document 2 are effective in suppressing the deterioration of the quality of the user experience, but it is also effective to determine whether to display avatars of other users based on the current state of the drawing processing capacity of a device such as an HMD or the current state of the user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-28789 A [Patent Document 2] JP 2000-215324 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, even when the technology of Patent Document 1 or Patent Document 2 is used, there is a problem in that the decision as to whether to display another user's avatar cannot be made based on the current drawing processing capability of a device such as an HMD or the current state of the user.

[0006] The present invention has been made in consideration of the above problems, and aims to provide an image processing device, a control method for an image processing device, and a program that can determine whether to display an avatar of another user based on the current state of drawing processing capability or the current state of the user. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the image processing device of the present invention is an image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, and is characterized in having a first acquisition means for acquiring information regarding the drawing processing capability of the image processing device as first information, a second acquisition means for acquiring information regarding the drawing of the avatars of the other users as second information, and a determination means for determining whether to display or hide the avatars of the other users based on the first information and the second information. Effect of the Invention

[0008] According to the present invention, it is possible to determine whether or not to display another user's avatar based on the current state of drawing processing capability or the current state of the user. [Brief description of the drawings]

[0009] [Figure 1]1 is a diagram illustrating an example of a hardware configuration of an image processing device according to a first embodiment. [Diagram 2] 1 is a diagram illustrating an example of a functional configuration of an image processing device according to a first embodiment. [Diagram 3] FIG. 4 is a diagram showing an example of display priorities in the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of a relationship between a display determination result of a different user avatar display determination unit, different user avatar display setting information, and an actual display on a user's image processing device in the first embodiment. [Diagram 5] FIG. 2 is a diagram for explaining an example in which an avatar of another user is changed and displayed in the first embodiment. [Figure 6] FIG. 2 is a diagram for explaining an example in which an avatar of another user is changed and displayed in the first embodiment. [Figure 7] 4 is a flowchart showing a flow of processing in the image processing device according to the first embodiment. [Figure 8] 4 is a diagram for explaining an example of a display determination result by a different user avatar display determination unit in the first embodiment. FIG. [Figure 9] 10 is a flowchart showing a flow of processing in an image processing device according to a second embodiment. [Figure 10] 13 is a flowchart showing a flow of processing in an image processing device according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment. For example, each part constituting the present invention can be replaced with any configuration that can exert a similar function. Also, any configuration may be added. Also, any two or more configurations (features) of each embodiment can be combined. Also, in each embodiment, the same configurations and processes are described with the same reference numerals.

[0011] First Embodiment Hereinafter, the first embodiment will be described with reference to FIG. 1 to FIG. 8. FIG. 1 is a diagram showing an example of a hardware configuration of an image processing device 101 according to the first embodiment. The image processing device 101 is an electronic device such as an HMD or smart glasses that provides a user with activities in a metaverse in which other users participate. In the first embodiment, the image processing device 101 will be described as an HMD. Therefore, in the following description, the image processing device 101 may be written as an "HMD". Also, the image processing device 101 may be written as a "virtual space" to mean the metaverse. Note that if a control device capable of communicating with an electronic device such as the above-mentioned HMD has the processing function of the image processing device 101 according to the first embodiment described later, it can also be treated as the image processing device 101 of the present invention. Such a control device may be, for example, a personal computer, a tablet terminal, a smartphone, or a game device. These points are the same in the second and third embodiments.

[0012] The image processing device 101 includes, as its components, a CPU 102, a ROM 103, a RAM 104, a sensing unit 105, a display unit 106, an operation unit 107, and a communication unit 108. In the image processing device 101, the components are connected to each other via a bus 109. The CPU 102 is an arithmetic processing device that performs overall control of the image processing device 101. The CPU 102 executes various programs stored in the ROM 103 and the like to perform various processes. The ROM 103 is a read-only non-volatile memory device that stores programs and parameters (e.g., image processing programs and initial data) that do not require modification. The RAM 104 is a memory for temporarily storing input information, arithmetic results in image processing, and the like. The RAM 104 also provides a working area for the CPU 102. The sensing unit 105 performs eye tracking and acquires gaze information of a user wearing an HMD.

[0013] The display unit 106 is a liquid crystal display or the like. The display unit 106 displays captured images, virtual objects, avatars, characters, and / or items (icons; content). The operation unit 107 is an operation unit including an operation member such as a power button or a dial. The communication unit 108 transmits and receives data to and from an external device by wired communication or wireless communication (wireless LAN or local 5G, etc.). The communication unit 108 is a device that complies with a communication standard such as Ethernet or IEEE902.11. In the following, a user wearing an HMD is described as a "this user", and other users participating in the same virtual space as this user are described as "other users".

[0014] 2 is a diagram showing an example of a functional configuration of the image processing device 101 according to the first embodiment. The image processing device 101 has a different user avatar information acquisition unit 201, a virtual space information acquisition unit 202, a different user avatar display setting information acquisition unit 203, an image processing device information acquisition unit 204, a user current situation information acquisition unit 205, and a different user avatar display determination unit 206. The processes performed by these functional configurations can be realized by the CPU 102 executing a program, so it can be said that the CPU 102 has these functional configurations. However, it is not necessary for the CPU 102 to realize all of the processes performed by these functional configurations. For example, the image processing device 101 may have a dedicated processing circuit that realizes one or more processes.

[0015] Each component of the image processing device 101 may exchange information with other components by any method. For example, the image processing device 101 may store information acquired or generated by each component in the RAM 104, as described below, to enable other components to acquire such information. That is, in the first embodiment, information is exchanged between the components via the RAM 104. Also, in the image processing device 101, each component may output acquired or generated information to other components without storing the information in the RAM 104.

[0016] Returning to the description of FIG. 2, the other user avatar information acquisition unit 201 (second acquisition means) acquires other user avatar information 207 and stores it in the RAM 104. The other user avatar information 207 (second information) is information related to the drawing of the avatar of the other user, such as position coordinate information, polygon count, and texture of the avatar of the other user. The virtual space information acquisition unit 202 (third acquisition information) acquires virtual space information 208 and stores it in the RAM 104. The virtual space information 208 (third information) is information related to the virtual space in which the user is participating, such as polygon count and texture of virtual objects constituting the virtual space, and other setting information of the virtual space. The other setting information of the virtual space is information specifying the location represented by the virtual space, such as information set by the creator or administrator, such as that the virtual space is a museum or a live music venue.

[0017] The other user avatar display setting information acquisition unit 203 (fifth acquisition means) acquires other user avatar display setting information 209 and stores it in the RAM 104. The other user avatar display setting information 209 (fifth information) is information set by other users, such as whether to permit or prohibit the display of their own avatar for this user. The image processing device information acquisition unit 204 (first acquisition means) acquires image processing device information 210 and stores it in the RAM 104. The image processing device information 210 (first information) is information related to the drawing processing capability of the image processing device 101, such as the amount of resources of the CPU 102 and RAM 104 available for drawing processing of avatars of other users, or the communication network speed of the image processing device 101.

[0018] The user current state information acquisition unit 205 (fourth acquisition means) acquires and stores the user current state information 211 in the RAM 104. The user current state information 211 (fourth information) is gaze information of the user indicating where the user wearing the HMD is focusing. As described above, the gaze information is acquired by eye tracking using the sensing unit 105. The other user avatar display determination unit 206 (determination means) determines whether to display or hide the avatars of other users based on various information stored in the RAM 104. The other user avatar display determination unit 206 determines the avatars of other users participating in the virtual space together with the avatar of the user.

[0019] FIG. 3 is a diagram showing an example of the display priority in the first embodiment. The display priority is a parameter that the other user avatar display determination unit 206 takes into consideration when determining whether to display or hide the avatar of another user, and is set using the other user avatar information 207. In the first embodiment, as shown in FIG. 3, the display priority is set according to a combination of the distance from the present user and other conditions. The distance from the present user refers to the distance from the present user's avatar to the other user's avatar in the virtual space, and can be "close" or "far". The distance from the present user is obtained from the position coordinate information of the other user's avatar included in the other user avatar information 207, the position coordinates of the present user's avatar, a threshold value, and the like. The other conditions refer to conditions related to the amount of information of the other user's avatar, and can be "none in particular", "the texture of the avatar does not match the surroundings", and "the amount of information of the avatar is large". The other conditions are obtained from the number of polygons and texture of the other user's avatar included in the other user avatar information 207, and the like.

[0020] A display priority of "A" is set for the avatars of other users who are "close" to the user and whose other conditions are "none". A display priority of "B" is set for the avatars of other users who are "close" to the user and whose other conditions are "avatar texture does not match the surroundings". A display priority of "C" is set for the avatars of other users who are "close" to the user and whose other conditions are "large amount of avatar information". A display priority of "D" is set for the avatars of other users who are "far" to the user and whose other conditions are "none". A display priority of "E" is set for the avatars of other users who are "far" to the user and whose other conditions are "avatar texture does not match the surroundings". A display priority of "F" is set for the avatars of other users who are "far" to the user and whose other conditions are "large amount of avatar information".

[0021] The display priority decreases in the order of "A", "B", "C", "D", "E", and "F". In other words, the display priority increases in the order of "F", "E", "D", "C", "B", and "A". The higher the display priority of another user's avatar, the more likely it is to be determined to be displayed, and the lower the display priority, the more likely it is to be determined to be hidden. Therefore, another user's avatar with a display priority of "A" is most likely to be determined to be displayed, and another user's avatar with a display priority of "F" is most likely to be determined to be hidden. The display priority may be set based only on the distance from the present user, or may be set based only on other conditions.

[0022] 4 is a diagram showing an example of the relationship between the display determination result of the other user avatar display determination unit 206, the other user avatar display setting information 209, and the actual display result of the present user. In the relationship shown in FIG. 4, the display determination result is the determination result of the other user avatar display determination unit 206 regarding the avatar of another user. As described above, the other user avatar display setting information 209 is information set by another user, such as whether to permit or prohibit the display of the user's own avatar for the present user. The actual display result of the present user is the actual display result of the avatar of another user on the display unit 106 of the HMD worn by the present user.

[0023] According to the relationship shown in FIG. 4, the avatar of another user that is determined to be "displayed" by the other user avatar display determination unit 206 and set to "permitted" by the other user is displayed on the display unit 106 of the HMD worn by the present user. On the other hand, the avatar of another user that is determined to be "displayed" by the other user avatar display determination unit 206 and set to "prohibited" by the other user is changed to an avatar in a state in which the alter-identification is impossible and displayed on the display unit 106 of the HMD worn by the present user. An example of an avatar in a state in which the alter-identification is impossible is a lightweight abstract avatar. The present user can recognize the existence of the alter-identification other user by the avatar changed in this way, but cannot identify who the other user is. In addition, the avatar of another user that is determined to be "not displayed" by the other user avatar display determination unit 206 is not displayed on the display unit 106 of the HMD worn by the present user, regardless of whether the avatar is set to "permitted" or "prohibited" by the other user.

[0024] 5 and 6 are diagrams for explaining an example in which the avatar of another user is changed and displayed. Here, an example in which the other user avatar display determination unit 206 determines that each of the avatars of two other users is "displayed" will be described. FIG. 5 shows a case in which both of the two other users have set the display of their own avatars to "permitted" for the present user. In this case, the avatar 501 of one other user and the avatar 502 of the other other user are both displayed on the display unit 106 of the HMD worn by the present user without being changed. FIG. 6 shows a case in which one of the two other users has set the display of his / her own avatar to "permitted" for the present user, and the other other user has set the display of his / her own avatar to "prohibited" for the present user. In this case, the avatar 601 of one other user is displayed on the display unit 106 of the HMD worn by the present user without being changed. In response to this, the avatar 602 of the other user is changed to an avatar in a state in which the origin of the alter-ego cannot be identified, and is displayed on the display unit 106 of the HMD worn by this user.

[0025] Fig. 7 is a flowchart showing the process flow of the image processing device 101 according to the first embodiment. The process flow of the image processing device 101 shown in the flowchart of Fig. 7 (a control method of the image processing device) is realized by the CPU 102 (computer) expanding a program stored in the ROM 103 into the RAM 104 and executing it. This also applies to the process flow of the image processing device 101 according to the second embodiment shown in the flowchart of Fig. 9 and the process flow of the image processing device 101 according to the third embodiment shown in the flowchart of Fig. 10. Hereinafter, the process in which the image processing device 101 determines whether the avatars of other users are "displayed" or "not displayed" will be described with reference to the flowchart of Fig. 7.

[0026] In step S701, the other user avatar information acquisition unit 201 acquires other user avatar information 207 and stores it in the RAM 104 (second acquisition step). In step S702, the virtual space information acquisition unit 202 acquires virtual space information 208 and stores it in the RAM 104 (third acquisition step). In step S703, the image processing device information acquisition unit 204 acquires the amount of available rendering resources as image processing device information 210 and stores it in the RAM 104 (first acquisition step). The amount of available rendering resources is the amount of resources of the CPU 102 and RAM 104 available for rendering the virtual space and avatars. The amount of available rendering resources is determined from the performance and capacity of the CPU 102 and RAM 104 and the usage rates of the CPU 102 and RAM 104 when the virtual space and avatars are not displayed.

[0027] In step S704, the other user avatar display determination unit 206 determines whether the other user's avatar should be "displayed" or "not displayed" based on various information stored in the RAM 104 (determination step). In this determination, the other user avatar display determination unit 206 first obtains the amount of resources available for avatar rendering processing (hereinafter abbreviated as "available resource amount") from the virtual space information 208 and the image processing device information 210. As a result, the amount of available resources is smaller if the virtual space is relatively complex, and is larger if the virtual space is relatively simple. Next, the other user avatar display determination unit 206 uses the other user avatar information 207 to obtain the amount of resources required for rendering and the display priority as shown in FIG. 3 for each of the avatars of all other users existing in the virtual space, in order of the distance from the present user's avatar.

[0028] Finally, the other user avatar display determination unit 206 selects an avatar to be displayed from among all other users' avatars present in the virtual space, taking into consideration the amount of available resources, the amount of resources required for drawing, and the display priority. The other user avatar display determination unit 206 then determines the avatar of the selected other user to be "displayed" and the avatar of the other user not selected to be "not displayed." At this time, the other user avatar display determination unit 206 may determine how to display the avatar of the other user, taking into consideration the other user avatar display setting information 209 acquired by the other user avatar display setting information acquisition unit 203, as shown in FIG. 4.

[0029] FIG. 8 is a diagram for explaining an example of a display determination result by the other user avatar display determination unit 206. In FIG. 8, 801 is the avatar of the present user, 802, 803, and 804 are avatars of other users determined to be "displayed", and 805 and 806 are avatars of other users determined to be "hidden". Return to the description of FIG. 7. In step S704, if the other user avatar display determination unit 206 determines that at least one avatar of other users is "displayed", the process proceeds to step S705. On the other hand, if the other user avatar display determination unit 206 determines that all avatars of other users are "hidden", the flowchart of FIG. 7 ends. In step S705, the CPU 102 displays the avatars of other users determined to be "displayed" in step S704 on the display unit 106 of the HMD worn by the present user. Then, the flowchart of FIG. 7 ends.

[0030] As described above, the image processing device 101 according to the first embodiment obtains the amount of available resources, the amount of resources required for drawing, and the display priority, using the other user avatar information 207, the virtual space information 208, and the image processing device information 210. Furthermore, the image processing device 101 selects an avatar to be displayed from among all the avatars of other users existing in the virtual space, by taking into consideration the amount of available resources, the amount of resources required for drawing, and the display priority. In this way, the image processing device 101 can determine whether to display the avatars of other users, based on the current state of its drawing processing capability.

[0031] Furthermore, the image processing device 101 does not display the avatars of other users determined to be "hidden" on the display unit 106 of the HMD worn by the present user, thereby reducing the load caused by displaying the avatars of other users and preventing a decrease in the quality of the user experience. Note that, unlike the first embodiment, even if the setting of the display priority is omitted, the image processing device 101 can determine whether or not to display the avatars of other users based on the current state of its drawing processing capability.

[0032] <Second embodiment> The second embodiment will be described below with reference to FIG. 9. Here, the differences from the first embodiment will be mainly described. The degradation of the quality of the user experience also depends on the network communication environment of the image processing device 101. If the network communication environment is unstable, there is a concern that it may take a long time to download the other user avatar information 207, and that the long time required to download the other user avatar information 207 may affect the download of the virtual space information 208. Therefore, in the second embodiment, a method for acquiring the communication network speed of the HMD as the image processing device information 210 will be described.

[0033] Fig. 9 is a flowchart showing a processing flow of the image processing device 101 according to the second embodiment. Hereinafter, with reference to the flowchart in Fig. 9, a process in which the image processing device 101 determines whether the avatar of another user is "displayed" or "not displayed" will be described. Steps S701 and S702 are similar to steps S701 and S702 shown in Fig. 7 described above, and therefore their description will be omitted. In step S901, the image processing device information acquisition unit 204 acquires the communication network speed of the image processing device 101 as image processing device information 210 and stores it in the RAM 104. The communication network speed is expressed in units such as bps (bits per second).

[0034] In step S902, the other user avatar display determination unit 206 determines whether the avatar of the other user is to be "displayed" or "hidden" based on various information stored in the RAM 104 (determination step). At this time, the other user avatar display determination unit 206 obtains a display priority by using the other user avatar information 207. Furthermore, the other user avatar display determination unit 206 selects an avatar to be displayed from among all the avatars of the other users existing in the virtual space by taking into consideration the communication network speed and the display priority. Then, the other user avatar display determination unit 206 determines the avatar of the selected other user to be "displayed" and determines the avatar of the other user not selected to be "hidden". At this time, the other user avatar display determination unit 206 may determine how to display the avatar of the other user by taking into consideration the other user avatar display setting information 209 acquired by the other user avatar display setting information acquisition unit 203 as shown in FIG. 4.

[0035] Specifically, for example, the other user avatar display determination unit 206 sets a threshold value for the communication network speed, and if the acquired communication network speed is below the threshold value, it determines that only the avatars of other users with a display priority of "A" are to be "displayed." In this case, the threshold value for the communication network speed may be any value. Also, the display priority used as a criterion when the other user avatar display determination unit 206 determines to "display" is not limited to "A" and may be any value. Furthermore, the other user avatar display determination unit 206 may set multiple threshold values ​​for the communication network speed to perform a stepwise determination of whether to "display" or "hide" the avatars of other users.

[0036] Returning to the description of Fig. 9. In step S902, if the other user avatar display determination unit 206 determines that at least one other user's avatar is "displayed", the process proceeds to step S705. On the other hand, if the other user avatar display determination unit 206 determines that all other users' avatars are "not displayed", the flowchart of Fig. 9 ends. Step S705 is the same as step S705 shown in Fig. 7 described above, and therefore description thereof will be omitted.

[0037] As described above, the image processing device 101 according to the second embodiment determines the display priority by using the acquired other user avatar information 207. Furthermore, the image processing device 101 selects an avatar to be displayed from among all other users' avatars existing in the virtual space by considering the communication network speed acquired as the image processing device information 210 and the display priority. In this way, the image processing device 101 can determine whether to display another user's avatar based on the current state of its drawing processing capability. That is, the image processing device 101 can determine whether to display another user's avatar by making an appropriate determination according to the communication environment.

[0038] Furthermore, the image processing device 101 does not display the avatars of other users determined to be "not displayed" on the display unit 106 of the HMD worn by the present user, thereby reducing the load caused by displaying the avatars of other users and suppressing deterioration in the quality of the user experience. In this way, in an unstable communication environment, the image processing device 101 can reduce the amount of communication data by limiting the avatars of other users to be displayed. Note that, although different from the second embodiment, even if the setting of the display priority and the acquisition of the virtual space information 208 are omitted, the image processing device 101 can determine whether or not to display the avatars of other users based on the current state of its drawing processing capability.

[0039] <Third embodiment> Hereinafter, the third embodiment will be described with reference to FIG. 10. Here, the differences from the first and second embodiments will be mainly described. In the third embodiment, the current state of the user is taken into consideration to determine whether to display the avatar of another user. FIG. 10 is a flowchart showing the flow of processing of the image processing device 101 according to the third embodiment. Hereinafter, with reference to the flowchart of FIG. 10, the process in which the image processing device 101 determines whether the avatar of another user is "displayed" or "not displayed" will be described. Steps S701 and S702 are the same as steps S701 and S702 shown in FIG. 7 or FIG. 9 described above, and therefore their description will be omitted. Step S703 is the same as step S703 shown in FIG. 7 described above or step S903 shown in FIG. 9 described above, and therefore its description will be omitted.

[0040] In step S1001, the user current state information acquisition unit 205 acquires the user current state information 211 and stores it in the RAM 104 (fourth step). As described above, the user current state information 211 is gaze information on where the user wearing the HMD is looking, and is acquired by eye tracking by the sensing unit 105. The gaze information is expressed as a vector. In step S1002, the other user avatar display determination unit 206 determines whether the avatars of other users are to be "displayed" or "not displayed" based on various information stored in the RAM 104 (determination step).

[0041] At this time, the other user avatar display determination unit 206 determines the display priority using the other user avatar information 207. However, if the user is, for example, paying attention to scenery or exhibits and not paying attention to the avatars of other users around him / her for a certain period of time, it can be said that the user is not interested in other users. Therefore, when the other user avatar display determination unit 206 detects from the user current situation information 211 that the user is not interested in other users, it changes other conditions of the display priority that are the basis for the determination so that the number of avatars of other users that are determined to be "displayed" is reduced.

[0042] Thereafter, the other user avatar display determination unit 206 selects an avatar to be displayed from among all other users' avatars present in the virtual space, taking into consideration the changed display priority. The other user avatar display determination unit 206 then determines the avatar of the selected other user to be "displayed" and determines the avatar of the unselected other user to be "not displayed." At this time, the other user avatar display determination unit 206 may determine how to display the avatar of the other user, taking into consideration the other user avatar display setting information 209 acquired by the other user avatar display setting information acquisition unit 203, as shown in FIG. 4.

[0043] Returning to the description of Fig. 10, in step S1002, if the other user avatar display determination unit 206 determines that at least one other user's avatar is "displayed", the process proceeds to step S705. In contrast, if the other user avatar display determination unit 206 determines that all other users' avatars are "not displayed", the flowchart of Fig. 10 ends. Step S705 is similar to step S705 shown in Fig. 7 or Fig. 9 described above, and therefore description thereof will be omitted.

[0044] As described above, the image processing device 101 according to the third embodiment obtains the display priority using the acquired other user avatar information 207. Furthermore, the image processing device 101 changes the display priority using the acquired user current situation information 211, and selects an avatar to be displayed from among all other users' avatars present in the virtual space by considering the changed display priority. In this way, the image processing device 101 can determine whether to display the avatar of another user based on the current situation of the user. That is, the image processing device 101 can determine whether to display the avatar of another user by making an appropriate determination according to the interest of the present user. Furthermore, the image processing device 101 does not display the avatar of another user determined to be "not displayed" on the display unit 106 of the HMD worn by the present user, thereby reducing the load caused by the display of the avatar of another user and suppressing the deterioration of the quality of the user experience. Note that, although different from the third embodiment, even if the acquisition of the virtual space information 208 and the image processing device information 210 is omitted, the image processing device 101 can determine whether to display the avatar of another user based on the current situation of the user.

[0045] Also, contrary to the above-mentioned example, if the user pays attention to the avatars of other users around him / her and does not pay attention to the surrounding scenery or exhibits for a certain period of time, it can be said that the user is interested in other users. Therefore, when the other user avatar display determination unit 206 detects from the user current situation information 211 that the user is interested in other users, it may change other conditions of the display priority that is the criterion for the determination so that the number of avatars of other users determined to be "hidden" is reduced. Furthermore, the image processing device 101 may detect the interest of the user in other users from the other user avatar information 207 and the user current situation information 211 as the user's current situation, and determine whether or not to display the avatars of other users based on the detected user's current situation. In this case, the setting of the display priority, the acquisition of the virtual space information 208, and the acquisition of the image processing device information 210 can be omitted.

[0046] <Other> Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, the other user avatar display determination unit 206 may also determine that a change to a virtual object near an avatar of another user determined to be "hidden" is also "hidden". In this way, it is possible to prevent an unnatural phenomenon in which, for example, when an avatar of another user determined to be "hidden" opens a door of a virtual object near the avatar, the door of the virtual object appears to open by itself to the user.

[0047] Furthermore, when the other user avatar display setting information 209 is information in which another user has set the display of his / her avatar for the present user to be "prohibited," the other user avatar display determination unit 206 may always determine the other user's avatar to be "hidden." In a similar case, regardless of whether the display determination result of the other user avatar display determination unit 206 is "display" or "hidden," the other user's avatar may always be prevented from being displayed on the display unit 106 of the HMD worn by the present user.

[0048] The other user avatar display determination unit 206 may also use other setting information of the virtual space included in the virtual space information 208 to determine whether to "display" or "hide" the avatars of other users. This allows the image processing device 101 to make an appropriate determination as to whether to display the avatars of other users depending on the location of the user. For example, if the virtual space is a place where the user wants to concentrate on exhibits such as a museum, the other user avatar display determination unit 206 may change other conditions of the display priority to increase the number of avatars of other users that are determined to be "hide". In addition, in the case of such a virtual space, the other user avatar display determination unit 206 may collectively determine that all avatars of other users are "hide". On the other hand, if the virtual space is a place where the user wants to feel the presence of other users around him / her such as a live music venue, the other user avatar display determination unit 206 may change other conditions of the display priority to increase the number of avatars of other users that are determined to be "display". In addition, in the case of such a virtual space, the other user avatar display determination unit 206 may collectively determine to "display" all the other users' avatars.

[0049] Furthermore, the other user avatar display determination unit 206 may change other conditions of display priority according to the number of polygons and texture of the virtual object included in the virtual space information 208. In this way, if the virtual space is relatively complex, the other user avatar display determination unit 206 can increase the number of other users' avatars determined to be "not displayed," and if the virtual space is relatively simple, the other user avatar display determination unit 206 can increase the number of other users' avatars determined to be "displayed."

[0050] In addition, if an avatar of another user that is not displayed on the display unit 106 of the HMD worn by the user is present within a predetermined distance of the user for a predetermined period of time or longer, the image processing device 101 may notify the user to that effect.

[0051] The present invention can also be realized by supplying a program that realizes one or more functions of each of the above-mentioned embodiments to a system or device via a network or a storage medium, and having one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0052] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) An image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: a first acquiring means for acquiring information relating to a rendering processing capability of the image processing device as first information; A second acquisition means for acquiring information related to the drawing of the avatar of another user as second information; and a determination means for determining whether to display or not display avatars of other users based on the first information and the second information. (Configuration 2) The image processing device according to configuration 1, wherein the first information is a communication network speed of the image processing device. (Configuration 3) The image processing device according to configuration 1 or 2, wherein the second information is position coordinates, number of polygons, and texture of an avatar of another user. (Configuration 4) The image processing device according to configuration 3, wherein the determination means determines whether to display or hide the avatars of other users based also on the display priority set from the second information. (Configuration 5) The image processing device described in Configuration 4, wherein the determination means sets the display priority using at least one of information on the position coordinates of other users' avatars, the number of polygons, and the texture contained in the second information. (Configuration 6) A third acquisition means for acquiring information about the virtual space as third information, 6. The image processing device according to configuration 4 or 5, wherein the determination means changes the display priority based on the third information. (Configuration 7) The image processing device according to configuration 6, wherein the third information is the number of polygons and texture of a virtual object configured in the virtual space. (Configuration 8) The image processing device according to configuration 6, wherein the third information is information for identifying a location represented by the virtual space. (Configuration 9) A third acquisition means for acquiring information about the virtual space as third information, the third information is information that identifies a location represented by the virtual space, The image processing device described in configuration 1, characterized in that the determination means makes a collective decision to display or hide the avatars of all other users depending on information identifying the location represented by the virtual space acquired as the third information. (Configuration 10) An image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: a first acquiring means for acquiring information relating to a rendering processing capability of the image processing device as first information; A second acquisition means for acquiring information related to the drawing of the avatar of another user as second information; A third acquisition means for acquiring information related to the virtual space as third information; and a determination means for determining whether to display or hide avatars of other users based on the first information, the second information, and the third information. (Arrangement 11) The image processing device according to arrangement 10, wherein the first information is the amount of CPU and memory resources of the image processing device available for rendering processing of the virtual space and avatars. (Configuration 12) The image processing device according to configuration 10 or 11, wherein the second information is position coordinates, number of polygons, and texture of an avatar of another user. (Configuration 13) The image processing device according to configuration 12, wherein the determination means determines whether to display or hide the avatars of other users based also on the display priority set from the second information. (Configuration 14) The image processing device described in Configuration 13, characterized in that the determination means sets the display priority using at least one of information on the position coordinates of other users' avatars, the number of polygons, and the texture contained in the second information. (Configuration 15) The image processing device according to configuration 13 or 14, wherein the determination means changes the display priority based on the third information. (Configuration 16) The image processing device according to any one of configurations 10 to 15, wherein the third information is the number of polygons and texture of a virtual object configured in the virtual space. (Configuration 17) The image processing device according to any one of configurations 10 to 15, wherein the third information is information for identifying a location represented by the virtual space. (Configuration 18) The third information is information for identifying a location represented by the virtual space, The image processing device described in configuration 10, characterized in that the determination means makes a collective decision to display or hide the avatars of all other users depending on information identifying the location represented by the virtual space acquired as the third information. (Configuration 19) An image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: A second acquisition means for acquiring information related to the drawing of the avatar of another user as second information; A fourth acquisition means for acquiring information related to a current state of the user as fourth information; and a determination means for determining whether to display or not display the avatars of other users based on the second information and the fourth information. (Configuration 20) The image processing device according to configuration 19, wherein the second information is position coordinates, number of polygons, and texture of another user's avatar. (Configuration 21) The image processing device according to configuration 19 or 20, wherein the fourth information is user's line of sight information obtained by eye tracking. (Configuration 22) The image processing device according to configuration 20, wherein the determination means determines whether to display or hide the avatars of other users based also on the display priority set from the second information. (Configuration 23) The image processing device described in Configuration 22, characterized in that the determination means sets the display priority using at least one of information on the position coordinates of other users' avatars, the number of polygons, and the texture contained in the second information. (Configuration 24) The image processing device according to configuration 22 or 23, wherein the determination means changes the display priority based on the fourth information. (Configuration 25) A third acquisition means for acquiring information about the virtual space as third information, 24. The image processing device according to claim 22, wherein the determination means changes the display priority based on the third information. (Configuration 26) The image processing device according to configuration 25, wherein the third information is the number of polygons and texture of a virtual object to be configured in the virtual space. (Configuration 27) The image processing device according to configuration 25, wherein the third information is information that identifies a location represented by the virtual space. (Configuration 28) A third acquisition means for acquiring information about the virtual space as third information, the third information is information that identifies a location represented by the virtual space, The image processing device described in configuration 19, characterized in that the determination means makes a collective decision to display or hide the avatars of all other users depending on information identifying the location represented by the virtual space acquired as the third information. (Configuration 29) A fifth acquisition means for acquiring information on the display settings of avatars of other users as fifth information, 29. The image processing device according to claim 1, wherein the determining means determines whether to display or not display the avatars of other users based on the fifth information as well. (Configuration 30) The image processing device according to configuration 29, wherein the fifth information is information set by another user as to whether the user is permitted to display or prohibited from displaying the other user's avatar. (Configuration 31) In the image processing device described in configuration 30, when the fifth acquisition means has acquired, as the fifth information, information indicating that the avatar of another user is prohibited from being displayed to the user, and the judgment means determines to display the avatar of another user, the judgment means changes the avatar of the other user to an avatar whose origin cannot be identified. (Configuration 32) The image processing device according to any one of claims 1 to 31, characterized in that when the determination means determines to hide the avatar of another user, it determines to hide changes to virtual objects present in the vicinity of the avatar of the other user. (Configuration 33) The image processing device according to any one of claims 1 to 32, characterized in that the determination means determines whether to display or not display avatars of other users who participate in the virtual space together with the user's avatar. (Method 1) A method for controlling an image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising the steps of: a first acquisition step of acquiring information relating to a rendering processing capability of the image processing device as first information; a second acquisition step of acquiring information relating to the rendering of the avatar of another user as second information; a determination step of determining whether to display or hide avatars of other users based on the first information and the second information. (Method 2) A method for controlling an image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising the steps of: a first acquisition step of acquiring information relating to a rendering processing capability of the image processing device as first information; a second acquisition step of acquiring information relating to the rendering of the avatar of another user as second information; a third acquisition step of acquiring information about the virtual space as third information; A control method for an image processing device, comprising: a determination process for determining whether to display or hide avatars of other users based on the first information, the second information, and the third information. (Method 3) A method for controlling an image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising the steps of: a second acquisition step of acquiring information relating to the rendering of the avatar of another user as second information; a fourth acquisition step of acquiring information related to a current state of the user as fourth information; a determination step of determining whether to display or not display avatars of other users based on the second information and the fourth information. (Program 1) A program for causing a computer to execute each of the means of the image processing device according to any one of configurations 1 to 33. [Explanation of symbols]

[0053] 101 Image processing device 201 Other user avatar information acquisition unit (second acquisition means) 204 Image processing device information acquisition unit (first acquisition means) 206 Other user avatar display determination unit (determination means) 207 Other user avatar information (second information) 210 Image processing device information (first information)

Claims

1. An image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: a first acquiring means for acquiring information relating to a drawing processing capability of the image processing device as first information; A second acquisition means for acquiring information relating to the drawing of the avatar of another user as second information; and a determination means for determining whether to display or not display the avatars of other users based on the first information and the second information.

2. 2. The image processing apparatus according to claim 1, wherein the first information is a communication network speed of the image processing apparatus.

3. 2 . The image processing device according to claim 1 , wherein the second information is a position coordinate, a number of polygons, and a texture of an avatar of another user.

4. 4. The image processing apparatus according to claim 3, wherein the determining means determines whether to display or not display the avatars of other users based on the display priority set from the second information as well.

5. The image processing device according to claim 4 , wherein the determination means sets the display priority using at least one of information on position coordinates of the avatar of another user, the number of polygons, and a texture contained in the second information.

6. A third acquisition means for acquiring information about the virtual space as third information, 5. The image processing apparatus according to claim 4, wherein the determining means changes the display priority based on the third information.

7. 7. The image processing apparatus according to claim 6, wherein the third information is the number of polygons and texture of a virtual object configured in the virtual space.

8. The image processing apparatus according to claim 6 , wherein the third information is information for identifying a location represented by the virtual space.

9. A third acquisition means for acquiring information about the virtual space as third information, the third information is information that identifies a location represented by the virtual space, The image processing device according to claim 1, characterized in that the determination means makes a collective decision to display or hide the avatars of all other users depending on information identifying the location represented by the virtual space acquired as the third information.

10. An image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: a first acquiring means for acquiring information relating to a drawing processing capability of the image processing device as first information; A second acquisition means for acquiring information relating to the drawing of the avatar of another user as second information; A third acquisition means for acquiring information related to the virtual space as third information; and a determination means for determining whether to display or hide the avatars of other users based on the first information, the second information, and the third information.

11. 11. The image processing device according to claim 10, wherein the first information is the amount of CPU and memory resources of the image processing device available for rendering the virtual space and avatars.

12. 11. The image processing apparatus according to claim 10, wherein the second information is position coordinates, the number of polygons, and texture of an avatar of another user.

13. 13 . The image processing apparatus according to claim 12 , wherein the determining means determines whether to display or not display the avatars of other users based on the display priority set from the second information as well.

14. The image processing device according to claim 13 , wherein the determination means sets the display priority using at least one of information on position coordinates of the avatar of another user, the number of polygons, and a texture, which are included in the second information.

15. 14. The image processing apparatus according to claim 13, wherein the determining means changes the display priority based on the third information.

16. 16. The image processing apparatus according to claim 10, wherein the third information is a polygon count and texture of a virtual object configured in the virtual space.

17. The image processing device according to claim 10 , wherein the third information is information for identifying a location represented by the virtual space.

18. the third information is information that identifies a location represented by the virtual space, The image processing device according to claim 10, characterized in that the determination means makes a collective decision to display or hide all avatars of other users depending on information identifying the location represented by the virtual space acquired as the third information.

19. An image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: A second acquisition means for acquiring information relating to the drawing of the avatar of another user as second information; A fourth acquisition means for acquiring information related to a current state of the user as fourth information; a determination means for determining whether to display or not display the avatar of another user based on the second information and the fourth information.

20. 20. The image processing device according to claim 19, wherein the second information is position coordinates, number of polygons, and texture of an avatar of another user.

21. The image processing device according to claim 19 , wherein the fourth information is information on a user's line of sight obtained by eye tracking.

22. 21. The image processing apparatus according to claim 20, wherein the determining means determines whether to display or not display the avatars of other users based also on the display priority set from the second information.

23. The image processing device according to claim 22 , wherein the determination means sets the display priority using at least one of information on position coordinates of the avatar of another user, the number of polygons, and a texture, which are included in the second information.

24. 23. The image processing apparatus according to claim 22, wherein the determining means changes the display priority based on the fourth information.

25. A third acquisition means for acquiring information about the virtual space as third information, 23. The image processing apparatus according to claim 22, wherein the determining means changes the display priority based on the third information.

26. 26. The image processing apparatus according to claim 25, wherein the third information is the number of polygons and texture of a virtual object configured in the virtual space.

27. 26. The image processing apparatus according to claim 25, wherein the third information is information for identifying a location represented by the virtual space.

28. A third acquisition means for acquiring information about the virtual space as third information, the third information is information that identifies a location represented by the virtual space, The image processing device according to claim 19, characterized in that the determination means makes a collective determination as to whether to display or hide the avatars of all other users depending on information identifying the location represented by the virtual space acquired as the third information.

29. a fifth acquisition means for acquiring information on a display setting of an avatar of another user as fifth information; 20. The image processing device according to claim 1, wherein the determining means determines whether to display or not display the avatars of other users based on the fifth information as well.

30. 30. The image processing apparatus according to claim 29, wherein the fifth information is information set by another user as to whether the user is permitted to display or prohibited from displaying the avatar of the other user.

31. The image processing device according to claim 30, characterized in that when the fifth acquisition means has acquired, as the fifth information, information indicating that the display of another user's avatar to the user is prohibited, and the determination means determines that the other user's avatar should be displayed, the determination means changes the other user's avatar to an avatar whose origin cannot be identified.

32. The image processing device according to any one of claims 1, 10 and 19, characterized in that when the determination means determines to hide the avatar of another user, it determines to hide changes to virtual objects present in the vicinity of the avatar of the other user.

33. 20. The image processing apparatus according to claim 1, wherein the determining means determines whether to display or not display avatars of other users who participate in the virtual space together with the user's avatar.

34. A method for controlling an image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: a first acquisition step of acquiring information relating to a drawing processing capability of the image processing device as first information; a second acquisition step of acquiring information relating to the drawing of the avatar of another user as second information; a determination step of determining whether to display or not display the avatar of another user based on the first information and the second information.

35. A method for controlling an image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: a first acquisition step of acquiring information relating to a drawing processing capability of the image processing device as first information; a second acquisition step of acquiring information relating to the drawing of the avatar of another user as second information; a third acquisition step of acquiring information about the virtual space as third information; A control method for an image processing device, comprising a determination process of determining whether to display or hide the avatars of other users based on the first information, the second information, and the third information.

36. A method for controlling an image processing device for allowing a user's avatar to participate in a virtual space together with avatars of other users, comprising: a second acquisition step of acquiring information relating to the drawing of the avatar of another user as second information; a fourth acquisition step of acquiring information related to a current state of the user as fourth information; a determination step of determining whether to display or not display the avatars of other users based on the second information and the fourth information.

37. 2. A program for causing a computer to execute each of the means of the image processing apparatus according to claim 1.

38. A program for causing a computer to execute each of the means of the image processing apparatus according to claim 10.

39. A program for causing a computer to execute each of the means of the image processing apparatus according to claim 19.

Images