Program, computer system, and recording image processing method

Abstract

This invention provides a program, a computer system, and a method for processing recording images that can generate recording images while appropriately protecting personal information from those recording images. [Solution] The program causes the computer to function as a virtual space generation means for generating a virtual space, a player object placement means for placing player objects associated with user identification information that can identify the user who owns them into the virtual space, an image generation means for generating recording images using a predetermined virtual camera based on the user's shooting operation, and an image control means for controlling the display manner of the player objects in the recording image in image-related processing performed using the recording image when the recording image includes player objects, based on settings made by the user who owns the player object or the administrator of the virtual space.

Description

Technical Field

[0001] The present invention relates to a program for processing recording images, a computer system, and a recording image processing method.

Background Art

[0002] In a game in which a user operates a player character arranged in a virtual space, there is a screen shot function for shooting and saving a game screen (for example, a two-dimensional image captured by a virtual camera arranged in the virtual space) as a recording image. In Patent Document 1 below, it is described that by associating metadata including screen shot additional information such as a player name with a screen shot image, the search for the screen shot image is facilitated. Further, Patent Document 2 below describes that by distributing a screen shot image captured in a virtual space in a format of original information for image generation instead of drawn data in a bitmap format, copyright infringement can be prevented.

[0003] In recent years, a metaverse that realizes the real three-dimensional space in a virtual space and performs activities equivalent to the real world in the virtual space has attracted attention. In such a virtual space in the metaverse, in order to perform activities equivalent to the real world, it is necessary to be more careful about the privacy of virtual objects arranged in the virtual space and identifying individuals therefrom than in games using conventional virtual spaces.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] The present invention aims to provide a program, a computer system, and a method for processing recording images that can appropriately protect personal information from recording images while generating recording images in a manner different from that described in Patent Document 2. [Means for solving the problem]

[0006] To solve the above problems, a program according to one aspect of the present invention causes a computer to function as a virtual space generation means for generating a virtual space, a player object placement means for placing a player object associated with user identification information that can identify the user who owns it in the virtual space, an image generation means for generating a recording image using a predetermined virtual camera based on a user's shooting operation, and an image control means for controlling the display manner of the player object in the recording image based on settings made by the user who owns the player object or the administrator of the virtual space, in image-related processing performed using the recording image when the recording image includes the player object.

[0007] Another aspect of the present invention relates to a computer system comprising a program storage unit that stores a program having the above configuration, and a computer that executes the program stored in the program storage unit.

[0008] Another aspect of the present invention relates to a recording image processing method comprising the steps of generating a virtual space and associating a player object with user identification information that can identify the user who owns it with the virtual space. The process includes the steps of: placing the player object in a virtual space; generating a recording image using a predetermined virtual camera based on a user's shooting operation; and, if the recording image includes the player object, controlling the display manner of the player object in the recording image in image-related processing performed using the recording image based on settings made by the user who owns the player object or the administrator of the virtual space. [Effects of the Invention]

[0009] According to the present invention, personal information can be appropriately protected from recording images while simultaneously generating recording images. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a block diagram showing the hardware configuration of a computer system to which a program according to one embodiment of the present invention is applied. [Figure 2] Figure 2 is a block diagram showing the functional configuration of the control unit of the computer system shown in Figure 1. [Figure 3] Figure 3 is a flowchart illustrating the flow of image-related processing in this embodiment. [Figure 4] Figure 4 shows examples of recording images before and after processing in this embodiment. [Modes for carrying out the invention]

[0011] [Embodiment] The following describes a program and computer system according to an embodiment of the present invention, with reference to the drawings. The following description will primarily focus on a VR-type computer system.

[0012] [Hardware configuration] Figure 1 is a block diagram showing the hardware configuration of a computer system to which a program according to one embodiment of the present invention is applied. The computer system 1 is configured such that a computer 2, such as a game device, and a server 3 are connected to each other via a communication network NW such as the Internet or a LAN (Local Area Network). Of these, the computer 2 is controlled by a CPU (Central Processing Unit) 10, etc. The system includes a processor, to which an operation input unit 12, a storage device 14, a ROM (Read Only Memory) 15, and a RAM (Random Access Memory) 16 are connected via a bus 11. The computer system 1 may include multiple computers 2.

[0013] ROM15 is a semiconductor memory such as a mask ROM or PROM, and stores the boot program for starting computer 2. RAM16 is composed of DRAM or SRAM, and reads and temporarily stores the program 14a that the CPU 10 should execute, and the data 14b required for its execution, according to the game's progress.

[0014] Furthermore, the CPU 10 is connected to an HMD interface 17, a graphics processing unit 19, an audio synthesis unit 20, an audio interface 21, a wireless communication control unit 22, and a network interface 23. The HMD interface 17, the light detection device interface 18, and the audio interface 21 are connected to a head-mounted display (hereinafter referred to as HMD) 30 and headphones 32, respectively.

[0015] The HMD30 can be worn on the user's head. The HMD30 comprises a pair of left and right display units, a motion acquisition unit, and a control unit. When the HMD30 is worn on the user's head, A display unit for the left eye and a display unit for the right eye are positioned in front of the user's left and right eyes, respectively. A light-blocking partition is provided between the display unit for the left eye and the display unit for the right eye.

[0016] The motion acquisition unit of the HMD 30 detects the movement of the user's head. For example, the motion acquisition unit includes an acceleration sensor, a gyro sensor, a light emission unit, a light sensor, etc. The acceleration sensor detects, for example, the orientation of the HMD 30 with respect to the direction of gravity, that is, the tilt of the user's head. The gyro sensor detects, for example, the angular velocity of the HMD, that is, the speed at which the user tilts or rotates their head. The motion information obtained from the acceleration sensor and the gyro sensor is sent to the graphic processing unit 19 and the audio synthesis unit 20 via the HMD interface 17.

[0017] When the HMD 30 includes a light emission unit, a light detection device 31 is connected to the computer 2. The light detection device 31 is connected to the CPU 10 via the light detection device interface 18. The light emission unit is, for example, a plurality of LED lights provided on the HMD 30 at intervals from each other. The light detection device 31 is fixedly arranged, for example, in front of the user, and detects the position of the light emitted from the light emission unit of the HMD 30. By detecting the position of the light emitted from the light emission unit, the light detection device 31 acquires the position information of the user's head with respect to the light detection device 31. The position information can be used to correct the above-mentioned motion information. The position information obtained from the light detection device 31 is sent to the graphic processing unit 19 and the audio synthesis unit 20 via the light detection device interface 18.

[0018] In addition, when the HMD 30 includes a light sensor, a light emission device is connected to the computer 2 instead of the light detection device 31. The position of the user's head is detected by the light sensor of the HMD 30 detecting the light emitted from the light emission device arranged around the user.

[0019] Also, the HMD 30 may be provided with an eye-tracking function for tracking the movement of the line of sight of the user wearing the HMD 30. In this case, the user can perform an input operation by the line of sight. For example, the HMD 30 identifies the position where the user is gazing. Also, the HMD 30 detects the user's blink or the change in the diameter of the pupil.

[0020] The control unit of the HMD 30 controls various devices of the HMD 30, such as transmitting data from the acceleration sensor and the gyro sensor to the computer 2 and transmitting signals to the light emission unit.

[0021] The graphic processing unit 19 draws a computer image including a three-dimensional virtual space or the like according to the instruction of the CPU 10. For example, the graphic processing unit 19 generates a pair of virtual two-dimensional images having a parallax with each other so that the user can view the three-dimensional virtual space in stereoscopy. That is, the graphic processing unit 19 generates left-eye and right-eye virtual two-dimensional images having a parallax with each other, which are taken by each of a pair of virtual cameras arranged in the virtual three-dimensional space. Then, this pair of virtual two-dimensional images is converted into a video format and sent to the HMD 30 via the HMD interface 17, and is displayed on the pair of display units respectively. Further, the graphic processing unit 19 changes a pair of virtual two-dimensional images sent to the HMD 30 in real time according to the motion information and position information about the user sent from the HMD 30 and the light detection device 31 so as to give the user a sense of presence as if entering the virtual space.

[0022] The audio synthesis unit 20 reproduces and synthesizes the digital sound data included in the data 14b according to the instruction of the CPU 10. The sound data reproduced and synthesized by the audio synthesis unit 20 is decoded into an analog format and then sent to the headphones 32 via the audio interface 21 and output. As a result, the user playing this game can listen to the reproduced sound. The headphones 32 can be integrally configured with the HMD 30. Further, the audio synthesis unit 20 can change the sound output from the headphones 32 in real time according to the motion information and position information sent from the HMD 30 and the light detection device 31 so as to give the user a sense of presence as if entering the virtual space. A speaker can be connected to the audio interface 21 instead of the headphones 32.

[0023] The wireless communication control unit 22 has a 2.4GHz band wireless communication module and is wirelessly connected to the controller 33 attached to the computer 2, enabling data transmission and reception. The user can input signals to the computer 2 by operating buttons or other controls provided on the controller 33. The controller 33 may also be equipped with a motion acquisition unit similar to that of the HMD 30. The motion acquisition unit of the controller 33 detects the movement of the user's hand, arm, or the part of the device to which the controller is attached. For example, the motion acquisition unit may include an accelerometer, gyroscope, light emitter, or light sensor.

[0024] The network interface 23 connects computer 2 to a communication network NW such as the Internet or a LAN, and enables communication with other computers 2 and server 3. By connecting computer 2 to other computers 2 via the communication network NW and sending and receiving data to each other, it becomes possible to synchronize within the same virtual space.

[0025] On the other hand, server 3 is equipped with a high-performance processor such as a CPU 40 that controls its operation, and a storage device 42, ROM 43, RAM 44, and network interface 45 are connected to this CPU 40 via a high-speed bus 41.

[0026] ROM43 of server 3 is a semiconductor memory such as a mask ROM or PROM, and stores basic programs necessary for the basic operation of server 3, such as a program that controls communication between server 3 and other computers 2.

[0027] Furthermore, RAM 44 is composed of DRAM or SRAM, and temporarily stores programs that the CPU 40 should execute, as well as data necessary for their execution, by reading them from storage device 42, etc.

[0028] The storage device 42 of server 3 is a large-capacity storage medium built into server 3. Server 3 stores programs 42a and various data 42b in the storage device 42. Server 3 and computer 2 communicate with each other via a communication network NW.

[0029] Computer 2 receives (downloads and installs) programs 42a and data 42b stored in the storage device 42 of Server 3 in order to perform (play) predetermined virtual space activities such as games based on user input. Furthermore, Computer 2 receives (downloads and installs) update programs and data from Server 3. As a result, programs 14a and data 14b are stored in the storage device 14.

[0030] Computer 2, based on the installed programs and data, manages the activities within the virtual space and outputs the accompanying visuals (images or audio output) to the HMD 30 and headphones 32, etc.

[0031] Each computer 2 and each user will be assigned a different user account. Each account will also be assigned a user ID (identification code) to distinguish it from the other user.

[0032] Server 3 manages each user's user ID and receives data related to each user ID from computer 2 and stores it in storage device 42.

[0033] [Functional Configuration of Computer Systems] Figure 2 is a block diagram showing the functional configuration of the control unit of the computer system shown in Figure 1. Figure 2 mainly shows the functions necessary to realize the computer system 1 described above. Computer 2, which constitutes computer system 1, operates as a computer equipped with a control unit 5a that includes a CPU 10, a storage device 14, a ROM 15, a RAM 16, a graphics processing unit 19, an audio synthesis unit 20, etc. Server 3 operates as a computer equipped with a control unit 5b that includes a CPU 40, a storage device 42, a ROM 43, and a RAM 44, etc. As shown in Figure 2, the control unit 5 of computer system 1, which is formed by the cooperation of the control unit 5a of computer 2 and the control unit 5b of server 3, performs the functions described below by executing programs 14a and 42a and controlling each processor.

[0034] In other words, the control unit 5 performs functions such as virtual space generation means 51, player object placement means 52, image generation means 53, and image control means 54. These functional blocks include some or all circuits, including integrated circuits. Therefore, these components 51 to 54 can each be considered as circuits.

[0035] The virtual space generation means 51 generates a virtual space. Various objects (virtual objects) such as fixed background objects and moving objects such as non-player characters are placed in this virtual space. The player object placement means 52 places player objects such as player characters owned by the user into the virtual space. In this embodiment, the virtual space is a three-dimensional virtual space, and the objects placed are also three-dimensional objects.

[0036] A player object is associated with user identification information that can identify the user who owns that player object. In this embodiment, the user identification information is a user ID having a predetermined string. In this embodiment, a player object is a player character operated by the user. Furthermore, a player object is generated based on the user's input to the controller 33. For example, a player object is generated using the user's personal data. For example, the personal data includes biometric data obtained by scanning the user's face.

[0037] The generated player object data is associated with user identification information and stored in the storage device 14 of the user's computer 2. Furthermore, the generated player object data, along with the user identification information, is sent to server 3 and stored in the server 3's storage device 42.

[0038] Furthermore, virtual cameras are placed within the virtual space. In this embodiment, the virtual cameras are a pair of left and right virtual cameras that enable stereoscopic viewing. When the user changes the orientation of the HMD30 by turning their head or by making a predetermined operation input using the controller33, the orientation and position of the virtual cameras change, and the image captured by the virtual cameras changes.

[0039] The image generation means 53 generates recording images using a virtual camera based on the user's shooting operation. For example, when the user performs a first shooting operation on the controller 33, the image generation means 53 generates an image (screenshot, still image) of the image currently being captured by the virtual camera. The system generates still images (video clips, videos) as recording images. For example, when the user performs a second shooting operation on the controller 33, the image generation means 53 generates images (video clips, videos) captured by the virtual camera during a predetermined period of time from the second shooting operation as recording images. Thus, in this embodiment, the recording images include still images and videos, and are a portion of the images captured by the virtual camera. If the HMD 30 has an eye-tracking function, gaze input operations can be used as the first and second shooting operations. For example, the image generation means 53 can focus on an object that the user is fixated on, take a still image by blinking once as the first shooting operation, and take a video by blinking twice as the second shooting operation.

[0040] The data for the recorded images is stored in the storage device 14 of the user's computer 2 or in the storage device 42 of the server 3. The data for the recorded images is stored in association with the user identification information of the user who took the image. For example, the data for the recorded images is stored in a personal folder associated with the user. The personal folder is located in the storage device 14 of computer 2 or in the storage device 42 of server 3. The stored recorded images can be displayed based on user operations. In the following, "display" of recorded images may include displaying still images and playing back videos.

[0041] In this embodiment, the recording image data is a part of the image data (program execution format data) that can be displayed by a program for executing activities in the virtual space. Therefore, the recording image data is configured to be displayed (reconstructed) by executing the program. In other words, the recording image data in this embodiment is data that can be displayed (reconstructed) even if the image data itself is not recorded.

[0042] For example, recorded image data includes virtual camera data to identify the position and orientation of virtual cameras placed in the virtual space, object IDs to identify the type of object placed in the virtual space, operation state data to identify the position coordinates and orientation of the object, lighting data to identify the position of light sources in the virtual space, and image display elements such as user interfaces displayed on the display screen. The object ID includes an identifier to identify whether or not it is a player object. On the other hand, recorded image data does not include data that is stored in advance for program execution (data prepared in advance for activities in the virtual space), such as data of the object itself, effect data, and virtual space data.

[0043] The image control means 54 controls the display mode of the recorded image in image-related processing using the generated recorded image. This will be explained in detail below.

[0044] [Image-related processing] The image control means 54, when a player object is included in the recording image, controls the display mode of the player object in the recording image in image-related processing performed using the recording image, based on the settings made by the user who owns the player object.

[0045] Figure 3 is a flowchart illustrating the flow of image-related processing in this embodiment. Below, as an example of image-related processing, display processing for selecting and viewing a recording image from the user's personal folder that captured the recording image is illustrated. The image control means 54 monitors whether or not there is an instruction to execute image-related processing (step S1). For example, the user operates the controller 33 to select a virtual operation button to open the recording image display screen for displaying the recording image from the menu screen, and selects the desired recording image from the thumbnail images displayed on the recording image display screen. When there is an operation input to select the desired recording image, the image control means 54 monitors whether or not there is an instruction to execute image-related processing (step S1). The image control means 54 determines that an instruction to execute image-related processing has been received.

[0046] If it is determined that an instruction to execute image-related processing has been received (Yes in step S1), the image control means 54 reads the selected (target) recording image (step S2). Furthermore, the image control means 54 determines whether or not there are player objects owned by other users in the read recording image (step S3).

[0047] The image control means 54 queries whether a player object is included in the objects contained in the recording image data. For example, the image control means 54 reads the object ID of the objects contained in the recording image data, and if there is an object whose object ID contains an identifier indicating a player object, it extracts that object ID.

[0048] If a player object is present in the recording image (Yes in step S3), the image control means 54 performs an approval process for the user who owns the player object included in the recording image regarding the execution of image-related processing (step S4). In this embodiment, the user who owns the player object can pre-set the approval content for the player object they own. More specifically, the approval content for the player object can be set when the player object is created. The user can pre-set whether or not to allow the use of the player object when an image-related processing using an image taken by another user is performed, if the player object they own is included in a recording image captured by another user. Approval setting information indicating the content of the approval setting is associated with the corresponding player object or the user identification information of the user who owns it, and is stored in the storage device 42 of the server 3.

[0049] Thus, if approval details are pre-set for player objects included in the recording image, the image control means 54 automatically performs approval processing according to the approval details for the player object set by the user who owns the player object before executing image-related processing.

[0050] In this embodiment, a user who owns a player object can set the approval setting for the player object they own to unset or to require approval each time when the player object is created. The approval setting for a player object can be set or changed after the player object has been created. If the approval setting for a player object included in the recording image is unset or set to require approval each time, the image control means 54 requests approval from the user who owns the player object when executing image-related processing. The image control means 54 suspends the execution of image-related processing until approval is obtained from the user.

[0051] The approval request is executed as a notification process within the virtual space activity that the user can view while running this program. The user who receives the approval request can view the notification content regarding the approval request within the virtual space activity. The notification content regarding the approval request includes information about the user who captured the recording image (owner of the recording image), the content of the recording image, and the content of the image-related processing (display processing, saving processing, etc.). The user who receives the approval request decides whether or not to approve the use of their player object in the image-related processing based on the notification content regarding the approval request, and enters their decision.

[0052] The image control means 54 reads user identification information associated with the player object contained in the recording image during the approval process and identifies the user who owns the player object. The image control means 54 identifies the player object or the user who owns it. The approval setting information for the assigned player object is read. The image control means 54 executes the approval process according to the read approval setting information (approved, denied, not set, or approved each time).

[0053] If the approval setting information is not set or approval is required each time, the image control means 54 obtains information about the decision content from the user who owns the player object. If the approval setting information is approved or the decision content is approved, the image control means 54 determines that approval has been obtained from the user (Yes in step S5). In this case, the image control means 54 performs image-related processing using the recording image as is (step S6). For example, in the display process, the image control means 54 displays the recording image on the HMD 30.

[0054] On the other hand, if the approval setting information is denied or the decision content is denied, the image control means 54 determines that approval from the user has not been obtained (No in step S5). In this case, the image control means 54 performs processing on the recording image (step S7).

[0055] Figure 4 shows examples of recording images before and after processing in this embodiment. G1 in Figure 4 is a recording image taken by user B. Recording image G1 includes player object PO1 attempting to enter a shop in the virtual space. Player object PO1 is a player object owned by user A. In Figure 4, the player object owned by user A is denoted as PO1(UA).

[0056] For example, when user B performs an operation to display the recording image G1 on the HMD30, the image control means 54 performs the approval process in order to execute the display process as an image-related process. If user A approves the display of the player object PO1 in the recording image G1, the image control means 54 displays the recording image G1 on the HMD30 as is. On the other hand, if user A denies the display of the player object PO1 in the recording image G1, the image control means 54 processes the player object PO1 in the recording image G1 as a processing process. In the example in Figure 4, the player object PO1 is replaced with a generic character (a stick figure with a circular face and the rest of its body made up of a combination of sticks). The image control means 54 then performs the display process on the processed recording image (processed image) G2. As a result, user B's HMD30 displays the recording image G2, in which a generic character is about to enter a shop in the virtual space.

[0057] [summary] In summary, this embodiment is a program that causes computer 2 to function as a virtual space generation means 51 for generating a virtual space, a player object placement means 52 for placing player objects associated with user identification information that can identify the user who owns them into the virtual space, an image generation means 53 for generating recording images using a predetermined virtual camera based on the user's shooting operation, and an image control means 54 for controlling the display manner of player objects in recording images based on settings made by the user who owns the player objects, in image-related processing performed using the recording images when the recording images include player objects.

[0058] [effect] According to the above program, when a player object owned by one user is included in a recording image generated by another user, the display manner of the player object in the recording image is controlled according to the settings for the display manner of the player object set by that user when image-related processing using the recording image is performed. Therefore, it is possible to prevent a user's player object from being unintentionally displayed in a recording image generated by another user. On the other hand, if a part of the recording image contains the player object of another user... If the use of the entire recording image is prohibited when it contains player objects that can identify the user, there is a risk that recording images cannot be generated in virtual spaces such as the metaverse, severely hindering the generation and use of recording images. In contrast, the above program can suppress the disruption to the generation and use of recording images by controlling the display manner of player objects included in the recording image. Therefore, it is possible to generate recording images while appropriately protecting personal information from them.

[0059] Furthermore, in this embodiment, the image control means 54 performs an approval process for the user who owns the player object included in the recording image before image-related processing using the recording image is performed, and performs the image-related processing according to the user's approval. More specifically, when the image control means 54 displays the recording image, if the display of the player object included in the recording image is not approved in the approval process for the execution of image-related processing, the image control means 54 processes the player object and displays the recording image. This suppresses the display of the player object by displaying the recording image, and prevents the leakage of personal information from the player object.

[0060] Furthermore, in this embodiment, the image control means 54 can automatically perform approval processing according to the approval content set by the user who owns the player object before the execution of image-related processing. As a result, the user who owns the player object does not need to perform an approval operation each time a recording image containing the player object is generated, or each time image-related processing using the recording image is performed, thereby saving the user who owns the player object time and effort.

[0061] On the other hand, in this embodiment, the image control means 54 can request approval from the user who owns the player object when executing image-related processing, and can suspend the execution of image-related processing until approval is obtained from the user. This can meet the needs of users who wish to sequentially approve or deny approval depending on the content of the recording image containing the player object owned by the user, or the content of the image-related processing using the recording image.

[0062] [Other embodiments] Although this embodiment has been described above, the present invention is not limited to the above embodiment, and various improvements, changes, and modifications are possible without departing from the spirit of the invention.

[0063] For example, in the above embodiment, the display process for viewing the recording image from the user's personal folder who captured the recording image was given as an example of image-related processing, but the display process is not limited to this. For example, the display process may include publishing within the virtual space, publishing outside the virtual space such as on social networking services (SNS), and providing to others within or outside the virtual space. Furthermore, the image-related processing is not limited to display processing as long as it is processing using the recording image. For example, the image-related processing may include saving processing, file conversion processing, tag search processing, object property display processing, preview processing before saving, etc.

[0064] The saving process is the process of saving the generated recording image to the storage device 14 of computer 2 or the storage device 42 of server 3. During the approval process (until approved by the user who owns the player object), the recording image is temporarily stored in the storage device 42 of server 3, but becomes unreadable. If the display of the player object contained in the image is denied by the user who owns the player object, the recording image saved in storage devices 14 and 42 is the processed recording image G2. Therefore, when subsequent image-related processing (display processing, etc.) is performed, the processed recording image G2 is the target of processing. In this case, Subsequent image-related processing may not require approval from the user who owns the player object. Alternatively, if only pre-approval is adopted, where approval is automatically performed based on settings predetermined by the user who owns the player object, all approval processes may be performed before saving the recording image. If unprocessed data is saved to server 3 and approval and processing are performed later, there is a possibility that the unprocessed recording image data may be leaked due to some trouble. By performing all approval and processing processes before saving the recording image, the leakage of unprocessed recording image data can be prevented.

[0065] The file conversion process is a process that converts the storage data format of the recording image. The program executable format data exemplified in the above embodiment is difficult to use outside of virtual spaces such as SNS. For this reason, the file conversion process converts the storage data format of the recording image from program executable format to a standard data format such as JPEG or MPEG. This makes the recording image widely usable. When the image control means 54 performs an operation to convert the storage data format of the recording image associated with a player object included in the recording image during the file conversion process, it decides whether or not to perform the conversion through an approval process.

[0066] The tag search process is the process of performing a search based on search tags contained in the recorded image. In this case, search tag data is associated with various objects placed in the virtual space. When the user selects an object in the virtual space or recorded image displayed on the HMD30, or when an object is included within the imaging area of ​​the virtual camera, the search tag of the corresponding object is displayed. The search tag may be displayed in the virtual space or in a predetermined tag display window, etc. By performing a predetermined operation input on the displayed search tag, it is possible to search for more detailed information about the object or the user who owns it, related information, etc. In the tag search process, when an operation to display a search tag associated with a player object contained in the recorded image is performed, or when the conditions for displaying a search tag are met, the image control means 54 decides whether or not to display the search tag through an approval process.

[0067] The object property display process is the process of displaying properties associated with objects included in a recorded image. In this case, various objects placed in the virtual space are associated with detailed information indicating the attributes, status, etc. of the object. When the user selects an object in the virtual space or recorded image displayed on the HMD30, the properties of the corresponding object are displayed. The properties may be displayed in the virtual space or in a designated property display window, etc. Search tags may be included in the displayed property content.

[0068] In the property display process, the image control means 54 determines, through an approval process, whether or not to display a property associated with a player object included in the recording image when an operation is performed to display that property. Thus, the display modes of the player object controlled by the image control means 54 may include the display mode of the player object itself, the display mode of the search tag, or the display mode of the property.

[0069] The pre-save preview process is a process that displays the recording image as a preview image before performing the operation to decide whether or not to save the recording image captured at the time of capturing the recording image (the operation to start the above-mentioned save process). In the preview process, when the operation to capture a recording image is performed, the image control means 54 determines by approval whether or not to display the player object in the captured recording image in the preview image.

[0070] Furthermore, in the above embodiment, the recording image data is used for the execution of activities in the virtual space. Although an example was shown where the data is displayable by executing the RAM (data in program execution format), the recording image data may also be general (standard format) image data such as JPEG or general (standard format) video data such as MPEG. For example, in the saving process, the image control means 54 may first configure the recording image as data that can be displayed by executing the program, and then, upon approval from the user who owns the player object containing the recording image, convert the recording image into general image data or video data and save it.

[0071] Furthermore, when the recording image data is general image data or video data, and subsequent image-related processing is performed using the recording image, the determination of whether or not a player object is included in the recording image may be performed by image processing that uses data of multiple player objects stored in the storage device 42 of the server 3 as a database to determine whether or not a player object corresponding to the recording image exists.

[0072] Furthermore, before converting the recording image into general image data or video data and saving it, information about player objects contained in the recording image may be extracted in advance, and the extracted player object information may be associated with the recording image data and saved. In this case, whether or not a player object is included in the recording image is determined by whether or not information about the player object is associated with the recording image data. Thus, the determination of whether or not a player object is included in the recording image may be made when the recording image is created or saved (after input of the operation to create or save the recording image), or when image-related processing begins (after input of the operation for image-related processing).

[0073] Furthermore, while the above embodiment exemplified the process of replacing the player object with a stick figure as a processing step, it is not limited to this. The processing step may also involve replacing the stick figure with a general-purpose character (a character prepared in advance as a non-player character, etc.). Alternatively, the processing step may involve blurring, filling, or other processing on the display area of ​​the player object. Alternatively, the processing step may involve erasing the player object.

[0074] Furthermore, the processed recording image may be marked with an indication that it has been processed (e.g., a stamp). Additionally, credit information indicating that the processed recording image has been processed may be stored on the blockchain. These post-processing steps allow third parties to verify that the processed recording image is a privacy-conscious image.

[0075] The settings for displaying recorded images are not limited to settings for whether or not to perform image-related processing, such as whether or not to display or save the image. For example, it may be possible to set conditions for allowing the display of player objects. In this case, for example, it may be possible to set settings such as approving publication within the virtual space while denying publication outside the virtual space. Also, for example, it may be possible to set settings such as approving the display of the player object itself while denying the display of search tags and properties. Furthermore, for example, it may be possible to set settings so that display is not permitted if a specific part of the player object (e.g., face, inside clothing, tattoos, etc.) is visible in the recorded image. Also, for example, it may be possible to set shooting locations for which display is permitted or denied. In this case, it may be possible to set settings so that display is not permitted if the location where the recorded image was taken (i.e., where the player object is) is a specific location. Furthermore, it may be possible to set the display method (processing) of the player object when display is not permitted.

[0076] Furthermore, in the above embodiment, in the approval process, the user who owns the player object The description includes both cases where approval is automatically performed based on settings pre-configured by the system, and cases where an approval request is made to the user who owns the player object each time image-related processing is performed; however, only one of these approval processes may be performed.

[0077] Furthermore, while the above embodiment exemplifies determining that user approval was not obtained when the setting content is denied or the decision content based on the approval request is denied during the approval process, it is not limited to this. For example, if the image control means 54 fails to obtain the decision content within a predetermined period after issuing an approval request to the user who owns the player object, it may determine that approval was not obtained from that user. It may also be possible to set how to handle cases where a decision on the approval request is not made within a predetermined period (approval, denial, approval depending on the content of the image-related processing, etc.).

[0078] Furthermore, a single user may be able to generate multiple player objects. In this case, multiple player objects are associated with a single user identification piece of information. If a single user identification piece of information common to multiple player objects is associated with them, the image control means 54 may determine the settings for all of the multiple player objects at once. Alternatively, if a single user identification piece of information common to multiple player objects is associated with them, the image control means 54 may determine the settings for each player object individually.

[0079] Furthermore, while the above embodiment uses a user ID as an example of user identification information, it is not limited to this, as long as it is information that can identify the user. For example, user identification information may be My Number, health insurance card number, etc. Also, user identification information is not limited to strings. For example, user identification information can be various types of data such as facial recognition data, voice recognition data, etc.

[0080] Furthermore, while the above embodiment uses a player character operated by the player as an example of a player object, it is not limited to this. For example, a player object may include an avatar, a vehicle, an item, an NFT (Non-Fungible Token), etc. Because NFTs are difficult to tamper with, special care must be taken to avoid them appearing in images. A player object may be an object operated by the player, or it may be an item owned by the player character (for example, something worn, a character that acts together with the player character, etc.).

[0081] A player object may be created by the user who owns it, or it may be transferred from another user (for example, it may be someone else's copyrighted work). A player object may or may not use personal data.

[0082] The above embodiment shows an example in which a player object includes biometric data obtained by scanning the face of the user who owns the player object as personal data, but is not limited to this. For example, the personal data used to generate a player object may include personal biometric data such as face, build, race, and gender; data such as user-identifying information, numbers or strings that identify an individual in virtual space activities or in the real world; name data; and data of an individual's copyrighted work created in virtual space or in the real world. For example, a player object may be generated randomly or based on predetermined rules from data such as numbers or strings that identify an individual. If the predetermined rules are known, it may be possible to identify or infer personal information from the player object, potentially leading to the leakage of personal information. In such cases, the leakage of personal information can be prevented by refusing to approve the request. A player object may also be generated by a predetermined program that generates player objects from data such as numbers or strings that identify an individual.

[0083] The biometric data used to generate player objects is the appearance obtained by scanning the face and entire body. This includes data or skeletal data. The biometric data may also be an individual's vocal data. Furthermore, when generating the player object, the scanned biometric data may be processed (e.g., its appearance may be stylized, or effects applied to the vocal data). As a processing step for the player object using biometric data (step S7 in Figure 3), the player object generated from the processed biometric data may be displayed.

[0084] A player object may be generated based on the personal or biometric data of a user other than the user who owns the player object. For example, a player object may be generated from the appearance data of a celebrity that the user who owns the player object is a fan of. In this case, the data of the player object is also associated with the user-identifying information of that celebrity.

[0085] In this case, the image control means 54 may perform an approval process for the execution of image-related processing not only for the user who owns the player object, but also for the user from whom the player object originated (the user whose personal data or biometric data was used) when executing image-related processing. For example, if an avatar of idol x1, which is a player object, is generated from the data of idol x1 belonging to idol group X, and is owned by user A, the settings for the display manner of idol x1's avatar may be determined by user A, idol x1, or the administrator (management company) of idol x1. For example, if a recording image captured by user B includes the avatar of idol x1 and idol y1 belonging to another idol group Y, the settings may be configured to deny the display of idol x1's avatar.

[0086] In other words, detailed approval conditions may be set for the display mode, such as whether the recorded image contains a specific object (including a player object), or whether the recorded image contains a specific fixed object (such as a specific shooting location or a specific background).

[0087] A player object may be a character wearing clothes created by the user, or a character possessing weapons, tools, etc., created by the user. Alternatively, a player object may be clothing, weapons, tools, etc. That is, if a generic character wearing clothing or weapons etc., created by the user, is included in a recording image taken by another user, an approval process for the user who created the clothing or weapons etc. may be performed when image-related processing using the recording image is executed, similar to the embodiment described above. A player object may also be a character that performs a predetermined action pattern (choreography, emote, etc.) generated by the user (a player character capable of performing the user's own action pattern). If the author of the player object or the author of the equipment or action pattern of the player object is different from the user who owns the player object, the player object may be associated with the author's user identification information in addition to, or instead of, the user identification information of the user who owns the player object.

[0088] In this case, the image control means 54 may perform an approval process for the execution of image-related processing not only for the user who owns the player object, but also for the author of the player object or the equipment or behavior patterns of the player object, when image-related processing is performed. These authors may set the approval settings when the player object is created or transferred, or they may decide individually whether to approve or deny approval each time image-related processing is performed. Furthermore, if the player object is an NFT, approval from an administrator such as an NFT seller may be required.

[0089] Furthermore, in the above embodiment, the image control means 54 controls the player object in the recording image. When a player object is included, in image-related processing performed using the recording image, an example was given in which the display manner of the player object in the recording image is controlled based on the settings of the user who owns the player object. However, in addition to or instead of this, the display manner of the player object may be controlled based on the settings of the virtual space administrator. The virtual space administrator may be the creator (manufacturer) of a game or other program that uses the virtual space, or the operator of a virtual space activity that uses the virtual space. This prevents the deployment of recording images that do not conform to the image publication policy of the virtual space operator, etc.

[0090] Furthermore, while the above embodiment shows a configuration in which the search tag or properties of a player object are displayed when the user selects a player object in the recording image, the user who owns the player object may also be able to set whether or not to display the search tag or properties. For example, the information searched using the search tag corresponding to the player object or the information displayed in the properties may include user information about the user who owns the player object as information associated with the player object.

[0091] In this case, the settings regarding the display mode of the recording image may include a setting for whether or not to allow the display of user information. When the image control means 54 displays user information corresponding to a player object included in the recording image along with the recording image during image-related processing, it may reward the user who allowed the display of user information. The more user information is allowed to be displayed, the more recording images become available for searching, thus improving convenience for other users. In return, the user who allowed the display of user information is rewarded. The reward may be, for example, virtual currency, points, or specific items usable in virtual space activities, or it may be a reward usable in the real world.

[0092] Furthermore, the configurations 51 to 54 of the computer system 1 in the above embodiment may be implemented in computer 2 or in server 3, respectively.

[0093] Furthermore, although the above embodiment shows an example applied to a VR-type game system, it is also applicable to AR-type, SR-type, or MR-type game systems. Moreover, the above embodiment can be applied to configurations with an HMD that has only one display unit, such as an AR-type game system. Furthermore, it is not limited to the above-described game system, as long as it is a game system capable of performing activities within a virtual space. For example, the above embodiment can be applied to a game system equipped with a general-purpose monitor instead of an HMD. For example, computer 2 may be a personal computer, a game device, a smartphone, a tablet terminal, etc.

[0094] The effects and benefits of the present invention will be similarly exhibited even when other embodiments described above are adopted. Furthermore, the above embodiments and other embodiments, as well as other embodiments, may be combined as appropriate.

[0095] [Summary of Disclosures] Each of the following items is a disclosure of a preferred embodiment.

[0096] [Item 1] A computer includes: virtual space generation means for generating a virtual space; player object placement means for placing player objects associated with user identification information that can identify the user who owns them into the virtual space; image generation means for generating recording images using a predetermined virtual camera based on the user's shooting operation; and, when the recording image includes the player objects, in image-related processing performed using the recording image, A program that functions as an image control means for controlling the display manner of the player object based on settings made by the user who owns the player object or the administrator of the virtual space.

[0097] [Item 2] The program described in item 1, wherein the image control means performs an approval process for the user who owns the player object included in the recording image before the image-related processing using the recording image is executed, and executes the image-related processing according to the content of the approval from the user.

[0098] [Item 3] The program described in item 2, wherein when displaying the recording image, if the display of the player object included in the recording image is not approved in the approval process for the execution of the image-related processing, the image control means processes the player object and displays the recording image.

[0099] [Item 4] The image control means automatically executes the approval process according to the approval content set for the player object by the user who owns the player object before the execution of the image-related processing, as described in item 2 or 3.

[0100] [Item 5] The image control means is a program according to any one of items 2 to 4, which requests approval from the user who owns the player object when executing the image-related processing, and suspends the execution of the image-related processing until approval is obtained from the user.

[0101] [Item 6] The image control means is a program according to any one of items 2 to 5, which performs an approval process for the execution of the image-related processing with respect to the author of the player object when the image-related processing is executed.

[0102] [Item 7] The aforementioned player object is a program described in any of items 1 to 6, including an object generated using personal data.

[0103] [Item 8] The aforementioned player object is the program described in item 7, which includes a character generated from biometric data.

[0104] [Item 9] The setting includes a setting item for whether or not to allow the display of user information relating to the user who owns the player object as information associated with the player object when the recording image containing the player object is displayed, and the image control means, in the image-related processing, grants a reward to the user who allowed the display of the user information when the user information corresponding to the player object contained in the recording image is displayed together with the recording image, according to any one of items 1 to 8.

[0105] [Item 10] The image control means controls one user-specific information common to a plurality of player objects. A program described in any of items 1 to 9, which determines the settings for each player object if a report is associated with it.

[0106] [Item 11] The image control means is a program according to any one of items 1 to 10, which determines the setting content for multiple player objects collectively when a single user identification piece of information common to multiple player objects is associated with them.

[0107] [Item 12] A computer system comprising: a program storage unit storing a program described in any of items 1 to 11; and a computer that executes the program stored in the program storage unit.

[0108] [Item 13] A recording image processing method comprising: generating a virtual space; placing a player object associated with user identification information that can identify the user who owns it in the virtual space; generating a recording image using a predetermined virtual camera based on a user's shooting operation; and, if the recording image includes the player object, controlling the display manner of the player object in the recording image based on settings made by the user who owns the player object or the administrator of the virtual space in image-related processing performed using the recording image. [Industrial applicability]

[0109] This invention is useful for generating recording images while appropriately protecting personal information from those recording images. [Explanation of Symbols]

[0110] 1. Computer System 51 Virtual space generation means 52 Player object placement means 53 Image generation means 54 Image control means

Claims

1. Computers, means for generating a virtual space, Player object placement means for placing player objects in the virtual space to which user identification information that can identify the user who owns them is associated, Image generation means that generates recording images using a predetermined virtual camera based on the user's shooting operation. A program that, when the recording image includes the player object, functions as an image control means in image-related processing performed using the recording image, which controls the display manner of the player object in the recording image based on settings made by the user who owns the player object or the administrator of the virtual space.

2. The program according to claim 1, wherein the image control means performs an approval process for the user who owns the player object included in the recording image before the image-related processing using the recording image is executed, and executes the image-related processing according to the content of the approval from the user.

3. The program according to claim 2, wherein, when displaying the recording image, if the display of the player object included in the recording image is not approved in the approval process for the execution of the image-related processing, the image control means processes the player object and displays the recording image.

4. The program according to claim 2, wherein the image control means automatically executes the approval process according to the approval content for the player object set by the user who owns the player object before the execution of the image-related processing.

5. The program according to claim 2, wherein the image control means requests approval from the user who owns the player object when executing the image-related processing, and suspends the execution of the image-related processing until approval is obtained from the user.

6. The program according to claim 2, wherein the image control means also performs an approval process for the execution of the image-related processing with respect to the author of the player object when the image-related processing is executed.

7. The program according to any one of claims 1 to 6, wherein the player object includes an object generated using personal data.

8. The program according to claim 7, wherein the player object includes a character generated from biometric data.

9. The aforementioned settings include a setting item that specifies whether or not to allow the display of user information relating to the user who owns the player object, as information associated with the player object, along with the recording image, when the recording image containing the player object is displayed. The program according to any one of claims 1 to 6, wherein the image control means, in the image-related processing, grants a reward to the user who permitted the display of the user information when the user information corresponding to the player object included in the recording image is displayed together with the recording image.

10. The program according to any one of claims 1 to 6, wherein the image control means determines the setting content for each player object when a single user identification piece of information common to a plurality of player objects is associated with it.

11. The program according to any one of claims 1 to 6, wherein the image control means determines the setting content for the multiple player objects collectively when a single user identification piece of information common to the multiple player objects is associated with the multiple player objects.

12. A program storage unit storing a program according to any one of claims 1 to 6, A computer system comprising a computer that executes a program stored in the program storage unit.

13. The steps to generate a virtual space, The steps include: placing a player object in the virtual space to which user identification information that can identify the user who owns it is associated; The process involves generating a recording image using a predetermined virtual camera based on the user's shooting operation, A recording image processing method, which includes, when the recording image includes the player object, controlling the display manner of the player object in the recording image based on settings made by the user who owns the player object or the administrator of the virtual space, in image-related processing performed using the recording image.

Images