Program and acoustic control device

Abstract

To improve user experience in a service provided in a virtual space.SOLUTION: A virtual space control part causes an object arranged in a virtual space to be operated on the basis of an operation signal indicating an operation of a predetermined part in a user's body. A display part 562 causes a display 61 to display a video of the virtual space. A reproduction part 563 reproduces a first sound associated with the object when parameters correlated to the speed of the object are a first predetermined value or more, and reproduces a second sound associated with the object when the parameters are a second predetermined value or more larger than the first predetermined value. When the second sound is reproduced, the reproduction part 563 continues reproduction of the second sound even when the parameters change from the second predetermined value to the first predetermined value.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a program and an acoustic control device.

Background Art

[0002] Some game programs reproduce sound effects in a virtual space (see, for example, Patent Document 1). In the example of Patent Document 1, gunshots and the like are reproduced during a shooting game.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When various services (such as games) are provided in a virtual space, from the perspective of user experience, it is required that voices suitable for the scene be reproduced.

[0005] An object of the present disclosure is to improve user experience in services provided in a virtual space.

Means for Solving the Problems

[0006] A first aspect is a program that causes a computer system having one computer or a plurality of cooperating computers to function as a virtual space control unit, a display unit, and a reproduction unit, the computer system being connected with a display and a speaker, the virtual space control unit operating an object arranged in the virtual space based on an operation signal indicating an operation of a predetermined part of the user's body, The display unit causes the display to display an image of the virtual space. The playback unit causes the speaker to play sound. When a parameter correlated with the speed of the object is equal to or greater than a first predetermined value, the playback unit plays a first sound associated with the object, and when the parameter is equal to or greater than a second predetermined value that is greater than the first predetermined value, the playback unit plays a second sound associated with the object. When the second sound is being played, the playback unit continues to play the second sound even if the parameter changes from the second predetermined value to the first predetermined value. It is a program.

[0007] In a first aspect, When the parameter is equal to or greater than a third predetermined value, the playback unit plays a sound associated with the object according to the speed of the object, while when the parameter is less than the third predetermined value, the playback unit may play a sound associated with the object according to the amount of movement of the object.

[0008] In the above aspect, A part of the object may be an object whose position is fixed in the virtual space.

[0009] A second aspect is an acoustic control device including a storage unit that stores the program of the above aspect, and a control unit that executes the program.

Advantages of the Invention

[0010] According to the present disclosure, the user experience can be improved.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0012] [First Embodiment] Hereinafter, an example in which a program is implemented as a game program and an acoustic control device is realized as a game device will be described. The game device is an example of a computer system having one computer or a plurality of cooperating computers.

[0013] The game by this game program is carried out in a virtual space (three-dimensional). This game program is implemented using the technology of Virtual Reality (abbreviation: VR). The VR technology is a technology that creates a virtual space similar to reality on a computer and allows a user to experience the feeling of being in that virtual space.

[0014] There is no limitation on the type of game or the like. In this embodiment, as an example of the game, a battle-type action game in which a player character attacks and defeats an enemy character will be described.

[0015] In this game program, a character such as a player character may hold a virtual medium within the game screen. Here, "holding" means, for example, that the character and the virtual medium are integrally displayed on the screen. Depending on the game, "holding" may be expressed as "equipping".

[0016] Examples of virtual media held by characters include weapons. In the following, it is assumed that the player character holds a firearm (e.g., a handgun) as virtual media. The player character can attack enemy characters with that firearm.

[0017] In this game, various sounds are reproduced as voices emitted by virtual sound sources in the virtual space. The virtual media held by the characters can be virtual sound sources in the virtual space. That is, in this game, voices are reproduced as sounds emitted by the virtual media. For example, a firearm (virtual media) may emit a firing sound (specifically, when a sound simulating the firing sound is reproduced).

[0018] 《Configuration of the Game Device》 〈Hardware Configuration〉 FIG. 1 is a block diagram showing the configuration of a game device 5 (an example of an acoustic control device). The game device 5 executes a predetermined game based on the operation of the user. A display 61, a speaker 62, and a controller 63 are connected to the game device 5.

[0019] As the game device 5, commercially available devices such as a personal computer, PlayStation (registered trademark), XBox (registered trademark), PlayStation Vita (registered trademark), and Nintendo Switch (registered trademark) can be used.

[0020] In the game device 5, the game progresses based on the installed game program and game data. The game devices 5 can perform data communication with each other using a communication network (not shown) or a short-range wireless communication device (not shown).

[0021] The game device 5 includes a network interface 51, a graphic processing unit 52, an audio processing unit 53, an operation unit 54, a storage unit 55, and a control unit 56. The network interface 51, the graphic processing unit 52, the audio processing unit 53, the operation unit 54, and the storage unit 55 are electrically connected to the control unit 56 via a bus 59.

[0022] The network interface 51 is communicably connected to the communication network in order to transmit and receive various data between, for example, other game devices 5 and an external server device (not shown).

[0023] The graphic processing unit 52 draws a game image including player characters and various objects in the virtual space in a video format according to the game image information output from the control unit 56.

[0024] As described above, this game program uses VR technology. The graphic processing unit 52 generates an image of a virtual space based on VR technology. Hereinafter, an image displaying the virtual space based on VR technology is named a "VR image".

[0025] The graphic processing unit 52 is connected to the display 61. The VR image is displayed on the display 61.

[0026] The display 61 is constituted by, for example, a liquid crystal display. The display 61 is housed in a housing that can be worn by a user (hereinafter referred to as user P). The display 61 of the present embodiment is a display device of a head-mounted display (hereinafter referred to as VR goggles 70).

[0027] FIG. 2 is a perspective view showing an example of the VR goggles 70. As shown in FIG. 2, the VR goggles 70 include a main body portion 71 and a headband 72.

[0028] The headband 72 is for mounting the VR goggles 70 on the head of user P (a player of the game). FIG. 3 is a schematic diagram of a state where user P wears the VR goggles 70. The VR goggles 70 are mounted on the head of user P with the main body portion 71 covering both eyes of user P.

[0029] The arrow shown in Fig. 2 (hereinafter referred to as vector Vd) indicates the direction that becomes the front of the user P when the VR goggles 70 are worn. In this game program, the vector Vd is the normal line of the display surface of the display 61.

[0030] The main body 71 has a display 61 fixed inside. In the VR goggles 70, with the VR goggles 70 worn on the head of the user P, the display surface of the display 61 faces both eyes of the user P.

[0031] The VR goggles 70 are provided with an acceleration sensor and a gyro sensor (not shown). Depending on the VR goggles 70, a geomagnetic sensor may also be provided (not shown). Depending on the VR goggles 70, a camera may also be provided.

[0032] The output of the sensors of the VR goggles 70 is output to the operation unit 54. By analyzing the output of the sensors of the VR goggles 70, information on the position and orientation of the head of the user P (in other words, the position and orientation of the display 61) can be obtained.

[0033] The audio processing unit 53 is connected to the speaker 62. The audio processing unit 53 reproduces digital game audio according to the instructions of the control unit 56. Specifically, the audio processing unit 53 converts the audio data output by the control unit 56 into an analog signal and outputs it to the speaker 62.

[0034] The audio processing unit 53 is configured to be able to output multi-channel audio in order to reproduce three-dimensional sound. Accordingly, a plurality of speakers 62 are connected to the game device 5. These speakers 62 may be arranged on the left and right, front and back, above, etc. of the listener (for example, the player of the game) so that three-dimensional sound can be reproduced.

[0035] For example, the speaker 62 includes a front left speaker (L speaker), a front right speaker (R speaker), a front center speaker (C speaker), a subwoofer (LFE speaker), a left surround speaker (SL speaker), and a right surround speaker (SR speaker).

[0036] The operation unit 54 is connected to the controller 63. The operation unit 54 transmits and receives signals to and from the controller 63. The game player inputs signals (such as commands and data) into the game device 5 by operating various operators such as the buttons of the controller 63.

[0037] In this embodiment, two controllers 63 are provided. One controller 63 is held in the right hand of the user P. The other controller 63 is held in the left hand of the user P. When the user P moves their arm or hand, the position of the held controller 63 changes. Depending on the form of the controller 63, the user P may also wear the controller 63.

[0038] The controller 63 includes a plurality of operators (such as buttons). The user P can input an operation signal into the operation unit 54 by operating various operators such as the buttons of the controller 63. By inputting the operation signal into the game device 5, the user P can give various instructions in the game. The operation unit 54 sends the operation signal to the control unit 56.

[0039] The controller 63 includes a sensor S that outputs an operation signal. For example, examples of the sensor S include an acceleration sensor, a gyro sensor, and a geomagnetic sensor. When the user P moves their arm or hand, the output (operation signal) of the sensor S changes. The operation unit 54 sends the operation signal to the control unit 56. Note that the sensor S outputs the operation signal at regular intervals. Each time the operation unit 54 receives the operation signal from the sensor S, it transmits the operation signal to the control unit 56.

[0040] As described above, since the controller 63 is held by the right (left) hand of the user P, the controller 63 will move in conjunction with the movement of the hand (arm) of the user P. Therefore, the operation signal output by the sensor S will indicate the movement of the hand (arm) of the user P. That is, by analyzing the operation signal, the movement of the hand (arm) of the user P can be analyzed.

[0041] In this game program, by changing the position and orientation of the controller 63, the player character C1 in the virtual space can be moved.

[0042] For example, when the user P holds the controller 63 in the right hand and changes the position and orientation of the right hand (right arm), the right hand (right arm) of the player character C1 in the virtual space will move in conjunction with the movement of the right hand (right arm) of the user P. Also, when the user P holds the controller 63 in the left hand and changes the position and orientation of the left hand (left arm), the left hand (left arm) of the player character C1 in the virtual space will move in conjunction with the movement of the left hand (left arm) of the user P.

[0043] That is, in this game program, when the user P changes the position and orientation of the controller 63, the hand (arm) of the player character C1 corresponding to the hand (arm) of the user P that holds the controller 63 will move in the virtual space.

[0044] In this game program, the operation signal includes information indicating the position and orientation of the controller 63, as well as the speed and angular velocity (parameters) of the controller 63. In this game program, the speed and angular velocity of the hand (arm) of the player character C1 in the virtual space are determined according to the speed and angular velocity of the hand (arm) of the user P.

[0045] The storage unit 55 is composed of an HDD, an SSD, a RAM, a ROM, etc. Various programs including game data and game programs are stored in the storage unit 55. Examples of game data can include virtual media and user account information.

[0046] The control unit 56 controls the operation of the game device 5. The control unit 56 includes a CPU (microcomputer) and a semiconductor memory. Programs, data, etc. for operating the CPU are stored in the semiconductor memory.

[0047] <Function Configuration of Control Unit 56> By executing the game program, the control unit 56 functions as a game progress unit 561 (virtual space control unit), a display unit 562, and a playback unit 563 (see FIG. 1).

[0048] (Game Progress Unit 561) The game progress unit 561 advances the game by operating the player character, non-player characters, and predetermined objects in the virtual space. In controlling the operation of non-player characters, the game progress unit 561 uses, for example, AI (artificial intelligence).

[0049] The game progress unit 561 controls the operation of the player character, etc. according to the instructions of the user P (player). The instructions of the user P are input to the operation unit 54 via the controller 63.

[0050] For example, the user P can instruct the game progress unit 561 to move the player character and attack the enemy (such as firing a gun) by operating the controller 63. For example, when the user P operates a predetermined button on the controller 63, the controller 63 outputs an operation signal indicating that the trigger of the firearm held by the player character has been pulled to the operation unit 54.

[0051] Furthermore, when the game progression unit 561 receives a movement signal from the controller 63 (operation unit 54), it acquires various information (position, orientation, speed, angular velocity, etc.) included in the movement signal. The game progression unit 561 converts the acquired various information into movement information (information relating to the position, orientation, speed, angular velocity, etc.) of the hand (arm) of the player character C1 using a predetermined method (for example, by referring to a conversion table).

[0052] The game progression unit 561 moves the hand (arm) of the player character C1 in the virtual space based on the converted movement information.

[0053] When the player character C1 holds an object in its hand (arm), the game progression unit 561 moves the object in accordance with the action information. For example, when the game progression unit 561 receives an operation signal from the operation unit 54 indicating that a predetermined operation (such as an operation to equip an object) has been performed, the game progression unit 561 causes the object to be held in the hand (arm) of the player character C1.

[0054] When the game progression unit 561 receives a movement signal to move the hand (arm) of the player character C1 while the player character C1 is holding an object in the hand (arm), the game progression unit 561 moves the object in accordance with the movement of the hand (arm). In other words, when an object is held in the hand (arm) of the player character C1, the hand (arm) of the player character C1 and the object move in conjunction with each other.

[0055] (Display section 562) The display unit 562 generates a VR image Sc by referring to information about the virtual space generated by the game progression unit 561. The display unit 562 outputs the generated VR image Sc to the graphics processing unit 52.

[0056] (Playback section 563) The playback unit 563 plays the sound necessary for providing the service (in this embodiment, providing a game). Specifically, the playback unit 563 selects the audio data to be played back and instructs the audio processing unit 53 to play back the audio data. The playback unit 563 also gives instructions regarding the volume of each speaker 62 to the audio processing unit 53.

[0057] The playback unit 563 plays the sound of virtual sound sources existing in the virtual space according to the progress of the game. In this game program, various audio data are prepared in advance for playing back sound. For example, in this game program, audio data such as the firing sound of firearms, the sound of bullets hitting, and the operating sound of levers are prepared.

[0058] In this game program, audio data is associated with each object. For example, the lever C2 (stick C22) has audio data of the first sound, the second sound, and the third sound described below associated with it.

[0059] Here, in this game program, objects that are partially movable are arranged according to the operation of the player P. For example, in the lever C2 as shown in FIG. 4, the main body C21 is fixed in position in the virtual space. Also, in the lever C2, the stick C22 is relatively movable with respect to the main body C21.

[0060] Specifically, in the virtual space, when the user P moves the controller 63 while the player character C1 holds the stick C22 of the lever C2, the stick C22 moves in the direction of the arrow in FIG. 4. That is, by moving the controller 63 while the player character C1 holds the stick C22 of the lever C2, the user P can operate (tilt) the stick C22.

[0061] In games, various mechanisms are set up to operate in a virtual space by operating a lever stick. For example, a mechanism may be set up so that a closed door opens by pushing a lever stick in a virtual area.

[0062] In this game program, a sound reproduction process is executed to give the user a feeling of operation when operating the stick C22 of the lever C2.

[0063] Example of operation FIG. 5 is a flowchart illustrating the sound playback process in the present game program.

[0064] First, the reproduction unit 563 determines whether or not the player character C1 is holding the stick C22 of the lever C2 (step S1).

[0065] For example, when the game progression unit 561 receives an operation signal indicating that a predetermined operation has been performed from the operation unit 54, it causes the hand (arm) of the player character C1 to hold the stick C22 of the lever C2 in the virtual space. When the player character C1 is holding the stick C22 in the virtual space, the reproduction unit 563 determines that the player character C1 is holding the stick C22.

[0066] The reproduction unit 563 waits in step S1 until it is determined that the player character C1 is holding the stick C22 (No in step S1).

[0067] When it is determined that the player character C1 is holding the stick C22 (Yes in step S1), the reproduction unit 563 determines whether or not the speed of the stick C22 is equal to or greater than a first speed (first predetermined value) (step S2). Here, since the player character C1 is holding the stick C22, the hand (arm) of the player character C1 and the stick C22 move in conjunction with each other. Therefore, the speed of the stick C22 is determined according to the movement signal output from the controller 63.

[0068] When the playback unit 563 determines that the speed of the stick C22 is less than the first speed (No in step S2), it waits in step S2 until it determines that the speed of the stick C22 is equal to or greater than the first speed.

[0069] When the playback unit 563 determines that the speed of the stick C22 is equal to or greater than the first speed (Yes in step S2), it determines whether the speed of the stick C22 is equal to or greater than the second speed (second predetermined value) (step S3). The second speed is higher than the first speed.

[0070] When the playback unit 563 determines that the speed of the stick C22 is less than the second speed (No in step S3), it plays the first audio (step S4). The first audio is, for example, the audio emitted when the lever stick is operated at a low speed. The playback unit 563 continues to play the first audio until it determines that the speed of the stick C22 is less than the second speed.

[0071] When the playback unit 563 determines that the speed of the stick C22 is equal to or greater than the second speed (Yes in step S3), it plays the second audio (step S5). The second audio is, for example, the audio emitted when the lever stick is operated at a high speed. Note that the first audio and the second audio may be different, or they may be the same audio passed through a low-pass filter with a predetermined cut-off frequency.

[0072] The playback unit 563 determines whether the speed of the stick C22 is equal to or greater than the first speed (step S6). When the playback unit 563 determines that the speed of the stick C22 is equal to or greater than the first speed (Yes in step S6), it continues to play the second audio (step S7). When the playback unit 563 determines that the speed of the stick C22 is less than the first speed (No in step S6), it stops playing the second audio (step S8).

[0073] Summarizing the above, in this aspect, in a program that causes a game device 5 (a computer system having one computer or a plurality of cooperating computers) to function as a game progress unit 561 (virtual space control unit), a display unit 562, and a playback unit 563, the game device 5 has a display 61 and a speaker 62 connected thereto. The game progress unit 561 operates a stick C22 (object) arranged in the virtual space based on an operation signal indicating the movement of the arm (a predetermined part of the body) of the user P. The display unit 562 displays the video of the virtual space on the display 61, and the playback unit 563 plays back the sound on the speaker 62. When the speed (parameter) of the stick C22 is equal to or higher than a first speed (a first predetermined value), the playback unit plays back a first sound associated with the lever C2 (stick C22). When the speed of the stick C22 is higher than the first speed and equal to or higher than a second speed (a second predetermined value), the playback unit plays back a second sound associated with the lever C2 (stick C22). When the second sound is being played back, the playback unit 563 continues to play back the second sound even if the speed of the stick C22 changes from the second speed to the first speed.

[0074] 《Effects of the First Embodiment》 As described above, in this embodiment, when the speed of the stick C22 becomes equal to or higher than the second speed and the second sound is played back, then even if the speed of the stick C22 subsequently becomes lower than the second speed and equal to the first speed, the playback of the second sound continues. As a result, when the stick C22 is operated slowly after being operated quickly, it is possible to give the user P a sense of operation (feeling of feedback) of the stick C22 rather than changing the sound. Therefore, the user experience can be improved.

[0075] [Second Embodiment] In the second embodiment, even when the stick C22 is being operated at a low speed, a sound playback process for giving the user a sense of operation when operating the stick C22 of the lever C2 is executed.

[0076] 《Operation Example》 FIG. 6 is a flowchart for explaining the voice reproduction process in this game program.

[0077] First, the playback unit 563 determines whether the player character C1 is gripping the stick C22 of the lever C2 (step S1). The playback unit 563 waits in step S1 until it determines that the player character C1 is gripping the stick C22 (No in step S1).

[0078] When the playback unit 563 determines that the player character C1 is gripping the stick C22 (Yes in step S1), it determines whether the speed of the stick C22 is equal to or higher than the third speed (third predetermined value) (step S11). The third speed is the speed at which the sensor S cannot detect the speed of the user P's hand (arm) when the user P moves the controller 63 slowly (at a low speed).

[0079] When the playback unit 563 determines that the speed of the stick C22 is equal to or higher than the third speed (Yes in step S11), it plays the third voice according to the speed of the stick C22 (step S12). The third voice is, for example, the voice emitted when the stick of the lever is operated. In step S12, the third voice processed according to the speed of the stick C22 is played.

[0080] When the playback unit 563 determines that the speed of the stick C22 is less than the third speed (No in step S11), it detects the movement amount of the stick C22 (step S13). Then, the playback unit 563 plays the third voice according to the movement amount of the stick C22 (step S14). In step S13, the third voice processed according to the movement amount of the stick C22 is played.

[0081] To summarize the above, in this aspect, when the speed (parameter) of the stick C22 (object) is equal to or higher than the third speed (third predetermined value), the playback unit 563 plays back the third audio associated with the lever C2 (stick C22) according to the speed of the stick C22. On the other hand, when the speed of the stick C22 is lower than the third speed, the third audio associated with the lever C2 (stick C22) is played back according to the movement amount of the stick C22.

[0082] 《Effects of the Second Embodiment》 As described above, in this embodiment, when the speed of the stick C22 is low, it may not be detected by the sensor S. In this case, the third audio is played back according to the movement amount of the stick C22. Thereby, even if the stick C22 is operated at a low speed, the user P can be given the operating feeling (sense of feedback) of the stick C22. Therefore, the user experience can be improved.

[0083] [Other Embodiments] The game may be a first-person perspective game or a third-person perspective game. In the case of a first-person perspective game, the display unit 562 does not necessarily need to display the player character C1 on the display 61.

[0084] The service provided in the virtual space is not limited to games. In the virtual space, various services such as the provision of a business transaction place and the provision of a communication place can be provided. That is, this program can be used for service provision in a so-called metaverse.

[0085] The body part for detecting the movement of the user P is not limited to the "hand", and may be the "leg", "head", etc. In this case, the sensor S may be attached to the body part for detecting the movement of the user P.

[0086] There is no limitation on the type of sensor for detecting the movement of the user P. Also, the position and orientation may be detected using means other than the sensor (for example, a camera).

[0087] In each of the above embodiments, the case where the parameter is the speed of the stick C22 has been described as an example, but the present invention is not limited to this. The parameter may be any parameter correlated with the speed of the stick C22, such as the acceleration, position, angular velocity, angular acceleration, or angle of the stick C22.

[0088] In each of the above embodiments, the case where the object is the lever C2 (stick C22) has been described as an example, but the present invention is not limited to this. The object may be any movable object (e.g., a vehicle, etc.) within the virtual space.

[0089] In each of the above embodiments, the playback unit 563 may change the audio to be played according to the type of the object. In this case, it is sufficient that the audio to be played is associated with the type of the object in the storage unit 55.

[0090] In the above embodiment, the playback unit 563 may change the first audio, the second audio, and the third audio according to the speed of the stick C22.

[0091] The program may be implemented to be executed by a plurality of cooperating computers (e.g., a server computer and a client computer). For example, when the game device 5 operates as a client connected to a server, the processing performed by the control unit 56 may be performed by the server, or the processing may be shared between the server and the client.

[0092] The audio data may be synthesized in real time in addition to being prepared in advance.

[0093] Even when these other embodiments are adopted, the effects of the present aspect are exhibited. Also, it is possible to appropriately combine the present embodiment with other embodiments and between other embodiments.

Explanation of Reference Numerals

[0094] 55 Storage unit 56 Control Unit (Computer) 61 Display 63 Controller 561 Game Progression Unit (Virtual Space Control Unit) 562 Display Unit 563 Reproduction Unit S Sensor P User C1 Player Character C2 Lever C21 Main Body C22 Stick (Object)

Claims

1. In a program that causes a computer system having one computer or a plurality of cooperating computers to function as a virtual space control unit, a display unit, and a playback unit, the computer system has a display and a speaker connected thereto, the virtual space control unit operates an object arranged in the virtual space based on an operation signal indicating an operation of a predetermined part of the user's body, the display unit displays an image of the virtual space on the display, the playback unit plays back sound through the speaker, when a parameter correlated with the speed of the object is equal to or greater than a first predetermined value, the playback unit plays back a first sound associated with the object, and when the parameter is equal to or greater than a second predetermined value greater than the first predetermined value, the playback unit plays back a second sound associated with the object, when the second sound is being played back, the playback unit continues to play back the second sound even if the parameter changes from the second predetermined value to the first predetermined value, Program.

2. In the program according to claim 1, when the parameter is equal to or greater than a third predetermined value, the playback unit plays back a sound associated with the object according to the speed of the object, while when the parameter is less than the third predetermined value, the playback unit plays back a sound associated with the object according to the amount of movement of the object, Program.

3. In the program according to claim 1, part of the object is an object whose position is fixed in the virtual space, Program.

4. a storage unit that stores any one of the programs according to claims 1 to 3, and a control unit that executes the program Audio control device.

Images