Program and sound control device

Abstract

To improve user experience in a service provided in a virtual space.SOLUTION: A virtual space control part causes an object arranged in a virtual space to be operated on the basis of an operation signal indicating an operation of a predetermined part in a user's body. A display part 562 causes a display 61 to display a video of the virtual space. A reproduction part 563 reproduces a first sound associated with the object when parameters correlated to the speed of the object are a first predetermined value or more, and reproduces a second sound associated with the object when the parameters are a second predetermined value or more larger than the first predetermined value. When the second sound is reproduced, the reproduction part 563 continues reproduction of the second sound even when the parameters change from the second predetermined value to the first predetermined value.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006] ,

[0001] The present disclosure relates to a program and an acoustic control device.

Background Art

[0002] Among game programs, there are those that reproduce sound effects in a virtual space (see, for example, Patent Document 1). In the example of Patent Document 1, gunshots and the like are reproduced during a shooting game.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When various services (such as games) are provided in a virtual space, from the perspective of the user experience, it is required that voices suitable for the scene be reproduced.

[0005] An object of the present disclosure is to improve the user experience in services provided in a virtual space. [[ID=4)]

Means for Solving the Problems

[0006] A first aspect is a program that causes a computer system having one computer or a plurality of cooperating computers to function as a virtual space control unit, a display unit, and a reproduction unit, the computer system being connected with a display and a speaker, the virtual space control unit operating an object arranged in the virtual space based on an operation signal indicating an operation of a predetermined part of the user's body, The display unit displays the image of the virtual space on the display. The playback unit plays sound to the speaker. The playback unit plays a first sound associated with the object if the parameter correlated with the speed of the object is equal to or greater than a first predetermined value, and plays a second sound associated with the object if the parameter is equal to or greater than a second predetermined value which is greater than the first predetermined value. When the second audio is being played, the playback unit continues to play the second audio even if the parameter changes from the second predetermined value to the first predetermined value. It is a program.

[0007] In the first embodiment, The playback unit may, when the parameter is equal to or greater than a third predetermined value, play back the sound associated with the object according to the speed of the object, while when the parameter is less than the third predetermined value, it may play back the sound associated with the object according to the amount of movement of the object.

[0008] In the above embodiment, The aforementioned object may be an object in which a part of its position is fixed within the virtual space.

[0009] A second embodiment includes a storage unit that stores the program of the above embodiment, A control unit that executes the aforementioned program, This is an acoustic control device equipped with [a specific feature / feature]. [Effects of the Invention]

[0010] This disclosure can improve the user experience. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the configuration of a game device. [Figure 2]It is a perspective view showing an example of a VR goggle. [Figure 3] It is a schematic diagram of a state where a user wears a VR goggle. [Figure 4] It is an example of a lever. [Figure 5] It is a flowchart for explaining the audio reproduction process according to the first embodiment. [Figure 6] It is a flowchart for explaining the audio reproduction process according to the second embodiment.

Mode for Carrying Out the Invention

[0012] [First Embodiment] Hereinafter, an example in which a program is implemented as a game program and an acoustic control device is realized as a game device will be described. The game device is an example of a computer system having one computer or a plurality of computers working together.

[0013] The game by this game program is implemented in a virtual space (three-dimensional). This game program is implemented using the technology of Virtual Reality (abbreviation: VR). The VR technology is a technology that creates a virtual space similar to reality on a computer and allows a user to experience the feeling of being in that virtual space.

[0014] There is no limitation on the type of game or the like. In the present embodiment, as an example of the game, a battle-type action game in which a player character attacks and defeats an enemy character will be described.

[0015] In this game program, a character such as a player character may hold a virtual medium within the game screen. Here, "holding" means, for example, that the character and the virtual medium are integrally displayed on the screen. Depending on the game, "holding" may be expressed as "equipping".

[0016] As an example of the virtual medium possessed by a character, a weapon can be cited. In the following, it is assumed that the player character possesses a firearm (e.g., a handgun) as a virtual medium. The player character can attack an enemy character with that firearm.

[0017] In this game, various sounds are reproduced as the sounds emitted by virtual sound sources in the virtual space. The virtual medium possessed by a character can be a virtual sound source in the virtual space. That is, in this game, a sound is reproduced as the sound emitted by the virtual medium. For example, a firearm (virtual medium) may emit a firing sound (specifically, when a sound simulating the firing sound is reproduced).

[0018] 《Configuration of the Game Device》 〈Hardware Configuration〉 FIG. 1 is a block diagram showing the configuration of a game device 5 (an example of an acoustic control device). The game device 5 executes a predetermined game based on the operation of the user. A display 61, a speaker 62, and a controller 63 are connected to the game device 5.

[0019] As the game device 5, for example, commercially available devices such as a personal computer, PlayStation (registered trademark), XBox (registered trademark), PlayStation Vita (registered trademark), Nintendo Switch (registered trademark), etc. can be used.

[0020] In the game device 5, the game progresses based on the installed game program and game data. The game devices 5 can perform data communication with each other using a communication network (not shown) or a short-range wireless communication device (not shown).

[0021] The game device 5 has a network interface 51, a graphic processing unit 52, an audio processing unit 53, an operation unit 54, a storage unit 55, and a control unit 56. The network interface 51, the graphic processing unit 52, the audio processing unit 53, the operation unit 54, and the storage unit 55 are electrically connected to the control unit 56 via a bus 59.

[0022] The network interface 51 is connected to the communication network in a communicative manner, for example, to send and receive various data with other game devices 5 or external server devices (not shown).

[0023] The graphics processing unit 52 renders game images, including player characters and various objects in the virtual space, in video format, according to the game image information output from the control unit 56.

[0024] As previously mentioned, this game program utilizes VR technology. The graphics processing unit 52 generates images of a virtual space based on VR technology. Hereafter, images displaying a virtual space based on VR technology will be referred to as "VR images".

[0025] The graphics processing unit 52 is connected to the display 61. The VR image is displayed on the display 61.

[0026] The display 61 is composed of, for example, a liquid crystal display. The display 61 is housed in a casing that can be worn by a user (hereinafter referred to as user P). The display 61 in this embodiment is a display device for a head-mounted display (hereinafter referred to as VR goggles 70).

[0027] Figure 2 is a perspective view showing an example of VR goggles 70. As shown in Figure 2, the VR goggles 70 comprises a main body 71 and a headband 72.

[0028] The headband 72 is for attaching the VR goggles 70 to the head of user P (the game player). Figure 3 is a schematic diagram of user P wearing the VR goggles 70. The VR goggles 70 are attached to user P's head with the main body 71 covering both of user P's eyes.

[0029] The arrow shown in Figure 2 (hereinafter referred to as vector Vd) indicates the direction that is in front of user P when wearing the VR goggles 70. In this game program, vector Vd is the normal to the display surface of the display 61.

[0030] The main unit 71 has a display 61 fixed inside. In the VR goggles 70, when the VR goggles 70 are worn on the user P's head, the display surface of the display 61 faces both of the user P's eyes.

[0031] The VR goggles 70 are equipped with an accelerometer and a gyroscope (not shown in the diagram). Some versions of the VR goggles 70 may also be equipped with a geomagnetic sensor (not shown in the diagram). Some versions of the VR goggles 70 may also be equipped with a camera.

[0032] The output of the sensors in the VR goggles 70 is output to the control unit 54. By analyzing the output of the sensors in the VR goggles 70, information about the position and orientation of the user P's head (in other words, the position and orientation of the display 61) can be obtained.

[0033] The audio processing unit 53 is connected to the speaker 62. The audio processing unit 53 plays digital game audio according to the instructions of the control unit 56. Specifically, the audio processing unit 53 converts the audio data output by the control unit 56 into an analog signal and outputs it to the speaker 62.

[0034] The audio processing unit 53 is configured to provide multi-channel audio output in order to reproduce three-dimensional sound. Accordingly, multiple speakers 62 are connected to the game device 5. These speakers 62 may be positioned to the left, right, front, back, and above the listener (e.g., the game player) to enable three-dimensional sound reproduction.

[0035] For example, speaker 62 includes a front left speaker (L speaker), a front right speaker (R speaker), a front center speaker (C speaker), a subwoofer (LFE speaker), a left surround speaker (SL speaker), and a right surround speaker (SR speaker).

[0036] The control unit 54 is connected to the controller 63. The control unit 54 sends and receives signals to and from the controller 63. The game player inputs signals (commands, data, etc.) to the game device 5 by operating various controls such as buttons on the controller 63.

[0037] In this embodiment, two controllers 63 are provided. One controller 63 is held in the user P's right hand. The other controller 63 is held in the user P's left hand. When user P moves their arm or hand, the position of the held controller 63 changes. Depending on the form of the controller 63, user P may also wear the controller 63.

[0038] The controller 63 is equipped with multiple controls (e.g., buttons). User P can input operation signals to the control unit 54 by operating various controls such as buttons on the controller 63. By inputting operation signals to the game device 5, User P can give various instructions within the game. The control unit 54 sends the operation signals to the control unit 56.

[0039] The controller 63 is equipped with a sensor S that outputs an operation signal. Examples of sensors S include an accelerometer, a gyroscope, and a geomagnetic sensor. When the user P moves their arm or hand, the output (operation signal) of the sensor S changes. The operation unit 54 sends the operation signal to the control unit 56. The sensor S outputs an operation signal at predetermined intervals. Each time the operation unit 54 receives an operation signal from the sensor S, it transmits the operation signal to the control unit 56.

[0040] As described above, since the controller 63 is held in the user P's right (left) hand, the controller 63 will move in conjunction with the user P's hand (arm). Therefore, the motion signal output by the sensor S will indicate the movement of the user P's hand (arm). In other words, by analyzing the motion signal, the movement of the user P's hand (arm) can be analyzed.

[0041] In this game program, the player character C1 can be controlled within the virtual space by changing the position and orientation of controller 63.

[0042] For example, if user P holds controller 63 in their right hand and changes the position or orientation of their right hand (right arm), the right hand (right arm) of player character C1 in the virtual space will move in conjunction with the movement of user P's right hand (right arm). Similarly, if user P holds controller 63 in their left hand and changes the position or orientation of their left hand (left arm), the left hand (left arm) of player character C1 in the virtual space will move in conjunction with the movement of user P's left hand (left arm).

[0043] In other words, in this game program, when user P changes the position or orientation of controller 63, the hand (arm) of player character C1, which corresponds to the hand (arm) on the side where user P is holding controller 63, moves within the virtual space.

[0044] In this game program, the motion signals include information indicating the position and orientation of controller 63, as well as the speed and angular velocity (parameters) of controller 63. In this game program, the speed and angular velocity of player character C1's hand (arm) in the virtual space are determined according to the speed and angular velocity of user P's hand (arm).

[0045] The memory unit 55 consists of an HDD, SSD, RAM, ROM, etc. The memory unit 55 stores various programs, including game data and game programs. Examples of game data include virtual media and user account information.

[0046] The control unit 56 controls the operation of the game device 5. The control unit 56 includes a CPU (microcomputer) and semiconductor memory. The semiconductor memory stores programs and data for operating the CPU.

[0047] <Functional configuration of the control unit 56> The control unit 56 functions as a game progress unit 561 (virtual space control unit), a display unit 562, and a playback unit 563 by executing a game program (see Figure 1).

[0048] (Game progress section 561) The game progression unit 561 advances the game by making player characters, non-player characters, and predetermined objects move within the virtual space. The game progression unit 561 uses, for example, AI (artificial intelligence) to control the movements of non-player characters.

[0049] The game progress unit 561 controls the movements of player characters and other elements according to the instructions of user P (player). User P's instructions are input to the control unit 54 via the controller 63.

[0050] For example, user P can instruct the game progress unit 561 to move the player character or attack enemies (such as firing a gun) by operating the controller 63. For instance, when user P operates a predetermined button on the controller 63, the controller 63 outputs an operation signal to the control unit 54 indicating that the trigger of a firearm held by the player character has been pulled.

[0051] Furthermore, when the game progress unit 561 receives an action signal from the controller 63 (operation unit 54), it acquires various information (position, orientation, speed, angular velocity, etc.) contained in the action signal. The game progress unit 561 converts the acquired information into action information (information regarding position, orientation, speed, angular velocity, etc.) of the player character C1's hand (arm) using a predetermined method (for example, by referring to a conversion table).

[0052] The game progress unit 561 moves the hand (arm) of player character C1 in the virtual space based on the converted motion information.

[0053] The game progress unit 561 operates the object in accordance with the action information if the player character C1 is holding an object in their hand (arm). For example, when the game progress unit 561 receives an operation signal from the operation unit 54 indicating that a predetermined operation (such as equipping an object) has been performed, it causes the player character C1 to hold the object in their hand (arm).

[0054] When the game progress unit 561 receives an action signal indicating that player character C1 is holding an object in their hand (arm), it moves the object in accordance with the movement of that hand (arm). In other words, when player character C1 is holding an object in their hand (arm), player character C1's hand (arm) and the object move in conjunction.

[0055] (Display section 562) The display unit 562 generates a VR image Sc by referring to information about the virtual space generated by the game progress unit 561. The display unit 562 outputs the generated VR image Sc to the graphics processing unit 52.

[0056] (Playback section 563) The playback unit 563 plays the audio necessary for providing the service (in this embodiment, providing the game). Specifically, the playback unit 563 selects the audio data to be played and instructs the audio processing unit 53 to play that audio data. The playback unit 563 also instructs the audio processing unit 53 on the volume of each speaker 62.

[0057] The playback unit 563 plays sounds from virtual sound sources that exist in the virtual space according to the progress of the game. In this game program, various sound data are pre-prepared for sound playback. For example, in this game program, sound data such as the sound of gunfire, the sound of bullets hitting, and the sound of levers moving are provided.

[0058] In this game program, audio data is associated with each object. For example, lever C2 (stick C22) is associated with the first, second, and third audio data described below.

[0059] In this game program, objects are placed where parts of the object move in response to the actions of player P. For example, in the lever C2 shown in Figure 4, the main body C21 is fixed in position in the virtual space. In addition, the stick C22 moves relative to the main body C21 of the lever C2.

[0060] Specifically, in the virtual space, when player character C1 is holding stick C22 of lever C2 and user P moves controller 63, stick C22 moves in the direction of the arrow in Figure 4. In other words, by moving controller 63 while player character C1 is holding stick C22 of lever C2, user P can operate (tilt) stick C22.

[0061] In games, manipulating a joystick can sometimes activate various mechanisms in a virtual space. For example, tilting the joystick in a virtual area might open a closed door.

[0062] This game program performs audio playback processing to give the user a sense of the controls when operating lever C2 and stick C22.

[0063] Example of operation Figure 5 is a flowchart illustrating the audio playback process in this game program.

[0064] First, the playback unit 563 determines whether or not the player character C1 is gripping the stick C22 of the lever C2 (step S1).

[0065] For example, when the game progress unit 561 receives an operation signal from the operation unit 54 indicating that a predetermined operation has been performed, it causes the player character C1 to grasp the stick C22 of the lever C2 in the virtual space. The playback unit 563 determines that the player character C1 is grasping the stick C22 in the virtual space if the player character C1 is grasping the stick C22.

[0066] The playback unit 563 waits in step S1 (No. of step S1) until it determines that the player character C1 is gripping the stick C22.

[0067] If the playback unit 563 determines that the player character C1 is gripping the stick C22 (Yes in step S1), it determines whether the speed of the stick C22 is equal to or greater than the first speed (first predetermined value) (step S2). Here, since the player character C1 is gripping the stick C22, the hand (arm) of the player character C1 and the stick C22 operate in conjunction. For this reason, the speed of the stick C22 is determined according to the operation signal output from the controller 63.

[0068] If the playback unit 563 determines that the speed of stick C22 is less than the first speed (No. in step S2), it waits in step S2 until it determines that the speed of stick C22 is equal to or greater than the first speed.

[0069] If the playback unit 563 determines that the speed of stick C22 is equal to or greater than the first speed (Yes in step S2), it determines whether the speed of stick C22 is equal to or greater than the second speed (second predetermined value) (step S3). The second speed is a speed that is faster than the first speed.

[0070] If the playback unit 563 determines that the speed of the stick C22 is less than the second speed (No in step S3), it plays the first sound (step S4). The first sound is, for example, a sound emitted when the lever stick is moved at a low speed. The playback unit 563 continues to play the first sound until it determines that the speed of the stick C22 is less than the second speed.

[0071] If the playback unit 563 determines that the speed of the stick C22 is equal to or greater than the second speed (Yes in step S3), it plays the second sound (step S5). The second sound is, for example, a sound emitted when the lever stick is moved at high speed. The first and second sounds may be different sounds, or they may be the same sound with a low-pass filter of a predetermined cutoff frequency applied.

[0072] The playback unit 563 determines whether the speed of the stick C22 is equal to or greater than the first speed (step S6). If the playback unit 563 determines that the speed of the stick C22 is equal to or greater than the first speed (Yes in step S6), it continues to play the second audio (step S7). If the playback unit 563 determines that the speed of the stick C22 is less than the first speed (No in step S6), it stops playing the second audio (step S8).

[0073] In summary, the present embodiment is a program that causes a game device 5 (a computer system having one computer or multiple cooperating computers) to function as a game progress unit 561 (virtual space control unit), a display unit 562, and a playback unit 563. The game device 5 is connected to a display 61 and a speaker 62. The game progress unit 561 operates a stick C22 (object) placed in the virtual space based on an operation signal indicating the movement of the user P's arm (a predetermined part of the body). The display unit 562 displays the image of the virtual space on the display 61. The playback unit 563 displays the audio and plays it back to the speaker 62. The playback unit plays a first audio associated with the lever C2 (stick C22) if the speed (parameter) of the stick C22 is equal to or greater than the first speed (first predetermined value). The playback unit plays a second audio associated with the lever C2 (stick C22) if the speed of the stick C22 is faster (greater) than the first speed (second predetermined value). If the playback unit 563 is playing the second audio, it continues to play the second audio even if the speed of the stick C22 changes from the second speed to the first speed.

[0074] Effects of the First Embodiment As described above, in this embodiment, once the speed of the stick C22 reaches the second speed or higher and the second sound is played, the playback of the second sound continues even if the speed of the stick C22 then becomes the first speed, which is slower than the second speed. This allows the user P to feel the operation (responsiveness) of the stick C22 rather than changing the sound when the stick C22 is operated quickly and then slowly. Therefore, the user experience can be improved.

[0075] [Second Embodiment] In the second embodiment, even when the stick C22 is being operated at a low speed, audio playback processing is performed to give the user a sense of operation when operating the stick C22 of the lever C2.

[0076] Example of operation Figure 6 is a flowchart illustrating the audio playback process in this game program.

[0077] First, the playback unit 563 determines whether or not the player character C1 is gripping the stick C22 of the lever C2 (step S1). The playback unit 563 waits in step S1 until it determines that the player character C1 is gripping the stick C22 (step S1 No).

[0078] If the playback unit 563 determines that the player character C1 is gripping the stick C22 (Yes in step S1), it determines whether the speed of the stick C22 is greater than or equal to the third speed (third predetermined value) (step S11). The third speed is the speed at which the sensor S cannot detect the speed of the user P's hand (arm) when the user P moves the controller 63 slowly (at a low speed).

[0079] If the playback unit 563 determines that the speed of the stick C22 is equal to or greater than the third speed (Yes in step S11), it plays a third sound according to the speed of the stick C22 (step S12). The third sound is, for example, a sound emitted when the lever stick is moved. In step S12, a third sound that has been processed according to the speed of the stick C22 is played.

[0080] If the playback unit 563 determines that the speed of the stick C22 is less than the third speed (No. in step S11), it detects the amount of movement of the stick C22 (step S13). Then, the playback unit 563 plays the third sound according to the amount of movement of the stick C22 (step S14). In step S13, the third sound, which has been processed according to the amount of movement of the stick C22, is played.

[0081] To summarize, in this embodiment, the playback unit 563 plays a third sound associated with lever C2 (stick C22) according to the speed of stick C22 (object) if the speed (parameter) of stick C22 is equal to or greater than the third speed (third predetermined value), while playing a third sound associated with lever C2 (stick C22) according to the amount of movement of stick C22 if the speed of stick C22 is less than the third speed.

[0082] Effects of the Second Embodiment As described above, in this embodiment, if the speed of the stick C22 is low, it may not be detected by the sensor S. In this case, a third sound is played according to the amount of movement of the stick C22. This allows the user P to feel the operation (feel) of the stick C22 even when it is operated at a low speed. Therefore, the user experience can be improved.

[0083] [Other embodiments] The game may be a first-person game or a third-person game. If it is a first-person game, the display unit 562 does not need to display the player character C1 on the display 61.

[0084] The services offered in virtual space are not limited to games. Various services can be provided in virtual space, such as platforms for commercial transactions and communication. In other words, this program can be used to provide services in what is known as the metaverse.

[0085] The body part used to detect user P's movement is not limited to the "hands," but could also be the "legs" or "head," etc. In this case, the sensor S should be attached to the body part used to detect user P's movement.

[0086] There are no limitations on the type of sensor used to detect user P's movement. Furthermore, position and orientation may be detected using means other than sensors (e.g., a camera).

[0087] In the embodiments described above, the example was given where the parameter is the velocity of stick C22, but it is not limited to this. The parameter can be any parameter that correlates with the velocity of stick C22, such as the acceleration, position, angular velocity, angular acceleration, or angle of stick C22.

[0088] In the embodiments described above, the example of the object being lever C2 (stick C22) was used, but the invention is not limited to this. The object can be any movable object in the virtual space (for example, a vehicle).

[0089] In each of the above embodiments, the playback unit 563 may change the sound to be played depending on the type of object. In this case, it is sufficient that the sound to be played is associated with the type of object in the storage unit 55.

[0090] In the above embodiment, the playback unit 563 may change the first sound, second sound, and third sound according to the speed of the stick C22.

[0091] The program may be implemented to be executed by multiple cooperating computers (for example, a server computer and a client computer). For example, if the game device 5 operates as a client connected to a server, the processing that was performed by the control unit 56 may be performed by the server, or the processing may be divided between the server and the client.

[0092] Audio data can be prepared in advance or synthesized in real time.

[0093] The effects and advantages of this embodiment will also be exhibited when these other embodiments are adopted. Furthermore, it is possible to combine this embodiment with other embodiments and with other embodiments as appropriate. [Explanation of Symbols]

[0094] 55 Storage section 56 Control Unit (Computer) 61 displays 63 Controllers 561 Game Management Unit (Virtual Space Control Unit) 562 Display section 563 Playback Department S sensor P User C1 Player Character C2 Lever C21 Main Unit C22 Stick (Object)

Claims

1. In a program that causes a computer system having one computer or multiple cooperating computers to function as a virtual space control unit, display unit, and playback unit, The aforementioned computer system has a display and speakers connected to it. The virtual space control unit operates objects placed in the virtual space based on motion signals indicating the movement of a predetermined part of the user's body. The display unit displays the image of the virtual space on the display. The playback unit plays sound to the speaker. The playback unit plays a first sound associated with the object if the parameter correlated with the speed of the object is equal to or greater than a first predetermined value, and plays a second sound associated with the object if the parameter is equal to or greater than a second predetermined value which is greater than the first predetermined value. When the second audio is being played, the playback unit continues playing the second audio if the parameter is equal to or greater than the first predetermined value. program.

2. In the program of claim 1, The playback unit plays the sound associated with the object according to the speed of the object when the parameter is equal to or greater than a third predetermined value, while playing the sound associated with the object according to the amount of movement of the object when the parameter is less than the third predetermined value. program.

3. In the program of claim 1, The aforementioned object is an object whose position is fixed in part within the virtual space. program.

4. A storage unit that stores any one of the programs from claim 1 to 3, The system comprises a control unit that executes the aforementioned program. Acoustic control device.

Images