Program and sound control device

Abstract

To improve user experience.SOLUTION: A sound control device includes: a virtual-space control section that controls a virtual space such that a virtual portion moves in the virtual space in response to a movement of a predetermined portion of a user; an image reproducing section that allows a display to reproduce an image in the virtual space; a sound generating section that, if a condition for reproducing effect sound responsive to a predetermined movement of the virtual portion is established, generates effect sound in response to a moving speed of the predetermined portion; and a sound reproducing section that allows a speaker to reproduce the effect sound generated by the sound generating section.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a program and an audio control device. [Background technology]

[0002] Among game programs, there are those that reproduce sound effects in a virtual space. In the example of Patent Document 1, gunshot sounds and the like are reproduced in a shooting game. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-094160 A Summary of the Invention [Problem to be solved by the invention]

[0004] When various services (eg, games) are provided in a virtual space, it is desirable to reproduce sounds appropriate for the scene from the viewpoint of user experience.

[0005] An object of the present disclosure is to improve the user experience in services provided in virtual spaces. [Means for solving the problem]

[0006] A first aspect is a program for causing a computer system having one computer or a plurality of computers working together to function as a virtual space control unit, an image playback unit, an audio generation unit, and an audio playback unit, The computer system is used with a display, a speaker, and a sensor; The sensor outputs a signal indicative of a movement of a predetermined part of the user's body; the virtual space control unit controls the virtual space such that a virtual part moves within the virtual space in response to the movement of the predetermined part indicated by the signal; The image playback unit plays an image of the virtual space on the display, the sound generation unit generates a sound effect according to a moving speed of the virtual body part when a condition for playing the sound effect according to a predetermined movement of the virtual body part is satisfied; The sound reproducing unit reproduces the sound effect generated by the sound generating unit through the speaker. It is a program.

[0007] In a first aspect, The computer system may be used in conjunction with a controller that is worn or held by the user and operated, The sensor may be included in the controller; The virtual space control unit may control the virtual space based on a signal output from the sensor and an operation input to the controller.

[0008] In a first aspect, The predetermined part may be a hand, The virtual part may be a virtual hand.

[0009] In a first aspect, A virtual object that can be held by the virtual hand may be placed in the virtual space, The virtual space control unit may control the virtual space such that, when a gripping condition is satisfied, the gripping operation is performed in which the virtual hand grips the virtual object in the virtual space. When the gripping condition is satisfied, the sound generation unit may generate a gripping sound, which is a sound effect corresponding to the gripping action, depending on the movement speed of the specified part during a gripping start period starting from the time when the gripping condition is satisfied.

[0010] In a first aspect, The grasping condition may be a condition that an operation for performing the grasping action is input to the controller with the virtual hand entering a predetermined range based on a position of the virtual object in the virtual space.

[0011] In a first aspect, The virtual object may be portable in a state where it is held by the virtual hand. The virtual space control unit may control the virtual space such that, when a carrying condition is satisfied after the grasping condition is satisfied in the virtual space, the carrying action is performed in which the virtual hand grasps and carries the virtual object, and When the carrying condition is met after the holding condition is met, the sound generation unit may generate a carrying sound, which is a sound effect corresponding to the carrying action, depending on the movement speed of the specified part during a carrying start period from the time when the holding condition is met to the time when the carrying condition is met.

[0012] In a first aspect, The carrying condition may be a condition that, while the virtual hand is continuing to grip the virtual object in the virtual space, a movement distance of the specified part after the gripping condition is satisfied reaches a specified distance.

[0013] The second aspect is A storage unit that stores a program according to the first aspect; A control unit that executes the program. It is an audio control device. Effect of the Invention

[0014] According to the present disclosure, it is possible to improve the user experience. [Brief description of the drawings]

[0015] [Figure 1]1 is a block diagram illustrating a configuration of a game device according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating an example of the appearance of a head mounted display. [Diagram 3] 1 is a schematic diagram illustrating a player wearing a head mounted display and a controller. FIG. [Figure 4] FIG. 11 is a schematic diagram for explaining a gripping operation. [Diagram 5] 11 is a flowchart illustrating a flow of sound processing in response to a gripping motion. [Figure 6] FIG. 11 is a schematic diagram for explaining a carrying operation. [Figure 7] 11 is a flowchart illustrating a flow of sound processing in response to a carrying operation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, the embodiment will be described in detail with reference to the drawings. In addition, the same or corresponding parts in the drawings are given the same reference numerals, and the description thereof will not be repeated. In addition, the following will describe an example in which the program is implemented as a game program, and the sound control device is implemented as a game device. The game device is an example of a computer system having one computer or multiple computers working together.

[0017] (Game Description) In the following description, a game refers to a game (video game) that is played in a virtual space. The virtual space is a three-dimensional game space, and is displayed as an image on a display. Objects are placed in the virtual space. Examples of objects include a player character operated by a player (a user who plays the game) and a non-player character operated by a computer.

[0018] The objects also include sound source objects that serve as virtual sound sources in the virtual space. Examples of sound source objects include objects that operate in the virtual space and objects that only output sound in the virtual space. Player characters and non-player characters can be sound source objects. Sound is played in the games described below. Sounds played in the games include sounds output from sound source objects, background music (BGM), sound effects, and the like.

[0019] In the game described below, the player character is a virtual player (virtual human) located in a virtual space, and moves in the virtual space according to the operation and movement of the player in the real world. Specifically, a virtual part moves in the virtual space according to the movement of a predetermined part of the player's body. The virtual part is a virtual part (a predetermined part of the player character's body) that corresponds to a predetermined part of the player. For example, the virtual hand of the player character (virtual hand) moves in the virtual space according to the movement of the player's hand in the real world.

[0020] In the following description, virtual objects that can be held by a virtual hand (the hand of the player character) are placed in the virtual space. The virtual objects can be carried while being held by the virtual hand. An example of a virtual object is a weapon equipped to the player character. Examples of weapons include a firearm such as a handgun, and a knife. The player character performs various actions such as a holding action and a carrying action. The holding action is an action in which the virtual hand holds a virtual object in the virtual space. The carrying action is an action in which the virtual hand holds and carries a virtual object.

[0021] For ease of explanation, the following will take as an example of a game a "competitive action game in which a player character attacks and defeats enemy characters." There is no limitation to the type of game. Also, as an example of a virtual space, a "virtual reality space in which virtual reality (VR) can be experienced" will be taken. Also, an example will be taken of a case in which an image of the virtual space is displayed on a display from a first-person perspective and the game progresses.

[0022] In the following, an example will be given in which the predetermined part of the user's body is the "user's right hand," the virtual part corresponding to the predetermined part is the "virtual right hand," and the virtual object that can be held by the virtual hand is a "handgun." The player character can equip the handgun in the virtual right hand and attack enemy characters with the handgun.

[0023] (Gaming device) 1 illustrates a configuration of a game device 10 according to an embodiment. The game device 10 executes a game in response to an operation by a user. In this example, the game device 10 is used together with a head mounted display 20, a speaker 30, and a controller 40, and is connected to these devices by wire or wirelessly. The game device 10 is an example of an audio control device.

[0024] The game device 10 may be, for example, a personal computer, PlayStation (registered trademark), XBox (registered trademark), PlayStation Vita (registered trademark), Nintendo Switch (registered trademark), or other commercially available device.

[0025] A game progresses on the basis of the installed game program and game data in the game device 10. The game devices 10 can communicate data with each other using a communication network (not shown) or a short-range wireless communication device (not shown).

[0026] [Head-mounted display] The head mounted display 20 includes a display 21 and a motion sensor 22 .

[0027] The display 21 reproduces (displays) images. For example, the display 21 is a liquid crystal display.

[0028] The motion sensor 22 outputs a motion signal indicating the movement of the player's head. For example, the motion sensor 22 includes an acceleration sensor that detects acceleration, a gyro sensor that detects angular velocity (changes in rotation and orientation), etc. The motion signal output from the motion sensor 22 is transmitted to the game device 10.

[0029] As shown in Fig. 2, the head mounted display 20 has a main body 25 and a headband 26. As shown in Fig. 3, the head mounted display 20 is worn on the head of the player 50 with the main body 25 covering both eyes of the player 50. A display 21 is fixed to the inside of the main body 25 (the side facing the both eyes of the player 50). The display surface of the display 21 faces both eyes of the player 50 when the head mounted display 20 is worn on the head of the player 50.

[0030] The head mounted display 20 attached to the head of the player 50 moves in conjunction with the movement of the head of the player 50. Therefore, based on the movement signal output from the movement sensor 22 included in the head mounted display 20, it is possible to derive "the position and orientation of the head of the player 50" and "the position and orientation of the display 21 included in the head mounted display 20."

[0031] Moreover, the vector Vd in the head mounted display 20 is a reference line of the image of the virtual space represented from the first person viewpoint. The vector Vd extends in the normal direction of the display surface of the display 21. The vector Vd can be derived from the position and orientation of the display 21 included in the head mounted display 20.

[0032] [Speaker] The speaker 30 reproduces sound (game sound in this example). In this example, a plurality of speakers 30 are provided so that three-dimensional sound can be reproduced.

[0033] 〔controller〕 The controller 40 is operated while being held or worn by the player 50. In this example, the controller 40 is operated while being held in the hand of the player 50. As shown in FIG.

[0034] The operator 41 is composed of various buttons and the like. The controller 50 outputs an operation signal corresponding to an operation (e.g., pressing a button) input to the operator 41 to the game device 10. With this configuration, the player 50 operates the operator 41 of the controller 40 to input an operation signal (a signal indicating a command, data, etc.) to the game device 10.

[0035] The motion sensor 42 outputs a motion signal indicating the movement of the player's hand. For example, the motion sensor 22 includes an acceleration sensor that detects acceleration, a gyro sensor that detects angular velocity (changes in rotation and orientation), etc. The motion signal output from the motion sensor 42 is transmitted to the game device 10.

[0036] The controller 40 held (or worn) in the hand of the player 50 moves in conjunction with the movement of the hand of the player 50. Therefore, the position and orientation of the hand of the player 50 can be derived based on the movement signal output from the movement sensor 42 included in the controller 40. The movement sensor 42 is an example of a sensor that outputs a signal indicating the movement of a predetermined part of the user's body.

[0037] 3, in this example, a first controller 40a and a second controller 40b are provided as the controller 40. The configuration of each of the first controller 40a and the second controller 40b is similar to that of the controller 40.

[0038] The first controller 40a is held and operated by the right hand 55 of the player 50, and moves in conjunction with the movement of the right hand 55 of the player 50. The second controller 40b is held and operated by the left hand 56 of the player 50, and moves in conjunction with the movement of the left hand 56 of the player 50. Therefore, the position and orientation of the right hand 55 of the player 50 can be derived based on the movement signal output from the movement sensor 42 included in the first controller 40a. Also, the position and orientation of the left hand 56 of the player 50 can be derived based on the movement signal output from the movement sensor 42 included in the second controller 40b.

[0039] [Hardware configuration of the game device] 1, the game device 10 includes a network interface 11, a graphics processing unit 12, an audio processing unit 13, an operation processing unit 14, a storage unit 15, and a control unit 16. The network interface 11, the graphics processing unit 12, the audio processing unit 13, the operation processing unit 14, and the storage unit 15 are electrically connected to the control unit 16 via a bus 17.

[0040] The network interface 11 is communicatively connected to a communication network (not shown) for transmitting and receiving various data to and from, for example, other game devices 10 and external server devices (not shown).

[0041] The graphics processing unit 12 is connected to the display 21 of the head mounted display 20 by wire or wirelessly. The graphics processing unit 12 renders game images including various objects in a virtual space in a moving image format according to game image information (image data) output from the control unit 16. The game images (images of the virtual space) rendered in a moving image format are displayed on the display 21 as a game screen.

[0042] The audio processing unit 13 is connected to the speaker 30 by wire or wirelessly. The audio processing unit 13 reproduces digital game sounds in accordance with instructions from the control unit 16. Specifically, the audio processing unit 13 converts the audio data output by the control unit 16 into an analog signal and outputs it to the speaker 30.

[0043] In this example, the audio processor 13 is configured to be capable of multi-channel audio output in order to reproduce three-dimensional sound. A plurality of speakers 30 are connected to the game device 10. The plurality of speakers 30 are arranged to the left and right, front and rear, above, etc. of a listener (player) in order to reproduce three-dimensional sound.

[0044] The operation processing unit 14 is connected to each of the head mounted display 20 and the controller 40 by wire or wirelessly. The operation processing unit 14 transmits and receives signals between each of the head mounted display 20 and the controller 40. In this example, the operation processing unit 14 transmits to the control unit 16 a motion signal output from the motion sensor 22 of the head mounted display 20, a motion signal transmitted from the controller 40 (a signal corresponding to an operation input to the operator 41), and a motion signal output from the motion sensor 42 of the controller 40.

[0045] The storage unit 15 stores various information and data. The storage unit 15 is composed of an HDD, an SSD, a RAM, a ROM, etc. For example, the storage unit 15 stores game data, various programs including game programs, etc. Examples of game data include game media, user account information, etc.

[0046] The storage unit 15 also stores various types of audio data indicating various types of sounds prepared in advance. The various types of audio data include audio data indicating sounds output from sound source objects, audio data indicating music (BGM) played in a virtual space, audio data indicating sound effects, and the like.

[0047] The control unit 16 controls the operation of the game device 10. The control unit 16 transmits and receives various information and data, and processes various information and data. The control unit 16 has a CPU (microcomputer) and a semiconductor memory. The semiconductor memory stores programs and data for operating the CPU.

[0048] [Functional configuration of the control unit] The control unit 16 of the game device 10 has a game progression unit 100, an image playback unit 101, a sound generation unit 102, and a sound playback unit 103. Specifically, the control unit 16 functions as the game progression unit 100, the image playback unit 101, the sound generation unit 102, and the sound playback unit 103 by executing a game program.

[0049] <Game Progression Section (Virtual Space Control Section)> The game progression unit 100 progresses a game that takes place in a virtual space. The game progression unit 100 is an example of a virtual space control unit that controls the virtual space.

[0050] In this example, the game progression unit 100 progresses the game that takes place in a virtual space by controlling (generating or updating) image data showing the virtual space so that the movements (movement of the head of the player 50) indicated in the movement signal output from the movement sensor 22, the movements (movement of the hands of the player 50) indicated in the movement signal output from the movement sensor 42, and the operations (operations input to the operators 41 of the controller 40) indicated in the operation signal output from the controller 40 are reflected in the virtual space.

[0051] Specifically, the game progression unit 100 controls the movement of the player character in the virtual space in accordance with the above movements and operations. The game progression unit 100 also controls the movement of other objects (such as non-player characters) in the virtual space in accordance with the progress of the game. For example, AI (artificial intelligence) is used to control the movement of other objects by the game progression unit 100.

[0052] Furthermore, as shown in Figures 4 and 6, the game progression unit 100 controls the virtual space so that a virtual part 61 (a specified part of the player character 60) moves within the virtual space in accordance with the "movement of a specified part 51 of the body of the player 50" indicated in the movement signal output from the movement sensor 42.

[0053] In this example, the game progression unit 100 controls the virtual space so that a virtual right hand 65 (the right hand of the player character 60) moves within the virtual space in accordance with the "movement of the right hand 55 of the player 50" indicated in the movement signal output from the movement sensor 42 included in the first controller 40a. The game progression unit 100 also controls the virtual space so that a virtual left hand 66 (the left hand of the player character 60) moves within the virtual space in accordance with the "movement of the left hand 56 of the player 50" indicated in the movement signal output from the movement sensor 42 included in the second controller 40b.

[0054] <Image playback section> The image playback unit 101 plays an image of the virtual space controlled by the game progression unit 100 on the display 21. In this example, the image playback unit 101 supplies image data (image data showing an image of the virtual space) controlled by the game progression unit 100 to the graphics processing unit 12. As a result, the image of the virtual space shown in the image data is played (displayed) on the display 21 of the head mounted display 20.

[0055] <Speech Generation Unit> The sound generation unit 102 generates sounds (game sounds) in the virtual space controlled by the game progression unit 100. In this example, the sound generation unit 102 generates sounds according to the situation in the virtual space (such as the movement of objects). Specifically, the sound generation unit 102 selects sound data to be played from sound data (various types of sound data prepared in advance) stored in the storage unit 15 according to the situation in the virtual space (the progress of the game in this example), and generates sound data indicating sound according to the situation in the virtual space based on the selected sound data (or new sound data obtained by synthesizing the selected multiple sound data).

[0056] In this example, the sound generation unit 102 performs a sound image localization process. The sound image localization process is a process of localizing the position of a sound source felt by a human (player) at a predetermined virtual position (for example, the position of a sound source object) in a virtual space.

[0057] In the sound image localization process, the sound generation unit 102 places a virtual microphone (not shown) in a virtual space and generates sound data including sound obtained by the virtual microphone. The virtual microphone is a virtual listener (sound collection point) and is a reference point for sound image localization in the sound image localization process. For example, the virtual microphone is placed near the player character and moves following the movement of the player character.

[0058] Specifically, in the sound image localization process, the sound generation unit 102 generates sound data according to the sound collection conditions of the virtual microphone (e.g., the position and direction of the virtual microphone relative to the sound source object in the virtual space, the volume and direction of the sound output from the sound source object, etc.) so that the sound played (output) from the speaker 30 includes sound obtained by the virtual microphone.

[0059] By processing the audio data generated by the sound image localization process by the audio processing unit 13, it becomes possible to output the sound output from a virtual sound source (sound source object) in the virtual space from the speaker 30 with an acoustic expression as if the sound was collected by a virtual microphone.

[0060] Furthermore, when a condition for playing back a sound effect corresponding to a predetermined movement of a virtual part (a predetermined part of the player character) in the virtual space is satisfied, the sound generation unit 102 generates a sound effect corresponding to the moving speed of a predetermined part of the user's body in the real world. Specifically, when the above condition is satisfied, the sound generation unit 102 generates a sound effect corresponding to the moving speed of the predetermined part within a predetermined period (a predetermined period starting or ending at the time when the above condition is satisfied) immediately before or after the above condition is satisfied. The generation of sound effects (sound effects corresponding to a predetermined movement of a virtual part) by the sound generation unit 102 will be described in detail later.

[0061] Audio playback section The audio reproducing unit 103 reproduces the audio generated by the audio generating unit 102 (audio including sound effects corresponding to a predetermined movement of a virtual body part) on the speaker 30. In this example, the audio reproducing unit 103 supplies the audio data generated by the audio generating unit 102 to the audio processing unit 13. As a result, the audio of the virtual space indicated by the audio data (audio including sound effects corresponding to a predetermined movement of a virtual body part) is reproduced (output) from the speaker 30.

[0062] [Processing related to gripping operation] Next, a process related to a gripping motion will be described with reference to Fig. 4. The gripping motion is a motion in which a virtual right hand 65 (virtual hand) grips a handgun 71 (virtual object 70) in a virtual space. The gripping motion is an example of a predetermined movement of the virtual part 61.

[0063] When a gripping condition, which is a condition for performing a gripping action, is satisfied, the game progression unit 100 controls the virtual space so that the gripping action is performed. Specifically, when the gripping condition is satisfied, the game progression unit 100 causes the virtual right hand 65 of the player character 60 to grip a handgun 71 in the virtual space.

[0064] When the gripping condition is satisfied, the sound generating unit 102 generates a gripping sound, which is a sound effect corresponding to a gripping action, according to the moving speed of the right hand 55 (predetermined part 51) of the player 50 during a gripping start period starting from the time point when the gripping condition is satisfied. The sound processing corresponding to the gripping action will be described in detail later.

[0065] The gripping condition is a condition that an operation for performing a gripping motion is input to the operator 41 of the controller 40 in a state where the virtual right hand 65 (virtual hand) enters within a predetermined range R1 based on the position Q1 of the handgun 71 (virtual object 70) in the virtual space. The gripping condition is an example of a condition for playing a sound effect corresponding to a predetermined movement of the virtual part 61.

[0066] [Gripping sound] In this example, the gripping sound is the sound (e.g., a "grab" sound) of the imaginary right hand 65 gripping the handgun 71 that is removably housed in a holster attached to the player character 60. The gripping sound is a sound effect that imitates the sound made when gripping an object (a handgun in this example) with a hand.

[0067] In the real world, the sound emitted when grasping an object with a hand changes depending on the moving speed of the hand moving toward the object to grasp the object. For example, the sound emitted when grasping an object with a hand tends to become louder and more intense as the moving speed of the hand moving toward the object increases. Also, the faster the moving speed of the hand moving toward the object increases, the faster the moving speed of the hand moving with the momentum (remaining momentum) immediately after grasping the object with that hand also increases.

[0068] The relationship between the "sound produced when an object is grasped by hand" and the "movement speed of the hand moving toward the object in order to grasp the object (movement speed in a predetermined period of time)" can be derived by experiment and simulation. Similarly, the relationship between the "sound produced when an object is grasped by hand" and the "movement speed of the hand moving with momentum immediately after grasping the object by hand (movement speed in a predetermined period of time)" can be derived by experiment and simulation.

[0069] [Sound processing according to gripping motion] Next, with reference to Fig. 5, an acoustic process (a process for generating sound effects) corresponding to a gripping action will be described. In this example, a gripping sound (a plurality of gripping sounds) corresponding to a gripping action of the virtual object 70 is prepared for each type of virtual object 70 and for each range of the moving speed of a predetermined part 51 in a gripping start period described later. Sound data (a plurality of sound data) indicating these gripping sounds is stored in the storage unit 15. Note that, in this example, the range of the moving speed of the virtual part 61 is three ranges, namely, a low-speed range (a range of moving speed higher than zero), a medium-speed range (a range of moving speed higher than the low-speed range), and a high-speed range (a range of moving speed higher than the medium-speed range).

[0070] <Step S10> The sound generation unit 102 determines whether or not the virtual part 61 is within a predetermined range R1 based on the position Q1 of the virtual object 70. If the virtual part 61 is within the predetermined range R1, the process of step S11 is performed.

[0071] 4, in this example, the predetermined range R1 is a spherical range centered on the position of the handgun 71, which is position Q1 of the virtual object 70, and is a range within which the virtual right hand 65, which is the virtual part 61, is deemed to be in a position where it can grip the handgun 71. The sound generation unit 102 acquires the position (e.g., coordinates) of the virtual right hand 65 in the virtual space, and when the position of the virtual right hand 65 is included within the predetermined range R1, it determines that the virtual right hand 65 has entered the predetermined range R1.

[0072] <Step S11> The sound generation unit 102 determines whether or not an operation for gripping the virtual object 70 has been input to the controller 40. If an operation for gripping the virtual object 70 has been input to the controller 40, the process of step S12 is performed, and if not, the process of step S10 is performed.

[0073] In this example, the operation for gripping handgun 71, which is virtual object 70, is an operation of pressing a predetermined button included in operator 41 of controller 40. Based on an operation signal (a signal corresponding to an operation input to operator 41) output from controller 40, sound generation unit 102 determines whether or not an operation for gripping handgun 71 (an operation of pressing a predetermined button) has been input to operator 41 of controller 40.

[0074] <Step S12> When an operation to grasp the virtual object 70 is input to the controller 40 while the virtual part 61 is within the predetermined range R1, the grasping condition is met. Note that the determination of whether the grasping condition is met (the processing of steps S10 and S11) may be performed by the game progression unit 100. When the grasping condition is met, the sound generation unit 102 identifies the type of virtual object 70 that is the target of the grasping operation. In this example, the sound generation unit 102 identifies the type of virtual object 70 that is the target of the grasping operation as a "handgun 71."

[0075] <Step S13> Next, the sound generation unit 102 derives the "movement speed of the predetermined part 51 of the body of the player 50 in the real world" during the grip start period starting from the time point when the grip condition is established.

[0076] In this example, the sound generation unit 102 derives the "movement speed of the right hand 55 of the player 50 in the real world" during the grip start period, based on the movement signal output from the movement sensor 42 of the first controller 40a. For example, the sound generation unit 102 derives the movement speed of the right hand 55 during the grip start period by integrating the "acceleration indicated in the movement signal output from the movement sensor 42" during the grip start period.

[0077] <Step S14> Next, the voice generation unit 102 determines whether the moving speed of the predetermined part 51 derived in step S13 (in this example, the moving speed of the right hand 55) falls within a predetermined "low speed range," "medium speed range," or "high speed range." If the moving speed of the predetermined part 51 falls within the low speed range, the process of step S15 is performed. If the moving speed of the predetermined part 51 falls within the medium speed range, the process of step S16 is performed. If the moving speed of the predetermined part 51 falls within the high speed range, the process of step S17 is performed.

[0078] <Step S15> When the moving speed of the predetermined part 51 (in this example, the moving speed of the right hand 55) is included in the low speed range, the voice generating unit 102 generates a gripping sound corresponding to the low speed. The voice reproducing unit 103 causes the speaker 30 to reproduce the "gripping sound corresponding to the low speed" generated by the voice generating unit 102.

[0079] In this example, the sound generation unit 102 selects a plurality of pieces of sound data corresponding to the type of virtual object 70 identified in step S12 (handgun 71 in this example) from among a plurality of pieces of sound data (sound data indicating gripping sounds) stored in the storage unit 15. Next, the sound generation unit 102 selects sound data indicating a "grip sound corresponding to low speed" corresponding to the low speed range from among the plurality of pieces of sound data (sound data prepared for each range of the moving speed of the virtual part 61 in the grip start period). Then, the sound reproduction unit 103 supplies the sound data indicating the "grip sound corresponding to low speed" selected by the sound generation unit 102 to the audio processing unit 13, thereby causing the speaker 30 to reproduce the grip sound corresponding to low speed.

[0080] <Step S16> When the moving speed of the predetermined part 51 (in this example, the moving speed of the right hand 55) is within the medium speed range, the voice generating unit 102 generates a gripping sound corresponding to the medium speed. The voice reproducing unit 103 causes the speaker 30 to reproduce the "gripping sound corresponding to the medium speed" generated by the voice generating unit 102. For example, the gripping sound corresponding to the medium speed is more intense and louder than the gripping sound corresponding to the low speed.

[0081] The sound generation unit 102 selects sound data indicating a "gripping sound corresponding to medium speed" that corresponds to the type of virtual object 70 (handgun 71 in this example) identified in step S12 and corresponds to the medium speed range from among a plurality of sound data (sound data indicating gripping sounds) stored in the storage unit 15. The selection of sound data indicating a "gripping sound corresponding to medium speed" by the sound generation unit 102 is similar to the selection of sound data indicating a "gripping sound corresponding to low speed" by the sound generation unit 102. The sound reproduction unit 103 supplies the sound data indicating the "gripping sound corresponding to medium speed" selected by the sound generation unit 102 to the audio processing unit 13, thereby causing the speaker 30 to reproduce the gripping sound corresponding to medium speed.

[0082] <Step S17> When the moving speed of the predetermined part 51 (in this example, the moving speed of the right hand 55) is included in the high speed range, the voice generating unit 102 generates a gripping sound corresponding to the high speed. The voice reproducing unit 103 causes the speaker 30 to reproduce the "gripping sound corresponding to the high speed" generated by the voice generating unit 102. For example, the gripping sound corresponding to the high speed is more intense and louder than the gripping sound corresponding to the medium speed.

[0083] The voice generating unit 102 selects, from among a plurality of voice data (voice data indicating gripping sounds) stored in the storage unit 15, voice data indicating a "grip sound corresponding to high speed" that corresponds to the type of virtual object 70 (handgun 71 in this example) identified in step S12 and corresponds to the high speed range. The selection of voice data indicating a "grip sound corresponding to high speed" by the voice generating unit 102 is similar to the selection of voice data indicating a "grip sound corresponding to low speed" by the voice generating unit 102. The voice reproducing unit 103 supplies the voice data indicating the "grip sound corresponding to high speed" selected by the voice generating unit 102 to the audio processing unit 13, thereby reproducing the grip sound corresponding to high speed on the speaker 30.

[0084] [Processing related to carrying operation] Next, the carrying motion will be described with reference to Fig. 6. The carrying motion is a motion in which the virtual right hand 65 (virtual hand) holds and carries a handgun 71 (virtual object 70). The carrying motion is an example of a predetermined movement of the virtual part 61.

[0085] After the gripping condition is satisfied in the virtual space, the game progression unit 100 controls the virtual space so that a carrying action is performed when a carrying condition, which is a condition for performing a carrying action, is satisfied. Specifically, when the carrying condition is satisfied, the game progression unit 100 controls the movement of virtual right hand 65 in conjunction with the movement of right hand 55 of player 50 so that virtual right hand 65 carries handgun 71 in a gripping state.

[0086] When the carrying condition is satisfied after the holding condition is satisfied, the sound generating unit 102 generates a carrying sound, which is a sound effect corresponding to the carrying action, according to the moving speed of the right hand 55 (predetermined part 51) during the carrying start period from the time when the holding condition is satisfied to the time when the carrying condition is satisfied. The sound processing corresponding to the carrying action will be described in detail later.

[0087] The carrying condition is a condition that, in a state where the virtual right hand 65 (virtual hand) continues to grip the handgun 71 (virtual object 70) in the virtual space, the moving distance of the right hand 55 (predetermined part 51) after the gripping condition is established reaches a predetermined distance. The carrying condition is an example of a condition for playing a sound effect according to a predetermined movement of the virtual part 61.

[0088] [Carrying noise] In this example, the carrying sound is the sound (e.g., a "whoosh" sound) of handgun 71 housed in a holster attached to player character 60 being removed from the holster while rubbing against the inner surface of the holster. The carrying sound is a sound effect that imitates the sound made when carrying (moving) an object (a handgun in this example) while rubbing it with the hand.

[0089] In the real world, the sound emitted when carrying an object while rubbing it with the hands changes depending on the moving speed of the hands that grasp and carry the object. For example, the sound emitted when carrying an object while rubbing it with the hands tends to become louder and more intense as the moving speed of the hands that grasp and carry the object increases. The relationship between the above-mentioned "sound emitted when carrying an object while rubbing it with the hands" and "moving speed of the hands that grasp and carry the object (moving speed in a predetermined period of time)" can be derived through experiments and simulations.

[0090] [Audio processing in response to carrying movements] Next, sound processing (processing for generating sound effects) corresponding to the carrying action will be described with reference to Fig. 7. In this example, carrying sounds (plurality of carrying sounds) corresponding to the carrying action of the virtual object 70 are prepared for each type of virtual object 70 and for each range of moving speed of a predetermined part 51 in a carrying start period described later. Sound data (plurality of sound data) indicating these carrying sounds is stored in the storage unit 15.

[0091] <Step S20> The voice generating unit 102 determines whether or not the gripping condition is satisfied. If the gripping condition is satisfied, the process of step S21 is performed.

[0092] <Step S21> The sound generation unit 102 starts measuring the moving distance of the virtual part 61 from the gripping position (the position when the gripping motion is started) of the virtual object 70. Next, the process of step S22 is performed.

[0093] In this example, the voice generation unit 102 acquires the position (e.g., coordinates) of the virtual right hand 65 when the virtual right hand 65, which is the virtual part 61, starts a gripping action to grip the handgun 71, which is the virtual object 70, and sets the position of the virtual right hand 65 as the "measurement start position of the movement distance." Then, the voice generation unit 102 tracks the position (e.g., coordinates) of the virtual right hand 65 in the virtual space, and derives the distance of the trajectory of the movement of the virtual right hand 65 as the "movement distance of the virtual right hand 65 from the grip position of the handgun 71." Alternatively, the voice generation unit 102 may derive the straight-line distance between the current position of the virtual right hand 65 in the virtual space and the measurement start position as the "movement distance of the virtual right hand 65 from the grip position of the handgun 71."

[0094] <Step S22> The sound generation unit 102 determines whether or not an operation for releasing the grip of the virtual object 70 has been input to the controller 40. If an operation for releasing the grip of the virtual object 70 has not been input to the controller 40, the process of step S23 is performed. On the other hand, if an operation for releasing the grip of the virtual object 70 has been input to the controller 40, the process ends.

[0095] In this example, the operation for releasing the grip of handgun 71, which is virtual object 70, is an operation of releasing the press of a predetermined button (a button pressed to grip handgun 71) included in operator 41 of controller 40. Sound generator 102 determines whether or not the operation for gripping handgun 71 (the operation of pressing a predetermined button) has been released, based on an operation signal (a signal corresponding to an operation input to operator 41) output from controller 40. If the operation for gripping handgun 71 is continued without being released, processing of step S23 is performed. On the other hand, if the operation for gripping handgun 71 is released, processing ends.

[0096] <Step S23> Next, the sound generation unit 102 determines whether or not the movement distance of the virtual part 61 (the virtual right hand 65 in this example) from the gripping position of the virtual object 70 (the handgun 71 in this example) has reached a predetermined distance. If the movement distance of the virtual part 61 from the gripping position of the virtual object 70 has reached the predetermined distance, the process of step S24 is performed, and if not, the process of step S22 is performed.

[0097] <Step S24> When the movement distance of the virtual part 61 from the gripping position of the virtual object 70 reaches a predetermined distance without an operation for releasing the grip of the virtual object 70 being input to the controller 40, the carrying condition is met. Note that the determination of whether or not the carrying condition is met (the processing of steps S21 to S23) may be performed by the game progression unit 100. When the gripping condition is met, the sound generation unit 102 identifies the type of virtual object 70 that is the target of the carrying action. In this example, the sound generation unit 102 identifies the type of virtual object 70 that is the target of the carrying action as a "handgun 71."

[0098] <Step S25> Next, the sound generating unit 102 derives the "movement speed of the predetermined part 51 of the body of the player 50 in the real world" during the carrying start period from the point when the gripping condition is met to the point when the carrying condition is met.

[0099] In this example, the sound generation unit 102 derives the "movement speed of the right hand 55 of the player 50 in the real world" during the start of carrying, based on the movement signal output from the movement sensor 42 of the first controller 40a. For example, the sound generation unit 102 derives the movement speed of the right hand 55 during the start of carrying, by integrating the "acceleration indicated in the movement signal output from the movement sensor 42" during the start of carrying.

[0100] <Step S26> Next, the voice generation unit 102 determines whether the moving speed of the predetermined part 51 derived in step S25 (in this example, the moving speed of the right hand 55) falls within a predetermined "low speed range", "medium speed range", or "high speed range". If the moving speed of the predetermined part 51 falls within the low speed range, the process of step S27 is performed. If the moving speed of the predetermined part 51 falls within the medium speed range, the process of step S28 is performed. If the moving speed of the predetermined part 51 falls within the high speed range, the process of step S29 is performed.

[0101] <Step S27> When the moving speed of the predetermined part 51 (in this example, the moving speed of the right hand 55) is included in the low speed range, the sound generation unit 102 generates a carrying sound corresponding to the low speed. The sound reproduction unit 103 causes the speaker 30 to reproduce the "carrying sound corresponding to the low speed" generated by the sound generation unit 102.

[0102] In this example, the sound generation unit 102 selects a plurality of pieces of sound data corresponding to the type of virtual object 70 identified in step S24 (handgun 71 in this example) from among a plurality of pieces of sound data (sound data indicating a carrying sound) stored in the storage unit 15. Next, the sound generation unit 102 selects sound data indicating a "carrying sound corresponding to a low speed" corresponding to the low speed range from among the plurality of pieces of sound data (sound data prepared for each range of the moving speed of the virtual part 61 in the carrying start period). Then, the sound reproduction unit 103 supplies the sound data indicating the "carrying sound corresponding to a low speed" selected by the sound generation unit 102 to the audio processing unit 13, thereby causing the speaker 30 to reproduce the carrying sound corresponding to the low speed.

[0103] <Step S28> When the moving speed of the predetermined part 51 (in this example, the moving speed of the right hand 55) is within the medium speed range, the sound generation unit 102 generates a carrying sound corresponding to the medium speed. The sound reproduction unit 103 causes the speaker 30 to reproduce the "carrying sound corresponding to the medium speed" generated by the sound generation unit 102. For example, the carrying sound corresponding to the medium speed is more intense and louder than the carrying sound corresponding to the low speed.

[0104] The sound generation unit 102 selects sound data indicating a "carrying sound corresponding to medium speed" that corresponds to the type of virtual object 70 (handgun 71 in this example) identified in step S24 and corresponds to the medium speed range from among a plurality of sound data (sound data indicating carrying sounds) stored in the storage unit 15. The selection of sound data indicating a "carrying sound corresponding to medium speed" by the sound generation unit 102 is similar to the selection of sound data indicating a "carrying sound corresponding to low speed" by the sound generation unit 102. The sound reproduction unit 103 supplies the sound data indicating the "carrying sound corresponding to medium speed" selected by the sound generation unit 102 to the audio processing unit 13, thereby causing the speaker 30 to reproduce the carrying sound corresponding to medium speed.

[0105] <Step S29> When the moving speed of the predetermined part 51 (in this example, the moving speed of the right hand 55) is included in the high speed range, the sound generation unit 102 generates a carrying sound corresponding to the high speed. The sound reproduction unit 103 causes the speaker 30 to reproduce the "carrying sound corresponding to the high speed" generated by the sound generation unit 102. For example, the carrying sound corresponding to the high speed is more intense and louder than the carrying sound corresponding to the medium speed.

[0106] The voice generating unit 102 selects, from among a plurality of voice data (voice data indicating carrying sounds) stored in the storage unit 15, voice data indicating a "carrying sound corresponding to high speed" that corresponds to the type of virtual object 70 (handgun 71 in this example) identified in step S24 and corresponds to the high speed range. The selection of voice data indicating a "carrying sound corresponding to high speed" by the voice generating unit 102 is similar to the selection of voice data indicating a "carrying sound corresponding to low speed" by the voice generating unit 102. The voice reproducing unit 103 supplies the voice data indicating the "carrying sound corresponding to high speed" selected by the voice generating unit 102 to the audio processing unit 13, thereby reproducing the gripping sound corresponding to high speed on the speaker 30.

[0107] [Overview of the embodiment] In summary, the program (game program) of the embodiment is a program that causes a computer system (game device 10) having one computer or multiple computers working together to function as a virtual space control unit (game progression unit 100), an image reproduction unit 101, a sound generation unit 102, and a sound reproduction unit 103. The computer system is used together with a display 21, a speaker 30, and a sensor (motion sensor 42). The sensor outputs a signal indicating the movement of a predetermined part 51 in the body of a user (player 50). The virtual space control unit controls the virtual space so that a virtual part 61 moves in the virtual space according to the movement of the predetermined part 51 indicated by the signal output from the sensor. The image reproduction unit 101 reproduces an image of the virtual space on the display 21. The sound generation unit 102 generates a sound effect according to the moving speed of the predetermined part 51 when a condition for reproducing a sound effect according to the predetermined movement of the virtual part 61 is established. The sound reproduction unit 103 reproduces the sound effect generated by the sound generation unit 102 on the speaker 30.

[0108] The acoustic control device (game device 10) of the embodiment includes a storage unit 15 that stores the above program, and a control unit 16 that executes the above program.

[0109] [Effects of the embodiment] In the game device 10 of the embodiment, when a condition for playing a sound effect corresponding to a predetermined movement of a virtual part in the virtual space (a virtual part that moves in conjunction with the movement of a predetermined part of the player's body) is met, the above sound effect is generated according to the moving speed of the predetermined part of the player's body in the real world. This can improve the user experience. For example, the reality of the sound effect can be improved, thereby improving the interest.

[0110] In addition, in the game device 10 of the embodiment, a gripping sound corresponding to a gripping action in which a virtual hand grips a virtual object in a virtual space and a carrying sound corresponding to a carrying action in which the virtual hand carries the virtual object while gripping it are generated separately. This can further improve the user experience. For example, the reality of the sound effect corresponding to a series of actions in which a gripping action is followed by a carrying action can be improved, thereby improving the interest.

[0111] (Other embodiments) In the above description, the following configuration or setting may be adopted.

[0112] The predetermined part of the player's (user's) body is not limited to the hand. For example, the predetermined part of the player's body may be the player's torso, the player's legs, or another part.

[0113] The sensor that outputs a signal indicating the movement of a predetermined part of the player's (user's) body is not limited to the motion sensor 42 of the controller 40. For example, such a sensor may be a combination of a camera (image sensor) that captures an image of a predetermined part of the player's body, and a recognition processing unit that recognizes the movement of the predetermined part based on the "image of the predetermined part" obtained by the camera.

[0114] The virtual object that can be held by the virtual hand is not limited to a firearm such as a handgun, etc. For example, such a virtual object may be another weapon such as a knife, may be armor worn by a player character, or may be another object (virtual object) other than a weapon or armor.

[0115] The voice data does not necessarily need to be prepared in advance. For example, the voice generation unit 102 may synthesize or generate voice data each time based on some data (e.g., voice data stored in the storage unit 15) or information.

[0116] The speaker 30 may be a stationary speaker, or a wearable speaker such as an earphone or a headphone.

[0117] The player character may be displayed or not displayed on the display. In other words, the player character may be included or not included in an image of the virtual space displayed (reproduced) on the display. For example, when an image of the virtual space is displayed on the display from a first-person perspective, only a virtual hand (virtual body part) that is a part of the player character may be displayed, or the virtual hand may not be displayed and only a handgun (virtual object) held by the virtual hand may be displayed.

[0118] The image of the virtual space may be an image from a first-person perspective or a third-person perspective.

[0119] In the above description, the game program may be implemented as a game program for a so-called online game. In this case, the processing performed in the control unit 16 of the game device 10 may be performed in a control unit of a server device (not shown), or may be shared between the game device 10 and the control unit of the server device. The game device 10 may be a mobile information terminal such as a smartphone or a tablet.

[0120] In the above description, a virtual space in which a game progresses (game space) has been given as an example of a virtual space, but the present invention is not limited to this. For example, the virtual space may be a virtual space in which avatars communicate with each other (metaverse space). In other words, the services provided in the virtual space are not limited to games. For example, the services provided in the virtual space may be the provision of a place for commercial transactions, the provision of a place for communication, etc.

[0121] Even when these other embodiments are adopted, the effects of the present invention are exhibited. In addition, this embodiment can be combined with other embodiments, and other embodiments can be combined with each other as appropriate. The above embodiments are essentially preferred examples, and are not intended to limit the scope of the present invention, its applications, or its uses. [Explanation of symbols]

[0122] 10 Game device (sound control device) 11 Network Interface 12 Graphics Processing Unit 13 Audio Processing Section 14 Operation Processing Section 20 Head-mounted display 21 Display 22 Motion Sensor 25 Main body 26 Headband 30 Speaker 40 Controller 40a First Controller 40b Second Controller 41 Controls 42 Motion Sensor (Sensor) 50 players (users) 51 Prescribed Part 55 Right hand (hand) 56 Left hand (hand) 60 Player characters (virtual users) 61 Virtual Body Parts 65 Virtual right hand (virtual hand) 66 Virtual left hand (virtual hand) 70 Virtual Objects 71 Handgun 100 Game progress section (virtual space control section) 101 Image playback unit 102 Voice Generation Unit 103 Audio playback section

Claims

1. A program for causing a computer system having one computer or a plurality of computers working together to function as a virtual space control unit, an image playback unit, an audio generation unit, and an audio playback unit, The computer system is used with a display, a speaker, and a sensor; The sensor outputs a signal indicative of a movement of a predetermined part of the user's body; the virtual space control unit controls the virtual space such that a virtual part moves within the virtual space in response to the movement of the predetermined part indicated by the signal; The image playback unit plays an image of the virtual space on the display, the sound generation unit generates a sound effect according to a moving speed of the virtual body part when a condition for playing the sound effect according to a predetermined movement of the virtual body part is satisfied; The sound reproducing unit reproduces the sound effect generated by the sound generating unit through the speaker. program.

2. In the program of claim 1, The computer system is used together with a controller that is worn or held by the user and operated; The sensor is included in the controller; The virtual space control unit controls the virtual space based on a signal output from the sensor and an operation input to the controller. program.

3. In the program of claim 2, the predetermined part is a hand, The virtual part is a virtual hand. program.

4. In the program of claim 3, a virtual object that can be held by the virtual hand is placed in the virtual space; the virtual space control unit, when a gripping condition is satisfied that is a condition for performing a gripping action in which the virtual hand grips the virtual object in the virtual space, controls the virtual space so that the gripping action is performed; When the gripping condition is satisfied, the sound generating unit generates a gripping sound, which is a sound effect corresponding to the gripping motion, in accordance with a moving speed of the predetermined part during a gripping start period starting from a time point when the gripping condition is satisfied. program.

5. In the program of claim 4, The gripping condition is a condition that an operation for performing the gripping motion is input to the controller in a state in which the virtual hand enters within a predetermined range based on a position of the virtual object in the virtual space. program.

6. In the program of claim 4, the virtual object is portable in a state where it is held by the virtual hand; the virtual space control unit, when a carrying condition is satisfied that is a condition for performing a carrying action in which the virtual hand carries the virtual object in a state of holding the virtual object after the grasping condition is satisfied in the virtual space, controls the virtual space so that the carrying action is performed; When the carrying condition is satisfied after the holding condition is satisfied, the sound generating unit generates a carrying sound, which is a sound effect corresponding to the carrying action, in accordance with a moving speed of the predetermined part during a carrying start period from the time point when the holding condition is satisfied to the time point when the carrying condition is satisfied. program.

7. In the program of claim 6, The carrying condition is a condition that, in a state in which the virtual hand continues to grip the virtual object in the virtual space, a moving distance of the predetermined part after the gripping condition is established reaches a predetermined distance. program.

8. A storage unit that stores the program according to any one of claims 1 to 7; A control unit that executes the program. Sound control device.

Images