Program and video / sound control device

Abstract

To alleviate a feeling of discomfort of a sound generated when interlocking a virtual microphone with a virtual camera.SOLUTION: A camera control part moves a virtual camera C on the basis of a motion signal according to a user's motion. A sound collection point control part executes interlocking control for interlocking a virtual microphone M with the virtual camera C so as to maintain predetermined positional relationships with respect to the position of the virtual camera C, and invasion avoidance control to maintain the position of the virtual microphone M at the position before the invasion to an invasion impossible area when the virtual microphone M invades the invasion impossible area by the interlocking control.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a program and an audio-visual control device.

Background Art

[0002] Patent Document 1 discloses a game system that realizes a virtual three-dimensional space in which a plurality of sound source objects, a virtual microphone, and a virtual camera are arranged. In this game system, an audio signal is generated based on the volume and localization of each sound when the sound reproduced from a plurality of sound source objects is received by the virtual microphone. Also, the virtual camera is arranged at the same position as the virtual microphone in the virtual three-dimensional space, and the image captured by the virtual camera is displayed on a monitor. The virtual camera and the virtual microphone move as the player character moves.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a game system such as Patent Document 1, when a boundary line that divides a part or all of the space into two regions is provided at a position corresponding to a wall in the virtual three-dimensional space, and a virtual microphone is present in one of the two regions divided by the boundary line, there is a technique for reducing the volume of sound generated in the other region. This can make the user feel as if the sound is blocked by a wall.

[0005] By the way, in such a game system, when the virtual camera penetrates inside a thick wall, the virtual microphone moves in conjunction with the virtual camera across the boundary line, and the sound in the region where the virtual microphone was arranged before crossing the boundary line suddenly becomes inaudible, which may cause a sense of discomfort.

[0006] An object of the present disclosure is to alleviate the sense of discomfort in sound caused by linking a virtual microphone with a virtual camera.

Means for Solving the Problem

[0007] A first aspect is a computer system having one computer or a plurality of cooperating computers, a video playback unit that plays back video obtained by a virtual camera arranged in a virtual space on a display, a camera control unit that moves the virtual camera based on an operation signal corresponding to a user's operation, an audio playback unit that plays back sound obtained by a virtual microphone arranged in the virtual space through a speaker, a linkage control that links the virtual microphone with the virtual camera so as to maintain a predetermined positional relationship with respect to the position of the virtual camera, and when the virtual microphone is to enter a predetermined non-entry area by the linkage control, an intrusion avoidance control that maintains the position of the virtual microphone at a position before entering the non-entry area, and functions as a sound collection point control unit that performs the above program.

[0008] In the first aspect, The non-entry area may be an area where intrusion by a player character is impossible.

[0009] A second aspect is a storage unit that stores the program of the first or second aspect, a control unit that executes the program, and a video and audio control device comprising the same.

Advantages of the Invention

[0010] According to the present disclosure, it is possible to alleviate the sense of discomfort in sound caused by linking a virtual microphone with a virtual camera.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and their description will not be repeated.

[0013] (Description of the game) The game in the following description is a game (video game) performed in a virtual space. The virtual space is a three-dimensional game space and is displayed on a display as an image based on VR (Virtual Reality) technology. In the following example, objects are arranged in the virtual space. Examples of objects include a player character operated by a user, a non-player character operated by a computer, and the like. Also, in the virtual space, partitions (shelters) such as walls, rocks, and doors that divide a part or all of the virtual space into two regions (spaces) are arranged. Among the two regions divided by the partition, the region on the side where the player character is not present becomes a region where intrusion by the player character is impossible.

[0014] In addition, the object includes a sound source object that becomes a virtual sound source in the virtual space. Examples of the sound source object include an object that operates in the virtual space, an object that only outputs sound in the virtual space, and the like.

[0015] In the game described below, sounds are played. The sounds played in the game include the sounds output from the sound source object, BGM (Background Music), sound effects, and the like. Examples of the sounds output from the sound source object include the sounds associated with the movement of the object such as footsteps and wind noise, and the sounds emitted by the object such as cries and lines.

[0016] Below, for the sake of convenience of explanation, as an example of the game, a "fighting action game in which the player character attacks and defeats the enemy character" is cited. The enemy character is an example of a non-player character. The player character and the enemy character are examples of the sound source object. Note that there is no limitation on the type of the game.

[0017] (Game device) FIG. 1 illustrates the configuration of the game device 10 of the embodiment. The game device 10 executes a game according to an operation by the user. The game device 10 has a display 21, a speaker 22, and a controller 23 externally connected or built-in. The game device 10 is an example of a computer system having one computer or a plurality of cooperating computers. Further, the game device 10 is an example of a video and audio control device.

[0018] For the game device 10, commercially available devices such as a personal computer, PlayStation (registered trademark), XBox (registered trademark), PlayStation Vita (registered trademark), Nintendo Switch (registered trademark), etc. can be used.

[0019] In the game device 10, the game progresses based on the installed game program and game data. The game devices 10 can also perform data communication with each other using a communication network (not shown) or a short-range wireless communication device (not shown).

[0020] 〔Hardware configuration of the game device〕 The game device 10 includes a network interface 11, a graphic processing unit 12, an audio processing unit 13, an operation unit 14, a storage unit 15, and a control unit 16. The network interface 11, the graphic processing unit 12, the audio processing unit 13, the operation unit 14, and the storage unit 15 are electrically connected to the control unit 16 via a bus 17.

[0021] The network interface 11 is communicably connected to a communication network (not shown), for example, to transmit and receive various data to and from other game devices 10 and external server devices (not shown).

[0022] The graphic processing unit 12 is connected to a display 21 (for example, a liquid crystal display). The graphic processing unit 12 draws a game image including various objects in a virtual space in a video format according to the game image information output from the control unit 16. The game image drawn in the video format is displayed on the display 21 as a game screen.

[0023] The audio processing unit 13 is connected to a speaker 22. The audio processing unit 13 reproduces digital game sounds according to the instructions of the control unit 16. Specifically, the audio processing unit 13 converts the audio data output by the control unit 16 into an analog signal and outputs it to the speaker 22.

[0024] The audio processing unit 13 is configured to be able to output multi-channel audio in order to perform three-dimensional sound reproduction. Accordingly, a plurality of speakers 22 are connected to the game device 10. These speakers 22 may be arranged around the listener (for example, the player of the game), such as on the left and right, front and back, and above, so that three-dimensional sound reproduction is possible.

[0025] The operation unit 14 is connected to the controller 23. The operation unit 14 transmits and receives operation signals corresponding to the actions (operations) of the user (i.e., the game player) to and from the controller 23. The game player inputs operation signals (commands, data, etc.) into the game device 10 by operating various operators such as the buttons of the controller 23. The controller 23 includes a plurality of operators and can transmit operation signals corresponding to the movements of the user's head and body to the operation unit 14. For example, when the user takes a forward-bending posture, that posture is reflected in the operation signal.

[0026] The storage unit 15 stores various kinds of information and data. The storage unit 15 is composed of an HDD, an SSD, a RAM, a ROM, etc. For example, the storage unit 15 stores game data, various programs including game programs, etc. Examples of game data include game media, user account information, etc.

[0027] In addition, the storage unit 15 stores various kinds of pre-prepared voice data. The various kinds of voice data include voice data indicating sounds output from sound source objects such as player characters and enemy characters, voice data indicating music (BGM) played in the game, voice data indicating sound effects, etc.

[0028] The control unit 16 controls the operation of the game device 10. The control unit 16 transmits and receives various kinds of information and data and processes various kinds of information and data. The control unit 16 has a CPU (microcomputer) and a semiconductor memory. Programs, data, etc. for operating the CPU are stored in the semiconductor memory.

[0029] 〔Functional Configuration of Control Unit〕 The control unit 16 has a game progress unit 100, a camera control unit 101, a video playback unit 102, a sound collection point control unit 103, and a voice playback unit 104. Specifically, the control unit 16 functions as the game progress unit 100, the camera control unit 101, the video playback unit 102, the sound collection point control unit 103, and the voice playback unit 104 by executing the game program.

[0030] <Game Progression Unit> The game progression unit 100 progresses the game performed in the virtual space according to the operation signal received by the operation unit 14.

[0031] In this example, the game progression unit 100 progresses the game by operating the player character C1 (see FIG. 2) in the virtual space according to the operation signal. For example, the game progression unit 100 moves the player character C1 in the virtual space according to the operation signal. Also, the game progression unit 100 operates non-player characters and predetermined objects (such as vehicles) in the virtual space. For example, the game progression unit 100 controls the operations of non-player characters and predetermined objects by AI (artificial intelligence). Then, the game progression unit 100 progresses the game according to the operations of the player character and non-player characters.

[0032] <Camera Control Unit> The camera control unit 101 controls the position of the virtual camera C (see FIG. 2) arranged in the virtual space according to the progression of the game by the game progression unit 100. The virtual camera C is a virtual viewer (viewpoint) and a reference point for reproducing the viewpoint of the player character C1. The virtual camera C has directivity.

[0033] In this example, the camera control unit 101 arranges the virtual camera C near the player character C1. Specifically, the virtual camera C is arranged on the head of the player character C1. Then, the camera control unit 101 controls the position of the virtual camera C so that the virtual camera C moves following the movement of the player character C1.

[0034] As described above, the game progression unit 100 moves the player character C1 according to the operation signal. Therefore, the camera control unit 101 also moves the virtual camera C based on the operation signal.

[0035] <Video Playback Unit> The video playback unit 102 generates video data according to the progress of the game. Then, the video playback unit 102 outputs the video data to the graphic processing unit 12. As a result, a video (game image in video format) corresponding to the progress of the game is displayed on the display 21.

[0036] Also, the video playback unit 102 generates video data including the video obtained by the virtual camera C, and outputs the video data to the graphic processing unit 12. As a result, the video playback unit 102 causes the display 21 to display a video including the video obtained by the virtual camera C (for example, the video viewed from the perspective of the player character C1).

[0037] 〈Sound collection point control unit〉 The sound collection point control unit 103 controls the position of the virtual microphone M (see FIG. 2) arranged in the virtual space according to the progress of the game. The virtual microphone M is a virtual listener (sound collection point) and is the reference point for sound image localization by the sound playback unit 104. The virtual microphone M has directivity.

[0038] In this example, the sound collection point control unit 103 usually performs interlocking control to interlock the virtual microphone M with the virtual camera C so that the position of the virtual microphone M maintains a positional relationship where it is the same as the position of the virtual camera C. That is, the sound collection point control unit 103 usually controls the position of the virtual microphone M so that the virtual microphone M moves following the movement of the virtual camera C. On the left side of FIG. 2, for the sake of explanation, the virtual microphone M and the virtual camera C are shown slightly separated, but actually, the virtual microphone M and the virtual camera C are at the same position point.

[0039] 〈Sound playback unit〉 The sound playback unit 104 generates sound data according to the progress of the game. Specifically, the sound playback unit 104 selects the sound data to be played back from the sound data (various types of sound data prepared in advance) stored in the storage unit 15 according to the progress of the game, and outputs the selected sound data (or new sound data obtained by synthesizing the selected multiple sound data) to the audio processing unit 13.

[0040] When the audio playback unit 104 outputs audio data, it instructs the audio processing unit 13 about its volume. The audio processing unit 13 converts the audio data output from the audio playback unit 104 into an analog signal of the instructed volume and outputs it to the speaker 22. As a result, a sound corresponding to the audio data generated by the audio playback unit 104 is reproduced from the speaker 22.

[0041] 《Audio Image Localization Processing》 Also, the audio playback unit 104 performs audio image localization processing. In the audio image localization processing, the audio playback unit 104 generates audio data including the sound obtained by the virtual microphone M (see FIG. 2) arranged in the virtual space.

[0042] Specifically, in the audio image localization processing, the audio playback unit 104 generates audio data according to the sound collection situation of the virtual microphone M (for example, the position and direction of the virtual microphone M with respect to the sound source object in the virtual space, the volume and direction of the sound output from the sound source object, etc.) so that the sound output from the speaker 22 becomes a sound including the sound obtained by the virtual microphone M.

[0043] By processing the audio data generated by the audio image localization processing in the audio processing unit 13, it becomes possible to output from the speaker 22 the sound output from the virtual sound source (sound source object) in the virtual space in an acoustic expression as if it were collected by the virtual microphone M. In this way, the audio playback unit 104 reproduces the sound obtained by the virtual microphone M from the speaker 22.

[0044] Also, as shown in FIG. 2, when a boundary line L that divides a part or all of the space into two regions R1 and R2 is set at a position corresponding to the wall W in the virtual space, and the virtual microphone M is in one of the two regions R1 and R2 divided by the boundary line L, the audio playback unit 104 reduces the volume of the sound generated in the other region. Thereby, it is possible to make the user feel as if the sound is blocked by the wall W. Note that the boundary line L is not limited to the wall W and may be at a position corresponding to other partitions (obstacles) such as rocks and doors.

[0045] Of the two regions R1 and R2 partitioned by the boundary line L, the region where the body (trunk) of the player character C1 is not present allows the virtual camera C to enter, but does not allow the body of the player character C1 to enter. In the example of FIG. 2, since the body of the player character C1 is in region R2, the body of the player character C1 cannot enter region R1. Specifically, the determination of whether the player character C1 hits the wall W is made based on the position of the center point of the body of the player character C1. Therefore, the body of the player character C1 cannot enter the interior of the wall W and region R1, but the head of the player character C1 and the virtual camera C can enter the interior of the wall W and region R1.

[0046] 〔Invasion of the virtual camera into the interior of the wall〕 Next, with reference to FIG. 2, the control of the position of the virtual microphone M when the virtual camera C enters the interior of the wall W will be described. Note that the same control as described below is performed when a boundary line L is provided at a position corresponding to another partition (shield) other than the wall W and the virtual camera C is made to enter the interior of the partition (shield).

[0047] As shown in FIG. 2, the camera control unit 101 may cause the virtual camera C to enter the interior of the wall W based on an operation signal corresponding to the user's operation.

[0048] The sound collection point control unit 103 normally performs interlocking control to interlock the virtual microphone M with the virtual camera C so as to maintain a positional relationship in which the position of the virtual microphone M is the same as that of the virtual camera C.

[0049] However, for example, as shown on the left side of FIG. 2, when the player character C1, the virtual camera C, and the virtual microphone M are in the region R2 and the head of the player character C1 is pressed against the wall W, the virtual camera C penetrates inside the wall W and enters the region R1 beyond the boundary line L. At this time, when the virtual microphone M enters the region R1 beyond the boundary line L in conjunction with the virtual camera C, the sound in the region R2 where the virtual microphone M was located before crossing the boundary line L suddenly stops being played, causing a sense of discomfort.

[0050] Therefore, in this embodiment, as shown on the right side of FIG. 2, when the virtual microphone M is to enter the region R1 (the position of the virtual microphone M indicated by the virtual line) by interlocking control, the sound collection point control unit 103 performs intrusion avoidance control to maintain the position of the virtual microphone M at the position before entering the region R1, that is, in the region R2.

[0051] 〔Sound Collection Point Control Process〕 Next, with reference to FIG. 3, the process of the sound collection point control unit 103 will be described.

[0052] 〈Step S11〉 When the user performs a predetermined operation (action) on the controller 23, the sound collection point control unit 103 determines whether the virtual microphone M will enter a predetermined non-intrusion region when the virtual microphone M is moved by the interlocking control according to the operation signal corresponding to the operation. For this determination, for example, ray casting can be used. If the virtual microphone M is to enter a predetermined non-intrusion region, the process proceeds to step S12. On the other hand, if the virtual microphone M is not to enter a predetermined non-intrusion region, the process proceeds to step S13. Here, the non-intrusion region is the other region of the two regions R1 and R2 partitioned by the boundary line L when the virtual microphone M is originally in one of the regions, that is, the region where the volume of the sound to be played is set to 0. For example, in the example of FIG. 2, since the virtual microphone M is originally in the region R2, the non-intrusion region is the region R1.

[0053] 〈Step S12〉 In step S12, the sound collection point control unit 103 performs intrusion avoidance control to maintain the position of the virtual microphone M at the position before intrusion into the non-intrusion area. That is, the sound collection point control unit 103 maintains the position of the virtual microphone M in the area where the virtual microphone M originally was among the two areas R1 and R2 partitioned by the boundary line L. For example, in the example of FIG. 2, the position of the virtual microphone M is maintained in the area R2. Specifically, as shown in FIG. 2, the position of the virtual microphone M is maintained at the position immediately before entering the wall W.

[0054] 〈Step S13〉 Next, in step S13, the sound collection point control unit 103 moves the virtual microphone M by interlocking control in response to the operation signal according to the operation by the user. That is, the sound collection point control unit 103 moves the virtual microphone M to the same position as the virtual camera C.

[0055] 〔Summary of the Embodiment〕 Summarizing the above, the game program of the embodiment causes a computer (control unit 16) to function as a video playback unit 102 that plays back on the display 21 a video obtained by a virtual camera C arranged in a virtual space, a camera control unit 101 that moves the virtual camera C based on an operation signal according to the user's operation, an audio playback unit 104 that plays back on the speaker 22 the sound obtained by a virtual microphone M arranged in the virtual space, an interlocking control that interlocks the virtual microphone M with the virtual camera C so as to maintain a predetermined positional relationship with respect to the position of the virtual camera C, and an intrusion avoidance control that maintains the position of the virtual microphone M at the position before intrusion into the non-intrusion area when the virtual microphone M is to intrude into a predetermined non-intrusion area by the interlocking control, as a sound collection point control unit 103.

[0056] The game device 10 of the embodiment includes a storage unit 15 that stores the above game program and a control unit 16 that executes the above game program.

[0057] 〔Effects of the Embodiment〕 As described above, in the embodiment, when the virtual microphone M is to enter a predetermined non-infringement area by interlocking control, the position of the virtual microphone M is maintained at the position before entering the non-infringement area. Therefore, by setting the area where the volume of the reproduced sound becomes 0 as the non-infringement area, it is possible to suppress the virtual microphone M from entering the area where the volume of the reproduced sound becomes 0 by interlocking control. Therefore, a sudden change in volume can be suppressed, and the sense of strangeness of the sound can be alleviated.

[0058] (Other Embodiments) In the above embodiment, the positional relationship between the virtual camera C and the virtual microphone M maintained during interlocking control is set such that the virtual camera C and the virtual microphone M are at the same position. However, the virtual microphone M may be separated from the virtual camera C by a predetermined distance greater than 0 in a predetermined direction. For example, the virtual camera C may be set at the position of the eyes of the player character C1, and the virtual microphone M may be set at the position of the ears of the player character C1.

[0059] Also, in the above description, the game program may be implemented as a game program for a so-called online game. In this case, the processing performed in the control unit 16 of the game device 10 may be performed in the control unit (not shown) of the server device, or may be shared between the control unit of the game device 10 and the control unit of the server device. The game device 10 may be a portable information terminal such as a smartphone or a tablet.

[0060] Also, in the above description, as an example of the virtual space, the virtual space in which the game progresses is given, but it is not limited to this. For example, the virtual space may be a virtual space (metaverse space) where avatars communicate with each other. For example, the services provided in the virtual space may be the provision of a place for business transactions, the provision of a place for communication, etc.

[0061] Even when these other embodiments are adopted, the functions and effects of the present invention are exhibited. Also, it is possible to appropriately combine this embodiment with other embodiments and among other embodiments. The above embodiments are essentially preferred examples and are not intended to limit the scope of the present invention, its applications, or its uses.

Explanation of Reference Numerals

[0062] 10 Game device (computer system, video and audio control device) 11 Network interface 12 Graphics processing unit 13 Audio processing unit 14 Operation unit 15 Storage unit 16 Control unit 17 Bus 21 Display 22 Speaker 23 Controller 100 Game progress unit 101 Camera control unit 102 Video playback unit 103 Sound collection point control unit 104 Audio playback unit C Virtual camera C1 Player character M Virtual microphone

Claims

1. A computer system having one computer or a plurality of cooperating computers, a video playback unit that plays back video obtained by a virtual camera disposed in a virtual space on a display; a camera control unit that moves the virtual camera based on an operation signal corresponding to a user's operation; an audio playback unit that plays back sound obtained by a virtual microphone disposed in the virtual space through a speaker; an interlocking control that interlocks the virtual microphone with the virtual camera so as to maintain a predetermined positional relationship with respect to the position of the virtual camera, and when the virtual microphone is to enter a predetermined non-invadable area by the interlocking control, a sound collection point control unit that functions to perform intrusion avoidance control to maintain the position of the virtual microphone at a position before intrusion into the non-invadable area Program.

2. In the program of Claim 1, The program is characterized in that the non-invadable area is an area where intrusion by a player character is impossible.

3. A storage unit that stores the program of Claim 1 or 2, and a control unit that executes the program Video and audio control device.

Images