Program and video / audio control device

Abstract

To alleviate a feeling of discomfort of a sound generated when switching positions of a virtual camera.SOLUTION: A camera control part executes camera position switching processing for switching positions of a virtual camera C from a first camera position CP1 to a second camera position CP2 away from a predetermined object in a virtual space farther than the first camera position CP1 when a predetermined condition is satisfied. A sound collection point control part executes interlocking control for interlocking a virtual microphone M with the virtual camera C so as to maintain predetermined positional relationships with respect to the position of the virtual camera C, and microphone position switching control for switching the positions of the virtual microphone M from a first microphone position MP1, which is a predetermined positional relationship with respect to the first camera position CP1 to a second microphone position MP2 closer to the object than a corresponding position MPC, which is a predetermined positional relationship with respect to the second camera position CP2 when performing the camera position switching processing.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This disclosure relates to a program and an audio-visual control device.

Background Art

[0002] Patent Document 1 discloses a game system that realizes a virtual three-dimensional space in which a plurality of sound source objects, a virtual microphone, and a virtual camera are arranged. In this game system, an audio signal is generated based on the volume and localization of each audio when the audio reproduced from a plurality of sound source objects is received by the virtual microphone. Also, the virtual camera is arranged at the same position as the virtual microphone in the virtual three-dimensional space, and an image captured by the virtual camera is displayed on a monitor.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a game system such as Patent Document 1, when the position of the virtual camera is switched from a first camera position to a second camera position that is farther from the first camera position than a predetermined object in the virtual space, if the virtual microphone is moved to the same position as the virtual camera in联动 with the virtual camera, the volume may suddenly change and a sense of discomfort may occur.

[0005] An object of this disclosure is to alleviate the sense of discomfort in the sound that occurs when the position of the virtual camera is switched

Means for Solving the Problems

[0006] A first aspect is a computer system having one computer or a plurality of computers operating in cooperation, A video playback unit that plays back images obtained from a virtual camera placed in a virtual space on a display, A camera control unit that performs a camera position switching process to switch the position of the virtual camera from a first camera position to a second camera position that is further away from a predetermined object in the virtual space than the first camera position, when predetermined conditions are met. A sound playback unit that plays back sound obtained by a virtual microphone placed in the virtual space to a speaker, The virtual microphone is configured to function as a sound collection point control unit that performs linked control to link with the virtual camera to maintain a predetermined positional relationship with the position of the virtual camera, and microphone position switching control to switch the position of the virtual microphone from a first microphone position that has the predetermined positional relationship with the first camera position to a second microphone position that is closer to the object than the corresponding position that has the predetermined positional relationship with the second camera position during the camera position switching process. It is a program.

[0007] In the first embodiment, The camera control unit executes a camera position restoration process to return the position of the virtual camera from the second camera position to the first camera position when the predetermined conditions are no longer met. The sound collection point control unit may further perform microphone position return control during the camera position return process, which returns the position of the virtual microphone from the second microphone position to the first microphone position.

[0008] In the first embodiment, The first camera position may be the viewpoint of the object, and the second camera position may be a position that captures the object within its field of view.

[0009] In the first embodiment, The first camera position and the second camera position may be positions that capture the object within the field of view.

[0010] A second embodiment includes a storage unit that stores the program of the first embodiment, A control unit that executes the aforementioned program, This is a video and audio control device equipped with [features / equipment]. [Effects of the Invention]

[0011] According to this disclosure, it is possible to mitigate the unnatural sound that occurs when switching the position of the virtual camera. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram illustrating the configuration of the game device according to the embodiment. [Figure 2] This is a schematic diagram illustrating the switching of the virtual camera and virtual microphone positions. [Figure 3] This flowchart illustrates the processing performed by the camera control unit and the sound collection point control unit. [Modes for carrying out the invention]

[0013] The embodiments will be described in detail below with reference to the drawings. Note that the same or corresponding parts in the drawings are denoted by the same reference numerals, and their descriptions will not be repeated.

[0014] (Game description) In the following description, "game" refers to a game played in a virtual space (video game). This virtual space is a three-dimensional game space, displayed as an image on a screen. In the following example, objects are placed within the virtual space. Examples of objects include player characters controlled by the user and non-player characters controlled by the computer.

[0015] Furthermore, objects include sound source objects that act as virtual sound sources within the virtual space. Examples of sound source objects include objects that operate within the virtual space, and objects that only output sound within the virtual space.

[0016] In the game described below, sounds are played. The sounds played in the game include the sounds output from sound source objects, BGM (Background Music), sound effects, and the like. Examples of the sounds output from sound source objects include the sounds associated with the movement of objects such as footsteps and wind sounds, and the sounds emitted by objects such as cries and lines.

[0017] Hereinafter, for the sake of convenience of explanation, as an example of a game, a "fighting action game in which a player character attacks and defeats an enemy character" is given. The enemy character is an example of a non-player character. The player character and the enemy character are examples of sound source objects. Note that there is no limitation on the type of the game.

[0018] (Game device) FIG. 1 illustrates the configuration of the game device 10 of the embodiment. The game device 10 executes a game according to an operation by a user. The game device 10 has a display 21, a speaker 22, and a controller 23 externally connected or built-in. The game device 10 is an example of a computer system having one computer or a plurality of computers working together. Also, the game device 10 is an example of a video and audio control device.

[0019] Commercially available devices such as, for example, a personal computer, PlayStation (registered trademark), XBox (registered trademark), PlayStation Vita (registered trademark), Nintendo Switch (registered trademark), etc. can be used as the game device 10.

[0020] In the game device 10, the game progresses based on the installed game program and game data. The game devices 10 can also perform data communication with each other using a communication network (not shown) or a short-range wireless communication device (not shown).

[0021] 〔Hardware configuration of game device〕 The game device 10 includes a network interface 11, a graphics processing unit 12, an audio processing unit 13, an operation unit 14, a storage unit 15, and a control unit 16. The network interface 11, graphics processing unit 12, audio processing unit 13, operation unit 14, and storage unit 15 are electrically connected to the control unit 16 via a bus 17.

[0022] The network interface 11 is connected to a communication network (not shown) so as to enable communication in order to send and receive various data with, for example, other game devices 10 or external server devices (not shown).

[0023] The graphics processing unit 12 is connected to the display 21 (for example, an LCD display). The graphics processing unit 12 renders game images, including various objects in the virtual space, in video format according to the game image information output from the control unit 16. The game images rendered in video format are displayed on the display 21 as the game screen.

[0024] The audio processing unit 13 is connected to the speaker 22. The audio processing unit 13 plays digital game sounds according to the instructions of the control unit 16. Specifically, the audio processing unit 13 converts the audio data output by the control unit 16 into an analog signal and outputs it to the speaker 22.

[0025] The audio processing unit 13 is configured to provide multi-channel audio output in order to reproduce three-dimensional sound. Accordingly, multiple speakers 22 are connected to the game device 10. These speakers 22 may be positioned to the left, right, front, back, and above the listener (e.g., the game player) to enable three-dimensional sound reproduction.

[0026] The control unit 14 is connected to the controller 23. The control unit 14 sends and receives signals to and from the controller 23. The game player inputs signals (commands, data, etc.) to the game device 10 by operating various controls such as buttons on the controller 23.

[0027] The storage unit 15 stores various types of information and data. The storage unit 15 is composed of HDD, SSD, RAM, ROM, etc. For example, the storage unit 15 stores various programs, including game data and game programs. Examples of game data include game media and user account information.

[0028] Furthermore, the memory unit 15 stores various types of pre-prepared audio data. These types of audio data include audio data indicating sounds output from sound source objects such as player characters and enemy characters, audio data indicating music (BGM) played in the game, and audio data indicating sound effects.

[0029] The control unit 16 controls the operation of the game device 10. The control unit 16 transmits and receives various information and data, and processes various information and data. The control unit 16 has a CPU (microcomputer) and semiconductor memory. The semiconductor memory stores programs and data for operating the CPU.

[0030] [Functional configuration of the control unit] The control unit 16 includes a game progression unit 100, a camera control unit 101, a video playback unit 102, a sound collection point control unit 103, and an audio playback unit 104. Specifically, the control unit 16 functions as the game progression unit 100, the camera control unit 101, the video playback unit 102, the sound collection point control unit 103, and the audio playback unit 104 by executing a game program.

[0031] <Game Management Team> The game management unit 100 manages the game played in the virtual space.

[0032] In this example, the game progression unit 100 advances the game by making the player character move in the virtual space in response to the actions of the user (i.e., the game player) using the game device 10. The game progression unit 100 also makes non-player characters and predetermined objects (such as vehicles) move in the virtual space. For example, the game progression unit 100 controls the actions of non-player characters and predetermined objects using AI (artificial intelligence). The game progression unit 100 then advances the game in response to the actions of the player character and non-player characters.

[0033] <Camera Control Unit> The camera control unit 101 controls the position of the virtual camera C (see Figure 2) placed in the virtual space according to the progress of the game. The virtual camera C is a virtual viewer (viewpoint) and is a reference point for reproducing the viewpoint of the player character C1 (see Figure 2). The virtual camera C is directional.

[0034] In this example, the camera control unit 101 positions the virtual camera C near the player character C1. Specifically, the virtual camera C is positioned within a predetermined range (a circular area) centered on the position of the player character C1. The camera control unit 101 then controls the position of the virtual camera C so that it moves in accordance with the movement of the player character C1.

[0035] <Video Playback Section> The video playback unit 102 generates video data according to the progress of the game. The video playback unit 102 then outputs the video data to the graphics processing unit 12. As a result, video (game images in video format) corresponding to the progress of the game is displayed on the display 21.

[0036] Furthermore, the video playback unit 102 generates video data including the video obtained by the virtual camera C and outputs this video data to the graphics processing unit 12. As a result, the video playback unit 102 displays the video, including the video obtained by the virtual camera C (for example, the video seen from the player character's perspective), on the display 21.

[0037] <Sound Collection Point Control Section> The sound collection point control unit 103 controls the position of the virtual microphone M (see Figure 2) placed in the virtual space according to the progress of the game. The virtual microphone M is a virtual listener (sound collection point) and is the reference point for sound image localization by the sound playback unit 104. The virtual microphone M is directional.

[0038] In this example, the sound collection point control unit 103 typically performs linked control to synchronize the virtual microphone M with the virtual camera C so that the virtual microphone M maintains a positional relationship with the virtual camera C. In other words, the sound collection point control unit 103 typically controls the position of the virtual microphone M so that it moves in accordance with the movement of the virtual camera C.

[0039] <Audio Playback Unit> The audio playback unit 104 generates audio data according to the progress of the game. Specifically, the audio playback unit 104 selects the audio data to be played from the audio data (various pre-prepared audio data) stored in the memory unit 15 according to the progress of the game, and outputs the selected audio data (or new audio data obtained by combining multiple selected audio data) to the audio processing unit 13.

[0040] When the audio playback unit 104 outputs audio data, it instructs the audio processing unit 13 on the volume. The audio processing unit 13 converts the audio data output from the audio playback unit 104 into an analog signal of the instructed volume and outputs it to the speaker 22. As a result, the speaker 22 plays a sound corresponding to the audio data generated by the audio playback unit 104.

[0041] Sound localization processing Furthermore, the audio playback unit 104 performs sound image localization processing. In sound image localization processing, the audio playback unit 104 generates audio data that includes sound obtained from a virtual microphone M (see Figure 2) placed in the virtual space.

[0042] Specifically, in sound image localization processing, the audio playback unit 104 generates audio data according to the sound collection conditions of the virtual microphone M (for example, the position and direction of the virtual microphone M relative to the sound source object in the virtual space, the volume and direction of the sound output from the sound source object, etc.) so that the sound output from the speaker 22 includes the sound obtained by the virtual microphone M.

[0043] The audio data generated by the sound image localization process is processed in the audio processing unit 13, making it possible to output sound from a virtual sound source (sound source object) in the virtual space to the speaker 22 with an acoustic representation as if it were collected by a virtual microphone M. In this way, the audio playback unit 104 reproduces the sound obtained by the virtual microphone M through the speaker 22.

[0044] [Switching the position of the virtual camera and virtual microphone] Next, referring to Figure 2, we will explain how to switch the positions of the virtual camera C and the virtual microphone M.

[0045] The camera control unit 101 executes a camera position switching process to switch the position of the virtual camera C from the first camera position CP1 to the second camera position CP2, which is further away from the player character C1 in the virtual space than the first camera position CP1, when predetermined conditions are met. The first camera position CP1 is the viewpoint of the player character C1. Therefore, when the virtual camera C is at the first camera position CP1, the image obtained by the virtual camera C is a first-person view image. The second camera position CP2 is a position that captures the player character C1 within its field of view. Therefore, when the virtual camera C is at the second camera position CP2, the image obtained by the virtual camera C is a third-person view image. In this embodiment, the predetermined conditions for executing the camera position switching process are the conditions that the player character C1 performs a predetermined action, such as engaging in combat with an enemy character, due to the game progression by the game progression unit 100. When the predetermined conditions are met and the virtual camera C is at the second camera position CP2, the game progress unit 100 may prevent the user's input to the controller 23 from being reflected in the game's progress. In other words, when the predetermined conditions are met and the virtual camera C is at the second camera position CP2, the user's input to the controller 23 may be disabled.

[0046] In this embodiment, the predetermined condition for executing the camera position switching process is defined as the player character C1 performing a predetermined action, such as engaging an enemy character, as a result of the game progression by the game progression unit 100. However, this predetermined condition may also be defined as the user performing a predetermined operation on the controller 23, or the player character C1's health value being below a predetermined threshold (for example, a threshold corresponding to death).

[0047] The sound collection point control unit 103 normally performs linked control to synchronize the virtual microphone M with the virtual camera C so that the virtual microphone M is in the same position as the virtual camera C.

[0048] However, as shown in Figure 2, when the camera position switching process is performed, if the position of the virtual microphone M is moved by the above-mentioned interlocking control from the first microphone position MP1, which is the same as the first camera position CP1, to the corresponding position MPC, which is the same as the second camera position CP2, the amount of sound obtained by the virtual microphone M will change abruptly, which may cause an unnatural sound. For example, if the sounds emitted by the player character C1 and the enemy character were being played back to the speaker 22, when the camera position switching process is performed, the virtual microphone M will move away from the player character C1 and the enemy character, and the volume will suddenly decrease, causing an unnatural sound.

[0049] Therefore, in this embodiment, during the camera position switching process, the sound collection point control unit 103 performs microphone position switching control to switch the position of the virtual microphone M from the first microphone position MP1, which is the same as the first camera position CP1, to the second microphone position MP2, which is closer to the player character C1 than the corresponding position MPC, which is the same as the second camera position CP2. The second microphone position MP2 is set to be closer to the player character C1 by a predetermined percentage (X%) of the distance between the corresponding position MPC and the player character C1, compared to the corresponding position MPC. The predetermined percentage is set to, for example, 50%.

[0050] [Sound collection point control processing] Next, referring to Figure 3, the processing of the camera control unit 101 and the sound collection point control unit 103 will be described. The camera control unit 101 and the sound collection point control unit 103 repeatedly perform the following processing at predetermined intervals.

[0051] <Step S11> The camera control unit 101 determines whether the predetermined conditions for executing the camera position switching process are met. If the predetermined conditions are met, the process proceeds to step S12. On the other hand, if the predetermined conditions are not met, the determination in step S11 is performed again. For example, if the predetermined condition is that the player character C1 is engaged with an enemy character, the camera control unit 101 determines that the player character C1 is engaged with an enemy character, then proceeds to step S12, and determines that the player character C1 is not engaged with an enemy character, then performs the determination in step S11 again.

[0052] <Step S12> In step S12, the camera control unit 101 performs a camera position switching process to switch the position of the virtual camera C from the first camera position CP1 to the second camera position CP2, which is further away from the player character C1 than the first camera position CP1. As a result, the image obtained by the virtual camera C switches from a first-person view to a third-person view.

[0053] Furthermore, at this time, the sound collection point control unit 103 performs microphone position switching control, which switches the position of the virtual microphone M from the first microphone position MP1, which is the same as the first camera position CP1, to the second microphone position MP2, which is closer to the player character C1 than the corresponding position MPC, which is the same as the second camera position CP2.

[0054] <Step S13> Next, in step S13, the camera control unit 101 determines whether the predetermined condition is still being met. If the predetermined condition is met, the determination in step S13 is repeated. On the other hand, if the predetermined condition is not met, the process proceeds to step S14. For example, if the predetermined condition is that the player character C1 is engaged with an enemy character, the camera control unit 101 determines that the player character C1 is still engaged with an enemy character, then the determination in step S13 is repeated. On the other hand, if it determines that the player character C1 is no longer engaged with an enemy character, the process proceeds to step S14.

[0055] <Step S14> When the predetermined conditions are no longer met, in step S14, the camera control unit 101 executes a camera position recovery process to return the position of the virtual camera C from the second camera position CP2 to the first camera position CP1. As a result, the image obtained by the virtual camera C returns from a third-person perspective to a first-person perspective.

[0056] Furthermore, during the camera position return process, the sound collection point control unit 103 performs microphone position return control to return the position of the virtual microphone M from the second microphone position MP2 to the first microphone position MP1.

[0057] [Summary of Embodiments] To summarize, the game program of the embodiment functions as a computer (control unit 16) which is configured as a video playback unit 102 that plays images obtained by a virtual camera C placed in the virtual space on a display 21, a camera control unit 101 that performs a camera position switching process to switch the position of the virtual camera C from a first camera position CP1 to a second camera position CP2 that is further away from a predetermined object in the virtual space than the first camera position CP1 when predetermined conditions are met, an audio playback unit 104 that plays sounds obtained by a virtual microphone M placed in the virtual space to a speaker 22, an interlocking control that links the virtual microphone M with the virtual camera C to maintain a predetermined positional relationship with the position of the virtual camera C, and a microphone position switching control that, during the camera position switching process, switches the position of the virtual microphone M from a first microphone position MP1 that has the predetermined positional relationship with the first camera position CP1 to a second microphone position MP2 that is closer to the object than the corresponding position MPC that has the predetermined positional relationship with the second camera position CP2.

[0058] The game device 10 of this embodiment includes a storage unit 15 for storing the above-mentioned game program and a control unit 16 for executing the above-mentioned game program.

[0059] [Effects of the Embodiment] As described above, in this embodiment, when the position of the virtual camera C is switched from the first camera position CP1 to the second camera position CP2, which is further from the player character C1 in the virtual space than the first camera position CP1, the sound collection point control unit 103 switches the position of the virtual microphone M from the first microphone position MP1, which is the same as the first camera position CP1, to the second microphone position MP2, which is closer to the player character C1 than the corresponding position MPC, which is the same as the second camera position CP2. With this configuration, the amount of movement of the virtual microphone M can be reduced compared to when the virtual microphone M is moved from the first microphone position MP1 to the corresponding position MPC during the camera position switching process. Therefore, changes in volume caused by sudden changes in the position of the virtual microphone M can be suppressed, and the feeling of unnaturalness in sound can be alleviated.

[0060] Furthermore, in this embodiment, when the predetermined conditions for executing the camera position switching process are no longer met, and the camera control unit 101 returns the position of the virtual camera C from the second camera position CP2 to the first camera position CP1, the sound collection point control unit 103 also returns the position of the virtual microphone M to the same first microphone position MP1 as the first camera position CP1. With this configuration, when the image obtained by the virtual camera C at the first camera position CP1 is a first-person view, the virtual microphone M can be placed in the same position as the virtual camera C. This makes the sound obtained by the virtual microphone M sound as if it were the sound heard by the player character C1. This enhances the sense of realism and improves the player's immersion in the game.

[0061] (Other embodiments) In the above embodiment, the positional relationship between the virtual camera C and the virtual microphone M during linked control was set so that the virtual camera C and the virtual microphone M are in the same position. However, the virtual microphone M may be set to be separated from the virtual camera C by a predetermined distance greater than 0 in a predetermined direction. For example, the virtual camera C may be at the position of the player character C1's eyes, and the virtual microphone M may be at the position of the player character C1's ears.

[0062] Furthermore, in the above embodiment, the first camera position CP1 is set as the viewpoint of the player character C1 (object), and the second camera position CP2 is set as a position within the player character C1's (object's) field of view. However, both the first camera position CP1 and the second camera position CP2 may be set as positions within the player character C1's (object's) field of view. In other words, the image obtained by the virtual camera C may be a third-person view image regardless of whether the virtual camera C is at the first camera position CP1 or the second camera position CP2. For example, the first camera position may be set behind the player character C1, and the second camera position may be set above the player character C1, or behind another character. Also, in multiplayer mode, the first camera position may be set behind the player character C1, and the second camera position may be set behind another player's player character or non-player character.

[0063] Furthermore, in the above description, the game program may be implemented as a game program for so-called online games. In this case, the processing performed in the control unit 16 of the game device 10 may be performed in the control unit of the server device (not shown), or it may be shared between the control unit of the game device 10 and the control unit of the server device. The game device 10 may be a portable information terminal such as a smartphone or tablet.

[0064] Furthermore, while the above explanation uses a virtual space where a game takes place as an example of a virtual space, it is not limited to this. For example, a virtual space could be a virtual space where avatars communicate with each other (a metaverse space). For example, the services provided in a virtual space could be the provision of a platform for commercial transactions, the provision of a platform for communication, etc.

[0065] The effects and benefits of the present invention will also be achieved when these other embodiments are adopted. Furthermore, it is possible to combine these embodiments with other embodiments, and with other embodiments as appropriate. The above embodiments are essentially preferred examples and are not intended to limit the scope of the present invention, its applications, or its uses. [Explanation of Symbols]

[0066] 10 game devices 11 Network Interfaces 12 Graphics Processing Unit 13 Audio Processing Unit 14 Control section 15 Storage section 16 Control Unit 17 Bus 21 displays 22 speakers 23 Controllers 100 Game Management Department 101 Camera Control Unit 102 Video Playback Unit 103 Sound Collection Point Control Unit 104 Audio playback unit C Virtual Camera C1 Player Character (Object) CP1 First Camera Position CP2 Second Camera Position M Virtual Microphone MP1 First microphone position MP2 2nd microphone position MPC compatible location

Claims

1. A computer system having one computer or multiple cooperating computers, A video playback unit that plays back images obtained from a virtual camera placed in a virtual space on a display, A camera control unit that performs a camera position switching process to switch the position of the virtual camera from a first camera position to a second camera position that is further away from a predetermined object in the virtual space than the first camera position, when predetermined conditions are met. A sound playback unit that plays back sound obtained by a virtual microphone placed in the virtual space to a speaker, The virtual microphone functions as a sound collection point control unit that performs linked control to link the virtual camera to maintain a predetermined positional relationship with the position of the virtual camera, and microphone position switching control during the camera position switching process, which switches the position of the virtual microphone from a first microphone position that has the predetermined positional relationship with the first camera position to a second microphone position that is closer to the object than the corresponding position that has the predetermined positional relationship with the second camera position, and is different from the first microphone position. The predetermined positional relationship is such that the virtual camera and the virtual microphone are in the same position. The first camera position is the viewpoint of the object, and the second camera position is the position that captures the object within its field of view. program.

2. A computer system having one computer or multiple cooperating computers, A video playback unit that plays back images obtained from a virtual camera placed in a virtual space on a display, A camera control unit that performs a camera position switching process to switch the position of the virtual camera from a first camera position to a second camera position that is further away from a predetermined object in the virtual space than the first camera position, when predetermined conditions are met. A sound playback unit that plays back sound obtained by a virtual microphone placed in the virtual space to a speaker, The virtual microphone functions as a sound collection point control unit that performs linked control to link the virtual camera to maintain a predetermined positional relationship with the position of the virtual camera, and microphone position switching control during the camera position switching process, which switches the position of the virtual microphone from a first microphone position that has the predetermined positional relationship with the first camera position to a second microphone position that is closer to the object than the corresponding position that has the predetermined positional relationship with the second camera position, and is different from the first microphone position. The predetermined positional relationship is such that the virtual camera and the virtual microphone are in the same position. The first camera position and the second camera position are positions that capture the object within the field of view. program.

3. In the program of claim 1 or 2, The camera control unit executes a camera position restoration process to return the position of the virtual camera from the second camera position to the first camera position when the predetermined conditions are no longer met. The sound collection point control unit further performs microphone position return control during the camera position return process, which returns the position of the virtual microphone from the second microphone position to the first microphone position. program.

4. A storage unit for storing the program according to claim 1 or 2, The system comprises a control unit that executes the aforementioned program. Video and audio control device.

Images