Moving sound generation system
The moving sound generation system addresses the lack of realistic sound feedback in virtual spaces by determining and reproducing moving sounds based on object mass, surface materials, and contact timing, enhancing user experience and spatial awareness.
Patent Information
- Application Number
- JP2022078595
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-05-12
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-10-06
AI Technical Summary
Conventional methods for generating moving sounds in virtual spaces fail to provide a realistic experience, as users cannot hear their own moving sounds or those of other objects, leading to a poor user experience.
A moving sound generation system that determines the volume, sound quality, and generation timing of moving sounds based on the mass of the moving object, material information of the contact surfaces, and the timing of contact, using virtual speakers to reproduce these sounds in virtual spaces.
Generates realistic moving sounds, enhancing user experience by allowing users to hear their own and other objects' movements, improving spatial awareness and realism.
Smart Images

Figure 0007579823000001 
Figure 0007579823000002 
Figure 0007579823000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a moving sound generation system that generates a moving sound when a moving object moves in a virtual space. Regarding the system. [Background technology]
[0002] Conventionally, a system has been proposed that generates the sound of a moving object moving in a virtual space. For example, moving objects (non-player characters, non-commercial objects, vehicles, etc.) in a virtual space In order to represent the movement of the sound source, a method of moving the sound source in a certain direction at a certain speed has been proposed. (See, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2018-201790 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, in the conventional method, the movement of the moving object (character) around the user is The sound of the object moving is played, which gives a certain degree of realism, but the user Since the moving sounds made by the moving object (character) itself or other moving objects are not generated, There was a problem that the user's experience in the virtual space was poor. For example, in the real world, When people move, they can hear the volume, quality, and rhythm of their own moving sounds, which can affect how they feel. It can infer the situation, such as what kind of ground environment it is in and how fast it is moving. In addition, the sound of approaching or passing other people or cars can be localized without relying on vision. This is an inference.
[0005] The present invention has been made in consideration of the above problems, and provides a method for simulating analogy and cognition in a real space by using a virtual It can generate realistic moving sounds so that they can be reproduced in a virtual space. The present invention aims to provide a moving sound generation system that can improve the user's bodily sensation during a moving scene. The target. [Means for solving the problem]
[0006] The moving sound generation system of the present invention is a system for generating moving sounds in which a moving object moves on a predetermined moving surface in a virtual space. A moving sound generation system for generating a moving sound when a moving object moves in the virtual space, a volume determination unit that determines the volume of the movement sound based on the mass of the body; Material information of the grounding portion of the moving body that comes into contact with the moving body and a sound quality determining unit that determines the sound quality of the moving sound based on material information of the moving surface; The timing of generating the moving sound is determined based on the timing at which the ground contact portion comes into contact with the moving surface. a sound generation timing determination unit that determines a sound generation timing for a moving sound in the virtual space; the volume determined by the sound quality determination unit, the sound quality determined by the sound timing determination unit, and a movement sound reproducing section for reproducing the movement sound of the moving object at the timing.
[0007] According to this configuration, a predetermined moving plane (e.g., When a moving object (such as an avatar) moves on the ground or floor, the sound of the movement is automatically generated. Dynamically generated. In this case, the volume of the moving sound is determined based on the mass of the moving object, The quality of the moving sound is determined based on the material information of the contact part and the moving surface. The timing of the sound of the movement is determined based on the timing of contact with the moving surface. In this way, the real-time analysis is performed for each moving object in the virtual space (including the moving object operated by the user). This makes it possible to generate realistic moving sounds, improving the user's experience in the virtual space.
[0008] In other words, this configuration allows analogy and cognition in the real world to be reproduced in the virtual world. In this case, the moving object itself can be made to sound realistic. It is possible to generate moving sounds that generate realism. By generating moving sounds, it is possible to localize and infer the sound images of the sound sources of other moving objects.
[0009] In the moving sound generating system of the present invention, the sound quality determining unit determines a moving object moving information based on the moving object moving information. Based on the above, a contact portion of the moving surface with which the ground contact portion of the moving object comes into contact is predicted. It is also possible to read in advance only the material information of the contact portion of the moving surface.
[0010] According to this configuration, when determining the quality of the moving sound, the moving sound with which the ground contacts the ground of the moving object is determined. The contact areas of the faces are predicted, and only the material information of the contact areas of the moving faces is loaded in advance. Therefore, compared to reading the material information of the entire moving surface, the processing for generating the moving sound is This can reduce the burden on the processor.
[0011] In the moving sound generating system of the present invention, the moving sound reproducing unit is A virtual speaker that reproduces the moving sound of the moving object is provided at a contact portion of the moving surface where the moving object comes into contact. It may be placed.
[0012] According to this configuration, the contact portion of the moving surface with which the ground contact portion of the moving object comes into contact is contacted with the moving sound of the moving object. A virtual speaker is set up to play the audio from the virtual speaker. By reproducing the moving sound of a moving object, a realistic moving sound can be generated.
[0013] In the moving sound generation system of the present invention, the volume determination unit determines a volume of a moving object in front of the moving object in the virtual space. When the mass of the moving body is not set, the moving body is calculated based on the volume of the moving body. The mass may be estimated.
[0014] According to this configuration, even if the mass of the moving object in the virtual space is not set, The mass of the moving object is estimated based on the volume of the moving object, and the volume of the moving sound can be appropriately determined. can.
[0015] In the moving sound generating system of the present invention, the sound generation timing determination unit Based on the timing when the potential energy of the ground contact portion becomes minimum, A timing of contact with the moving surface may be determined.
[0016] According to this configuration, the potential energy of the ground contact portion of the moving object is minimized. Therefore, the timing when the grounding part of the moving object comes into contact with the moving surface can be appropriately determined, and Therefore, the timing of generating the moving sound can be appropriately determined.
[0017] The method of the present invention is executed in a moving sound generating system, and the moving sound generating system generates a moving sound of a predetermined moving surface in a virtual space. A method for generating a moving sound when a moving object moves on the virtual space, determining a volume of the movement sound based on a mass of the moving object; Material information of the grounding portion of the moving body that contacts the surface and the determining a quality of the moving sound based on material information of the moving surface; The timing of generating the moving sound is determined based on the timing at which the ground contact portion comes into contact with the moving surface. determining a volume, a sound quality, and a reproduction volume in the virtual space determined in the determining step; and reproducing the moving sound of the moving object at a sound timing.
[0018] This method, like the above system, allows the user to control the movement of each moving object (user) in the virtual space. It is possible to generate realistic moving sounds for the virtual The user's experience in the space is improved.
[0019] The program of the present invention is a program for controlling a moving object moving on a predetermined moving surface in a virtual space. A program for generating a moving sound, the program being configured to cause a computer to A process of determining a volume of the moving sound based on a mass of a body; and material information of the grounding portion of the moving body that contacts the grounding portion of the moving surface. A process for determining the quality of the moving sound based on material information; A process for determining a timing for generating the moving sound based on a timing of contact with the moving surface. In the virtual space, the volume, sound quality, and pronunciation timing determined by the processing are and reproducing the moving sound of the moving object.
[0020] This program also allows the individual moving objects (users) in the virtual space to move in the same way as the above system. It is possible to generate realistic moving sounds for moving objects (including those operated by the user) This improves the user's experience in the virtual space. Effect of the Invention
[0021] According to the present invention, it is possible to generate realistic moving sounds, and to improve the user's It is possible to improve the physical sensation. [Brief description of the drawings]
[0022] [Figure 1] 1 is a block diagram showing a configuration of a moving sound generation system according to an embodiment of the present invention. [Diagram 2] FIG. 11 is an explanatory diagram showing an example of determining sound quality of a moving sound. [Diagram 3] FIG. 11 is an explanatory diagram showing an example of prediction of a contact portion of a moving surface; [Figure 4] FIG. 11 is an explanatory diagram showing an example of the placement of virtual speakers. [Diagram 5] FIG. 11 is an explanatory diagram showing another example of the placement of virtual speakers. [Figure 6] 4 is a sequence diagram for explaining the operation of the moving sound generation system according to the embodiment of the present invention. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0023] A moving sound generating system according to an embodiment of the present invention will be described below with reference to the drawings. In the embodiment, a moving object ( We will use the example of a system that generates the sound of a character (such as an avatar) moving.
[0024] The configuration of the moving sound generation system of the present embodiment will be described with reference to the drawings. 1 is a block diagram showing a configuration of a moving sound generating system according to an embodiment of the present invention. The sound generation system 1 includes a user device 2 and a server device 3 that are connected to each other via a network N. It is composed.
[0025] The user device 2 is a device operated by a user, and is, for example, a head-mounted display. The user device 2 is equipped with a head-mounted display 4 (HMD) and a controller 5. The user device 2 is not limited to a device having a personal computer display 4. It may also be equipped with a computer, a smartphone, a tablet terminal, etc.
[0026] The head mounted display 4 is worn on the user's head and covers the field of view of both eyes of the user. This type of display device is a type of display device. A user wearing the head-mounted display 4 The image displayed on the head-mounted display 4 can be seen. There is no particular restriction on the display mode of the display 4, and it can be a virtual space with depth (virtual reality space). Alternatively, the object may be displayed at any position on the virtual plane. The object may be displayed.
[0027] The controller 5 is an input interface that accepts operational inputs from a user. Although not shown here, for example, the controller 5 has at least one operation button. It has various built-in sensors that detect the direction and movement (acceleration, rotation, etc.) of the controller 5. By inputting operations from the controller 5, the user can move the avatar in the virtual reality space. In addition to the controller, the user can also move around in the real world. The avatar may be moved in synchronization with the movement of the user.
[0028] The server device 3 includes a control unit 6 and a storage unit 7. For example, the control unit 6 includes a processor The storage unit 7 is composed of a large-capacity memory.
[0029] The control unit 6 includes a volume determination unit 8, a sound quality determination unit 9, a sound generation timing determination unit 10, and a moving sound determination unit. The memory unit 7 stores user information (e.g., user account information) ), avatar information (e.g., avatar type, size, mass, material of grounded part, etc.) ,Movement information (e.g., avatar position (coordinates), movement direction, movement speed, movement posture (bipedal walking) walking, bipedal walking, quadrupedal walking, wheeled walking, etc.), world information (e.g., heavy Force acceleration, material of the moving surface, etc.) are stored.
[0030] The types of avatars include, for example, human, animal, insect, plant, food, fantasy creatures, etc. Object type (dragon, Pegasus, yeti, etc.), robot type, object type with wheels (car character This includes companies such as makers, etc.
[0031] The materials of the avatar's grounded parts can be, for example, bare feet, cloth (cloth of socks, cloth of tabi socks, slip-ons, etc.) cloth), rubber (shoe rubber, running shoe rubber, rubber boot rubber, tie Rubber from the saddle, etc.), plants such as trees and grasses (wood for geta, straw for straw sandals, etc.), leather (leather for leather shoes, leather for suri Examples of materials used include leather (such as the leather in shoes), metal (such as the metal in wheels), etc.
[0032] The material of the moving surface, for example, outdoors, can be asphalt, concrete, or tiles. Ruins, cobblestones, puddles, dirt, sand, gravel, rocks, marshes, grasslands, trees, grass, moss, frost, snow, hail, fallen leaves For example, indoors, flooring, carpet, tile carpet, This includes tatami mats, cork, carpets, rugs, mats, etc.
[0033] The volume determination unit 8 determines the volume of the moving sound based on the mass of the moving body (avatar) in the virtual space. For example, the volume of a movement sound is determined so that the larger the mass, the louder the sound. The volume determination unit 8 can also determine the weight of the moving body (avatar) in the virtual space. The volume of the moving sound may be determined based on (= mass x gravitational acceleration). You can determine the volume of the movement sound so that the higher the volume, the louder the sound will be. The determination unit 8 determines, based on the z-coordinate of the swing-up position of the grounded part of the moving object (the avatar's foot), You can also determine the volume of the movement sound. For example, the higher the swing position, the louder the volume. In addition, the volume of the movement sound can be determined.
[0034] In addition, when the mass of the moving object in the virtual space is not set, the volume determination unit 8 The mass of the moving object may be estimated based on the volume of the moving object. By estimating that the mass of the moving object is larger, the sound volume will be louder as the volume increases. Thus, the volume of the movement sound can be determined.
[0035] The sound quality determination unit 9 determines the quality of the ground contact part (the avatar's feet) of the moving object that comes into contact with the moving surface in the virtual space. Based on the material information and the material information of the moving surface (ground or floor surface) that the ground contact part of the moving object comes into contact with, Determines the quality of movement sounds.
[0036] FIG. 2 is an explanatory diagram showing an example of determining the quality of a moving sound. As shown in FIG. The material of the part of the body that touches the ground (the avatar's feet) is "barefoot" and the material of the moving surface (ground) is "Athletic". If the sound quality is "Falto", the sound quality of the movement sound is determined to be "sound quality a". The data of a, b, etc. may be prepared in advance as a library. The material information of the grounded part of the moving object (the avatar's feet) and the moving surface that the grounded part of the moving object comes into contact with (the ground The quality of the moving sound may be calculated each time based on the material information of the object (surface, floor, etc.).
[0037] Further, the sound quality determination unit 9 determines a moving part where the grounded part of the moving object comes into contact based on the moving information of the moving object. The contact area of the moving surface is predicted, and only the material information of the contact area of the predicted moving surface is read in advance. Good too.
[0038] FIG. 3 is an explanatory diagram showing an example of how sound quality of a moving sound is determined. In FIG. 3, a moving body (avatar A) The contact area of the moving surface where the left foot is in contact is illustrated as R1, and the contact area of the moving surface where the right foot is in contact is illustrated as R2. The contact part of the surface is shown as R2. In this case, the movement information of the moving object (avatar A) Based on the avatar's position, movement direction, movement speed, movement posture, etc., the contact area of the moving object The contact points R3, R4, and R5 of the moving surface where the object comes into contact can be predicted. Only the material information of the contact parts R3, R4, and R5 is preloaded.
[0039] The sound generation timing determination unit 10 determines a sound generation timing based on the timing when the ground contact portion of the moving object comes into contact with the moving surface. For example, the sound generation timing determination unit 10 determines the sound generation timing of the moving sound based on the moving sound. It is determined whether the grounded part of the body is in contact with the moving surface, and based on the result of the determination, the moving sound is The timing of the pronunciation may be determined.
[0040] In addition, the sound generation timing determination unit 10 determines whether the potential energy of the grounded part of the moving object is at a minimum. Based on the timing, the timing when the grounded portion of the moving object comes into contact with the moving surface may be determined. In this case, the potential energy of the grounded part of the moving body is calculated based on the following formula: can be done. Potential energy = avatar mass x gravitational acceleration x height of the moving body's contact point Here, the height of the ground contact part of the moving body is the ground contact part of the moving body (for example, the left and right feet) is the z-coordinate of the lowest point or center of gravity of the
[0041] In addition, when the mass and gravitational acceleration of the avatar are constant, the sound generation timing determination unit 10 The moving part of the moving object moves based on the timing when the height of the moving object's grounding part becomes minimum. The timing of contact with the moving surface may be determined.
[0042] The moving sound reproducing unit 11 reproduces the volume and sound quality determined by the moving sound determining unit in the virtual space. The sound quality and the sound timing determined by the sound generation timing determination unit 10 are output to the mobile device. For example, the moving sound of the moving object is played at the contact part of the moving surface where the ground part of the moving object comes into contact. By installing a virtual speaker that reproduces the sound of a moving object, the moving sound of the moving object can be reproduced. This can be done.
[0043] FIG. 4 is an explanatory diagram showing an example of the placement of virtual speakers. In the example of FIG. 4, The contact part R1 of the moving surface that the left foot of the robot A is in contact with and the contact part R2 of the moving surface that the right foot is in contact with are A virtual speaker S is installed in each contact area R2.
[0044] 5 is an explanatory diagram showing another example of the placement of virtual speakers. In the example of FIG. The contact area of the moving surface where both feet of avatar A are in contact, and the bipedal running avatar The contact area of the moving surface where one foot of avatar B is in contact, and the foot of avatar C who is walking on four legs is in contact A virtual speaker S is placed at each contact point of the moving surface.
[0045] The moving sound generation system 1 configured as above will be described with reference to the sequence diagram of FIG. The operation will be explained below.
[0046] When using the moving sound generation system 1 of this embodiment, first, login information is entered into the user device 2. When the user inputs login information (S1), the input login information is transmitted from the user device 2 to the server device 3. The user authentication is successful (S2), and the server device 3 performs user authentication processing (S3). Then, the server device 3 sends a login permission to the user device 2 (S4).
[0047] Next, the user who has been permitted to log in operates the user device 2 to create an avatar in the virtual reality space. The avatar is then moved in the virtual space (S6), and the avatar's movement information is generated (S7). is sent from the user device 2 to the server device 3 (S7).
[0048] In the server device 3, the volume of the moving sound is determined based on the mass of the moving object in the virtual space. Then, (S8) the material information of the grounding part of the moving object that contacts the moving surface in the virtual space and the grounding part of the moving object are calculated. The quality of the moving sound is determined based on the material information of the moving surface that the part comes into contact with (S9). In addition, the timing of the sound of the moving object is determined based on the timing of contact between the grounded part of the moving object and the moving surface. Then, the selection is determined (S10).
[0049] Next, a request to install a virtual speaker is sent from the server device 3 to the user device 2 (S1 1) A moving sound of the moving object is reproduced at a contact portion of the moving surface where the grounded portion of the moving object comes into contact. A virtual speaker is then set up (S12).
[0050] Then, when the sound generation timing determined in step S10 comes (S13), The server device 3 sends a request to play the moving sound to the user device 2 (S14). The moving sound with the volume and quality determined in steps 8 and 9 is played on the virtual speaker set in step 12. It is played in the car (S15).
[0051] According to the moving sound generation system 1 of the present embodiment, a virtual space (for example, a VR space) A moving object (e.g., an avatar) moves on a predetermined moving surface (e.g., the ground or floor) in a certain area. When a moving object (such as a robot) moves, the sound of the object moving is automatically generated. In this case, the sound of the object moving is based on the mass of the moving object. The volume of the moving sound is determined based on the material information of the contact part of the moving object and the moving surface. The sound quality of the moving object is determined based on the timing when the grounded part of the moving object comes into contact with the moving surface. In this way, the timing of the sound is determined for each moving object (user) in the virtual space. It is possible to generate realistic moving sounds for the virtual This improves the user's experience in the space (see Figure 5).
[0052] That is, according to the moving sound generation system 1 of the present embodiment, analogy and It is possible to generate realistic moving sounds so that cognition can be reproduced in virtual space. In this case, the moving object itself can generate moving sounds to improve the reality. In addition, by generating the moving sounds of all moving objects, it is possible to define the sound images of the sound sources of other moving objects. The position can be inferred.
[0053] In this embodiment, when determining the quality of the moving sound, the moving object is determined based on the contact of the ground part of the moving object with the ground. The contact area of the moving surface is predicted, and only the material information of the contact area of the moving surface is read in advance. Therefore, compared to reading the material information of the entire moving surface, it is much easier to generate moving sounds. This can reduce the processing load.
[0054] In this embodiment, the contact portion of the moving surface with which the ground contact portion of the moving object comes into contact is A virtual speaker is set up to play the moving sound. By playing the moving sound of a moving object from a remote location, it is possible to generate a realistic moving sound. do.
[0055] In this embodiment, the mass of the moving object in the virtual space is not set. However, the mass of the moving object is estimated based on the volume of the moving object, and the volume of the moving sound is appropriately determined. It is possible.
[0056] In this embodiment, the timing at which the potential energy of the ground contact portion of the moving body becomes minimum Based on this, it is possible to appropriately determine the timing when the grounding portion of the moving object comes into contact with the moving surface. Therefore, the timing of generating the moving sound can be appropriately determined.
[0057] Although the embodiments of the present invention have been described above by way of example, the scope of the present invention is not limited to these. The present invention is not intended to be limited to the scope of the present invention, and may be modified or altered according to the purpose within the scope of the claims. It is possible. [Industrial Applicability]
[0058] As described above, the moving sound generation system according to the present invention can generate realistic moving sounds. This has the effect of improving the user's experience in the virtual space, It is useful as a virtual reality space system, etc. [Explanation of symbols]
[0059] 1. Moving sound generation system 2. User Device 3. Server equipment 4. Head-mounted display 5. Controller 6 Control section 7 Memory section 8 Volume Determination Section 9 Sound quality determining section 10. Sound generation timing determination section 11 Moving sound playback section N Network
Claims
1. A moving sound generation system that generates a moving sound when a moving object moves on a predetermined moving surface in a virtual space, a volume determination unit that determines a volume of a moving sound generated when a moving object moves on a moving surface in the virtual space based on a gravitational acceleration in the virtual space; a sound quality determination unit that determines a sound quality of the moving sound based on material information of a grounded portion of the moving body that contacts a moving surface in the virtual space and material information of the moving surface with which the grounded portion of the moving body contacts; a sound generation timing determination unit that determines a sound generation timing of the moving sound based on a timing at which a ground contact portion of the moving object comes into contact with the moving surface; a movement sound reproducing unit that reproduces, in the virtual space, a movement sound of the moving object at a volume determined by the volume determination unit, a sound quality determined by the sound quality determination unit, and a sound timing determined by the sound timing determination unit; Equipped with A movement sound generation system, characterized in that the volume determination unit determines the volume of the moving object so that the volume of the moving sound of the moving object becomes louder as the gravitational acceleration in the virtual space increases.
2. The movement sound generation system according to claim 1 , wherein the sound quality determination unit predicts the contact portion of the moving surface with which the grounding portion of the moving body will come into contact based on the movement information of the moving body, and pre-loads only the material information of the predicted contact portion of the moving surface.
3. The movement sound generating system according to claim 1 or 2, wherein the movement sound reproducing unit installs a virtual speaker that reproduces the movement sound of the moving object at a contact portion of the moving surface with which the ground contact portion of the moving object comes into contact.
4. The moving sound generation system according to any one of claims 1 to 3, wherein the sound generation timing determination unit determines the timing at which the grounding portion of the moving body contacts the moving surface based on the timing at which the potential energy of the grounding portion of the moving body becomes minimum.
5. A method for generating a moving sound when a moving object moves on a predetermined moving surface in a virtual space, the method being executed in a moving sound generating system, comprising: determining a volume of the moving sound when a moving object moves on a moving surface in the virtual space based on gravitational acceleration in the virtual space; determining a sound quality of the moving sound based on material information of a grounded portion of the moving body that contacts a moving surface in the virtual space and material information of the moving surface with which the grounded portion of the moving body contacts; determining a timing for generating the moving sound based on a timing at which a ground contact portion of the moving object comes into contact with the moving surface; reproducing, in the virtual space, the sound of the moving object with the volume, sound quality, and sound timing determined in the step; Including, A method according to claim 1, wherein in the step of determining a volume of the moving sound, the volume of the moving body is determined such that the volume of the moving sound of the moving body becomes louder as the gravitational acceleration in the virtual space increases.
6. A program for generating a moving sound when a moving object moves on a predetermined moving surface in a virtual space, On the computer, determining a volume of the moving sound when a moving object moves on a moving surface in the virtual space based on gravitational acceleration in the virtual space; A process of determining a sound quality of the moving sound based on material information of a grounded portion of the moving object that contacts a moving surface in the virtual space and material information of the moving surface with which the grounded portion of the moving object contacts; A process of determining a timing of generating the moving sound based on a timing of contact of a ground contact portion of the moving object with the moving surface; a process of reproducing the moving sound of the moving object in the virtual space with the volume, sound quality, and sound timing determined in the process; Run the command, A program characterized in that, in the process of determining the volume of the moving sound, the volume of the moving sound of the moving object is determined so that the volume of the moving sound of the moving object becomes louder as the gravitational acceleration in the virtual space increases.
Citation Information
Patent Citations
Game apparatus and game program
JP2005218706A
Program, information storage medium, and image generation system
JP2010022646A
Game program and game device
JP2018201790A
Information processing device, information processing method, and program
WO2018186178A1
