Information processing device and information processing method

Abstract

An information processing device 12 comprises a reception unit 50, an answer acquisition unit 54, and an output unit 56. The reception unit 50 of the information processing device 12 receives data relating to a basic sound effect. The answer acquisition unit 54 of the information processing device 12 acquires data relating to a variation sound effect with respect to the basic sound effect, the data being generated by inputting input information including the data relating to the basic sound effect to generative AI. The output unit 56 of the information processing device 12 outputs the data relating to the variation sound effect generated by the generative AI.

Description

Information Processing Apparatus and Information Processing Method

[0001] The present disclosure relates to data processing technology, and particularly to an information processing apparatus and an information processing method.

[0002] In games, various sound effects are output. Game sound designers have had to create a large number of sound effects for each sound generation factor and combination of sound source objects.

[0003] Up to now, the burden on sound designers to create a large number of sound effects has been great. One object of the present disclosure is to provide a technology for efficiently creating a large number of sound effects.

[0004] To solve the above problems, an information processing apparatus according to an aspect of the present disclosure includes a CPU configured as follows. The CPU receives data of a basic sound effect, obtains data of a variation sound effect for the basic sound effect generated by inputting input information including the data of the basic sound effect to a music generation AI, and outputs the data of the variation sound effect.

[0005] Another aspect of the present disclosure is an information processing method. In this method, a computer receives data of a basic sound effect, obtains data of a variation sound effect for the basic sound effect generated by inputting input information including the data of the basic sound effect to a generation AI, and outputs the data of the variation sound effect.

[0006] Note that any combination of the above components and those obtained by converting the expression of the present disclosure among a system, a computer program, a recording medium storing the computer program, etc. are also effective as aspects of the present disclosure.

[0007] According to the present disclosure, it is possible to assist in efficiently creating a large number of sound effects.

[0008] It is a diagram showing the configuration of the information processing system of the first embodiment. It is a block diagram showing the functional blocks of the information processing apparatus of the first embodiment. It is a flowchart showing the operation of the information processing apparatus of the first embodiment. It is a block diagram showing the functional blocks of the information processing apparatus of the second embodiment.

[0009] <First Embodiment> Figure 1 shows the configuration of the information processing system 10 of the first embodiment. The information processing system 10 is a system that assists a user in efficiently creating a large number of sound effects. The user of the first embodiment is a sound designer (also called a sound creator) who creates various sound effects in a game. The information processing system 10 comprises an information processing device 12, a display device 14, a speaker 16, and a server 18.

[0010] The functions of the components in the information processing device 12 and the server 18 may be realized in circuits or processing circuits, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, which are configured or programmed to realize the functions described herein. A processor is considered to be a circuit or processing circuit that includes transistors and other circuits. A processor may also be a programmed processor that executes a program stored in memory.

[0011] In this specification, circuits, units, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0012] If the hardware is a processor that is considered to be a type of circuit, then the circuit, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0013] The information processing device 12 is an information processing device operated by a user. The information processing device 12 may be a PC, a tablet terminal, or a smartphone.

[0014] The display device 14 displays various types of information, including text and images, in accordance with the control of the information processing device 12. The display device 14 may include a liquid crystal display or an organic EL display. The speaker 16 outputs sound in accordance with the control of the information processing device 12.

[0015] Server 18 is an information processing device that stores the music generation AI model and provides a music generation service using the music generation AI to the information processing device 12, which acts as a client. The music generation AI model is a mathematical model created by machine learning and can also be described as a function approximator. Information processing device 12 is connected to server 18 via a communication network 20, which includes LAN, WAN, the Internet, etc.

[0016] The music generation AI of the first embodiment can also be called generative artificial intelligence, and it generates and outputs sound effects that are variations of a basic sound effect that has been input. For example, the music generation AI may compare the frequency waveform of the input basic sound effect with a learning set to classify the basic sound effect, and based on the classification result, it may generate sound effects that are variations of the basic sound effect. The variation sound effect can also be called a sound effect that is an arrangement of the basic sound effect. For example, the variation sound effect may be one in which at least one of the following has been changed in the basic sound effect: pitch (frequency of the sound wave), loudness (amplitude of the sound wave), or timbre (waveform of the sound wave).

[0017] As a variation, server 18 may store a Large Language Model (LLM) as a music generation AI. The LLM may generate information on variations of sound effects (e.g., sequences of notes) for input basic sound effect information (e.g., sequences of notes) according to the modification instructions indicated by the input prompt.

[0018] Figure 2 is a block diagram showing the functional blocks of the information processing device 12 according to the first embodiment. Each block shown in the block diagrams of this specification can be realized in hardware terms by a computer processor (CPU, etc.), memory and other elements, electronic circuits, and mechanical devices, and in software terms by a computer program, etc., but here we are depicting functional blocks that are realized through the cooperation of these. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by combinations of hardware and software.

[0019] The information processing device 12 comprises a processing unit 30, a storage unit 32, and a communication unit 34. The processing unit 30 performs various data processing operations. The storage unit 32 stores data that is referenced or updated by the processing unit 30. The communication unit 34 communicates with external devices according to a predetermined communication protocol. The processing unit 30 transmits data to the display device 14 and the speaker 16 via the communication unit 34. The processing unit 30 also sends and receives data with the server 18 via the communication unit 34.

[0020] The memory unit 32 includes an audio asset memory unit 40. The audio asset memory unit 40 stores audio assets. The audio assets of the first embodiment include data for multiple sound effects that are played in a game or the like. The sound effects include sounds that occur when a first object collides with or comes into contact with a second object. For example, the sound effects include the sound of a user character's hand colliding with a glass window. The sound effects also include the sound of a user character's footsteps when walking down a corridor or when walking up stairs.

[0021] The processing unit 30 includes a reception unit 50, a prompt generation unit 52, a response acquisition unit 54, and an output unit 56. A computer program implementing the functions of these multiple functional blocks may be installed in the storage of the information processing device 12. The processor of the information processing device 12 may perform the functions of these multiple functional blocks by reading this computer program into main memory and executing it.

[0022] The reception unit 50 receives basic sound effect data, which will serve as the basis for generating variations of sound effects. The prompt generation unit 52 generates input information (hereinafter also referred to as "prompt") that includes the basic sound effect data. In addition to the basic sound effect data, the prompt may also include instructions (text, etc.) for the music generation AI.

[0023] The response acquisition unit 54 inputs the prompt generated by the prompt generation unit 52 to the music generation AI on the server 18. The response acquisition unit 54 acquires data from the server 18 for variations of the basic sound effect generated by the music generation AI. The output unit 56 outputs the data for the variations of the sound effect acquired by the response acquisition unit 54.

[0024] Figure 3 is a flowchart illustrating the operation of the information processing device 12 of the first embodiment. The operation of the information processing system 10 of the first embodiment will be described below with reference to Figure 3.

[0025] The audio asset storage unit 40 of the information processing device 12 stores sound effect data created in advance by the user. The user selects sound effect data from the sound effect data stored in the audio asset storage unit 40 to be used as the basis for generating variations. The receiving unit 50 of the information processing device 12 reads the sound effect data selected by the user from the audio asset storage unit 40 as the basic sound effect data (S10).

[0026] The prompt generation unit 52 of the information processing device 12 generates a prompt that includes data for a basic sound effect (S11). The prompt generation unit 52 may also include in the prompt instructions (text, etc.) to arrange the basic sound effect and generate multiple variations of the basic sound effect.

[0027] The response acquisition unit 54 of the information processing device 12 sends the prompt generated in S11 to the server 18 and inputs the prompt to the music generation AI (S12). The server 18 inputs the prompt sent from the information processing device 12 to the music generation AI. Based on the prompt, the music generation AI may generate data for multiple variations of the basic sound effect by gradually changing at least one of the pitch, volume, and timbre of the basic sound effect.

[0028] Server 18 transmits data of multiple sound effect variations generated by the music generation AI to the information processing device 12. The response acquisition unit 54 of the information processing device 12 receives the data of multiple sound effect variations transmitted from Server 18 and stores it in the audio asset storage unit 40 (S13). The output unit 56 of the information processing device 12 transmits the data of the sound effect variations acquired in S13 to the speaker 16, causing the sound effect variations to be output from the speaker 16 (S14).

[0029] For example, the output unit 56 of the information processing device 12 may display a list of multiple sound effect variations stored in the audio asset storage unit 40 on the display device 14. The output unit 56 may also output a sound effect variation selected by the user from among the multiple sound effect variations stored in the audio asset storage unit 40 from the speaker 16.

[0030] According to the information processing device 12 of the first embodiment, it is possible to efficiently create a large number of sound effects necessary for the presentation of games and the like by utilizing generation AI, thereby reducing the burden on sound designers.

[0031] In S10 of Figure 3, the reception unit 50 of the information processing device 12 may further receive information (hereinafter also referred to as "modification instruction information") that the user has input, which instructs how to modify the basic sound effect. The modification instruction information may include instructions for changes to the material of the object that serves as the sound source for the basic sound effect (e.g., glass, cloth, marble, etc.). For example, the modification instruction information may include instructions that the material of the object that serves as the sound source for the basic sound effect (here, footsteps) is carpet, but that the material should be changed to marble for the variation sound effect. The modification instruction information may also include instructions on how to change at least one of the pitch, volume, and timbre of the sound in the basic sound effect (e.g., the degree or range of the change).

[0032] In S11 of Figure 3, the prompt generation unit 52 of the information processing device 12 may generate a prompt that includes change mode instruction information. In S12 of Figure 3, the response acquisition unit 54 of the information processing device 12 may send the prompt that includes change mode instruction information to the server 18 and further input the change mode instruction information to the music generation AI. The music generation AI generates sound effect data for variations of the basic sound effect according to the change mode indicated by the change mode instruction information.

[0033] In S13 of Figure 3, the response acquisition unit 54 may acquire data of sound effect variations that reflect the change patterns indicated by the change pattern instruction information, which have been generated by the music generation AI. According to the information processing device 12 of the first embodiment, it is possible to automatically generate a large number of sound effect variations that reflect the change patterns instructed by the user, thereby reducing the burden on the sound designer.

[0034] The present disclosure has been described above based on the first embodiment. The first embodiment is illustrative, and it will be understood by those skilled in the art that various modifications are possible in combinations of each component and each processing process, and that such modifications are also within the scope of the present disclosure.

[0035] <Second Embodiment> The second embodiment of this disclosure will be described focusing on the differences from the first embodiment. Components of the second embodiment that are the same as or equivalent to components of the first embodiment will be denoted by the same reference numerals as those of the first embodiment, and explanations that overlap with the description of the first embodiment will be omitted as appropriate.

[0036] The information processing system 10 of the second embodiment has the same configuration as the information processing system 10 of the first embodiment shown in Figure 1. However, since the information processing device 12 of the second embodiment has a function to create and store a model of the generated AI, the information processing system 10 of the second embodiment may be a standalone configuration of the information processing device 12.

[0037] Figure 4 is a block diagram showing the functional blocks of the information processing device 12 of the second embodiment. In addition to the functional blocks of the information processing device 12 of the first embodiment, the information processing device 12 of the second embodiment includes a training data storage unit 42, a model storage unit 44, a training data acquisition unit 58, and a learning unit 60.

[0038] The training data storage unit 42 stores training data used for machine learning to create a model for the music generation AI. The training data includes multiple sets of sound effect data and information indicating the characteristics of the sound effect (hereinafter also called "tags"). Tags are also called labels and may include at least one of the following: the type of sound effect (e.g., "footsteps," "sound of glass breaking," etc.) and information about the object that is the sound source (e.g., material, hardness, speed, etc.). The model storage unit 44 stores data of the music generation AI model generated by the learning unit 60.

[0039] The training data acquisition unit 58 acquires multiple sets of sound effect data and tags associated with each other, which are input or specified by the user, as training data, and stores them in the training data storage unit 42.

[0040] The learning unit 60 reads out the teacher data stored in the teacher data storage unit 42, and creates data of a model of a music generation AI that generates data of variation sound effects for the data of basic sound effects by performing known machine learning based on the read teacher data. The music generation AI generated by the learning unit 60 may classify the features of the input basic sound effects and generate data of variation sound effects for the basic sound effects based on the classification result, as described in the first embodiment. The learning unit 60 stores the created data of the model of the music generation AI in the model storage unit 44.

[0041] The response acquisition unit 54 of the information processing apparatus 12 according to the second embodiment inputs a prompt including data of basic sound effects to the music generation AI stored in the teacher data storage unit 42, and acquires data of variation sound effects generated by the music generation AI.

[0042] According to the information processing apparatus 12 of the second embodiment, in addition to the effects produced by the information processing apparatus 12 of the first embodiment, by using a music generation AI based on the user's voice assets (for example, a plurality of sound effects used in past games, etc.), it has the effect of making it easier to obtain data of variation sound effects obtained by appropriately arranging the basic sound effects.

[0043] As described above, the present disclosure has been described based on the second embodiment. It is understood by those skilled in the art that the second embodiment is an example, and various modifications are possible for the combination of each component and each processing process, and such modifications are also within the scope of the present disclosure.

[0044] In the second embodiment, the learning unit 60 of the information processing apparatus 12 performs supervised learning. As a modification, the learning unit 60 may perform semi-supervised learning. That is, the learning unit 60 may create a music generation AI by performing learning using both tagged sound effect data and untagged sound effect data.

[0045] Any combination of the above-described embodiments and modifications is also useful as an embodiment of the present invention. The new embodiments resulting from these combinations possess the combined effects of each of the embodiments and modifications. Furthermore, it will be understood by those skilled in the art that the functions to be performed by each component described in the claims can be achieved by each component shown in the embodiments and modifications individually or in combination thereof.

[0046] The technical concepts described in the above embodiments and modifications can be expressed as follows: [Technology 1] An information processing device comprising a circuit configured to the following, wherein the circuit receives basic sound effect data, acquires variation sound effect data for the basic sound effect generated by inputting input information including the basic sound effect data to a music generation AI, and outputs the variation sound effect data. [Technology 2] The information processing device according to Technology 1, wherein the circuit further receives information indicating a modification of the basic sound effect, and the circuit further inputs the indicated information to the music generation AI to acquire variation sound effect data reflecting the modification. [Technology 3] The information processing device according to Technology 2, wherein the indicated information indicates a modification of the material of the object that serves as the sound source for the basic sound effect. [Technology 4] The information processing device according to Technology 1, wherein the circuit acquires a plurality of sets of sound effect data and information indicating characteristics related to the sound effect as training data, and the circuit creates a model of the music generation AI by learning based on the training data. [Technical 5] The information processing apparatus according to Technical 4, wherein the information describing the features includes information about the material of the object that serves as the sound source for the sound effect. [Technical 6] An information processing method comprising: a computer receiving basic sound effect data, inputting input information including the basic sound effect data into a music generation AI to obtain data of a variation sound effect to the basic sound effect, and outputting the data of the variation sound effect.

[0047] The technology disclosed herein can be applied to information processing devices, information processing systems, and the like.

[0048] 10 Information processing system, 12 Information processing device, 16 Speaker, 18 Server, 50 Reception unit, 52 Prompt generation unit, 54 Response acquisition unit, 56 Output unit, 58 Training data acquisition unit, 60 Learning unit.

Claims

1. An information processing device comprising a CPU configured as follows, wherein the CPU receives basic sound effect data, acquires variation sound effect data for the basic sound effect generated by inputting input information including the basic sound effect data to a music generation AI, and outputs the variation sound effect data.

2. The information processing apparatus according to claim 1, wherein the CPU further receives information indicating the manner in which the basic sound effect should be modified, and the CPU further inputs the information to the music generation AI to obtain data of the variation sound effect that reflects the modified manner.

3. The information processing apparatus according to claim 2, wherein the information to be instructed instructs a change in the material of the object that serves as the sound source for the basic sound effect.

4. The information processing apparatus according to any one of claims 1 to 3, wherein the CPU acquires a plurality of sets of sound effect data and information indicating characteristics related to the sound effect as training data, and the CPU creates a model of the generative AI by learning based on the training data.

5. The information processing apparatus according to claim 4, wherein the information describing the features includes information about the material of the object that serves as the sound source for the sound effect.

6. An information processing method comprising: a computer receiving basic sound effect data; acquiring variation sound effect data for the basic sound effect generated by inputting input information including the basic sound effect data into a generating AI; and outputting the variation sound effect data.

Images