Information processing device

Abstract

To provide an information processing device for mixing effect sound natural for a listener with music without applying a load on audio equipment.SOLUTION: An effect sound mixing system has an effect sound information embedding device and an effect sound mixing device. A control section of the effect sound information embedding device has: a music feature acquisition section 111 for acquiring a feature of music to be obtained by analyzing audio data of the music; an effect sound determination section 112 for determining a type of the effect sound to be mixed with the music and a method for mixing the effect sound based on the feature of the music; and a data generation section 113 for embedding the effect sound information indicating the determined type of the effect sound and the mixing method in the audio data of the music so as to generate embedded audio data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device. [Background technology]

[0002] There is known a technology that mixes sound effects into music to create the atmosphere of a live concert venue. For example, Patent Document 1 discloses a karaoke sound effect system, in which the type of sound effect is set according to the genre of the music, and the output mode of the sound effect (the number of people clapping and cheering) is set according to the size of the selected live concert venue. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-70999 Summary of the Invention [Problem to be solved by the invention]

[0004] At a live concert venue, the volume and volume of the audience's cheers and applause will vary depending on the characteristics of the song, such as its rhythm and melody. By mixing sound effects that correspond to the characteristics of the song into the music, it is possible to make the listener feel more like they are at a live concert.

[0005] The process of analyzing music to obtain its characteristics and the process of determining sound effects that match the characteristics of the music are heavy-duty processes, and are better performed on devices with high processing power, such as smartphones, rather than on audio equipment such as car stereos.

[0006] One example of a problem that the present invention aims to solve is to mix sound effects into music that sound more natural to the listener without placing a load on the audio equipment. [Means for solving the problem]

[0007] In order to solve the above problem, the invention described in claim 1 comprises a music feature acquisition unit that acquires features of a music piece obtained by analyzing audio data of the music piece; a sound effect determination unit that determines the type of sound effect to be mixed into the music piece and how to mix the sound effects based on the features of the music piece; and a data generation unit that embeds sound effect information indicating the determined type of sound effect and how to mix the sound effects into the audio data of the music piece, and generates embedded audio data.

[0008] The invention recited in claim 2 comprises a sound effect information reading unit that reads sound effect information from embedded audio data generated by embedding sound effect information indicating the type of sound effect to be mixed into music and how the sound effect is to be mixed into audio data of the music, a storage unit that stores audio data of the sound effects, and a mixing unit that mixes the audio data of the sound effects into the embedded audio data based on the read sound effect information.

[0009] The invention recited in claim 4 is an information processing method executed by a computer, comprising: a sound effect information reading step of reading sound effect information from embedded audio data generated by embedding sound effect information indicating types of sound effects to be mixed into music and how the sound effects are to be mixed into the audio data of the music; and a mixing step of mixing audio data of sound effects into the embedded audio data based on the read sound effect information.

[0010] The invention described in claim 5 causes the information processing method described in claim 4 to be executed by a computer.

[0011] The invention as recited in claim 6 stores the information processing program as recited in claim 5. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating a sound effect mixing system according to an embodiment of the present invention. [Figure 2]FIG. 2 is a diagram illustrating a control unit 110 according to an embodiment of the present invention. [Figure 3] FIG. 10 is a diagram showing an example of how sound effects are mixed. [Figure 4] 1 is a diagram illustrating a control unit 210 of a sound effect mixing apparatus 200 according to an embodiment of the present invention. [Figure 5] 1 is a diagram illustrating the relationship between a sound effect information embedding device 100 and a sound effect mixing device 200. FIG. [Figure 6] FIG. 2 is a diagram illustrating an example of a processing operation in the sound effect mixing system according to an embodiment of the present invention. [Figure 7] FIG. 1 is a diagram illustrating an example of the configuration of a sound effect mixing system. [Figure 8] FIG. 1 is a diagram illustrating an example of the configuration of a sound effect mixing system. [Figure 9] FIG. 10 is a diagram illustrating an example of embedding sound effect information. DETAILED DESCRIPTION OF THE INVENTION

[0013] An information processing device according to one embodiment of the present invention includes a music feature acquisition unit that acquires music features obtained by analyzing the audio data of the music; a sound effect determination unit that determines, based on the music features, the type of sound effect to be mixed into the music and how the sound effects will be mixed; and a data generation unit that embeds sound effect information indicating the determined type of sound effect and how the sound effects will be mixed into the audio data of the music to generate embedded audio data. In this manner, in this embodiment, embedded audio data with embedded sound effect information is generated in advance. This makes it possible to separate a device that performs high-load processes such as analyzing music and determining sound effects from a device that performs processing to mix sound effects into music. As a result, it is possible to mix sound effects into music in a way that sounds more natural to the listener without placing a burden on the audio equipment.

[0014] An information processing device according to one embodiment of the present invention includes a sound effect information reading unit that reads sound effect information from embedded audio data generated by embedding sound effect information indicating the type of sound effect to be mixed into music and how the sound effect is to be mixed into the audio data of the music, a storage unit for storing sound effect audio data, and a mixing unit that mixes the sound effect audio data with the embedded audio data based on the read sound effect information. In this manner, in this embodiment, embedded audio data with embedded sound effect information is generated in advance. This makes it possible to mix sound effects into music that sound more natural to the listener without placing a load on the audio equipment.

[0015] The sound effect information may be embedded in an area of ​​the audio data that is inaudible to humans, so that listeners can listen to the music without feeling uncomfortable even if sound effect information is embedded in the audio data.

[0016] Furthermore, an information processing method according to one embodiment of the present invention is an information processing method executed by a computer, and includes: a sound effect information reading step of reading sound effect information from embedded audio data generated by embedding sound effect information indicating the type of sound effect to be mixed into music and how the sound effect is to be mixed into the audio data of the music, and a mixing step of mixing audio data of the sound effect into the embedded audio data based on the read sound effect information. As such, in this embodiment, embedded audio data with embedded sound effect information is generated in advance. This makes it possible to mix sound effects into music that sound more natural to the listener without placing a load on the audio equipment.

[0017] An information processing program according to an embodiment of the present invention causes a computer to execute the above-described information processing method, thereby enabling the computer to mix sound effects into music that sound more natural to the listener without placing a load on the audio equipment.

[0018] Furthermore, a computer-readable storage medium according to an embodiment of the present invention stores the information processing program, which allows the information processing program to be distributed as a standalone program in addition to being incorporated into a device, and makes it easy to perform version upgrades, etc. [Example]

[0019] <Sound effect mixing system> 1 is a diagram showing a sound effect mixing system according to an embodiment of the present invention, which includes a sound effect information embedding device 100 and a sound effect mixing device 200.

[0020] The sound effect information embedding device 100 includes a control unit 110, a storage unit 120, and an audio signal output unit 130. The control unit 110 is configured by, for example, a computer having a CPU and the like.

[0021] The storage unit 120 stores audio data of songs. The storage unit 120 is, for example, a storage device such as a hard disk or memory.

[0022] The audio signal output unit 130 outputs an audio signal of a song. The audio signal output unit 130 generates an audio signal of a song from audio data of a song stored in the storage unit 120, a CD (Compact Disc), the cloud, or the like, or from audio data generated by the control unit 110, and outputs the generated audio signal of the song.

[0023] 2 is a diagram showing a control unit 110 according to an embodiment of the present invention. The control unit 110 has a music feature acquisition unit 111, a sound effect determination unit 112, a data generation unit 113, and an output processing unit 114.

[0024] The music feature acquisition unit 111 acquires music feature such as the volume of the music, the position of the beats in the music, the number of beats per unit time (e.g., BPM (Beats Per Minute)), the time signature of the music, the number of types of chords used in the music, the number of chords per unit time, the clarity of the chords, the power of each band, the position of the chorus of the music, etc.

[0025] The music feature acquisition unit 111 may acquire music feature by analyzing the music, or music feature obtained by a prior analysis may be stored in the storage unit 120 or the cloud, and the music feature acquisition unit 111 may acquire music feature stored in the storage unit 120 or the cloud. The music feature acquisition unit 111 may also acquire music feature from tag information attached to music data stored in the storage unit 120, a CD, or the like.

[0026] The sound effect determination unit 112 determines the type of sound effect to be mixed into the music and how to mix the sound effects based on the music characteristics acquired by the music characteristic acquisition unit 111. Examples of sound effects include first sound effects such as cheers and applause that occur at the beginning and end of a song at a live venue, second sound effects such as environmental sounds (bustling sounds) that are constantly generated at a live venue, and third sound effects such as clapping in time with the rhythm and beat of the music at a live venue. Each sound effect may also be provided in multiple versions depending on the size of the live venue. FIG. 3 is a diagram showing an example of how sound effects are mixed. In the example shown in FIG. 3, first sound effects (such as cheers and applause) are mixed into the beginning and end of the music. Second sound effects (such as environmental sounds) begin to be mixed before the music starts and continue to be mixed throughout the music being played. Third sound effects (such as clapping) are mixed into the music while it is being played.

[0027] The data generation unit 113 embeds sound effect information, which indicates the type and mixing method of the sound effects determined by the sound effect determination unit 112, into the audio data of the music piece to generate embedded audio data. In other words, the data generation unit 113 does not mix the audio data of the sound effects themselves into the audio data of the music piece, but rather embeds sound effect information, which indicates what kind of sound effects are to be mixed and how, into the audio data of the music piece itself by overwriting part of the audio data of the music piece, thereby generating embedded audio data with the embedded sound effect information. Here, the audio data of the music piece into which sound effect information is embedded is uncompressed audio data such as PCM format, and if a compressed file of the audio data of the music piece (e.g., an MP3 file) is stored in the storage unit 120, the data generation unit 113 decompresses this compressed file to generate uncompressed audio data, and then embeds the sound effect information.

[0028] The output processing unit 114 generates an audio signal from the embedded audio data generated by the data generating unit 113 , and the audio signal output unit 130 outputs the generated audio signal to the sound effect mixing device 200 .

[0029] The sound effect mixing device 200 includes a control unit 210, a storage unit 220, an audio signal input unit 230, and a mixing unit 240. The control unit 210 is configured by, for example, a computer having a CPU and the like.

[0030] The storage unit 220 stores audio data for sound effects. When there are multiple types of sound effects, the storage unit 220 stores audio data for each type. The storage unit 220 is, for example, a storage device such as a hard disk or memory.

[0031] The audio signal input unit 230 receives the audio signal output from the sound effect information embedding unit 100 .

[0032] The mixer 240 mixes the audio signal input to the audio signal input unit 240 with the audio signal generated from the audio data of the mixed sound stored in the storage unit 120, and outputs the mixed audio signal.

[0033] 4 is a diagram showing the control unit 210 of the sound effect mixing device 200 according to an embodiment of the present invention. The control unit 210 includes a sound effect information reading unit 211 and a mixing processing unit 212.

[0034] The sound effect information reading unit 211 reads sound effect information from the audio signal input to the audio signal input unit 230. If the audio signal output from the sound effect embedding device 100 is an audio signal of embedded audio data, sound effect information has been embedded in this audio signal. The sound effect information reading unit 211 reads this embedded sound effect information from the audio signal.

[0035] Based on the sound effect information read by the sound effect information reading unit 211, the mixing processing unit 212 mixes the audio signal input to the audio signal input unit 230 with an audio signal generated from audio data of the sound effect using the mixing unit 240, and outputs the mixed audio signal. The sound effect information indicates the type of sound effect to be mixed into the music and the mixing method. For example, the mixing processing unit 212 reads audio data of the sound effect type indicated in the sound effect information from the storage unit 220, generates an audio signal of the sound effect from the read audio data of the sound effect, and outputs the generated audio signal of the sound effect to the mixing unit 240 based on the mixing method indicated in the sound effect information. The output audio signal of the sound effect is mixed in the mixing unit 240 with the audio signal input to the audio signal input unit 230 based on the mixing method indicated in the sound effect information.

[0036] 5, in this embodiment, a sound effect mixing device 200 receives an audio signal of embedded audio data generated by a sound effect information embedding device 100 and mixes sound effects into the received audio signal. Then, for example, the mixed audio signal with the sound effects mixed therein is converted into an analog signal by a DAC (Digital Analog Converter), this analog signal is amplified by an amplifier, and sound based on the amplified analog signal is output from a speaker.

[0037] As described above, in this embodiment, embedded audio data with embedded sound effect information is generated in advance, so it is possible to separate a device (sound effect information embedding device 100) that performs high-load processes such as analyzing music and determining sound effects from a device (sound effect mixing device 200) that mixes sound effects into music. Therefore, in this embodiment, high-load processes are performed by a device with high processing power, such as a smartphone, thereby avoiding placing a load on audio equipment such as car audio. It is possible to mix sound effects that match the characteristics of music into music without placing a load on audio equipment such as car audio. As a result, it is possible to mix sound effects into music that sound more natural to the listener without placing a load on the audio equipment.

[0038] Furthermore, in this embodiment, only audio signals are sent from the sound effect output device 100 to the sound effect mixing device 200, and sound effects can be mixed into music without communicating any other information between the sound effect output device 100 and the sound effect mixing device 200.

[0039] 6 is a diagram showing an example of processing operations in the sound effect mixing system according to this embodiment. The music feature acquisition unit 111 acquires the features of the music (step S601). The sound effect determination unit 112 determines the type of sound effect to be mixed into the music and how to mix the sound effects based on the music features acquired by the music feature acquisition unit 111 (step S602). The data generation unit 113 embeds sound effect information indicating the type of sound effect and how to mix the sound effects determined by the sound effect determination unit 112 into the audio data of the music to generate embedded audio data (step S603). The output processing unit 114 generates an audio signal from the embedded audio data generated by the data generation unit 113, and the audio signal output unit 130 outputs the generated audio signal to the sound effect mixing device 200 (step S604). The sound effect information reading unit 211 reads sound effect information from the audio signal input to the audio signal input unit 230 (step S605). The mixing processor 212 mixes the sound effect with the audio signal input to the audio signal input unit 230 based on the sound effect information read by the sound effect information reader 211 (step S606).

[0040] <Configuration of sound effect mixing system> The sound effect mixing system according to this embodiment can be configured, for example, by a smartphone and a car audio device as shown in FIG. 7(A), or by a smartphone and a home audio device as shown in FIG. 7(B). In this case, as shown in FIG. 7, the smartphone has the function of a sound effect information embedding device 100, and the car audio device or home audio device has the function of a sound effect mixing device 200. In this way, it is possible to mix sound effects into music that are more natural for the listener without placing a load on the car audio device or home audio device. Furthermore, only audio signals are sent from the smartphone to the car audio device or home audio device, making it possible to mix sound effects into music without communicating any other information between the smartphone and the car audio device or home audio device.

[0041] 7, when a car audio system or a home audio system has the function of sound effect mixing device 200, mixing unit 240 may also perform processing to multi-channelize the audio signal and output the multi-channel audio signal. In this way, by changing the audio system connected to the smartphone, it becomes possible to realize a system with any channel configuration.

[0042] Furthermore, the sound effect mixing system according to this embodiment may be configured with a cloud and a smartphone, as shown in Fig. 8(A). In this case, as shown in Fig. 7, the cloud has the functions of the sound effect information embedding device 100, and the smartphone has the functions of the sound effect mixing device 200. In this way, it is possible to avoid placing a load on the smartphone. Note that, as shown in Fig. 8(B), the entire sound effect mixing system may be configured to be included in the smartphone.

[0043] <Embedding sound effect information> Fig. 9 is a diagram showing an example of embedding sound effect information. As shown in Fig. 9, in this embodiment, sound effect information is embedded in audio data by rewriting part of the audio data with the sound effect information. Fig. 9 shows, as an example, audio data with a quantization bit rate of 16 bits.

[0044] The sound effect mixing device 200 may have a function (audio data editing unit) of cutting out the portion where the sound effect information is embedded from the audio signal of the embedded audio data after reading the sound effect information, and may output the audio signal from which the portion where the sound effect information is embedded has been cut out to a DAC, etc. This allows the listener to listen to the music without feeling unnatural.

[0045] Furthermore, the data generating unit 113 may embed sound effect information in a region of the audio data that is inaudible to humans. In this way, even if sound effect information is embedded in the audio data, the listener can listen to the music without feeling uncomfortable. For example, in the example shown in FIG. 9, the sound effect information is embedded in the least significant bit portion of the audio data. That is, in the example shown in FIG. 9, the sound effect information is embedded in a small signal region where the sound is too quiet for humans to hear. Furthermore, the sound effect information may be embedded in the audio data as an ultra-high frequency signal that is almost inaudible to humans.

[0046] The present invention has been described above in terms of preferred embodiments thereof. While the present invention has been described herein with reference to specific examples, various modifications and variations can be made to these examples without departing from the spirit and scope of the present invention as set forth in the claims. [Explanation of symbols]

[0047] 100 Sound effect information embedding device 110 control section 111 Music feature acquisition unit 112 Sound effect determination section 113 Data Generation Unit 114 Output Processing Unit 120 Storage section 130 Audio signal output section 200 Sound effect mixing device 210 Control Unit 211 Sound effect information reading unit 222 Mixing Processing Unit 220 Storage section 230 Audio signal input section 240 Mixing section

Claims

1. a music feature acquisition unit that acquires features of a music piece obtained by analyzing audio data of the music piece; a sound effect determination unit that determines the type of sound effect to be mixed with the music piece and how to mix the sound effect based on the characteristics of the music piece; and a data generating unit that embeds the determined sound effect information, which indicates the type of sound effect and how it is mixed, into the audio data of the music as an ultra-high frequency signal that is almost inaudible to humans, and generates embedded audio data.

2. a sound effect information reading unit that reads sound effect information from embedded audio data generated by embedding sound effect information indicating the type of sound effect to be mixed into the music piece and how the sound effect is to be mixed into the audio data of the music piece; a storage unit for storing audio data of sound effects; a mixing unit that mixes audio data of sound effects with the embedded audio data based on the read sound effect information, The information processing device, wherein the sound effect information is embedded in the audio data of the music as an ultra-high frequency signal that is almost inaudible to humans.

3. 1. A computer-implemented information processing method, comprising: a sound effect information reading step of reading the sound effect information from embedded audio data generated by embedding sound effect information indicating the types of sound effects to be mixed into the music piece and how the sound effects are to be mixed into the audio data of the music piece; a mixing step of mixing the embedded audio data with audio data of sound effects based on the read sound effect information, The information processing method, wherein the sound effect information is embedded in the audio data of the music piece as an ultra-high frequency signal that is almost inaudible to humans.

4. An information processing program that causes a computer to execute the information processing method according to claim 3.

5. A computer-readable storage medium storing the information processing program according to claim 4.

Images