Streaming sound production system

Abstract

To provide a system that records a live musical performance in a musical performance venue and produces a sound source for distribution in a short time.SOLUTION: A sound source production system comprises: a first microphone 1 which gathers data on music played in a musical performance venue; an arbitrary number of second microphones 2 which are two microphones or more and provided facing mutually different directions in the musical performance venue, and gather reverberation data; and a recording device 3. The recording device 3 comprises: data acquisition means 301 which acquires the data that the microphones 1, 2 gather; sound source separation means 302 which separates the data on music, part by part; editing means 303 which edits the data on the respective separated parts and the reverberation data; low-frequency data gathering means 304 which gathers non-audible low-frequency vibration data; and sound source production means 305 which synthesizes the respective data after the editing and adds the low-frequency vibration data to the synthesized data to produce a sound source for distribution.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This relates to a system that records live performances at concert venues and creates audio sources for distribution in a short period of time. [Background technology]

[0002] These are difficult times for those involved in the music industry. CD and DVD sales are declining due to the spread of online streaming and smartphones. Revenues from online streaming are also declining due to the availability of free video streaming sites. Furthermore, the number of music programs on television has drastically decreased. Where should people in the music industry look for a way out? There are two possibilities.

[0003] One option is to spread the music in some way. However, studio recording is costly. Also, even if you incur the expense of producing CDs, if they don't sell, you'll end up with a large amount of inventory and could end up in the red. Unless the song is a huge hit, producing CDs is not profitable. If that's the case, you should consider creating them cheaply and distributing them online rather than on CDs.

[0004] The other is to make live performances the focus of an artist's activities. Neither CDs nor television can support artists anymore. In this situation, live performances are the last resort for artists. The performers are immersed in the music during a live performance, each trying to convey their passion for the music to the audience. Meanwhile, the audience forgets their everyday worries for the moment and concentrates on the live performance. Not a single person in the audience looks like they're attending a wake. Everyone is trying to take the performers' feelings seriously. At a live performance, performers and audience share time and space with music as the one and only medium.

[0005] As such, the inventor has always wanted people to rediscover the power of music through live performances. However, for people who are busy with work, study, or caring for loved ones, attending a live concert can be difficult. Therefore, he wanted people to be able to enjoy the immersive feel of a live performance through online streaming. Of course, artists also have to make a living, so they have no choice but to charge for streaming. However, he wanted to keep the cost as low as possible. To achieve this, he wanted to produce streaming audio at low cost and deliver it at a low price.

[0006] Incidentally, there are already services available that allow recording at low cost and in a short period of time (for example, Non-Patent Document 1). This involves bringing sound recording equipment to the performance venue and carrying out the entire process from recording to recording at the venue, and is also known as live recording. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Live Recording | Band Recording, Music Video Production, Live Video Production by Ato4sound Factory (ato4sound.com), [Searched January 31, 2023], Internet<URL:https: / / band.ato4sound.com / liverec / > Summary of the Invention [Problem to be solved by the invention]

[0008] CDs and DVDs of live performances have been around for a long time. However, they tend to lack realism and impact. With the advances in digital technology today, music has become nothing more than information and data. It is processed digitally to sound beautiful and pleasant to the ear. This is fine if it is simply background music. However, for a live performance, noise is also necessary. In the analog era, information that is discarded or smoothed out by current digital technology was retained. In this way, the sound was richer because noise was also retained. After all, noise is necessary.

[0009] Well, there are two types of noise. The first is reverberation that occurs in live venues, and the second is low frequencies in the inaudible range. The latter are low frequencies below 13 Hz that were cut out in conventional CDs. Perhaps it was thought that cutting them out would be acceptable because they are inaudible. However, the inventor believes that long wavelength waves may have meaning in music. Performers have a strong desire to convey their feelings to the audience through their performance. It is difficult to say what the scientific basis for this is, but it may be that the performer's heartbeat and breathing at this time are carried on long wavelength waves and transmitted to the audience as the performer's feelings. In this regard, in the service described in Non-Patent Document 1, low-frequency data in the inaudible range is not added to the sound source. Now, imagine an outdoor heavy metal concert. Young people are pumping their hands in the air, exploding with energy and intoxication. But if the deep bass was cut out from the roaring sound, if it was just mid- to high-range sounds, would young people be able to get into the music? This is highly questionable.

[0010] Based on the above, the present invention aims, first, to record live performances on the spot and produce audio sources for distribution at low cost and in a short period of time, and second, to eliminate excessive processing of original audio data using digital technology. [Means for solving the problem]

[0011] The distribution sound source production system for solving the problems of the present invention comprises: a first microphone for collecting data on a piece of music being performed at a performance venue; two or more second microphones, each facing a different direction in the performance venue, for collecting reverberation data; Equipped with recording equipment installed in the venue, The recording device a data acquisition means for acquiring data collected by the first microphone and the second microphone; a sound source separation means for separating the music data into parts; editing means for editing the separated data of each part and the reverberation data; Inaudible Low Frequency Data a low frequency data collection means for collecting the The edited data are synthesized, and Low Frequency Data a distribution sound source production means for producing a distribution sound source by adding the above to the synthesized data; The present invention is characterized by comprising:

[0012] This allows you to record live performances at live venues and create audio files for distribution in a short period of time. Low Frequency Data The produced sound source also includes reverberation data, so the sense of realism is overwhelmingly superior to content that has been subjected to excessive digital processing.

[0013] The present invention further comprises an audience reaction detection device for detecting audience reactions, Based on the reaction, it may be determined whether or not to create a sound source for distribution from the played music piece, and the sound source may be transmitted to the recording device.

[0014] This is a kind of popularity contest, and songs that are popular with the audience are given priority for recording. This is because a large number of songs are performed at live shows, and it may not be possible to distribute all of them. [Effects of the Invention]

[0015] When a live performance is recorded at a concert venue, the recording process is done immediately on the spot, allowing for the production of audio for distribution quickly and inexpensively. Digital technology tends to remove noise excessively, but it is important to remove noise moderately ( Low Frequency Data This preserves the sound and reverberation data, so the sense of realism is not lost. Since it is possible to objectively grasp the audience's reaction during the performance, it is possible to select the most popular songs to distribute from a large number of performances. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram illustrating a system configuration of a first embodiment. [Figure 2] FIG. 10 is a diagram illustrating a system configuration of a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] A system according to an embodiment of the present invention will now be described. This system involves bringing sound-collecting equipment such as microphones and recording devices to a live venue, inputting audio data in real time during the performance, and then creating audio sources for distribution. Although a live performance offers not only musical enjoyment but also visual enjoyment, the present invention does not take into consideration images (including both moving and still images).

[0018] (First embodiment) A basic embodiment of the present invention (hereinafter referred to as the present system) will be described with reference to FIG. Figure 1 shows the configuration of this system. This system consists of a first microphone 1 that collects data on the music being performed in the concert hall, two or more second microphones 2 that are installed facing different directions in the concert hall and collect reverberation data, a recording device 3, and a distribution server 4. While Figure 1 shows one first microphone 1 and two second microphones 2, there can be more than this number. For example, in a venue where the stage is surrounded by audience seating on all sides, two second microphones 2 would not be enough. The recording device 3 acquires live performance data, separates and edits each part, and then extracts the reverberation data, which is noise, from the inaudible data. Low Frequency Data This is a device that adds these to ultimately generate a stereo sound source (2mix). The distribution server 4 stores the audio source for distribution created by the recording device 3, and transmits it to the audio playback device 5 such as the viewer's mobile terminal or personal computer in response to access from the viewer.

[0019] The recording device 3 includes a data acquisition means 301 , a sound source separation means 302 , an editing means 303 , a low-frequency data collection means 304 , and a sound source production means 305 . The data acquisition means 301 receives data collected by the first microphone 1 and the second microphone 2 . The sound source separation means 302 separates the audio data of the music piece picked up by the first microphone 1 into parts. The editing means 303 edits the data of each part separated by the sound source separation means 302 and the reverberation data collected by the second microphone 2 . The low frequency data collection means 304 is a non-audible Low Frequency Data It can acquire data, modify the acquired data, or generate it using some algorithm. The sound source production means 305 synthesizes each piece of data edited by the editing means 303, and Low Frequency Data This is then added to the synthesized data to create a stereo sound source for distribution, a process known as trackdown (or mixdown). The recording device 3 also includes communication means for receiving data from microphones 1 and 2 and sending and receiving data via the Internet, operation means necessary for operation by the recording staff, screen display means, and storage means for storing the progress of the work and the final product, but descriptions of these will be omitted.

[0020] Next, we will explain the processing steps for tracking down a live performance into a stereo sound source (2mix) for distribution.

[0021] A first microphone 1 is placed in an appropriate location, such as the center between the stage and the audience seats, and the data picked up by the microphone 1 is transmitted to a recording device 3 (usually installed at the rear end of the audience seats). Any means of transmitting the sound source can be used as long as the required sound quality is ensured. For example, LINE (registered trademark) can be used to transmit audio data. The recording staff may use headphones (not shown) to monitor the sounds collected by the first microphone 1. The staff will carry out the following recording work, referring to the monitored sounds and their memories of the sounds.

[0022] A second microphone 2, pointed in a different direction (left or right), picks up sounds from the wings of the stage, the right and left sides of the audience seats, etc. It can also record audience cheers and noise caused by the building's structure and materials. This includes sounds other than the performance, such as sounds reflected by the walls, floor, and ceiling, sounds reflected off those reflections, and sounds generated by these reflections, known as echoes or reverberation. If necessary, a low-frequency microphone can be used as this second microphone 2, or it can be used in conjunction with a regular microphone. This reverberation is also transmitted to the recording device 3 in real time.

[0023] The music data acquired by the data acquisition means 301 of the recording device 3 is separated into parts by the sound source separation means 302 . Unlike studio recording, where vocals, bass, drums, etc. are recorded separately, this separation process is essential in this system.

[0024] The editing means 303 of the recording device 3 edits the data for each separated part. The present invention places importance on not processing the original music data more than necessary. However, if the data is left as is, it may be difficult to listen to due to issues such as volume balance. Therefore, editing work is performed, such as adjusting the volume and sound quality of each vocal and instrument within the necessary range. Since editing is essential in this system, the process of separating the original music data into parts was necessary. One of the features of this system is that it does not edit the audio data excessively, leaving noise intact. However, if the vocals are difficult to hear or the drums overwhelm the rest of the music, the original music data cannot be distributed as is. It goes without saying that some degree of editing is necessary. It's all about balance.

[0025] The low-frequency data collection means 304 also prepares inaudible low-frequency data. This low-frequency data may be the low-frequency portion contained in the entire music data before separation, or the low-frequency portion contained in one or more separated parts. It may also be the low-frequency portion itself cut from the original data, or it may be edited by emphasizing or weakening the cut portion. Alternatively, it may be data automatically generated by software independent of the original music data.

[0026] The reverberation data acquired by the data acquisition means 301 is also edited by the editing means 303 if necessary. At that time, low frequency parts may be cut from the original reverberation data. The cut parts may be edited as appropriate. The original reverberation data may be separated into two or more parts (for example, applause and cheers from the audience and sound reflections from facilities, etc.) and edited separately for each part. Editing reverberation data becomes necessary when the reverberation is too strong and makes it difficult to hear. It is possible to remove reverberation using digital technology, but this results in a bland sound and diminishes the sense of realism. However, cutting the reverberation increases the purity of the content, so it is difficult to say which is better. Here again, balance is an issue. One of the goals of this system is to create a sound that is close to the memory of being there as an audience member by achieving a good balance and mixing.

[0027] The sound source production means 305 synthesizes the data for each part that has been separated and edited, such as adjusting the volume and frequency band, with separately edited reverberation data and collected and appropriately edited low-frequency data to produce stereo sound source (2mix) data. The produced stereo sound source data is stored in the distribution server 4 as distribution content. This process of track-down can be carried out using equipment called a mixer, or it can also be done using dedicated software.

[0028] The above process is almost the same as the recording process disclosed in Non-Patent Document 1. However, it is possible to use existing technologies (e.g., "Sony Group Portal | Technology | Stories | AI Sound Source Separation (sony.com)" and "AI Sound Source Separation (sony.com)").<URL:https: / / www.sony.com / ja / SonyInfo / technology / stories / AI_Sound_Separation / > ') and low frequency component The difference is that additional data has been added. The latter in particular has the effect of enriching the original sound. Compared to regular CDs, which cut off frequencies below 13Hz, the bass has been strengthened, creating a more realistic sound. In order to deliver high quality sound, a high compression rate is required. Low Frequency Data This is because the file size of the audio data is large since it also contains the audio data.

[0029] With this system, live performance data is edited at the live venue, so audio files for distribution can be produced in a short time. In the earliest cases, distribution can be made on the same day or the next day. This means that the audience can listen to the performance again while the afterglow of the day is still fresh. In addition, many of the tasks involved in separating, editing, and synthesizing the original music data are performed by software rather than by human hands, which also reduces production costs. In other words, this system can provide live performances quickly and at low cost.

[0030] An example of how the present invention can be used is to use a live performance as an opportunity to release new songs, and immediately create and distribute music content. When tickets go on sale for a live concert, you can announce that a new song will be released and that a live recording of the song will be made. It is likely that many people will attend the concert in the hope of witnessing the first performance of the new song and possibly having their cheers recorded.

[0031] Another possibility is to try the following: after a performance, ask whether it is okay to broadcast the performance. If the audience's reaction is not good, play the same piece again. Or, if the audience gets excited and asks "one more time," play the same piece again and use this as the audio source for broadcast. The audience can choose the performance to be recorded.

[0032] Second Embodiment There is an existing system for streaming music performed at live shows. Usually, only a portion of the songs performed is streamed. Audience members are not informed of which songs will be streamed, and they look forward to what they will download. Usually, the decision on which songs to stream is made unilaterally by the artists or other organizers of the live show. In this embodiment, performances that are judged to be well-received by the audience are ranked based on their reactions, and a predetermined number of the top performances are selected for distribution. The method for producing audio sources for distribution is almost the same as in the first embodiment. However, since different volumes and sound qualities for each song can be difficult to listen to, adjustments must be made. This process is performed on the stereo sound source after the trackdown of each song is complete.

[0033] As shown in FIG. 2, this embodiment is characterized in that an audience reaction detection device 6 is added to the system configuration of FIG. The audience response detection device 6 in Fig. 2 comprises one or more cameras 601 that capture images of the audience seats, a computer 602 that performs judgment processing, and communication means 603 that transmits the judgment results to the recording device 3 or the like. The computer 602 may be installed in the live venue, or it may be located outside and connected to the equipment at the live venue via a communication line such as the Internet.

[0034] Each spectator will be asked to carry a portable light-emitting device such as a penlight or glow stick. When the audience is moved or excited, they will wave penlights and other light sticks. Therefore, a camera 601 (or multiple cameras) captures the audience seats. A computer 602 then counts the penlights and other light sticks that are being waved by the audience in the captured video. Since stationary lights may be emergency lights in the live venue, only light sticks that move a certain distance and return are counted. The ratio of the number of moving lights to the number of audience members is used as the audience popularity of the song. The songs are arranged in descending order of popularity, and only the number of songs scheduled for distribution is targeted for distribution. Detecting moving light from camera footage is possible using existing technology, so details will be omitted here.

[0035] There are several other ways to detect audience reactions besides those mentioned above. One method is based on the applause of the audience. This can be achieved using audio acquired from the second microphone 2 or similar. Here, not only absolute volume but also the change in volume before and after applause is used as a basis for judgment. The songs are then sorted in descending order of volume, and only the number of songs planned for distribution is targeted for distribution. This can also be easily achieved using existing technology.

[0036] The above first and second embodiments are merely examples. The key features of this invention are that it records music performed at a live venue and immediately produces audio for distribution, that the audio for distribution contains a balanced amount of noise, and that it takes into account the reaction of the audience.

[0037] There are various ways to sell the produced audio material. You can purchase a card with a QR code (registered trademark) that can access a content sales server at a music store or convenience store, and download the content from this content distribution server. You can also purchase it from a website set up by the artist, or you can purchase it within an app that operates an online shop. [Industrial Applicability]

[0038] The songs performed at the live concert are distributed in a short time, either on the same day or the next day, and because they are full of realism, it is expected to attract demand from audiences who are still excited about the live concert. Even if they were not in attendance on the day, it also has the effect of attracting viewers who have heard the powerful sound source to the actual live concert. [Explanation of symbols]

[0039] 1: First microphone 2: Second microphone 3: Recording equipment 4: Distribution server 5: (User) audio playback device 6: Audience reaction detection device 7: (User) portable light-emitting device

Claims

1. a first microphone for collecting data on music performed at a concert venue; two or more second microphones, any number of which may be arranged facing different directions in the performance venue, for collecting reverberation data; A system comprising a recording device, The recording device a data acquisition means for acquiring data collected by the first microphone and the second microphone; a sound source separation means for separating the music data into parts; editing means for editing the separated data of each part and the reverberation data; low frequency data collection means for collecting inaudible low frequency data; a sound source producing means for synthesizing the edited data and adding the low-frequency data to the synthesized data to produce a sound source for distribution; A sound source production system for distribution, comprising:

2. The distribution sound source production system described in claim 1, characterized in that the low-frequency data collection means either extracts the inaudible low-frequency data contained in the entire data of the song or one or more arbitrary parts as is, or strengthens or weakens the extracted data, or generates new data.

3. 2. The sound source production system for distribution according to claim 1, wherein the editing means separates the reverberation data into two or more parts and edits each part, and the reverberation data synthesized by the sound source production means is data edited for each part.

4. Equipped with an audience reaction detection device that detects audience reactions, 2. The system for producing a sound source for distribution according to claim 1, wherein a decision is made as to whether or not to produce a sound source for distribution from the played music piece based on the reaction.

5. The distribution sound source production system described in claim 4, characterized in that the audience reaction detection device measures the number of portable light-emitting devices carried by each audience member that make a predetermined movement, and detects the degree of reaction based on the ratio between the measured number and the number of audience members.

6. 5. The sound source production system for distribution according to claim 4, wherein the audience reaction detection device detects the quality of the reaction based on the volume of the audience's applause and the change in volume before and after the applause.

Images