Acoustic wave processing device, acoustic wave processing method, and data structure

The sonic processing device and method address the limitations of conventional digital sound sources by converting sound into multiple frequency intensities and synthesizing sine waves to replicate the human ear's processing, achieving high-quality sound reproduction.

JP2024162908APending Publication Date: 2024-11-21JAPAN INTELLECTUAL PROPERTY RESEARCH INSTITUTE CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023078890
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Conventional digital sound sources fail to reproduce the quality of analog sound sources due to the limitations of sampling theory, which chops sounds on the time axis and subjects them to digital processing that the human ear does not undergo, leading to a perceived loss of natural sound quality.

Method used

A sonic processing device and method that converts sound waves into multiple frequency intensities at predetermined intervals, storing and reproducing these as sine waves with adjusted intensities to mimic the human ear's natural processing, using electronic or physical resonators to extract and synthesize frequency signals.

Benefits of technology

Enables high-quality recording and reproduction of sound by imitating the human ear's natural processing, providing a more natural sound experience compared to conventional digital methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162908000001_ABST
    Figure 2024162908000001_ABST
Patent Text Reader

Abstract

To provide an acoustic wave processing device and an acoustic wave processing method that record and reproduce a sound with high quality.SOLUTION: A recording / playback system is composed of: a recording device comprising an input part that inputs an acoustic wave as an analog signal, a converter that converts the analog signal input in the input part into multiple frequency intensities (spectra) for each predetermined time interval, and a storage part that stores the frequency intensity for each predetermined time interval as digital information; and a reproducing device comprising an input part that inputs digital information of a predetermined number of frequency intensities for each predetermined time interval, a sine wave generation part that generates a sine wave of the predetermined number of frequencies, an adjustment part that adjusts, for each frequency, an output intensity of the predetermined number of sine waves generated by the sine wave generation part based on the digital information, and a synthesis part that synthesizes the predetermined number of sine waves whose output intensity is adjusted by the adjustment part, and configured to adjust the output intensity of the predetermined number of sine waves for each predetermined time interval and carry out synthesis and output.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a sonication device, a sonication method, and a data structure. [Background technology]

[0002] There are two technologies for recording and playing back music and other sound waves: analog sound source technology, which physically records sound waves onto a record, and digital sound source technology, which converts sound waves into digital information using methods such as PCM (Pulse Code Modulation) and PDM (Pulse Density Modulation) that divide the sound waves on the time axis, and then writes the information to an information medium using AD (Analog to Digital) conversion.

[0003] Digital audio sources are perfectly compatible with the digital information exchanged in the modern information society, and were considered to have replaced analog audio sources. Currently, there are two main technologies being considered to improve digital audio sources. One is the DVD Audio standard, which aims to improve the specs of PCM, and the other is the Super Audio standard, which changes the format from PCM to PDM. The goal of these improvements is to expand the playable frequency range and reduce quantization noise, which has already been achieved more than adequately.

[0004] However, even though the recording and playback technology of digital sound sources has already been developed to the extent that it can be said to be over-specified, digital sound sources have not been evaluated as being able to reproduce the quality of analog sound sources. Moreover, there are persistent voices from users that say that the quality of analog sound sources is lost in digital sound sources. Under these circumstances, a new digital sound source that can reproduce the quality of analog sound sources is desired. The drawback of conventional digital sound sources is the drawback of the sampling theory that "chops up sound on the time axis and converts it into a PCM signal." The human ear does not sample and hear the sound. The first problem is that we are listening to chopped up sounds that the human ear does not sample, but there is also a serious problem that derives from this. That is, the moment music is converted into a PCM signal, the PCM signal can be freely digitally processed using the extremely convenient mathematics of sampling theory. All digital sound sources currently being made are naturally subjected to this kind of digital processing. In other words, we are listening to sounds that have been subjected to mathematical processing that the human brain never does. People with good ears feel uncomfortable with this and reject digital sound sources.

[0005] The human ear detects the vibrations of many resonating membranes in the cochlea through the auditory nerve. The fundamental flaw of conventional digital sound sources is that they completely ignore this "thing that nature does." Therefore, the only way to create a correct digital sound source that does not impair the naturalness of analog sound sources such as conventional records is to mimic exactly what the ear actually does, "divide the sound on the frequency axis instead of chopping it up on the time axis," and then digitize that spectrum information.

[0006] The reason why this invention is called a "new digital sound source" is because it completely denies the premise that "digital sound source = PCM sound source (the idea of ​​chopping up sound is exactly the same for PDM sound sources)" which nobody has questioned up until now. So, first let's look at why all conventional digital sound sources take the PCM sound source as a given.

[0007] Patent Document 1 describes an audio recording and playback device in which an A / D converter digitally converts an input audio signal to generate a digital signal, performs audio coding processing to generate audio coded data, and stores the data in a storage unit. The audio coded data is generated by performing compression coding processing on an audio-emphasizing digital signal based on an ADPCM (Adaptive Differential Pulse Code Modulation) audio compression algorithm. In other words, Patent Document 1 describes a technology for performing a type of digital processing in which an audio signal is converted into a PCM signal by an A / D converter, and then a specific voice is recognized and emphasized by formant detection.

[0008] Patent Document 2 describes an invention in which an audio signal is converted into a PCM signal by an A / D converter, and then an audio signal of a specific frequency is detected to control the start and restart of recording.

[0009] In both Patent Document 1 and Patent Document 2, the basic premise is that a PCM signal is used to detect a specific frequency. And because the PCM signal that has undergone some processing is used as a digital sound source as is, the sound quality of the digital sound source pointed out above is not overcome. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] JP 2006-78882 A [Patent Document 2] JP 2011-198348 A Summary of the Invention [Problem to be solved by the invention]

[0011] The present invention has been made in view of the above circumstances, and aims to provide an acoustic processing device and method that faithfully mimics what the human ear does, thereby enabling high-quality sound recording and playback through a new type of digitization. [Means for solving the problem]

[0012] The invention described in claim 1 is an ultrasonic processing device comprising an input means for inputting a sound wave as an analog signal, a conversion means for converting the input analog signal into a plurality of frequency intensities at predetermined time intervals, and a memory means for storing the frequency intensities as digital data at predetermined time intervals. The invention described in claim 2 is an ultrasonic processing device comprising an input means for inputting digital data of multiple frequency intensities for each predetermined time interval, an oscillation means for generating sine waves of multiple frequencies, an adjustment means for adjusting the output intensity of the sine waves based on the digital data, a synthesis means for synthesizing the sine waves of the multiple frequencies whose output intensities have been adjusted, and an output means for outputting a waveform resulting from the synthesis of the sine waves. The invention described in claim 3 is a sound wave processing method which executes an input step of inputting a sound wave as an analog signal, a conversion step of converting the input analog signal into a plurality of frequency intensities at predetermined time intervals, and a storage step of storing the frequency intensities as digital data at predetermined time intervals. The invention described in claim 4 is an acoustic wave processing method which executes an input step of inputting digital data of multiple frequency intensities for each predetermined time interval, an oscillation step of generating sine waves of multiple frequencies, an adjustment step of adjusting the output intensities of the sine waves based on the digital data, a synthesis step of synthesizing the sine waves of the multiple frequencies whose output intensities have been adjusted, and an output step of outputting the waveform resulting from the synthesis of the sine waves. The invention described in claim 5 is a data structure for playing sound in a playback device, the data structure including data of multiple frequency intensities for each predetermined time interval, the playback device generating sine waves corresponding to multiple frequencies, inputting the frequency intensity data for each time interval, and generating sound data by synthesizing a waveform obtained by multiplying the frequency intensity data by sine wave data corresponding to the multiple frequencies. The invention described in claim 6 is the data structure described in claim 5, wherein the frequency intensities are frequency intensities of 120 or more frequencies per time interval. Effect of the Invention

[0013] According to the present invention, there is provided a sound processing device and a sound processing method for recording and reproducing sound with high quality. [Brief description of the drawings]

[0014] [Figure 1] 1 is a diagram showing the overall configuration of a recording / playback system to which the present embodiment is applied; [Diagram 2] FIG. 2 is a diagram showing the functional configuration of a recording device of the recording and playback system. [Diagram 3] FIG. 2 is a diagram showing the functional configuration of a playback device of the recording and playback system. [Figure 4] FIG. 2 is a diagram showing data output by a recording device of the recording and playback system. [Diagram 5] FIG. 2 is a diagram illustrating the process by which a playback device of a recording and playback system generates sound. [Figure 6] FIG. 4 is a diagram showing a processing flow of the recording device. [Figure 7] FIG. 11 is a diagram showing a processing flow of the playback device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] The recording and reproducing system according to the present invention will be described with reference to the drawings. Note that the present invention is not limited to the following embodiment.

[0016] First Embodiment FIG. 1 is a diagram showing the overall configuration of a recording / playback system to which this embodiment is applied. The recording and playback device in this embodiment has a control unit 11 that controls the operation of the entire device, a non-volatile memory unit 12 that records data, a display unit 13 that is used to display a user interface screen and the like, an operation reception unit 14 that receives user operations, a speaker 15 that reproduces electrical signals as sound, a microphone 16 that converts sound into an electrical signal, and a communication interface (=communication IF) 17.

[0017] The control unit 11 in this embodiment has a CPU (=Central Processing Unit) 101, a ROM (=Read Only Memory) 102 in which firmware and the like are recorded, and a RAM (=Random Access Memory) 103 used as a work area. The control unit 11 functions as a so-called computer. The ROM 102 is a non-volatile semiconductor memory. The storage unit 12 is composed of a non-volatile rewritable semiconductor memory or the like. The storage unit 12 stores, for example, data on the frequency intensity of sounds recorded by a microphone 16, and the like.

[0018] The display unit 13 is composed of, for example, a liquid crystal display or an organic EL display. Information that supports user operations is displayed on the display unit 13. The operation reception unit 14 is composed of, for example, a touch sensor arranged on the surface of the display unit 13, and switches and buttons arranged on the housing. The communication interface 17 is, for example, a wireless device that complies with wireless LAN (=Local Area Network), Bluetooth (registered trademark), and mobile communication standards. Incidentally, the control unit 11 and each unit are connected via a bus 18 and signal lines (not shown).

[0019] Fig. 2 is a diagram showing the functional configuration of the recording device of the recording and playback system. The functional modules shown in Fig. 2 are composed of hardware and software controlled through the execution of a program by the CPU 101 (see Fig. 1).

[0020] The recording device 20 shown in FIG. 2 comprises an input unit 21, an amplifier unit 22, a conversion unit 23, a recording function control unit 24, a storage unit 25, and an output unit 26. The input unit 21 is, for example, a microphone having a sound collection function, and the amplifier unit 22 is an amplifier having a function of amplifying the collected voice or music signal. The conversion unit 23 is composed of a resonator that extracts a signal of a specific frequency from the amplified voice or music signal by an electronic resonance circuit. The electronic resonance circuit is prepared for each of a plurality of frequencies. For example, when the resonators are composed of N resonators from resonator 1 to resonator 2, . . . resonator N, intensity signals for N different frequencies are generated. The generated intensity signals are then converted into digital data by an AD converter that converts analog to digital. Such conversion is repeated at small time intervals, for example, every 50 ms (1 second / 20 Hz), so that when the number of time intervals is M, the recording function control unit 24 causes the storage unit 25 to record frequency intensities for M time intervals for N different frequencies. The output unit 26 further outputs the recorded data onto a CD or the like.

[0021] 3 is a diagram showing the functional configuration of a playback device 30 of the recording and playback system. The playback device shown in FIG. 3 comprises an input section 31, an oscillation section 32, an adjustment section 33, a synthesis section , an output section 35, and a playback function control section . The input unit 31 has an input function of reading recorded data. The oscillator unit 32 has a function of generating sine waves of different frequencies, and in this embodiment, a plurality of oscillators generate sine waves of different frequencies. The adjuster unit 33 adjusts the volume of the generated sine waves to match the frequency intensity, and the synthesizer unit 34 synthesizes the waveforms of the adjusted frequencies. The output unit 35 outputs the signal from, for example, a speaker.

[0022] FIG. 4 is a diagram showing data output by the recording device of the recording and playback system. The recording function control unit 24 stores the frequency intensity calculated by the conversion unit 23 of the recording device for each frequency in the storage unit 25 by time. The recording function control unit 24 can output for any number of frequencies (N) and any number of time intervals (M). For ease of explanation, the example of FIG. 4 shows data for three frequencies and four time intervals. The volume and phase of the first initial time are zero. The N×M matrix of data for N frequencies and M times (start times of the time intervals) may be reversed in the vertical and horizontal directions. Here, the "frequency intensity" is a variable corresponding to the weight when the sine wave of each frequency oscillated by the oscillation circuit of each frequency is multiplied as a weight in the adjustment unit 33 during playback and output. For example, the frequency intensity in FIG. 4 is obtained by normalizing the effective value of the output voltage of each resonant circuit by a reference voltage during recording. Here, the reference voltage is the effective value of the output voltage obtained when a sine wave (frequency = f) of a unit voltage is input to a resonant circuit (resonant frequency = f).

[0023] Since the reference voltage of each resonant circuit (resonant frequency=f) differs depending on the resonant frequency f, it is necessary to measure it in advance for each resonant circuit and store it in the storage unit 25. Normalization means dividing the output voltage (effective value) of each resonant circuit obtained during recording by the reference voltage (effective value) of the resonant circuit. In this way, the recording function control unit 24 stores the frequency intensities obtained for N frequencies f and M times t in the storage unit 25 as N×M digital data. Here, N is the number of frequencies when the audible range is divided by a predetermined frequency interval (e.g., 0.0025 octave), and M is the number of start times when the recording time is divided by a predetermined time interval (e.g., 50 mS).

[0024] Because the overall volume of each song differs, the N x M frequency intensities are finally normalized by dividing them by the maximum value among them. As a result, all frequency intensity values ​​become dimensionless numbers less than 1 and greater than 0. For the sake of explanation, Fig. 4 shows an example of frequency intensity output when frequencies f = 20 Hz, 25 Hz, and 30 Hz and times t (in 50 ms units) = 0, 1, 2, and 3.

[0025] In practical terms, in order to record high-quality sound, the number of frequencies N should be large. The human cochlea has about 4000 resonators, and it is said that we can hear a difference of 0.0025 octaves. For this reason, it is ideal to set the number of frequencies N to 4000 and prepare 4000 resonators with the same frequencies as the human cochlea. In terms of how far these 4000 frequencies can be thinned out while maintaining a certain level of sound quality, it is desirable to thin out the frequencies to 120 or more. The reason for this is as follows. The audible range that the human ear can hear is said to be between 20Hz and 20,000Hz. The highest frequency in the audible range is about 1000 times the lowest frequency. Here, a sound one octave higher is a sound with a frequency twice as high, and the range of the audible range is close to 1024, which is the 10th power of 2. In other words, the audible range can be said to be approximately a 10-octave range. And since one octave is divided into 12 scales, 10 octaves are divided into 120 scales. In other words, N=120 is the minimum number of frequencies required to correctly reproduce the difference between semitones. For this reason, it is desirable that the minimum number of frequencies required is 120. In other words, it is desirable that the frequency intensity data used for recording and playback has a data structure having 120 or more frequency intensities.

[0026] FIG. 5 is a diagram for explaining the process in which the reproducing device of the recording and reproducing system generates sound. The oscillator 32 of the reproducing device of the recording and reproducing system generates sine waves for each of a plurality of different frequencies (N). For the sake of simplicity, FIG. 5 shows an example in which the oscillator 32 generates sine waves with three cycles of 20 Hz, 25 Hz, and 30 Hz. When the reproducing control unit 36 ​​inputs the frequency intensity data shown in FIG. 4 to the input unit 31, the adjustment unit 33 adjusts the volume of the sine wave of each frequency. The synthesis unit 34 generates a synthetic waveform by overlapping the sine waves of each frequency. In this case, the phase changes from moment to moment from the time when the sine wave was first generated, but it is not reset every time and changes with consistency.

[0027] In Fig. 4 and Fig. 5, for the sake of simplicity, three types of frequencies are used (N=3): 20Hz, 25Hz, and 30Hz. In the graph of Fig. 5, the horizontal axis is time in mS. The left side shows the oscillation waveforms of the three frequencies. When the music starts, all frequencies start oscillating from phase zero, and continue to oscillate until the music ends. This oscillation waveform is multiplied by the weight of the volume, which changes every 50mS, to produce the output waveform on the right. The time t is counted up every 50mS in the order of 1, 2, and 3. Normally, music starts from a silent state, so in this example, the volume of all frequencies is zero in the first 50mS. The music starts at 50mS, and the volume of 20Hz increases in the order of 0.5, 0.75, and 1, the volume of 25Hz increases in the order of 0.25, 0.5, and 0.75, and the volume of 30Hz increases in the order of 0.125, 0.25, and 0.5 every 50mS. In the case of 20Hz, the vibration period is 50mS, so 50mS is the timing when the waveform becomes exactly zero, but in the case of 25Hz, the vibration period is 40mS, so the waveform becomes discontinuous in the volume change every 50mS, but if you change the volume at short intervals of 50mS in the first place, the volume change will gradually rise and fall, so some discontinuity will not be noticeable. Even if the waveform is somewhat discontinuous, the phase is the same as the oscillation waveform. These output waveforms are added together to form the composite waveform at the bottom, which is played as a sound wave from the speaker.

[0028] Since the human ear does not hear the waveform, but the resonance strength of the frequency, it cannot detect the slight waveform discontinuity that occurs at 50mS intervals. Moreover, the sum of all frequency discontinuities should cancel each other out rather than reinforce each other statistically, so it can be almost ignored. What is more important is that the phase of all frequencies is consistent, as mentioned above, and this makes it possible to reproduce a natural sound without any discomfort.

[0029] [Recording device processing] FIG. 6 is a diagram showing a process flow of the recording device 20. As shown in FIG. 6, the recording function control unit 24 starts controlling the recording device upon receiving a start instruction from the user (START). Next, the recording function control unit 24 starts measuring the elapsed time (step 101). Next, the input unit 21 starts acquiring ambient music data (step 102). The amplifier unit 22 amplifies the input music data (step 103). The converter unit 23 inputs the amplified music data to a resonant circuit corresponding to N frequencies (step 104). The recording function control unit 24 judges whether a predetermined time (ΔT) has elapsed (step 105). If the predetermined time has not elapsed, steps 102 to 104 are continued. When the predetermined time has elapsed, the converter unit 23 measures the output voltage of the resonant circuit corresponding to the N frequencies, and stores the measured output voltage as a temporary file in the storage unit 12 or RAM 103 (step 106). Next, it is judged whether the music has ended (step 107). While the music has not ended, the above steps 102 to 106 are repeated. After the music has ended, the output voltage value stored in the primary file is normalized by the reference voltage of each frequency, and then normalized by the maximum value of all the results to calculate the frequency intensity (step 108). In other words, the conversion unit 23 also has an A / D conversion function that measures the spectrum of multiple frequencies and converts an analog signal into multiple frequency intensity data. Then, the recording function control unit 24 stores the frequency intensity for each time in the memory unit 25 (step 109). When the above processing is completed, the processing of the recording device 20 ends (END).

[0030] [Playback Device Processing] FIG. 7 is a diagram showing a process flow of the playback device 30. The playback function control unit 36 ​​shown in FIG. 3 starts controlling the playback device upon receiving a start instruction from the user (START). Next, the playback function control unit 36 ​​starts measuring the elapsed time (step 201). Next, the oscillation unit 32 starts oscillating sine waves of multiple frequencies (step 202). The input unit 31 acquires frequency intensity data stored in the storage unit 12 of FIG. 1 or a CD medium (not shown) (step 203). The adjustment unit 33 weights the sine waves of all frequencies oscillated by the oscillation unit 32 by their respective frequency intensities to generate a waveform of the volume for each frequency (step 204). The synthesis unit 34 sums up the sine waves of multiple frequencies to synthesize them into one waveform (step 205). The synthesized waveform is output as a sound wave (step 206). Next, it is determined whether a predetermined time (ΔT) has elapsed (step 207). If ΔT has not elapsed, the process returns to step 204. When ΔT has elapsed, it is determined whether the recorded data has ended (step 208). If the music has not ended, the process returns to step 203. If the music has ended, the process of the playback device 30 ends (END).

[0031] In this embodiment, the conversion unit 23 is described as being composed of a resonator that extracts a specific frequency signal from the amplified sound by an electronic resonant circuit, but it is not necessarily limited to an electronic resonant circuit, and the conversion unit 23 may be a physical resonator, that is, a tuning fork prepared for each of a plurality of frequencies. If the frequency intensity can be extracted as an electrical signal from the analog signal input to the tuning fork for each frequency, a recording device of the same level as that using an electronic resonant circuit can be constructed. Therefore, the conversion unit 23 may include not only an electronic resonant circuit but also a physical resonator.

[0032] In this embodiment, music has been used as an example, but the sound to be recorded and played back does not need to be limited to music, but can be human vocal music, human conversation, sounds from the natural environment, noise, or any other sound that can be recorded and played back.

[0033] In this embodiment, the recording and playback of sounds that humans can hear is described, but it is also possible to record and play sounds outside the range of human hearing. For example, it is possible to record and play sounds for animals such as dogs that can hear higher frequencies than the range of human hearing. [Explanation of symbols]

[0034] 1...recording / playback system, 11...control unit, 12...storage unit, 13...display unit, 14...operation reception unit, 15...speaker, 16...microphone, 17...communication interface, 18...bus, 20...recording device, 21...input section, 22...amplification section, 23...conversion section, 24...recording function control section, 25...storage section, 26...output section, 30...playback device, 31...input section, 32...oscillation section, 33...adjustment section, 34...synthesizing section, 35...output section, 36...playback function control section, 101...CPU, 102...ROM, 103...RAM

Claims

1. An input means for inputting sound waves as analog signals; A conversion means for converting the input analog signal into a plurality of frequency intensities at predetermined time intervals; A storage means for storing the frequency intensity as digital data for each predetermined time interval; A sonication device comprising:

2. an input means for inputting digital data of a plurality of frequency intensities at predetermined time intervals; An oscillator for generating sine waves of multiple frequencies; an adjusting means for adjusting an output intensity of the sine wave based on the digital data; A synthesizing means for synthesizing the sine waves of the plurality of frequencies whose output intensities have been adjusted; an output means for outputting a waveform resulting from the synthesis of the sine waves; A sonication device comprising:

3. an input step of inputting a sound wave as an analog signal; A conversion step of converting the input analog signal into a plurality of frequency intensities at predetermined time intervals; a storing step of storing the frequency intensity as digital data for each predetermined time interval; A sonication method that performs

4. An input step of inputting digital data of a plurality of frequency intensities for each predetermined time interval; An oscillation step of generating sine waves of multiple frequencies; an adjusting step of adjusting an output intensity of the sine wave based on the digital data; a synthesizing step of synthesizing the sine waves of the multiple frequencies whose output intensities have been adjusted; an output step of outputting a waveform resulting from the synthesis of the sine waves; A sonication method that performs

5. A data structure for playing back sound in a playback device, comprising: The data includes a plurality of frequency intensity data for each predetermined time interval, The playback device has a data structure for generating sine waves corresponding to a plurality of frequencies, inputting the frequency intensity data for each time interval, and generating sound data by synthesizing a waveform obtained by multiplying the frequency intensity data by sine wave data corresponding to the plurality of frequencies.

6. The data structure of claim 5 , wherein the frequency intensities are frequency intensities of 120 or more frequencies per time interval.

Citation Information

Patent Citations

  • Voice recording and reproducing device

    JP2006078882A

  • Sound recording device

    JP2011198348A