Acoustic correction apparatus, acoustic correction method, and storage medium

Abstract

The invention relates to an acoustic correction apparatus, an acoustic correction method, and a storage medium. The sound correction device comprises a sound box and a millimeter wave radar. The sound equipment is configured to send original audio information to a target object; the millimeter wave radar is configured to obtain original audio information emitted by the sound equipment and sound vibration information generated on the target object; converting the sound vibration information into actual audio information received by the target object; and correcting the actual audio information based on the original audio information, so that the actual audio information is consistent with the original audio information. According to the method, the actual audio information heard by a listener can be kept consistent with the original audio information sent by the sound equipment as much as possible, so that the listener can receive high-quality audio information.

Description

Technical Field

[0001] The present disclosure relates to the technical field of sound control, and in particular to an acoustic correction device, an acoustic correction method, and a storage medium. Background Art

[0002] When audio products make sounds indoors, the audio information forms a complex sound field from the audio to the various receiving points in the room. For any receiving point, the audio information received is roughly divided into the direct audio, near-reflected audio, and reverberated audio of the audio. Since near-reflected audio and reverberated audio are affected by the indoor environment, the walls and objects in the room absorb, reflect, and resonate the audio information, resulting in changes in indoor acoustics, and users cannot hear high-quality music. In order to correct the indoor acoustic parameters, it is necessary to collect the audio information received at the user's location, compare and analyze it, and then perform repairs.

[0003] In the related art, an audio collector (such as a microphone, mobile phone, etc.) is added to the listener's location to collect the audio information heard by the listener for analysis. Since the indoor environment is relatively stable, the use rate of the audio collector is extremely low after the indoor acoustic correction, making the audio collector easy to be idle or lost. When the indoor environment changes and the indoor acoustic correction is required again, it will cause inconvenience to the listener. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an acoustic correction device and an acoustic correction method. Based on the frequency response difference between the original output audio and the received audio, the frequency response parameters of the original output audio are used as the reference frequency response, and the frequency response parameters of the audio output audio are adjusted to make the frequency response of the received audio consistent with the original reference frequency response, thereby correcting and aligning the sound pressure level, delay, phase and other parameters of the received audio, so that the audio information received at the target object is high-quality audio information, further improving the user experience.

[0005] In a first aspect, the present disclosure provides an acoustic correction device, the acoustic correction device comprising: a sound system and a millimeter wave radar;

[0006] The speaker is configured as follows:

[0007] Sending raw audio information to the target object;

[0008] The millimeter wave radar is configured as follows:

[0009] Acquiring original audio information emitted by the speaker and sound vibration information generated on the target object;

[0010] converting the sound vibration information into actual audio information received by the target object;

[0011] Based on the original audio information, the actual audio information is corrected so that the actual audio information is consistent with the original audio information.

[0012] As an implementation method of the present disclosure in some embodiments, the sound system is specifically configured as follows:

[0013] The sending of original audio information to the target object comprises:

[0014] A frequency modulated continuous wave signal is sent to the target object; the frequency modulated continuous wave signal is a frequency modulated continuous signal.

[0015] As an implementation method of an embodiment of the present disclosure in some embodiments, the millimeter wave radar is specifically configured as follows:

[0016] The converting the sound vibration information into actual audio information received by the target object comprises:

[0017] receiving a reflected continuous wave signal reflected by the target object;

[0018] Acquire the time difference and phase difference between the frequency modulated continuous wave signal and the reflected continuous wave signal;

[0019] According to the time difference and the phase difference, obtaining the vibration displacement velocity of the reflected continuous wave signal;

[0020] Acquire a time domain signal corresponding to the reflected continuous wave signal according to the vibration displacement velocity of the reflected continuous wave signal;

[0021] Performing Fourier transform on the time domain signal to obtain actual audio information received by the target object; the actual audio information includes: frequency and amplitude.

[0022] As an implementation method of an embodiment of the present disclosure in some embodiments, the millimeter wave radar is further configured as follows:

[0023] The correcting the actual audio information based on the original audio information so that the actual audio information is consistent with the original audio information further includes:

[0024] Acquire a standard frequency response curve corresponding to the original audio information and an actual frequency response curve corresponding to the actual audio information;

[0025] The actual frequency response curve is corrected based on the standard frequency response curve so that the actual audio information is consistent with the original audio information.

[0026] As an implementation method of an embodiment of the present disclosure in some embodiments, the millimeter wave radar is specifically configured as follows:

[0027] The correcting the actual frequency response curve based on the standard frequency response curve so that the actual audio information is consistent with the original audio information includes:

[0028] When the frequency of the actual audio information is greater than a preset threshold, performing inverse Fourier transform processing on the standard frequency response curve and the actual frequency response curve to obtain waveform data of the standard frequency response curve and the actual frequency response curve in the time domain;

[0029] Based on the waveform data of the standard frequency response curve and the actual frequency response curve in the time domain, the actual frequency response curve is adjusted to make the actual audio information consistent with the original audio information.

[0030] As an implementation method of an embodiment of the present disclosure in some embodiments, the millimeter wave radar is further specifically configured as follows:

[0031] The correcting the actual frequency response curve based on the standard frequency response curve so that the actual audio information is consistent with the original audio information also includes:

[0032] When the frequency of the actual audio information is less than or equal to a preset threshold, the actual frequency response curve is reversely adjusted based on the standard frequency response curve.

[0033] As an implementation method of an embodiment of the present disclosure in some embodiments, the millimeter wave radar is further configured as follows:

[0034] Acquire direct audio received by the target object and reflected audio received by the target object;

[0035] Calculating the delay of the reflected audio according to the time interval between the original audio information and the actual audio information;

[0036] Calculating phase information of the reflected audio based on the delay of the reflected audio;

[0037] Acquiring phase information of a preset inverted audio according to the phase information of the reflected audio;

[0038] The phase information of the target corrected audio received by the target object is obtained based on the phase information of the direct audio, the phase information of the reflected audio, and the phase information of the preset inverted audio; the phase information of the direct audio is consistent with the phase information of the original audio information.

[0039] As an implementation method of an embodiment of the present disclosure in some embodiments, the millimeter wave radar is further specifically configured as follows:

[0040] The step of obtaining the phase information of the preset inverted audio according to the phase information of the reflected audio comprises:

[0041] Obtain the frequency and wavelength of the reflected audio;

[0042] Calculating a phase step coefficient according to the frequency of the reflected audio, the wavelength of the reflected audio, and the delay of the reflected audio;

[0043] The phase information of the reflected audio and the phase step coefficient are used to calculate the phase information of the preset anti-phase audio.

[0044] In a second aspect, an acoustic correction method is provided, which is applied to an acoustic correction device, wherein the acoustic correction device includes: an acoustic system and a millimeter wave radar; the method includes:

[0045] Sending original audio information to the target object through the speaker;

[0046] Acquiring original audio information emitted by the speaker and sound vibration information generated on the target object through the millimeter wave radar;

[0047] converting the sound vibration information into actual audio information received by the target object;

[0048] The actual audio information is corrected based on the original audio information to make the actual audio information consistent with the original audio information.

[0049] As an implementation method of the embodiment of the present disclosure in some embodiments, sending the original audio information to the target object through the speaker includes:

[0050] A frequency modulated continuous wave signal is sent to the target object; the frequency modulated continuous wave signal is a frequency modulated continuous signal.

[0051] As an implementation method of an embodiment of the present disclosure in some embodiments, converting the sound vibration information into actual audio information received by the target object includes:

[0052] receiving a reflected continuous wave signal reflected by the target object;

[0053] Acquire the time difference and phase difference between the frequency modulated continuous wave signal and the reflected continuous wave signal;

[0054] According to the time difference and the phase difference, obtaining the vibration displacement velocity of the reflected continuous wave signal;

[0055] Acquire a time domain signal corresponding to the reflected continuous wave signal according to the vibration displacement velocity of the reflected continuous wave signal;

[0056] Performing Fourier transform on the time domain signal to obtain actual audio information received by the target object; the actual audio information includes: frequency and amplitude.

[0057] As an implementation method of an embodiment of the present disclosure in some embodiments, the correcting the actual audio information based on the original audio information so that the actual audio information is consistent with the original audio information further includes:

[0058] Acquire a standard frequency response curve corresponding to the original audio information and an actual frequency response curve corresponding to the actual audio information;

[0059] The actual frequency response curve is corrected based on the standard frequency response curve so that the actual audio information is consistent with the original audio information.

[0060] As an implementation method of an embodiment of the present disclosure in some embodiments, the correcting the actual frequency response curve based on the standard frequency response curve so that the actual audio information is consistent with the original audio information includes:

[0061] When the frequency of the actual audio information is greater than a preset threshold, performing inverse Fourier transform processing on the standard frequency response curve and the actual frequency response curve to obtain waveform data of the standard frequency response curve and the actual frequency response curve in the time domain;

[0062] Based on the waveform data of the standard frequency response curve and the actual frequency response curve in the time domain, the actual frequency response curve is adjusted to make the actual audio information consistent with the original audio information.

[0063] As an implementation method of the embodiment of the present disclosure in some embodiments, the correcting the actual frequency response curve based on the standard frequency response curve so that the actual audio information is consistent with the original audio information also includes:

[0064] When the frequency of the actual audio information is less than or equal to a preset threshold, the actual frequency response curve is reversely adjusted based on the standard frequency response curve.

[0065] As an implementation method of an embodiment of the present disclosure in some embodiments, the method further includes: acquiring direct audio received by the target object and reflected audio received by the target object;

[0066] Calculating the delay of the reflected audio according to the time interval between the original audio information and the actual audio information;

[0067] Calculating phase information of the reflected audio based on the delay of the reflected audio;

[0068] Acquiring phase information of a preset inverted audio according to the phase information of the reflected audio;

[0069] The phase information of the target corrected audio received by the target object is obtained based on the phase information of the direct audio, the phase information of the reflected audio, and the phase information of the preset inverted audio; the phase information of the direct audio is consistent with the phase information of the original audio information.

[0070] As an implementation method of an embodiment of the present disclosure in some embodiments, acquiring the phase information of the preset inverted audio according to the phase information of the reflected audio includes:

[0071] Obtain the frequency and wavelength of the reflected audio;

[0072] Calculating a phase step coefficient according to the frequency of the reflected audio, the wavelength of the reflected audio, and the delay of the reflected audio;

[0073] The phase information of the reflected audio and the phase step coefficient are used to calculate the phase information of the preset anti-phase audio.

[0074] In a third aspect, a computer-readable storage medium is provided, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the acoustic correction method shown in the second aspect is implemented.

[0075] Compared with the related art, the technical solution provided by the embodiment of the present disclosure has the following advantages: sending original audio information to the target object through the speaker; obtaining the original audio information emitted by the speaker and the sound vibration information generated on the target object through the millimeter-wave radar; converting the sound vibration information into actual audio information received by the target object; correcting the actual audio information based on the original audio information so that the actual audio information is consistent with the original audio information. Since the acoustic correction device includes: a speaker and a millimeter-wave radar, that is, the speaker and the millimeter-wave radar are integrated in the same device, in the acoustic correction device, the actual audio information received by the target object is corrected based on the original audio information emitted by the speaker through the millimeter-wave radar, so that the actual audio information is consistent with the original audio information, avoiding the problem in the related art that the audio collector and the speaker are in different positions and the audio collector is used less frequently and is easily idle or lost, further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0078] Figure 1A This is a schematic diagram of the sound field when the speaker is sounding indoors;

[0079] Figure 1B A schematic diagram of an application scenario of an acoustic correction method in related technology;

[0080] Figure 1C A schematic diagram of an application scenario of the acoustic correction method according to one or more embodiments of the present disclosure;

[0081] Figure 1D is a hardware configuration block diagram of a control device 100 according to one or more embodiments of the present disclosure;

[0082] Figure 2 One of the flowcharts of an acoustic correction method provided in an embodiment of the present disclosure;

[0083] Figure 3A One of the example diagrams of the application principle of an indoor acoustic correction method provided by an embodiment of the present disclosure;

[0084] Figure 3B A second diagram showing an example of the application principle of the indoor acoustic correction method provided by an embodiment of the present disclosure;

[0085] Figure 4 A schematic diagram of a spectrum between a millimeter wave transmission signal and a reflected signal provided in an embodiment of the present disclosure;

[0086] Figure 5A A schematic diagram of a time domain signal corresponding to a reflected continuous wave signal provided in an embodiment of the present disclosure;

[0087] Figure 5B A schematic diagram of a frequency domain signal corresponding to a reflected continuous wave signal provided in an embodiment of the present disclosure;

[0088] Fig. 6A A second flowchart of an acoustic correction method provided in an embodiment of the present disclosure;

[0089] Figure 6B A schematic diagram of a frequency response curve of an EQ correction method provided in an embodiment of the present disclosure;

[0090] Fig. 7A A third flowchart of an acoustic correction method provided in an embodiment of the present disclosure;

[0091] Figure 7B A schematic diagram of a time domain signal of actual audio information and original audio information provided by an embodiment of the present disclosure;

[0092] Figure 7C A schematic diagram of a pre-echo waveform provided in an embodiment of the present disclosure;

[0093] Fig.7D A common waveform diagram provided for an embodiment of the present disclosure;

[0094] Fig. 8A A schematic diagram of the audio correction principle of an acoustic correction method provided by an embodiment of the present disclosure;

[0095] Figure 8B A waveform correction schematic diagram of an indoor acoustic correction method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0096] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0097] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0098] The terms "first", "second", "third", etc. in the specification and claims of the present disclosure and the above drawings are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0099] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0100] Glossary:

[0101] Direct audio: The sound that is transmitted directly from the sound source to the audience is called direct sound, and it is also the first sound heard by the audience. In the sound system, the unprocessed sound signal is also called direct sound. Audio refers to the sound waves with a frequency between 20HZ and 20kHz that can be heard by the human ear.

[0102] Reflected sound is divided into early reflected sound and late reflected sound. Early reflected sound is also called near-reflected sound. It is the sound emitted by the sound source that is reflected 1 to 2 times by the surrounding interface (walls, ceilings, and the ground) and all reflected sounds that are later than the direct sound by less than 50m / s belong to early reflected sound.

[0103] Reverberation, in full English, refers to the fact that when sound waves propagate indoors, they are reflected by obstacles such as walls, ceilings, and floors, and each reflection is absorbed by the obstacles. In this way, when the sound source stops making sound, the sound waves will be reflected and absorbed multiple times in the room before finally disappearing. We feel that there are still several sound waves mixed and continuing for a period of time after the sound source stops making sound (the phenomenon of sound continuation still existing after the indoor sound source stops making sound). This phenomenon is called reverberation, and this period of time is called reverberation time.

[0104] Millimeter wave radar refers to a radar sensor that works in the millimeter wave frequency band. It has the characteristics of wide frequency band, short wavelength, and large atmospheric propagation loss.

[0105] Sound pressure level: the sound pressure range audible to human ears is 2×10 -5 Pa~20Pa, the corresponding sound pressure level range is 0~120dB. Therefore, introducing the concept of sound pressure level makes it easy to describe sound pressure with large linear changes.

[0106] Frequency response refers to the phenomenon that when an audio signal with a constant voltage output is connected to a system, the sound pressure generated by the sound increases or decreases with the change of frequency, and the phase changes with the frequency. This change relationship between sound pressure and phase and frequency is called frequency response, and the unit is decibel (Db). It can be understood as the degree of response of the system to different frequencies. It is also called the effective frequency range, which reflects the main frequency range of the sound work and measures the ability of a piece of equipment to evenly reproduce high, medium and low frequency signals.

[0107] Loudness refers to the strength of sound, that is, how loud the sound is. Based on the subjective judgment of the human ear on sound, the sounds can be arranged in a sequence from soft to loud. The loudness mainly depends on the sound intensity and is also related to the frequency of the sound.

[0108] When the audio product produces sound indoors, refer to Figure 1A As shown in the figure, the audio information forms a complex sound field from the speaker to each receiving point in the room. For any receiving point, the audio information received is roughly divided into the direct audio of the speaker (refer to Figure 1A middle dashed line), near reflection audio, reverberation audio (refer to Figure 1AAs shown by the solid line in the middle). Since near-reflected audio and reverberant audio are affected by the indoor environment, the indoor walls and objects absorb audio information, reflect it multiple times, and resonate it, causing acoustic changes in the room, so users cannot hear high-quality music. In order to correct the indoor acoustic parameters, it is necessary to collect the audio information received at the user's location, compare and analyze it, and then perform repairs.

[0109] In traditional indoor acoustic correction, an audio collector (such as a microphone, mobile phone, etc.) is added to the listener's location to collect the audio information heard by the listener and analyze it. Figure 1B As shown, Figure 1B Schematic diagram of an application scenario of an acoustic correction method provided in the related art, the scenario consists of an audio product 101, a connecting line 104, and a microphone 102. When indoor acoustic correction is required, the audio product 101 is connected to the microphone 102, and the microphone 102 is placed at the position of the listener 103 to collect the audio information heard by the listener 103, feed it back to the audio product 101 for analysis, and then output the corrected audio information.

[0110] Because the indoor environment is relatively stable, the sound, people, walls, etc. will not change much. After the indoor acoustics are corrected, the microphone used as an audio collector is rarely used and is easy to be idle or lost. When the indoor environment changes and the indoor acoustics need to be corrected again, it will cause trouble to the user because the microphone cannot be found.

[0111] In view of the shortcomings of the above-mentioned method, the embodiment of the present disclosure provides an acoustic correction method, which sends original audio information to the target object through a speaker; obtains the original audio information emitted by the speaker and the sound vibration information generated on the target object through a millimeter-wave radar; converts the sound vibration information into actual audio information received by the target object; and corrects the actual audio information based on the original audio information so that the actual audio information is consistent with the original audio information. Since the acoustic correction device includes: a speaker and a millimeter-wave radar, that is, the speaker and the millimeter-wave radar are integrated in the same device, in the acoustic correction device, the actual audio information received by the target object is corrected based on the original audio information emitted by the speaker through the millimeter-wave radar, so that the actual audio information is consistent with the original audio information, thereby avoiding the problem in the related art that the audio collector and the speaker are in different positions and the audio collector is used less frequently and is easily idle or lost, further improving the user experience.

[0112] The acoustic correction method provided in the embodiments of the present disclosure can be implemented based on an acoustic correction device, or a functional module or functional entity in the acoustic correction device. The acoustic correction device can be a customized audio device. For example, a millimeter-wave radar is built into the audio device, and a slave device is no longer required to act as an audio collector. The operation is simple and it is not easy to be idle or lost. Among them, the core functions of the millimeter-wave radar include ranging, speed measurement, azimuth measurement, micro-motion detection, and 4D imaging. The millimeter-wave radar can be built into the audio device, so that it is not easy to be lost after being idle for a long time, and its characteristics can be used as a built-in audio collector.

[0113] For example, Figure 1C As shown, Figure 1C A schematic diagram of an application scenario of an acoustic correction method provided in an embodiment of the present disclosure, in Figure 1C In the embodiment, the user can operate the acoustic correction device 200 through the control device 100 or the smart device 300. For example, the user plays an audio file or adjusts the playback volume through the control device 100 or the smart device 300. The acoustic correction device 200 includes a speaker and a built-in millimeter-wave radar. The speaker plays the original audio information (i.e., the standard audio information). The standard audio information is transmitted to the surface of the target object, causing the surface of the target object to produce weak sound vibration information. The weak vibration distance of the target object surface is not very significant to the frequency information of the radar wave emission, but the phase change of the reflected radar wave signal is very obvious. The speed change of the round-trip amplitude can be calculated based on the phase change. In the embodiment of the present disclosure, the millimeter-wave radar phase velocity measurement method is used to identify the weak sound vibration information on the surface of the target object, and obtain the audio information received at the location of the target object.

[0114] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the acoustic correction device includes infrared protocol communication or Bluetooth protocol communication, and at least one of other short-range communication methods, to control the acoustic correction device 200 wirelessly or wired. The user may input user commands through buttons on the remote controller, voice input, control panel input, etc. to control the acoustic correction device 200. Figure 1D FIG. 1 is a block diagram showing an exemplary configuration of the control device 100. Figure 1DAs shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a storage device, and a power supply. The control device 100 can receive the user's input operation instructions, and convert the operation instructions into instructions that can be recognized and responded to by the acoustic correction device 200, and play the role of an interactive intermediary between the user and the acoustic correction device 200. The communication interface 130 is used to communicate with the outside, including a wireless network communication technology WIFI chip, a Bluetooth module, NFC (Near Field Communication) or at least one of an alternative module. The user input / output interface 140 includes a microphone, a touch panel, a sensor, a button or at least one of an alternative module.

[0115] In some embodiments, the acoustic correction device 200 can also be controlled by a smart device 300. For example, the acoustic correction device 200 is controlled by an application program running on the smart device 300. The smart device 300 can install software applications with the acoustic correction device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. Among them, the smart device can be a smart phone, a smart tablet, a personal computer (PC), a tablet computer, a laptop computer, a mainframe computer, etc. with a smart control function, and the embodiments of the present disclosure do not specifically limit this.

[0116] In some embodiments, the acoustic correction device 200 may not use the above-mentioned smart device 300 to receive instructions, but may receive user control through voice or touch.

[0117] In order to explain this scheme in more detail, the following will be combined in an exemplary manner Figure 2 To explain, it is understandable that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. The method for acoustic correction provided in the embodiment of the present disclosure shall prevail.

[0118] like Figure 2 As shown, the acoustic correction method is applied to an acoustic correction device, and the acoustic correction device includes: a speaker and a millimeter wave radar, wherein the millimeter wave radar can be built into the speaker to form an acoustic correction device. This method uses a built-in millimeter wave radar inside the speaker to perform indoor acoustic correction, and no longer requires a slave device to act as an audio collector, and the operation is simple. The method specifically includes the following steps S21-S24:

[0119] S21. Sending original audio information to a target object through the speaker.

[0120] In some embodiments, the above step S21 (sending the original audio information to the target object through the speaker) can be implemented as follows:

[0121] Sends a frequency modulated continuous wave signal to the target object.

[0122] The frequency modulated continuous wave signal is a frequency modulated continuous signal.

[0123] Specifically, in the acoustic correction device, the original audio information is played by the speaker, and the original audio information is transmitted to the surface of the target object, so that the surface of the target object generates weak sound vibration information. The original audio information can be a frequency modulated continuous wave signal, that is, a frequency modulated continuous signal. Frequency modulated continuous wave signals are divided into multiple types according to different frequency modulation methods, such as triangle wave, sawtooth wave, sine wave, etc.

[0124] In the disclosed embodiment, the distance measurement of the millimeter wave radar mainly utilizes the time delay between the transmitted signal and the echo signal, and combines the millimeter wave propagation speed and the relative speed of the target object and the radar to calculate the relative distance between the millimeter wave radar and the target object. The millimeter wave radar in the acoustic correction equipment is explained by taking the continuous wave radar as an example. The continuous wave radar transmits a continuous wave signal. The signal transmitted by the continuous wave radar can be a single-frequency continuous wave (CW) or a frequency modulated continuous wave (FMCW). There are also many frequency modulation methods, and the common ones are triangular wave, sawtooth wave, coded modulation or noise frequency modulation. Among them, since the single-frequency continuous wave radar can only be used for speed measurement and cannot measure distance, and the FMCW radar can measure both distance and speed, and has obvious advantages in close-range measurement, so the frequency modulated continuous wave can be used in the disclosed embodiment. In addition, for FMCW, the received echo frequency has the same change law as the transmitted frequency, both of which are triangular wave laws, but there is a time difference. This tiny time difference can be used to calculate the target distance.

[0125] S22. Acquire original audio information emitted by the speaker and sound vibration information generated on the target object through the millimeter wave radar.

[0126] The original audio information includes audio parameters such as sound pressure level, delay, and phase.

[0127] Specifically, in the acoustic correction equipment, the original audio information is played by the speakers. Since the essence of sound generation is obtained by the vibration of an object, when the original audio information is transmitted to the surface of the target object, weak sound vibration information will be generated on the surface of the target object.

[0128] S23: Convert the sound vibration information into actual audio information received by the target object.

[0129] Specifically, the weak vibration distance of the target object surface is not very significant to the frequency information of the radar wave emission, but the phase change of the reflected radar wave signal is very obvious, and the speed change of the round-trip amplitude can be calculated based on the phase change. In the embodiment of the present disclosure, the millimeter wave radar phase speed measurement method is used to identify the weak sound vibration information on the surface of the target object, and the actual audio information received at the location of the target object is obtained.

[0130] For example, refer to Figure 3A As shown, Figure 3A This is an example diagram of the application principle of the indoor acoustic correction method provided in an embodiment of the present disclosure. The speaker 301 has a built-in millimeter wave radar 302. The audio information is transmitted from the speaker 301 to the target audio receiving point in the room (i.e., the listener's position 303) to form a complex sound field. For the target audio receiving point, the audio information received is roughly divided into the direct audio of the speaker, the reflected radar wave 304, and the reflected audio clutter 305 in the room.

[0131] The method of collecting audio information by millimeter wave radar is to collect sound vibration information and convert it into audio spectrum information. The principle is to use millimeter radar waves to send a series of continuous frequency-modulated directional millimeter waves to the target object. The frequency is controlled by the modulation voltage. The waveform can be a triangular wave, and multiple millimeter wave reflection points are established on the target object.

[0132] In some embodiments, the above step S23 (converting the sound vibration information into actual audio information received by the target object) can be implemented in the following manner:

[0133] receiving a reflected continuous wave signal reflected by the target object;

[0134] Acquire the time difference and phase difference between the frequency modulated continuous wave signal and the reflected continuous wave signal;

[0135] According to the time difference and the phase difference, obtaining the vibration displacement velocity of the reflected continuous wave signal;

[0136] Acquire a time domain signal corresponding to the reflected continuous wave signal according to the vibration displacement velocity of the reflected continuous wave signal;

[0137] Performing Fourier transform on the time domain signal to obtain actual audio information received by the target object;

[0138] Specifically, a reflected continuous wave signal reflected by a target object is received, and the time difference and phase difference between the frequency modulated continuous wave signal and the reflected continuous wave signal are obtained. According to the time difference and phase difference between the frequency modulated continuous wave signal and the reflected continuous wave signal, the vibration displacement velocity of the reflected continuous wave signal is calculated. According to the vibration displacement velocity of the reflected continuous wave signal, a time domain signal corresponding to the reflected continuous wave signal is obtained, and the time domain signal is Fourier transformed to obtain the actual audio information received by the target object. The actual audio information includes not only the sound pressure level, delay, and phase of the audio, but also frequency and amplitude.

[0139] For example, refer to Figure 3B As shown, Figure 3B The second example of the application principle of the indoor acoustic correction method provided by the embodiment of the present disclosure is that the millimeter wave radar transmits a triangular wave and receives the triangular wave signal reflected from the reflection point. By comparing the transmitted signal with the reflected signal, the millimeter wave return time difference (t), wavelength (λ), frequency (f), phase difference (ω) and other information can be obtained. Figure 4 As shown, Figure 4 It is a spectrum diagram between the millimeter wave transmission signal and the reflected signal, where spectrum 401 represents the millimeter wave transmission signal, spectrum 402 is the reflected signal of the target object vibration position 1, spectrum 403 is the reflected signal of the target object vibration position 2, and spectrum 404 is the reflected signal of the target object static position; AB represents the phase difference between the reflected signal of the target object static position and the target object vibration position 1, and BC represents the phase difference between the reflected signal of the target object static position and the target object vibration position 2. The vibration displacement velocity can be calculated based on the millimeter wave return time difference (t), wavelength (λ), frequency (f), phase difference (ω) and other information. Since the phase difference △ω=2πft=2π*(v / λ)*t, the vibration displacement velocity v=λω / (2πt) is obtained. When the reflection point moves toward the radar direction, the vibration displacement velocity v is the positive velocity. When the reflection point moves away from the radar direction, the vibration displacement velocity v is the negative velocity. Among them, the positive and negative velocity values ​​change with time to form a set of time domain signals, such as Figure 5A As shown, Figure 5A The figure is a schematic diagram of the time domain signal corresponding to the reflected continuous wave signal. The time domain signal is transformed into a frequency domain signal by Fourier transform, such as Figure 5B As shown, Figure 5B This is a schematic diagram of the frequency domain signal corresponding to the reflected continuous wave signal, from which audio information can be obtained.

[0140] S24. Based on the original audio information, correct the actual audio information to make the actual audio information consistent with the original audio information.

[0141] Specifically, in the indoor acoustic correction system, since there is a frequency response difference between the original audio information emitted by the speaker and the actual audio information received by the target object, the frequency response parameters of the original audio information output by the speaker are used as the reference frequency response, and the frequency response of the actual audio information received by the target object is adjusted to make the frequency response parameters of the actual audio information consistent with the reference frequency response of the original audio information, thereby correcting and aligning the sound pressure level, delay, phase and other parameters of the actual audio information, so that the actual audio information received at the target object is high-quality audio information.

[0142] In the disclosed embodiment, original audio information is sent to the target object through a speaker; the original audio information emitted by the speaker and the sound vibration information generated on the target object are obtained through a millimeter-wave radar; the sound vibration information is converted into actual audio information received by the target object; based on the original audio information, the actual audio information is corrected so that the actual audio information is consistent with the original audio information. Since the acoustic correction device includes: a speaker and a millimeter-wave radar, that is, the speaker and the millimeter-wave radar are integrated in the same device, in the acoustic correction device, the actual audio information received by the target object is corrected based on the original audio information emitted by the speaker through the millimeter-wave radar, so that the actual audio information is consistent with the original audio information, thereby avoiding the problem in the related art that the audio collector and the speaker are in different positions and the audio collector is used less frequently and is easily idle or lost, further improving the user experience.

[0143] Due to the complex indoor acoustic environment, the audio information (actual audio information) heard at the location of the target object (reflection point) has been disturbed and needs to be corrected.

[0144] Traditional indoor acoustic correction generally uses sound compensation, that is, EQ (Equaliser) calibration, to lower or increase the sound pressure level of the problem frequency and adjust the frequency response. This traditional indoor acoustic correction cannot correct all problems. For example, some low sound pressure level frequency bands are caused by room reflections offsetting direct sound. When the direct sound pressure level is increased, the reflected sound is also increased, offsetting this part of the sound compensation.

[0145] Fig. 6A is a flow chart of another acoustic correction method provided by an embodiment of the present disclosure. Figure 2 In some embodiments, this embodiment mainly describes the process of step S24 (correcting the actual audio information based on the original audio information so that the actual audio information is consistent with the original audio information).

[0146] S61: Acquire a standard frequency response curve corresponding to the original audio information and an actual frequency response curve corresponding to the actual audio information.

[0147] S62: Correct the actual frequency response curve based on the standard frequency response curve to make the actual audio information consistent with the original audio information.

[0148] Specifically, after the actual audio information at the location of the target object is collected, it is compared with the standard frequency response curve, and based on the parameters corresponding to the standard frequency response curve, such as sound pressure level, delay, phase and other parameters, the actual frequency response curve corresponding to the actual audio information is corrected.

[0149] Exemplarily, the correction method may be a hybrid indoor acoustic correction method of phase correction and EQ correction, wherein the frequency response range of the audio is differentiated, and the high frequency part (for example, above 400 Hz) is mainly corrected by phase correction, and EQ correction is used as a supplement. The low frequency part (for example, below 400 Hz) is mainly corrected by EQ, and phase correction is used as a supplement.

[0150] Phase correction can be understood as: there is a delay in sound propagation, that is, phase difference, and phase difference can easily weaken the sound. By correcting the phase and keeping the phase consistent, the waveform cancellation can be avoided and the volume can be increased.

[0151] EQ correction can be understood as pulling the frequency response curve of the target object back to the standard value by reversely compensating the sound pressure level for the frequency band with poor frequency response curve. Figure 6B As shown, Figure 6B The frequency response curve diagram of the EQ correction method is shown in FIG. 601 , wherein curve 602 represents the standard audio information, curve 603 represents the audio information before correction, and curve 603 represents the audio information after correction. If the sound pressure level of a certain frequency band is too high, the sound pressure level of this frequency band when the audio is input is reduced. On the contrary, if the sound pressure level of a certain frequency band is too low, the sound pressure level of this frequency band when the audio is input is increased.

[0152] further, Fig. 7A is a flow chart of another acoustic correction method provided by an embodiment of the present disclosure. Fig. 6A In some embodiments, this embodiment mainly describes the process of step S62 (correcting the actual frequency response curve based on the standard frequency response curve so that the actual audio information is consistent with the original audio information).

[0153] When the frequency of the actual audio information is greater than a preset threshold, it is implemented through steps S71-S72; when the frequency of the actual audio information is less than or equal to the preset threshold, it is implemented through step S73.

[0154] S71, performing inverse Fourier transform processing on the standard frequency response curve and the actual frequency response curve to obtain waveform data of the standard frequency response curve and the actual frequency response curve in the time domain.

[0155] S72: Based on the waveform data of the standard frequency response curve and the actual frequency response curve in the time domain, adjust the actual frequency response curve so that the actual audio information is consistent with the original audio information.

[0156] Among them, the inverse Fourier transform processing is to convert the frequency domain signal into the time domain signal.

[0157] Specifically, the standard frequency response curve and the actual frequency response curve are subjected to inverse Fourier transform processing to obtain their waveform data in the time domain, and based on the waveform data in the time domain, the parameters of the actual frequency response curve are adjusted so that the frequency response parameters of the actual audio information are consistent with the frequency response parameters of the original audio information.

[0158] For example, high-frequency sounds are the main source of room reflections, and the human ear has a certain ability to distinguish between direct sounds and room reflections. Since high-frequency sounds are highly directional, the effect of changing the sound pressure level is not very obvious for direct sounds and reflections above 400Hz. At the same time, synchronous changes in reflections will also have a canceling effect. Therefore, high-frequency EQ correction is only used as an auxiliary correction, mainly using phase correction. By correcting the phase and maintaining phase consistency, waveform cancellation can be avoided and the volume can be increased.

[0159] At higher frequencies, the time domain behavior may be inconsistent (for example, sound delay, resulting in phase difference). For the high-frequency sound difference, the embodiment of the present disclosure uses inverse Fourier transform (i.e., frequency domain is converted to time domain) to refer to Figure 7B As shown, Figure 7B It is a schematic diagram of the time domain signals of the actual audio information and the original audio information, where OA represents the phase difference between the actual audio information and the original audio information, Ot1 represents the delay between the actual audio information and the original audio information. The shift data of the audio waveform in the time domain is analyzed and the delay of this frequency response area of ​​the audio is adjusted to make corrections.

[0160] After solving the consistent time domain behavior, there may still be some late reverberation noise in the room that causes subtle sound coloration. The disclosed embodiment generates a pre-echo signal through audio transform coding, which will make the sound more stereoscopic and improve the phase distortion caused by algorithms and other reasons, making the sound field more regular. The quantization processing of the audio signal after step transformation will produce a ringing effect, namely pre-echo.

[0161] S73: Based on the standard frequency response curve, reversely adjust the actual frequency response curve.

[0162] Specifically, for the low-frequency sound below 400Hz, its directivity is weak and it is not easy to cause indoor sound reflection. The correction solution mainly uses EQ correction. In the interval where the low-frequency frequency response curve is not ideal, the frequency response parameters of the audio output audio are adjusted in reverse to lower or increase the sound pressure level to compensate for the sound quality at the user's listening position.

[0163] For example, if the sound pressure level of a certain frequency band is high, the sound pressure level of this frequency band when the audio is input is reduced. Conversely, if the sound pressure level of a certain frequency band is low, the sound pressure level of this frequency band when the audio is input is increased.

[0164] Additionally, there is out-of-phase behavior between the speakers. Figure 7C and Fig.7D As shown, Figure 7C is a schematic diagram of a pre-echo waveform, wherein waveform AB represents a pre-echo and waveform CD represents a post-echo; Fig.7D It is a general waveform diagram, and waveform EF represents the rear echo. When processing the sound correction of the low frequency part, the phase between the full-range speaker and the subwoofer in the frequency band around the crossover point is adjusted to maintain the minimum phase and linear phase.

[0165] When the transmitted audio A of the indoor sound system is transmitted to the listener's position, part of the audio is direct audio D, and part of the audio is reflected by objects such as the indoor walls and ceilings to become reflected audio C transmitted to the listener's position. The direct audio D and the reflected audio C are mixed and superimposed at the listener's position. In the traditional way, the reflected audio C is weakened by physical solutions, which usually involve modifying indoor objects with sound-absorbing materials, such as adding attracting materials to walls and ceilings, adding carpets to the floor, adding soundproof curtains to windows, etc. However, this solution will increase the user's cost and requires the user to have some acoustic expertise. The implementation process is not simple and easy to operate.

[0166] Since the acoustic correction device includes: a speaker and a millimeter-wave radar, that is, the indoor acoustic correction method provided by the embodiment of the present disclosure integrates the speaker and the millimeter-wave radar in the same device. Therefore, in the acoustic correction device, the actual audio information received by the target object is corrected by the millimeter-wave radar based on the original audio information emitted by the speaker, so that the actual audio information is consistent with the original audio information, thereby avoiding the problem in the related art that the audio collector and the speaker are in different positions and the audio collector is used less frequently and is easily idle or lost, thereby further improving the user experience.

[0167] Reference Fig. 8A As shown, Fig. 8A is a flow chart of another acoustic correction method provided by an embodiment of the present disclosure. Figure 2This embodiment mainly describes the method for weakening the reflected audio.

[0168] S81. Send original audio information to the target object.

[0169] It should be noted that the implementation method of step S81 is the same as that of step S21.

[0170] S82: Acquire direct audio received by the target object and reflected audio received by the target object.

[0171] Specifically, actual audio information received by the target object is obtained, where the actual audio information includes direct audio and reflected audio.

[0172] S83. Calculate the delay of the reflected audio according to the time interval between the original audio information and the actual audio information.

[0173] Specifically, the delay is the interval between the millimeter-wave radar sending out the waveform and receiving the reflected waveform. The millimeter-wave radar can directly obtain the time of sending the waveform and the time of receiving the reflected waveform, thereby calculating the time interval.

[0174] S84. Acquire phase information of a preset inverted audio according to the phase information of the reflected audio.

[0175] Specifically, the phase calculation formula is: ω=2πft. The phase information of the reflected audio is calculated according to the time interval, and then the phase information of the preset inverted audio is obtained according to the phase information of the reflected audio.

[0176] In some embodiments, the step (obtaining the phase information of the preset inverted audio according to the phase information of the reflected audio) can be implemented as follows:

[0177] Obtain the frequency and wavelength of the reflected audio;

[0178] Calculating a phase step coefficient according to the frequency of the reflected audio, the wavelength of the reflected audio, and the delay of the reflected audio;

[0179] The phase information of the reflected audio and the phase step coefficient are used to calculate the phase information of the preset anti-phase audio.

[0180] For example, refer to Figure 8B As shown, Figure 8BFigure 3 is a waveform correction diagram of the indoor acoustic correction method. The audio information at the listener's position is collected through the millimeter-wave radar audio, and compared with the original transmitted audio A to obtain the reflected audio information at the listener's position. The phase information ω(n) of the reflected audio can be obtained by calculating the delay of the reflected audio. The phase information ω(n+1) of an estimated anti-phase audio F is inferred from the phase information ω(n) of the reflected audio C, so that it can be reversely offset with the reflected audio that will appear in the future to obtain the corrected audio B.

[0181] The estimated phase ω(n+1) of the antiphase echo F = the phase ω(n) of the reflected sound C – the phase step coefficient λ(n)*the phase ω(n) of the reflected sound C, that is, ω(n+1)=ω(n)-λ(n)*ω(n), where the phase step coefficient λ(n) changes with the difference N between the transmitted sound A and the corrected sound B.

[0182] S86. Acquire the phase information of the target corrected audio received by the target object according to the phase information of the direct audio, the phase information of the reflected audio, and the phase information of the preset anti-phase audio.

[0183] The phase information of the direct audio is consistent with the phase information of the original audio information.

[0184] Specifically, since the phase information of the direct audio is consistent with the phase information of the original audio information, the phase information of the target corrected audio received by the target object is obtained based on the phase information of the direct audio, the phase information of the reflected audio, and the phase information of the preset inverted audio.

[0185] Exemplarily, the best received audio at the listener's position is corrected audio B = direct sound D + reflected sound C - estimated anti-phase echo F. To obtain the best corrected audio B, the calculation process of pre-elimination of reflected sound is repeated many times until the difference N is minimized, that is, the estimated anti-phase echo F is repeatedly provided, and the phase step coefficient λ(n) is repeatedly adjusted until the difference N between the transmitted sound A and the corrected sound B is minimized (when the difference N becomes larger, the phase step coefficient λ is adjusted in the opposite direction to make the difference N smaller). At this time, the reflected sound C in the room is weakened to the optimal state.

[0186] In the disclosed embodiment, original audio information is sent to the target object through a speaker; the original audio information emitted by the speaker and the sound vibration information generated on the target object are obtained through a millimeter-wave radar; the sound vibration information is converted into actual audio information received by the target object; based on the original audio information, the actual audio information is corrected so that the actual audio information is consistent with the original audio information. Since the acoustic correction device includes: a speaker and a millimeter-wave radar, that is, the speaker and the millimeter-wave radar are integrated in the same device, in the acoustic correction device, the actual audio information received by the target object is corrected based on the original audio information emitted by the speaker through the millimeter-wave radar, so that the actual audio information is consistent with the original audio information, thereby avoiding the problem in the related art that the audio collector and the speaker are in different positions and the audio collector is used less frequently and is easily idle or lost, further improving the user experience.

[0187] The embodiments of the present disclosure also provide a storage medium containing computer executable instructions, which, when executed by a computer processor, implement the various processes of the method provided in any of the above embodiments and can achieve the same technical effect. To avoid repetition, they are not described here.

[0188] The computer-readable storage medium may be a read-only memory device (ROM), a random access memory device (RAM), a magnetic disk or an optical disk, etc.

[0189] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion in some embodiments is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. An acoustic correction device, characterized in that: The acoustic correction equipment includes: a speaker and a millimeter wave radar; The speaker is configured as follows: Sending raw audio information to the target object; The millimeter wave radar is configured as follows: Acquiring original audio information emitted by the speaker and sound vibration information generated on the target object; converting the sound vibration information into actual audio information received by the target object; Based on the original audio information, the actual audio information is corrected so that the actual audio information is consistent with the original audio information.

2. The acoustic correction device according to claim 1, characterized in that: The speaker is specifically configured as follows: The sending of original audio information to the target object comprises: A frequency modulated continuous wave signal is sent to the target object; the frequency modulated continuous wave signal is a frequency modulated continuous signal.

3. The acoustic correction device according to claim 2, characterized in that: The millimeter wave radar is specifically configured as follows: The converting the sound vibration information into actual audio information received by the target object comprises: receiving a reflected continuous wave signal reflected by the target object; Acquire the time difference and phase difference between the frequency modulated continuous wave signal and the reflected continuous wave signal; According to the time difference and the phase difference, obtaining the vibration displacement velocity of the reflected continuous wave signal; Acquire a time domain signal corresponding to the reflected continuous wave signal according to the vibration displacement velocity of the reflected continuous wave signal; Performing Fourier transform on the time domain signal to obtain actual audio information received by the target object; the actual audio information includes: frequency and amplitude.

4. The acoustic correction device according to claim 1, characterized in that: The millimeter wave radar is further configured as: The correcting the actual audio information based on the original audio information so that the actual audio information is consistent with the original audio information further includes: Acquire a standard frequency response curve corresponding to the original audio information and an actual frequency response curve corresponding to the actual audio information; The actual frequency response curve is corrected based on the standard frequency response curve so that the actual audio information is consistent with the original audio information.

5. The acoustic correction device according to claim 4, characterized in that: The millimeter wave radar is specifically configured as follows: The correcting the actual frequency response curve based on the standard frequency response curve so that the actual audio information is consistent with the original audio information includes: When the frequency of the actual audio information is greater than a preset threshold, performing inverse Fourier transform processing on the standard frequency response curve and the actual frequency response curve to obtain waveform data of the standard frequency response curve and the actual frequency response curve in the time domain; Based on the waveform data of the standard frequency response curve and the actual frequency response curve in the time domain, the actual frequency response curve is adjusted to make the actual audio information consistent with the original audio information.

6. The acoustic correction device according to claim 4, characterized in that: The millimeter wave radar is also specifically configured as: The correcting the actual frequency response curve based on the standard frequency response curve so that the actual audio information is consistent with the original audio information also includes: When the frequency of the actual audio information is less than or equal to a preset threshold, the actual frequency response curve is reversely adjusted based on the standard frequency response curve.

7. The acoustic correction device according to claim 1, characterized in that: The millimeter wave radar is further configured as: Acquire direct audio received by the target object and reflected audio received by the target object; Calculating the delay of the reflected audio according to the time interval between the original audio information and the actual audio information; Calculating phase information of the reflected audio based on the delay of the reflected audio; Acquiring phase information of a preset inverted audio according to the phase information of the reflected audio; The phase information of the target corrected audio received by the target object is obtained according to the phase information of the direct audio, the phase information of the reflected audio, and the phase information of the preset inverted audio; the phase information of the direct audio is consistent with the phase information of the original audio information.

8. The acoustic correction device according to claim 7, characterized in that: The millimeter wave radar is also specifically configured as: The step of obtaining the phase information of the preset inverted audio according to the phase information of the reflected audio comprises: Obtain the frequency and wavelength of the reflected audio; Calculating a phase step coefficient according to the frequency of the reflected audio, the wavelength of the reflected audio, and the delay of the reflected audio; The phase information of the reflected audio and the phase step coefficient are used to calculate the phase information of the preset anti-phase audio.

9. An acoustic correction method, characterized in that: Applied to an acoustic correction device, the acoustic correction device includes: an acoustic system and a millimeter wave radar; the method includes: Sending original audio information to the target object through the speaker; Acquiring original audio information emitted by the speaker and sound vibration information generated on the target object through the millimeter wave radar; converting the sound vibration information into actual audio information received by the target object; The actual audio information is corrected based on the original audio information to make the actual audio information consistent with the original audio information.

10. The method according to claim 9, characterized in that The sending of original audio information to the target object comprises: A frequency modulated continuous wave signal is sent to the target object; the frequency modulated continuous wave signal is a frequency modulated continuous signal.

Images