A recording method, device, equipment and storage medium

Abstract

The present application discloses a method, device, equipment and storage medium for recording. The specific technical solution is applied to a device provided with dual microphones, including: respectively obtaining first sound data in the environment collected by the first microphone and the second microphone; performing noise reduction processing on the first sound data collected by the first microphone to obtain second sound data; separating the first sound data collected by the second microphone according to the second sound data to obtain environmental noise data; determining the recorded sound data according to the environmental noise data and a preset environmental sound threshold. According to the embodiments of the present application, the effect of recording under different environmental sound conditions is guaranteed to meet the user's sound requirements for audio or video recording and enhance the user experience.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a recording method, device, equipment and storage medium. Background Art

[0002] With the development of mobile terminal technology, more and more microphones on smart mobile terminal devices are using dual-microphone design. The recording conditions of dual microphones can bring better sound effects to the recording and reception of smart mobile terminals.

[0003] However, in actual applications, due to changes in recording environment conditions, when recording using dual microphones, it is often not possible to successfully record appropriate sounds, and the user's sound requirements for audio or video recording cannot be met. Summary of the invention

[0004] The embodiments of the present application provide a recording method, apparatus, device and computer storage medium, which can ensure that appropriate sound and recording effects can be recorded under different environmental sound conditions, so as to meet the user's sound requirements for audio or video recording and improve the user experience.

[0005] In a first aspect, an embodiment of the present application provides a recording method, which is applied to a device provided with dual microphones, and the method includes:

[0006] Respectively obtain first sound data in the environment collected by the first microphone and the second microphone;

[0007] Performing noise reduction processing on the first sound data collected by the first microphone to obtain second sound data;

[0008] Separating the first sound data collected by the second microphone according to the second sound data to obtain environmental noise data;

[0009] The recorded sound data is determined according to the environmental noise data and a preset environmental sound threshold.

[0010] Optionally, determining the recorded sound data according to the ambient noise data and a preset ambient sound threshold includes:

[0011] When the environmental noise data exceeds a preset environmental sound threshold, the second sound data and the far-field sound data are synthesized into recorded sound data.

[0012] Optionally, before synthesizing the second sound data and the far-field sound data into recorded sound data, the method further includes:

[0013] Sound optimization is performed on the environmental noise data to obtain optimized environmental noise data.

[0014] Optionally, performing sound optimization on the environmental noise data includes:

[0015] The sound is optimized based on the dynamic range adjustment (DRC) for the environmental noise data.

[0016] Optionally, determining the recorded sound data according to the ambient noise data and a preset ambient sound threshold value further includes:

[0017] When the environmental noise data does not exceed the preset environmental sound threshold, the second sound data is used as the recorded sound data.

[0018] Optionally, the separating the first sound data collected by the second microphone according to the second sound data to obtain the environmental noise data includes:

[0019] The first sound data collected by the second microphone is subtracted from the second sound data to obtain the environmental noise data.

[0020] Optionally, the first sound data includes environmental noise data and voice data.

[0021] In a second aspect, an embodiment of the present application provides a recording device, which is applied to a device provided with dual microphones, and the device includes:

[0022] An acquisition module, used to respectively acquire first sound data in an environment collected by the first microphone and the second microphone;

[0023] A processing module, configured to perform noise reduction processing on the first sound data collected by the first microphone to obtain second sound data;

[0024] a separation module, configured to separate the first sound data collected by the second microphone according to the second sound data to obtain environmental noise data;

[0025] The determination module is used to determine the recorded sound data according to the environmental noise data and a preset environmental sound threshold.

[0026] In a third aspect, an embodiment of the present application provides a recording device, the device comprising: a first microphone, a second microphone, a processor, and a memory storing computer program instructions;

[0027] When the processor executes the computer program instructions, the first aspect and the optional recording method of the first aspect are implemented.

[0028] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the method for recording as in the first aspect and the optional recording method of the first aspect is implemented.

[0029] The recording method, device, equipment and computer storage medium of the embodiments of the present application can use dual microphones to simultaneously collect external environmental sound data when recording audio or video, and obtain second sound data, that is, pure voice data with noise removed, after performing noise reduction processing on the first sound data obtained by the first microphone. Then, the first sound data and the second sound data obtained by the second microphone are separated to obtain environmental noise data. According to the relationship between the environmental noise data and the preset environmental sound threshold, the recorded sound data in the corresponding environment is determined. Based on this, the recorded first sound data can be adjusted and processed according to different environmental conditions, which can ensure the effect of recording under different environmental sound conditions, thereby meeting the user's sound requirements for audio or video recording and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] Figure 1 is a flowchart of a recording method provided in some embodiments of the present application;

[0032] Figure 2 is a flowchart of a recording method provided in some other embodiments of the present application;

[0033] Figure 3 is a schematic diagram of the structure of a recording device provided in some other embodiments of the present application;

[0034] Figure 4 It is a schematic diagram of the hardware structure of the recording device provided in other embodiments of the present application. DETAILED DESCRIPTION

[0035] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0036] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0037] Nowadays, smart terminal devices are usually equipped with dual microphones. However, when recording video or audio, the related technologies mostly use two microphones set at different positions to perform noise reduction and other processing on the recorded sound, and enhance the voice in the recorded sound to achieve high-quality sound recording effects, etc.

[0038] However, in actual applications, the recording requirements are actually different in different recording environments. In some environments, such as a relatively quiet environment or an environment where you need to focus on the voice of the near-field user during recording, you only need to record the voice data, and you need to eliminate other noises in the recording environment using the methods in the relevant technology; in other environments, such as performance scenes such as concerts and music concerts, for the recording terminal, the music sound of the concert belongs to the environmental noise. If you still use the methods in the relevant technology to perform noise reduction and other processing, the recorded sound cannot highly restore the singing scene, resulting in poor sound effects of the recorded audio and video.

[0039] In order to solve the problems of the prior art, the embodiments of the present application provide a recording method, device, equipment and computer storage medium, which can ensure the recording effect under different environmental sound conditions to meet the user's sound requirements for audio or video recording and improve the user experience.

[0040] The following describes the recording method, device, equipment and computer storage medium provided according to the embodiments of the present application in conjunction with the accompanying drawings. It should be noted that these embodiments are not intended to limit the scope of the present application.

[0041] The following first introduces the recording method provided in the embodiment of the present application.

[0042] Figure 1 FIG. 1 is a flow chart of a recording method provided by an embodiment of the present application. Figure 1 As shown, in an embodiment of the present application, the recording method is applied to a device provided with dual microphones, and may include the following steps:

[0043] S101: Acquire first sound data in an environment collected by a first microphone and a second microphone respectively.

[0044] Here, the sound data collected by the first microphone and the second microphone may both be first sound data in the environment, that is, the two microphones collect the same sound data.

[0045] The first sound data in the environment may include all sound data in the live environment during recording.

[0046] In some embodiments of the present application, the first sound data may include voice data and environmental noise data.

[0047] In some embodiments of the present application, the voice data may include near-field sound data, i.e., near-field voice data. The environmental noise data may include far-field sound data, i.e., other sound data in addition to the near-field sound data, for example, far-field music sound, environmental wind sound, and other noise data.

[0048] Here, far field and near field can refer to the distance between the sound source and the microphone array. Sound waves are a kind of vibration wave. After the sound source vibrates, the medium around the sound source vibrates, and the sound wave spreads around with the medium, so it is a spherical wave. The near field regards the sound wave as a spherical wave, which considers the amplitude difference between the signals received by the microphone array elements. The far field regards the sound wave as a plane wave, which ignores the amplitude difference between the signals received by each array element, and approximately considers that there is a simple time delay relationship between each received signal. There is no absolute standard for the division of near field and far field. It is generally believed that when the distance between the sound source and the central reference point of the microphone array is much greater than the signal wavelength, it is the far field; otherwise, it is the near field.

[0049] It is understandable that, in practical applications, the specific positions of the first microphone and the second microphone on the device may not be limited.

[0050] S102: Perform noise reduction processing on the first sound data collected by the first microphone to obtain second sound data.

[0051] S103: Separate the first sound data collected by the second microphone according to the second sound data to obtain environmental noise data.

[0052] S104: Determine recorded sound data according to the environmental noise data and a preset environmental sound threshold.

[0053] In summary, the recording method in the embodiment of the present application can use dual microphones to simultaneously collect external environmental sound data when recording audio or video, and obtain second sound data, that is, pure voice data with noise removed, after performing noise reduction processing on the first sound data obtained by the first microphone. Then, the first sound data and the second sound data obtained by the second microphone are separated to obtain environmental noise data. According to the relationship between the environmental noise data and the preset environmental sound threshold, the recorded sound data in the corresponding environment is determined. Based on this, the effect of recording under different environmental sound conditions can be guaranteed, thereby meeting the user's sound requirements for audio or video recording and improving the user experience.

[0054] In order to better illustrate the recording method of the embodiment of the present application, the implementation of the method is further described in detail below. In some embodiments of the present application, such as Figure 2 As shown, Figure 2 : is a flowchart of a recording method provided by another embodiment of the present application. The method can be specifically implemented as follows:

[0055] S201: Acquire first sound data collected by a first microphone.

[0056] S202: Acquire first sound data collected by the second microphone.

[0057] S203: Perform noise reduction processing on the first sound data collected by the first microphone to obtain second sound data.

[0058] In some embodiments of the present application, the first microphone and the second microphone record the same external environment sound, that is, the first sound data. The first sound data collected by the first microphone is processed by the noise reduction module to obtain the second sound data, that is, the pure near-field voice data.

[0059] In some embodiments of the present application, the noise cancellation process can eliminate ambient wind noise, device circuit base noise, and other ambient sounds. These eliminated sounds can be regarded as ambient noise data.

[0060] S204: Subtract the first sound data collected by the second microphone from the second sound data to obtain environmental noise data.

[0061] In some embodiments of the present application, when separating the first sound data collected by the second microphone according to the second sound data, a subtractor can be used to subtract the first sound data collected by the second microphone from the second sound data to obtain ambient noise data.

[0062] Based on the above steps, the near-field voice data and the environmental noise data in the recorded first sound data can be separated, so that the two types of sound data can be further processed according to environmental conditions.

[0063] S205: When the environmental noise data exceeds a preset environmental sound threshold, sound optimization is performed on the environmental noise data to obtain optimized environmental noise data.

[0064] In some embodiments of the present application, the preset ambient sound threshold may be the energy level of the audio signal, that is, the absolute power value. At least one preset ambient sound threshold may be preset based on experience.

[0065] Exemplarily, the preset ambient sound threshold is set to a default value of -60 decibel milliwatts (dbm).

[0066] Exemplarily, the preset ambient sound threshold may include multiple ones, for example, the preset ambient sound threshold may include -60dbm, -80dbm, and -90dbm. Specifically, the corresponding preset ambient sound threshold may be determined in response to the configuration operation of the user.

[0067] In some embodiments of the present application, sound optimization is performed on environmental noise data based on a dynamic range control (DRC) algorithm.

[0068] In some embodiments of the present application, sound optimization may be performed on the environmental noise data based on a DRC and / or recursive filter (Infinite Impulse Response, IIR) algorithm.

[0069] S206: Synthesize the optimized second sound data and the far-field sound data into recorded sound data.

[0070] S207: When the environmental noise data does not exceed the preset environmental sound threshold, the second sound data is used as the recorded sound data.

[0071] In some embodiments of the present application, after the recorded sound data is determined, the sound data may be saved as an audio file, or may be saved as the sound data of a corresponding video file.

[0072] Therefore, in summary, the recording method in the embodiment of the present application can use dual microphones to simultaneously collect the same environmental sound data when recording audio or video. After the first sound data obtained by the first microphone is subjected to noise reduction processing, pure voice data is obtained. Then, according to the first sound data and the second sound data obtained by the second microphone, the environmental noise data is separated. According to the relationship between the environmental noise data and the preset environmental sound threshold, the recorded sound data under the corresponding environmental conditions is determined. Based on this, the recorded first sound data can be adjusted and processed according to different environmental conditions to ensure the effect of recording under different environmental sound conditions, thereby meeting the user's sound requirements for audio or video recording and improving the user experience.

[0073] Based on the recording method provided in the above embodiment, the present application also provides a specific implementation of the recording device. Please refer to the following embodiment.

[0074] Figure 3 FIG. 1 is a schematic diagram of a recording device provided by another embodiment of the present application. Figure 3 As shown, in some embodiments of the present application, the recording device can be applied to a device provided with dual microphones, specifically including:

[0075] An acquisition module 301 is used to respectively acquire first sound data in an environment collected by a first microphone and a second microphone;

[0076] A processing module 302 is used to perform noise reduction processing on the first sound data collected by the first microphone to obtain second sound data;

[0077] A separation module 303, configured to separate the first sound data collected by the second microphone according to the second sound data to obtain environmental noise data;

[0078] The determination module 304 is used to determine the recorded sound data according to the environmental noise data and a preset environmental sound threshold.

[0079] Figure 3 Each module / unit in the device shown has the function of realizing Figure 1 and Figure 2 The functions of each step in the process can achieve the corresponding technical effects, which will not be described in detail here for the sake of brevity.

[0080] In summary, the recording device in the embodiment of the present application can be used to execute the recording method described in the above embodiment. Based on the recording method, the recorded first sound data can be adjusted and processed according to different environmental conditions to ensure the recording effect under different environmental sound conditions, thereby meeting the user's sound requirements for audio or video recording and improving the user experience.

[0081] Based on the recording method provided in the above embodiment, the present application also provides a specific implementation of the recording device. Please refer to the following embodiment.

[0082] Figure 4 This is a schematic diagram of the hardware structure of the recording device provided in some embodiments of the present application.

[0083] The recording device may include a first microphone (not shown in the figure), a second microphone (not shown in the figure), a processor 401, and a memory 402 storing computer program instructions.

[0084] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0085] The memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 402 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 402 is a non-volatile solid-state memory. In a specific embodiment, the memory 402 includes a read-only memory (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0086] The processor 401 implements any one of the recording methods in the above embodiments by reading and executing the computer program instructions stored in the memory 402 .

[0087] In one example, the recording device may further include a communication interface 403 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.

[0088] The communication interface 403 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0089] Bus 410 includes hardware, software or both, and the parts of the recording device are coupled to each other. For example, but not limitation, the bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnect (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0090] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0091] The recording device can execute the recording method in the embodiment of the present application, thereby realizing the combination Figure 1 and Figure 2 Describe the recording method.

[0092] In addition, in combination with the recording method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the recording methods in the above embodiment is implemented.

[0093] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0094] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0095] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0096] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0097] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A recording method, characterized in that: The method is applied to a device provided with dual microphones, and the method comprises: Acquire first sound data in an environment collected by a first microphone and a second microphone respectively, the first sound data including voice data and environmental noise data, the voice data including near-field sound data, that is, near-field voice data, and the environmental noise data including far-field sound data, that is, other sound data except the near-field sound data; Performing noise reduction processing on the first sound data collected by the first microphone to obtain second sound data; Separating the first sound data collected by the second microphone according to the second sound data to obtain environmental noise data; Determine the recorded sound data according to the ambient noise data and the preset ambient sound threshold; wherein, determining the recorded sound data according to the ambient noise data and the preset ambient sound threshold includes: When the ambient noise data exceeds a preset ambient sound threshold, synthesizing the second sound data and the far-field sound data into recorded sound data; When the environmental noise data does not exceed the preset environmental sound threshold, the second sound data is used as the recorded sound data.

2. The method according to claim 1, characterized in that Before synthesizing the second sound data and the far-field sound data into recorded sound data, the method further includes: Sound optimization is performed on the environmental noise data to obtain optimized environmental noise data.

3. The method according to claim 2, characterized in that Performing sound optimization on the environmental noise data, including: The sound is optimized based on the dynamic range adjustment (DRC) for the environmental noise data.

4. The method according to claim 1, characterized in that: The step of separating the first sound data collected by the second microphone according to the second sound data to obtain the environmental noise data includes: The first sound data collected by the second microphone is subtracted from the second sound data to obtain the environmental noise data.

5. The method according to claim 1, characterized in that The first sound data includes environmental noise data and voice data.

6. A recording device, characterized in that: The device is applied to a device provided with dual microphones, and the device comprises: An acquisition module, used to respectively acquire first sound data in an environment collected by a first microphone and a second microphone, wherein the first sound data includes voice data and environmental noise data, wherein the voice data includes near-field sound data, i.e., near-field voice data, and the environmental noise data includes far-field sound data, i.e., other sound data except the near-field sound data; A processing module, configured to perform noise reduction processing on the first sound data collected by the first microphone to obtain second sound data; a separation module, configured to separate the first sound data collected by the second microphone according to the second sound data to obtain environmental noise data; A determination module, used to determine the recorded sound data according to the environmental noise data and a preset environmental sound threshold; Wherein, the determination module is specifically used for: When the ambient noise data exceeds a preset ambient sound threshold, synthesizing the second sound data and the far-field sound data into recorded sound data; When the environmental noise data does not exceed the preset environmental sound threshold, the second sound data is used as the recorded sound data.

7. A recording device, characterized in that: The device comprises: a first microphone, a second microphone, a processor, and a memory storing computer program instructions; When the processor executes the computer program instructions, the recording method as described in any one of claims 1 to 5 is implemented.

8. A computer storage medium, characterized in that The computer storage medium stores computer program instructions, which, when executed by a processor, implement the recording method as described in any one of claims 1 to 5.

Images