Song processing method, electronic equipment, vehicle and storage medium

By acquiring and analyzing audio signals in intelligent singing mode, combining noise information and energy characteristics, dynamically adjusting subbands and music playback, the problem of low vocal detection accuracy in vehicle environments is solved and the user's singing experience is improved.

CN118942435APending Publication Date: 2024-11-12NIO TECH ANHUI CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411008821.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing singing software has low vocal detection accuracy due to noise interference in vehicle environments, and cannot effectively distinguish between high-noise and low-noise environments, affecting the user's singing experience.

Method used

By acquiring audio signals in intelligent singing mode, determining noise information and energy characteristics, adjusting sub-band frequency division points, combining time domain energy and frequency domain energy thresholds to determine whether the user's voice exists, and dynamically adjusting the original singing voice and accompaniment music Play, realize intelligent playback.

Benefits of technology

Accurately detecting user's voice in intelligent singing mode improves singing experience in vehicle environments and reduces the impact of noise interference on vocal detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942435A_ABST
    Figure CN118942435A_ABST
Patent Text Reader

Abstract

The invention is suitable for the field of vehicles, and provides a song processing method, an electronic device, a vehicle and a storage medium, and the method comprises the steps: obtaining an audio signal in an intelligent singing mode; determining noise information and energy feature information of the audio signal; determining whether sound of a user exists or not according to the noise information and the energy feature information; under the condition that the sound of the user exists, playing accompaniment music of the current song; under the condition that the sound of the user does not exist, the accompaniment music of the current song and the sound of the original song are played, whether the sound of the user exists or not can be accurately detected in the intelligent singing mode, intelligent playing can be carried out according to whether the sound of the user exists or not, and the singing experience of the user is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of vehicle technology, and in particular, relates to a song processing method, electronic equipment, vehicle and storage medium. Background Art

[0002] There are many kinds of singing software, but existing singing software usually has only two singing modes, one is to play the original singer's vocals of the song while singing, and the other is to choose to turn off the original singer's vocals while singing. When the original singer's vocals are played when singing, if the user is also singing, the original singer's vocals may interfere with the user's singing. When the original singer's vocals are turned off when singing, if the user is not singing, only music can be played, affecting the user's singing experience. In addition, when the singing software is applied to the singing scene in the vehicle, since there will be noise when the vehicle is stopped or driving, it will affect the accuracy of the detection of the user's voice. For example, when the noise is high, there will be missed detection, and when the noise is low, there will be false detection. Summary of the invention

[0003] The embodiments of the present application provide a song processing method, electronic device, vehicle and storage medium, which can accurately detect whether there is a user's voice in an intelligent singing mode, and perform intelligent playback based on whether there is a user's voice, thereby improving the user's singing experience.

[0004] In a first aspect, the present application provides a song processing method, comprising:

[0005] In the intelligent singing mode, obtain the audio signal;

[0006] Determining noise information and energy characteristic information of the audio signal;

[0007] Determining whether there is a user's voice according to the noise information and the energy characteristic information;

[0008] In the presence of the user's voice, output the accompaniment music of the current song;

[0009] In the absence of the user's voice, the accompaniment music and the original singer's voice of the current song are output.

[0010] In some embodiments, the energy characteristic information includes frequency domain energy characteristics and time domain energy characteristics;

[0011] The determining of the noise information and energy characteristic information of the audio signal comprises:

[0012] Determining the noise information and the frequency domain energy characteristics according to the frequency domain information of the audio signal;

[0013] Determining the time domain energy feature according to the time domain information of the audio signal;

[0014] Determining whether there is a user's voice according to the noise information and the energy characteristic information includes:

[0015] According to the noise information and the preset rules, respectively determine the time domain energy threshold and the frequency domain energy threshold;

[0016] It is determined whether the user's voice exists according to the time domain energy feature, the frequency domain energy feature, the time domain energy threshold, and the frequency domain energy threshold.

[0017] In some embodiments, the frequency domain energy features include sub-band energy features of a plurality of target sub-bands;

[0018] The determining the noise information and the frequency domain energy feature according to the frequency domain information of the audio signal comprises:

[0019] Converting the audio signal into a frequency domain signal, and determining the noise information according to the frequency domain signal;

[0020] adjusting the frequency division points of the plurality of target sub-bands at least according to the noise information;

[0021] The sub-band energy characteristics of the plurality of target sub-bands are determined respectively according to the frequency domain signal.

[0022] In some embodiments, the sub-band energy thresholds of the multiple target sub-bands included in the frequency domain energy threshold are further determined according to the noise information and the preset rule;

[0023] The determining whether there is a user's voice according to the time domain energy feature, the frequency domain energy feature, the time domain energy threshold, and the frequency domain energy threshold comprises:

[0024] When the time domain energy feature is less than the time domain energy threshold, determining that there is no user's voice;

[0025] When the time domain energy feature is greater than the time domain energy threshold and the sub-band energy of each target sub-band is greater than the corresponding sub-band energy threshold, determining that the user's voice exists;

[0026] In a case where the time domain energy feature is greater than the time domain energy threshold and there is at least one target sub-band whose sub-band energy is less than the corresponding sub-band energy threshold, it is determined that there is no user's voice.

[0027] In some embodiments, the noise information is a noise level;

[0028] The determining the noise information according to the frequency domain signal comprises:

[0029] Determine the differential feature and the frequency roll-off coefficient according to the frequency domain signal, and determine the noise level of the audio signal from a plurality of preset noise levels according to the differential feature and the frequency roll-off coefficient; wherein,

[0030] The differential feature is obtained by subtracting frequency domain signals of two adjacent time units in a target frequency portion, and the target frequency portion is a frequency portion in a frequency signal whose frequency is greater than a frequency threshold.

[0031] In some embodiments, the plurality of preset noise levels include a first noise level and a second noise level, and the noise corresponding to the first noise level is smaller than the noise corresponding to the second noise level;

[0032] Determining the noise level of the audio signal from a plurality of preset noise levels according to the differential characteristics of the frequency domain signal and the frequency roll-off coefficient, comprising:

[0033] When the differential characteristic is less than or equal to the differential characteristic threshold, or the frequency roll-off coefficient is less than or equal to the frequency roll-off coefficient threshold, determining that the noise level is the first noise level;

[0034] When the differential characteristic is greater than the differential characteristic threshold and the frequency roll-off coefficient is greater than the frequency roll-off coefficient threshold, the noise level is determined to be the second noise level.

[0035] In some embodiments, the plurality of preset noise levels include a first noise level and a second noise level, and the noise corresponding to the first noise level is smaller than the noise corresponding to the second noise level;

[0036] Adjusting the frequency division points of the plurality of target sub-bands at least according to the noise information includes:

[0037] determining a spectral centroid based on the frequency domain signal;

[0038] The frequency division points of the plurality of target sub-bands are adjusted according to the noise level and the spectral centroid; wherein,

[0039] In response to the first noise level and the spectral centroid being less than a spectral centroid threshold, reducing the frequency value of the frequency division point of the preset sub-frequency band to determine the multiple target sub-frequency bands;

[0040] In response to the first noise level and the spectral centroid being greater than or equal to the spectral centroid threshold, increasing the frequency value of the frequency division point of the preset sub-frequency band to determine the multiple target sub-frequency bands;

[0041] In response to the second noise level, the plurality of target sub-frequency bands are maintained as sub-frequency bands determined at a previous moment before the current moment.

[0042] In some embodiments, the step of playing the accompaniment music of the current song in the presence of the user's voice includes:

[0043] In the case where there is a user's voice, and in the case where the playing state of the current song is to play the accompaniment music of the current song and the voice of the original singer, within a first preset time length, the voice of the original singer in the music is gradually reduced to a first preset value, and the accompaniment music of the current song is played;

[0044] In the absence of the user's voice, playing the accompaniment music and the original singer's voice of the current song includes:

[0045] In the absence of the user's voice and when the playback state is to play the accompaniment music of the current song, within a second preset time length, the original singer's voice is gradually increased to a second preset value, and the accompaniment music of the current song and the original singer's voice are played, and the first preset value is less than the second preset value.

[0046] In some embodiments, the method further comprises:

[0047] When there is a user's voice, the user's voice is output after noise is removed from the audio signal.

[0048] In a second aspect, an embodiment of the present application provides a song processing device, comprising:

[0049] An acquisition module is used to acquire audio signals in the intelligent singing mode;

[0050] A first determination module, used to determine noise information and energy characteristic information of the audio signal;

[0051] A second determination module, used to determine whether there is a user's voice according to the noise information and the energy characteristic information;

[0052] The first output module is used to output the accompaniment music of the current song in the presence of the user's voice;

[0053] The second output module is used to output the accompaniment music and the original singer's voice of the current song in the absence of the user's voice.

[0054] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the computer program.

[0055] In a fourth aspect, an embodiment of the present application provides a vehicle, comprising: the electronic device, the sound receiving device and the speaker as described above, the electronic device being communicatively connected with the sound receiving device and the speaker, the sound receiving device being used to collect audio signals and send the audio signals to the electronic device, the speaker being used to output the accompaniment music of the current song, or, to output the accompaniment music of the current song and the sound of the original singer.

[0056] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the methods described above is implemented.

[0057] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the electronic device to execute any of the methods described above.

[0058] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0059] The song processing method provided in the embodiment of the present application obtains an audio signal in an intelligent singing mode; determines the noise information and energy characteristic information of the audio signal; determines whether there is a user's voice based on the noise information and the energy characteristic information; plays the accompaniment music of the current song when the user's voice exists; and plays the accompaniment music of the current song and the original singer's voice when the user's voice does not exist. This method can accurately detect whether there is a user's voice in the intelligent singing mode, and can perform intelligent playback based on whether there is a user's voice, thereby improving the user's singing experience.

[0060] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0062] Figure 1 A schematic diagram of a song processing method for implementing the present application;

[0063] Figure 2 A flowchart of a method for determining whether there is singing of a user according to the audio signal provided in an embodiment of the present application;

[0064] Figure 3 A structural schematic diagram of a song processing device provided in an embodiment of the present application;

[0065] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0067] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0068] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0069] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrases "if it is determined" or "if it is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce detected" or "in response to detecting" depending on the context.

[0070] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0071] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0072] Before introducing the embodiments of the present application, a brief introduction to the technical problems in the related art is given:

[0073] There are many kinds of existing singing software, but the existing singing software usually has only two singing modes, one is to play the original singer's voice when singing, and the other is to choose to turn off the original singer's voice when singing. When the original singer's voice is played when singing, if the user is also singing, the original singer's voice may interfere with the user's singing. When the original singer's voice is turned off when singing, if the user is not singing, only music can be played, affecting the user's singing experience. In addition, when the singing software is applied to the singing scene in the vehicle, there will be noise when the vehicle is stopped or driving, and the loudness difference of the noise is as high as 70dB of the noise dynamic range. The traditional human voice detection technology cannot take into account the high noise and low noise situations well, and it is easy to miss the detection when the noise is high and misdetection when the noise is low, that is, the detection accuracy is not high. In addition, the car cockpit There are unfavorable acoustic conditions such as small space and strong sound wave reflection. When amplifying human voice under no-microphone conditions (i.e., there is no handheld microphone to collect sound, and the sound is collected by the microphone in the user's environment), excessive acoustic echo cancellation is required to better avoid howling. Therefore, the obtained human voice will have a large loss in sound quality. This loss will bring difficulties to human voice detection. The specific reasons include: compared with the ideal human voice signal, the energy difference between each frequency becomes larger, that is, some frequencies will have very low energy values, and the frequency range in which this loss occurs is unpredictable; in addition, the loss frequency range of different people's voices is inconsistent. Although the frequency energy distribution of noise is relatively easy to calculate, the loss of each frequency of human voice is unknown, which makes the situation of noise masking human voice at each frequency complicated, thereby causing a decrease in the accuracy of human voice detection.

[0074] Based on the technical problems of the related art, the embodiments of the present application provide a song processing method that can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices. Taking the electronic device as a vehicle-mounted device as an example, the electronic device can be a vehicle computer. In some embodiments, the electronic device can also be a device embedded in the vehicle before leaving the factory. In some embodiments, the electronic device can also be a device installed in the vehicle later, or the electronic device can be a portable device that can be used on the vehicle in a mobile manner.

[0075] The functions implemented by the song processing method provided in the embodiment of the present application can be implemented by calling program codes by the processor of the electronic device, wherein the program codes can be stored in a computer storage medium.

[0076] The present application embodiment provides a song processing method. Figure 1 A flowchart of a song processing method provided for the implementation of this application is shown in FIG. Figure 1 As shown, the song processing method includes:

[0077] Step S101, in the intelligent singing mode, obtain the audio signal.

[0078] In an embodiment of the present application, the vehicle may provide a user interface, and the user may select a singing mode through the user interface. If the user selects the intelligent singing mode, the electronic device will be in the intelligent singing mode.

[0079] In some embodiments, the electronic device can obtain the last memorized singing mode and determine the last memorized singing mode as the current singing mode. If the last memorized singing mode is an intelligent singing mode, the electronic device will be in the intelligent singing mode.

[0080] In some embodiments, a voice interface can be provided on the vehicle, and the voice interface can be connected to the electronic device for communication, and the voice of the user can be acquired through the voice interface, and the singing mode can be selected through the voice.

[0081] In an embodiment of the present application, the singing mode may include at least one of: intelligent singing mode, original singing mode, and accompaniment mode.

[0082] In the embodiment of the present application, the original singing mode is to play the accompaniment music and the original singer's voice in the song when playing the song, the accompaniment mode is to play only the accompaniment music of the song, and the smart singing mode is to play the accompaniment music of the current song if there is a user's voice, and to play the accompaniment music of the current song and the original singer's voice if there is no user's voice. The accompaniment music of the current song and the original singer's voice can be stored in two files. When the accompaniment music of the current song and the original singer's voice need to be played, the electronic device can obtain the corresponding content from the two files respectively and play them; of course, the accompaniment music of the current song and the original singer's voice can also be a whole, and when they need to be played, they can be played directly.

[0083] In an embodiment of the present application, the electronic device can be connected to the audio input device for communication. When the intelligent singing mode is triggered, the electronic device can turn on the audio input device to collect audio signals through the audio input device. The audio input device can also be called a sound receiving device. The sound receiving device can include a microphone. The microphone can be connected to the electronic device for communication wirelessly or by wire. Taking the wireless connection as an example, the electronic device can be connected to the microphone for communication via Bluetooth, or can be connected to the microphone for communication via a radio frequency signal, or can be connected to the microphone for communication via an infrared signal. The microphone can be used to obtain sound to obtain an audio signal, and the audio signal can be sent to the electronic device, so that the electronic device obtains the audio signal. In some embodiments, the microphone can be embedded in the vehicle.

[0084] In the embodiment of the present application, the audio signal is an electrical signal that describes sound, which represents the characteristics of the sound wave. The characteristics of the sound wave may include: the frequency, amplitude and waveform of the sound wave, etc. The audio signal may be an analog signal or a digital signal. By converting the sound wave into an analog signal or a digital signal, subsequent processing can be facilitated.

[0085] In the embodiment of the present application, the audio signal may include: the user's voice and / or environmental noise, etc. The environmental noise may include: the noise generated by the vehicle when it is stationary or moving.

[0086] Step S102: determining noise information and energy characteristic information of the audio signal.

[0087] In an embodiment of the present application, the audio signal can be analyzed to determine noise information and energy characteristic information. When analyzing the audio signal, signal processing technology can be used to analyze the audio signal to determine the noise information and energy characteristic information of the audio signal.

[0088] In some embodiments, the noise information and energy characteristic information of the audio signal may be determined by a machine learning algorithm.

[0089] In the embodiment of the present application, the energy characteristic information is used to determine the strength and clarity of the signal, etc. The energy characteristic information may include: time domain energy characteristics and frequency domain energy characteristics. The noise information may be an indicator for quantifying the noise level and characteristics, and the noise information may be a noise level, which may be represented by different grades or values ​​to characterize the specific situation of the environmental noise.

[0090] Step S103: determining whether there is a user's voice according to the noise information and the energy characteristic information.

[0091] In the embodiment of the present application, a comprehensive judgment can be made as to whether the user's voice is present based on the noise information and the energy characteristic information.

[0092] In the embodiment of the present application, the energy characteristic information can be compared with the corresponding energy characteristic threshold to determine whether the user's voice exists. If the energy characteristic information is lower than the corresponding energy characteristic threshold, it can be considered that the user's voice does not exist, and if the energy characteristic information is greater than the energy characteristic threshold, it can be considered that the user's voice exists.

[0093] Step S104, in the presence of the user's voice, play the accompaniment music of the current song.

[0094] In the embodiment of the present application, when there is the user's voice, it can be considered that the user is singing. At this time, in order to avoid the original singer's interference with the user's singing, only the accompaniment music of the current song is played, and the original singer's voice of the current song is not played.

[0095] Step S105, in the absence of the user's voice, playing the accompaniment music of the current song and the original singer's voice.

[0096] In the embodiment of the present application, if there is no user's voice, it is considered that the user is not singing, and the accompaniment music of the current song and the original singer's voice are played.

[0097] The method provided by the embodiment of the present application obtains an audio signal in an intelligent singing mode; determines the noise information and energy characteristic information of the audio signal; determines whether there is a user's voice based on the noise information and the energy characteristic information; plays the accompaniment music of the current song when the user's voice exists; and plays the accompaniment music of the current song and the original singer's voice when the user's voice does not exist. This method can accurately detect whether there is a user's voice in the intelligent singing mode, and can perform intelligent playback based on whether there is a user's voice, thereby improving the user's singing experience.

[0098] In some embodiments, the energy feature information may include: frequency domain energy features and time domain energy features. Step S102 may be implemented by the following steps:

[0099] Step S1021: Determine the noise information and the frequency domain energy feature according to the frequency domain information of the audio signal.

[0100] In the embodiment of the present application, the audio signal is usually a time domain signal. In order to obtain noise information and frequency domain energy characteristics, the audio signal can be Fourier transformed to convert the time domain audio signal into a frequency domain signal. The Fourier transform can be a fast Fourier transform FFT.

[0101] In the embodiment of the present application, after the audio signal is converted into frequency domain information, the amplitude and phase of each frequency domain component can be calculated to determine the frequency domain energy characteristics. The noise signal can be represented by a noise gear, and the noise gear can include: a first noise gear and a second noise gear, the first noise gear can be considered as a low noise gear, and the second noise gear can be a high noise gear. The noise information can be judged by a noise threshold.

[0102] In the embodiment of the present application, the frequency domain energy feature may be a sub-band energy feature of multiple target sub-bands.

[0103] In the embodiment of the present application, the number of target sub-bands can be configured, and illustratively, the number can be configured to 3, and the 3 sub-target bands can correspond to a low frequency band, a mid-frequency band, and a high frequency band. In other embodiments, it can also be configured to two, for example, a low frequency band and a high frequency band.

[0104] In some embodiments, step S1021, determining the noise information and the frequency domain energy feature according to the frequency domain information of the audio signal, can be implemented by the following steps:

[0105] Step S1: convert the audio signal into a frequency domain signal, and determine the noise information according to the frequency domain signal.

[0106] In the embodiment of the present application, a fast Fourier transform (FFT) can be used to convert the audio signal in the time domain into a frequency domain signal. The formula of the fast Fourier transform can be expressed as:

[0107] X(f) = FFT(x[n]);

[0108] Among them, x[n] is the time domain audio information, X(f) is the frequency domain signal, and FFT is the fast Fourier transform.

[0109] In an embodiment of the present application, a differential feature and a frequency roll-off coefficient can be determined based on the frequency domain signal, and the noise level of the noise of the audio signal can be determined from multiple preset noise levels based on the differential feature and the frequency roll-off coefficient, wherein the differential feature is obtained by subtracting the frequency domain signals of two adjacent time units in a target frequency part, and the target frequency part is a frequency part in the frequency signal whose frequency is greater than a frequency threshold.

[0110] In an embodiment of the present application, there may be multiple preset noise gears. For example, the preset noise gears may be set to two, and the two noise gears may include: a first noise gear and a second noise gear, and the noise corresponding to the first noise gear is less than the noise corresponding to the second noise gear. The preset noise gear may be set to three, and the three noise gears may include: a first noise gear, a second noise gear, and a third noise gear. The first noise gear, the second noise gear, and the third noise gear may correspond to a low gear, a mid-range, and a high gear, respectively. Of course, in other embodiments, there may be other division methods.

[0111] In an embodiment of the present application, the differential feature can help capture the dynamic change characteristics of the audio signal. The target frequency part is the frequency part in the frequency signal whose frequency is greater than the frequency threshold, and the target frequency part can be considered as the high frequency and intermediate frequency parts. Since the intermediate frequency and high frequency parts contain more details and change information, especially in speech and music, these parts have an important impact on sound quality and clarity, and many noise types have strong energy in the intermediate frequency and high frequency parts, so by analyzing the differential features of these frequency parts, noise can be better identified and distinguished. The time unit is used to segment the audio signal. The audio signal can be segmented based on the time unit, and the frequency information of each time unit can be obtained after segmentation. Each unit obtained can be independently subjected to FTT transformation. After the change, the frequency domain signals of two adjacent time units can be subtracted to obtain the differential feature.

[0112] In the embodiment of the present application, the frequency roll-off coefficient is an indicator that describes the energy distribution interval of the audio in the frequency dimension. The frequency information can be expressed as: a curve or spectrum diagram of the frequency changing over time. The calculation of the frequency roll-off coefficient requires processing the curve or spectrum diagram of the frequency changing over time, and accumulating and calculating the energy in the direction from low frequency to high frequency. When the accumulation reaches a preset proportion of all the energy in the spectrum diagram, the frequency point index at this time is the obtained frequency roll-off coefficient. The frequency distribution characteristics of the audio can be analyzed according to the frequency roll-off coefficient to distinguish between noise signals and singing signals.

[0113] In some embodiments, when calculating the frequency roll-off coefficient, the obtained frequency roll-off coefficient is also appropriately smoothed to reduce the influence of noise on the calculation result.

[0114] In the embodiment of the present application, the size relationship between the differential feature and the differential feature threshold, as well as the size relationship between the frequency roll-off coefficient and the frequency roll-off coefficient threshold can be compared to determine the noise level of the audio signal.

[0115] In the embodiment of the present application, when the differential feature is greater than the differential feature threshold and the frequency roll-off coefficient is greater than the frequency roll-off coefficient threshold, the noise level is determined to be the second noise level.

[0116] In the embodiment of the present application, the differential characteristic threshold and the frequency roll-off coefficient threshold can be obtained based on experiments.

[0117] In the embodiment of the present application, it is possible to analyze whether the differential feature is greater than the differential feature threshold and whether the frequency roll-off coefficient is greater than the frequency roll-off coefficient threshold. If both the differential feature and the frequency roll-off coefficient meet the threshold conditions, the noise level is determined to be the second noise level. The second noise level can be a high noise level,

[0118] In the embodiment of the present application, when the differential characteristic is less than or equal to the differential characteristic threshold or the frequency roll-off coefficient is less than or equal to the frequency roll-off coefficient threshold, the noise level is determined to be the first noise level.

[0119] Step S2: adjusting the frequency division points of the plurality of target sub-bands at least according to the noise information.

[0120] In the embodiment of the present application, the method of adjusting the frequency division points of multiple target sub-bands is different for different noise information. If the noise information is the first noise level, it is necessary to adjust according to the spectral centroid, and if the noise information is the second noise level, the frequency division points of the target sub-bands are not adjusted, so as to maintain the multiple target sub-bands as the sub-bands determined at the last moment before the current moment.

[0121] It should be noted that if, for other purposes, the noise information is divided into more than 2 noise gears, in order to achieve the relevant purposes of this application, according to actual conditions, the scheme of actually summarizing more than 2 noise gears into 2 noise gears still falls within the scope of this application. For other purposes, the noise information is divided into gear A, gear B, gear C and gear D in order from small to large according to the size of the corresponding noise. When achieving the relevant purposes of this application, gear A can be considered as the first noise gear, and gear B, gear C and gear D can be considered as the second noise gear.

[0122] In the embodiment of the present application, step S2 can be implemented by the following steps:

[0123] Step S21, determining the spectral centroid according to the frequency domain signal.

[0124] In the embodiment of the present application, the spectral centroid is the weighted average frequency of the spectrum, which can be used to represent the main frequency components of the audio signal. It is usually calculated by weighted average, and the formula is:

[0125] Spectral centroid = Σ(f*P(f)) / ΣP(f), where f is the frequency and P(f) is the power spectral density corresponding to the frequency.

[0126] Step S22: adjusting the frequency division points of the plurality of target sub-bands according to the noise level and the spectrum centroid.

[0127] In an embodiment of the present application, in response to the first noise level and the spectral centroid being less than a spectral centroid threshold, the frequency value of the division point of the preset sub-band is reduced to determine the multiple target sub-bands; in response to the first noise level and the spectral centroid being greater than or equal to the spectral centroid threshold, the frequency value of the division point of the preset sub-band is increased to determine the multiple target sub-bands; in response to the second noise level, the multiple target sub-bands are maintained as the sub-bands determined at the previous moment before the current moment.

[0128] In the embodiment of the present application, the frequency value of increase or decrease can be configured. For example, the frequency value of each increase or decrease can be set to 50HZ. Of course, in some embodiments, it can also be configured to 100HZ.

[0129] In the embodiment of the present application, in the case of the first noise level, that is, when the noise is low, the frequency value of the crossover point in the initial sub-band is dynamically adjusted according to the spectral centroid, thereby realizing the dynamic change of the sub-band. Since the frequency value of the crossover point is adjusted in the case of low noise, the sub-band setting can be made more reasonable, so that the voice attributes of the singer can be better captured, and the appropriate sound detection threshold conditions can be set, thereby improving the accuracy of the algorithm detection.

[0130] In the embodiment of the present application, when the noise information is the second noise level, it can be considered that the noise is high, and the sub-frequency band is the sub-frequency band before the current moment. That is, when the noise is high, the sub-frequency band is not dynamically adjusted.

[0131] Exemplarily, the preset sub-frequency bands include: low frequency band, mid-frequency band, and high frequency band, low frequency band: 0HZ-1000HZ, mid-frequency band: 1000HZ-2000HHZ, high frequency band: 2000HZ and above, the crossover points include: 1000HZ and 2000HZ, if you want to adjust the frequency value of the crossover point, for example, reduce it by 50HZ, then the adjusted crossover points are 950HZ and 1950HZ, then the target sub-frequency bands are: low frequency band: 0HZ-950HZ, mid-frequency band: 950HZ-1950HZ, high frequency band: above 1950Hz.

[0132] The method provided in the embodiment of the present application determines the spectral centroid according to the frequency domain signal and adjusts the crossover points of the multiple target sub-bands according to the noise level and the spectral centroid to achieve dynamic adjustment of the crossover points. After the crossover points are adjusted, multiple target sub-bands are obtained, so that the sub-band energy characteristics of the multiple target sub-bands can be calculated.

[0133] Step S3: determining sub-band energy characteristics of the plurality of target sub-bands respectively according to the frequency domain signal.

[0134] In an embodiment of the present application, after determining the target sub-band, the corresponding frequency domain signal can be screened out to calculate its energy, so that the energy of each target sub-band can be obtained. Continuing with the above example, the sub-band energy characteristics corresponding to the range of 0HZ-950HZ can be determined, as well as the sub-band energy characteristics corresponding to the range of 950HZ-1950HZ, and the sub-band energy characteristics corresponding to the range above 1950Hz can be determined.

[0135] Step S1022: Determine the time domain energy feature according to the time domain information of the audio signal.

[0136] In an embodiment of the present application, the time domain information of the audio signal can be represented as waveform data, and the time domain energy characteristics can be obtained by calculating the short-time energy, for example, calculating the energy of each frame.

[0137] In the embodiment of the present application, the time domain energy feature may be time domain mean square energy, which indicates the energy of a signal in the time domain. The time domain mean square energy may be obtained by calculating the square of the amplitude of the signal and averaging the square.

[0138] In some embodiments, step S103, determining whether there is a user's voice according to the noise information and the energy characteristic information, includes:

[0139] Step S1031, determining a time domain energy threshold and a frequency domain energy threshold respectively according to the noise information and a preset rule.

[0140] In the embodiment of the present application, the preset rule may be a set corresponding relationship, and different noise levels may correspond to different time domain energy thresholds and frequency domain energy thresholds. For example, the first noise level corresponds to the first time domain energy threshold and the first frequency domain energy threshold, and the second noise level corresponds to the second time domain energy threshold and the second frequency domain energy threshold.

[0141] In the embodiment of the present application, different noise levels have corresponding energy characteristics. In the embodiment of the present application, when the noise corresponding to the first noise level is smaller than the noise corresponding to the second noise level, the first time domain energy threshold may be smaller than the second time domain energy threshold, and the first frequency domain energy threshold is smaller than the second frequency domain energy threshold.

[0142] In the embodiment of the present application, the time domain energy threshold can be calculated based on the sample data, a noise sample can be collected, and then the time domain energy of the noise sample is calculated to obtain a noise energy value. When there are multiple noise samples, the average value of the noise energy value can be calculated to obtain the time domain energy of the noise sample, and the time domain energy can be determined as the time domain energy threshold. For example, taking the first time domain mean square energy threshold corresponding to the first noise gear as an example, a noise sample of the first noise gear can be obtained, and then the noise sample of the first noise gear is calculated to obtain the first time domain energy threshold.

[0143] Similarly, the frequency domain energy threshold can also be calculated based on the sample data of the noise.

[0144] In an embodiment of the present application, after calculating the time domain energy threshold and frequency domain energy threshold corresponding to the noise level, a correspondence between each noise level and the time domain energy threshold and the frequency domain energy threshold can be established to obtain a preset rule. After the noise information is determined, the time domain energy threshold and the frequency domain energy threshold can be determined respectively based on the preset rule.

[0145] Step S1032: determine whether there is a user's voice according to the time domain energy feature, the frequency domain energy feature, the time domain energy threshold, and the frequency domain energy threshold.

[0146] In the embodiment of the present application, the size relationship between the time domain energy threshold and the time domain energy feature can be compared, and the size relationship between the frequency band energy feature and the frequency band energy threshold can be compared at the same time. Through comprehensive analysis, it is determined whether the user's voice exists.

[0147] In some embodiments, when the time domain energy feature is less than the time domain energy threshold, it is determined that the user's voice is not present; when the time domain energy feature is greater than the time domain energy threshold and the sub-band energy of each target sub-band is greater than the corresponding sub-band energy threshold, it is determined that the user's voice is present; when the time domain energy feature is greater than the time domain energy threshold and there is at least one target sub-band whose sub-band energy is less than the corresponding sub-band energy threshold, it is determined that the user's voice is not present.

[0148] In an embodiment of the present application, if the sub-band energy of each sub-band is greater than the sub-band energy threshold of the corresponding sub-band, it indicates that there is a strong sound signal within the frequency domain range, and if the sub-band energy of the sub-band is less than the sub-band energy threshold of the corresponding sub-band, it is determined that there is no strong sound signal within the frequency domain range.

[0149] In the embodiment of the present application, by comprehensively considering the time domain energy characteristics and the sub-band energy, it can be determined whether the user's voice exists in the audio.

[0150] In some embodiments, the step of playing the accompaniment music of the current song in the presence of the user's voice includes:

[0151] In the presence of the user's voice, and when the playback state of the current song is to play the accompaniment music and the original singer's voice of the current song, within a first preset time length, the voice of the original singer in the music is gradually reduced to a first preset value, and the accompaniment music of the current song is played.

[0152] In the embodiment of the present application, if the current song is a song stored in the electronic device, the playing status can be queried through the query tool. If the current song is an online song, the playing status of the current song can be queried by calling the API interface.

[0153] In the embodiment of the present application, the playing status may include: playing the accompaniment music of the current song and the original singer's voice or playing the accompaniment music of the current song.

[0154] In the embodiment of the present application, the first preset duration can be configured, and illustratively, can be configured as 2S. The number of reductions can be set, and can be set to 4 times. For example, the original voice of the current song is gradually reduced 4 times within 2S.

[0155] In the embodiment of the present application, the first preset value may be 0% of the original singer's voice, that is, within the first preset time, the proportion of the original singer's voice is gradually reduced until it completely disappears.

[0156] In some embodiments, in the absence of the user's voice, playing the accompaniment music and the original singer's voice of the current song includes:

[0157] In the absence of the user's voice and when the playback state is to play the accompaniment music of the current song, within a second preset time length, the original singer's voice is gradually increased to a second preset value, and the accompaniment music of the current song and the original singer's voice are played, and the first preset value is less than the second preset value.

[0158] In the embodiment of the present application, the second preset duration can be configured, and the first preset duration and the second preset duration can be the same. In some embodiments, they can also be different. For example, the second preset duration can be configured as 2S, and the number of increases can be set, for example, it can be set to 4 times, and the second preset value can be 100%. That is, the voice of the original singer of the current song is gradually increased to 100% within the second preset duration.

[0159] In the embodiment of the present application, gradually lowering the original singer's voice or gradually increasing the original singer's voice helps to create a more layered and dynamic musical performance, allowing the audience to better experience the musical work.

[0160] In some embodiments, while executing step S104, the method further includes:

[0161] When there is a user's voice, the user's voice is output after noise is removed from the audio signal.

[0162] In the embodiment of the present application, noise elimination technology can be used to extract noise information, thereby obtaining the user's voice after noise is removed.

[0163] In the embodiment of the present application, the noise spectrum in the sound signal can be estimated and subtracted from the total signal to obtain the user's voice after the noise is removed.

[0164] In some embodiments, a deep learning model may be used to remove noise.

[0165] The method provided in the embodiment of the present application outputs the user's voice after noise is removed from the audio signal, which can make the user's voice clearer.

[0166] According to the above-mentioned embodiments, the present application further provides a method for determining whether there is a user's voice according to the audio signal, taking the user's voice as singing as an example. Figure 2 A flowchart of a method for determining whether there is singing of a user according to the audio signal provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the method includes:

[0167] After obtaining the audio signal, feature extraction is performed: short-time mean square energy in the time domain, and the audio is transformed from the time domain to the frequency domain through FFT. After transformation to the frequency domain, the extracted features include: spectral centroid, first-order FFT differential feature, and frequency roll-off coefficient. Noise information can be obtained by performing noise estimation through the first-order FFT differential feature and the frequency roll-off coefficient. It can be determined whether the noise information is a high noise gear or a low noise gear, and the time domain short-time mean square energy threshold A and the sub-band energy threshold B are set according to the noise gear. Then it is determined whether the time domain short-time mean square energy is greater than the time domain short-time mean square energy threshold. If not, it is determined that the user is not singing. If so, the audio is dynamically divided into three sub-bands according to the spectral centroid, and the sub-band energy of each sub-band is calculated. The energy of each sub-band is compared with the corresponding sub-band energy threshold B. If there is a sub-band energy less than the corresponding sub-band energy threshold B, it is determined that the user is not singing. If all sub-band energies are greater than the corresponding sub-band energy threshold B, it is determined that the user is singing.

[0168] In the embodiment of the present application, when the audio is dynamically divided into frequency bands according to the spectral centroid, if the spectral centroid is low, the frequency division point of the frequency band is also set to be low, that is, the frequency division point is reduced; if the spectral centroid is high, the frequency division point of the frequency band is also set to be high, that is, the frequency division point is increased. By dynamically adjusting the frequency division point, it can be ensured that singers with different vocal ranges can be captured with more reasonable frequency division sub-band characteristics. When the audio is dynamically divided into frequency bands according to the spectral centroid, it is only performed when the noise information is at a low noise level.

[0169] In an embodiment of the present application, the sub-frequency band can be divided into three sub-frequency bands, corresponding to bass, middle tone, and treble. For example, when the calculated spectral centroid is small, the two crossover points of the three sub-frequency bands are respectively reduced by a certain Hz. When the calculated spectral centroid is large, the two crossover points of the three sub-frequency bands are respectively increased by a certain Hz.

[0170] In the embodiment of the present application, different noise levels correspond to different time domain mean square energy thresholds and sub-band energy thresholds. After the noise level is determined, the time domain mean square energy threshold and sub-band energy threshold can be determined.

[0171] In the embodiment of the present application, a differential feature threshold and a frequency roll-off coefficient threshold can be set, and noise information can be determined by comparing the first-order FFT differential feature, the frequency roll-off coefficient, the differential feature threshold, and the frequency roll-off coefficient threshold. If the first-order FFT differential feature is greater than the differential feature threshold and the frequency roll-off coefficient is greater than the frequency roll-off coefficient threshold, it is considered to be a high noise gear. If the first-order FFT differential feature is greater than the differential feature threshold and the frequency roll-off coefficient is greater than the frequency roll-off coefficient threshold, it is considered to be a low noise gear.

[0172] In the embodiment of the present application, the differential characteristic threshold and the frequency roll-off coefficient threshold of each vehicle model can be obtained according to actual measurement. Different vehicle models may have different differential characteristic thresholds and frequency roll-off coefficient thresholds.

[0173] The method provided in the embodiment of the present application can detect and identify the user's voice more accurately, thereby realizing an intelligent singing mode.

[0174] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0175] According to the aforementioned embodiments, the embodiments of the present application provide a song processing device, and the modules included in the device, as well as the units included in the modules, can be implemented by a processor in a computer device; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing) or a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0176] The present application embodiment provides a song processing device, Figure 3 A structural diagram of a song processing device provided in an embodiment of the present application, such as Figure 3 As shown, the song processing device 300 includes:

[0177] An acquisition module 301 is used to acquire an audio signal in an intelligent singing mode;

[0178] A first determination module 302, configured to determine noise information and energy characteristic information of the audio signal;

[0179] A second determination module 303, configured to determine whether there is a user's voice according to the noise information and the energy characteristic information;

[0180] The first output module 304 is used to output the accompaniment music of the current song in the presence of the user's voice;

[0181] The second output module 305 is used to output the accompaniment music and the original singer's voice of the current song in the absence of the user's voice.

[0182] In some embodiments, the energy characteristic information includes frequency domain energy characteristics and time domain energy characteristics;

[0183] The first determination module includes:

[0184] A first determining unit, configured to determine the noise information and the frequency domain energy feature according to the frequency domain information of the audio signal;

[0185] A second determining unit, configured to determine the time domain energy feature according to the time domain information of the audio signal;

[0186] The second determination module includes:

[0187] A third determining unit, configured to determine a time domain energy threshold and a frequency domain energy threshold respectively according to the noise information and a preset rule;

[0188] The fourth determination unit is used to determine whether there is a user's voice according to the time domain energy feature, the frequency domain energy feature, the time domain energy threshold, and the frequency domain energy threshold.

[0189] In some embodiments, the frequency domain energy features include sub-band energy features of a plurality of target sub-bands;

[0190] The first determining unit includes:

[0191] A first determining subunit, configured to convert the audio signal into a frequency domain signal, and determine the noise information according to the frequency domain signal;

[0192] an adjusting subunit, configured to adjust the frequency division points of the plurality of target sub-frequency bands at least according to the noise information;

[0193] The second determining subunit is configured to respectively determine the sub-band energy characteristics of the plurality of target sub-bands according to the frequency domain signal.

[0194] In some embodiments, the song processing device is further used to further determine the sub-band energy thresholds of the multiple target sub-bands included in the frequency domain energy threshold according to the noise information and the preset rule;

[0195] The fourth determining unit includes:

[0196] A third determining subunit is used to determine that there is no user's voice when the time domain energy feature is less than the time domain energy threshold;

[0197] A fourth determination subunit, configured to determine that there is a user's voice when the time domain energy feature is greater than the time domain energy threshold and the sub-band energy of each target sub-band is greater than the corresponding sub-band energy threshold;

[0198] The fifth determination subunit is configured to determine that there is no user's voice when the time domain energy feature is greater than the time domain energy threshold and there is at least one target sub-band whose sub-band energy is less than the corresponding sub-band energy threshold.

[0199] In some embodiments, the noise information is a noise level;

[0200] The determining the noise information according to the frequency domain signal comprises:

[0201] Determine the differential feature and the frequency roll-off coefficient according to the frequency domain signal, and determine the noise level of the audio signal from a plurality of preset noise levels according to the differential feature and the frequency roll-off coefficient; wherein,

[0202] The differential feature is obtained by subtracting frequency domain signals of two adjacent time units in a target frequency portion, and the target frequency portion is a frequency portion in a frequency signal whose frequency is greater than a frequency threshold.

[0203] In some embodiments, the plurality of preset noise levels include a first noise level and a second noise level, and the noise corresponding to the first noise level is smaller than the noise corresponding to the second noise level;

[0204] Determining the noise level of the audio signal from a plurality of preset noise levels according to the differential characteristics of the frequency domain signal and the frequency roll-off coefficient, comprising:

[0205] When the differential characteristic is less than or equal to the differential characteristic threshold, or the frequency roll-off coefficient is less than or equal to the frequency roll-off coefficient threshold, determining that the noise level is the first noise level;

[0206] When the differential characteristic is greater than the differential characteristic threshold and the frequency roll-off coefficient is greater than the frequency roll-off coefficient threshold, the noise level is determined to be the second noise level.

[0207] In some embodiments, the plurality of preset noise levels include a first noise level and a second noise level, and the noise corresponding to the first noise level is smaller than the noise corresponding to the second noise level;

[0208] Adjusting the frequency division points of the plurality of target sub-bands at least according to the noise information includes:

[0209] determining a spectral centroid based on the frequency domain signal;

[0210] The frequency division points of the plurality of target sub-bands are adjusted according to the noise level and the spectral centroid; wherein,

[0211] In response to the first noise level and the spectral centroid being less than a spectral centroid threshold, reducing the frequency value of the frequency division point of the preset sub-frequency band to determine the multiple target sub-frequency bands;

[0212] In response to the first noise level and the spectral centroid being greater than or equal to the spectral centroid threshold, increasing the frequency value of the frequency division point of the preset sub-frequency band to determine the multiple target sub-frequency bands;

[0213] In response to the second noise level, the plurality of target sub-frequency bands are maintained as sub-frequency bands determined at a previous moment before the current moment.

[0214] In some embodiments, the first output module includes:

[0215] A first output unit is used to, in the case where there is a user's voice and the current song is in the playing state of playing the accompaniment music of the current song and the voice of the original singer, gradually reduce the voice of the original singer in the music to a first preset value within a first preset time length, and play the accompaniment music of the current song;

[0216] The second output module includes:

[0217] The second output unit is used to gradually increase the voice of the original singer to a second preset value within a second preset time length and play the accompaniment music of the current song and the voice of the original singer in the absence of the user's voice and when the playing state is to play the accompaniment music of the current song, and the first preset value is smaller than the second preset value.

[0218] In some embodiments, the song processing device further includes:

[0219] The third output module is used to output the user's voice after noise is removed from the audio signal when the user's voice exists.

[0220] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0221] in addition, Figure 3 The song processing device shown can be a software unit, a hardware unit, or a combination of software and hardware units built into an existing vehicle, or it can be integrated into the vehicle as an independent pendant, or it can exist as an independent terminal device.

[0222] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0223] Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 3 of this embodiment may include: at least one processor 30 ( Figure 4 Only one processor 30 is shown in the figure), a memory 31, and a computer program 32 stored in the memory 31 and executable on at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above-mentioned method embodiments are implemented, for example Figure 1 Steps S101 to S105 in the illustrated embodiment. Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 3 Functions of modules 301 to 305 are shown.

[0224] Exemplarily, the computer program 32 may be divided into one or more modules / units, one or more modules / units are stored in the memory 31, and are executed by the processor 30 to complete the present application. One or more modules / units may be a series of computer program 32 instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3.

[0225] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program 32. When the computer program 32 is executed by the processor 30, the steps in the above-mentioned method embodiments can be implemented.

[0226] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the vehicle can implement the steps in the above-mentioned method embodiments when the computer program product is executed.

[0227] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. According to this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program 32, and the computer program 32 can be stored in a computer-readable storage medium, and the computer program 32 can implement the steps of the above-mentioned various method embodiments when executed by the processor 30. Among them, the computer program 32 includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the terminal, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0228] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0229] An embodiment of the present application further provides a vehicle, comprising: the electronic device, sound receiving device and speaker provided in the above embodiment, the electronic device is communicatively connected with the sound receiving device and the speaker, the sound receiving device is used to collect audio signals and send the audio signals to the electronic device, and the speaker is used to output the accompaniment music of the current song, or, output the accompaniment music of the current song and the sound of the original singer.

[0230] In some embodiments, the sound receiving device includes: a vehicle-mounted microphone, and the vehicle-mounted microphone is embedded in the vehicle.

[0231] In the embodiment of the present application, a singing scene without a microphone can be realized by setting a car microphone.

[0232] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0233] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0234] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0235] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

[0236] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0237] The user personal information processed by the applicant will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, which may involve the user's account information, device information, driving information, vehicle information or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0238] The Applicant attaches great importance to the security of user personal information and has adopted reasonable and feasible security protection measures that comply with industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

Claims

1. A song processing method, characterized in that: include: In the intelligent singing mode, obtain the audio signal; Determining noise information and energy characteristic information of the audio signal; Determining whether there is a user's voice according to the noise information and the energy characteristic information; In the presence of the user's voice, output the accompaniment music of the current song; In the absence of the user's voice, the accompaniment music and the original singer's voice of the current song are output.

2. The method according to claim 1, characterized in that The energy characteristic information includes frequency domain energy characteristics and time domain energy characteristics; The determining of the noise information and energy characteristic information of the audio signal comprises: Determining the noise information and the frequency domain energy characteristics according to the frequency domain information of the audio signal; Determining the time domain energy feature according to the time domain information of the audio signal; Determining whether there is a user's voice according to the noise information and the energy characteristic information includes: According to the noise information and the preset rules, respectively determine the time domain energy threshold and the frequency domain energy threshold; It is determined whether the user's voice exists according to the time domain energy feature, the frequency domain energy feature, the time domain energy threshold, and the frequency domain energy threshold.

3. The method according to claim 2, characterized in that The frequency domain energy features include sub-band energy features of multiple target sub-bands; The determining the noise information and the frequency domain energy feature according to the frequency domain information of the audio signal comprises: Converting the audio signal into a frequency domain signal, and determining the noise information according to the frequency domain signal; adjusting the frequency division points of the plurality of target sub-bands at least according to the noise information; The sub-band energy characteristics of the plurality of target sub-bands are determined respectively according to the frequency domain signal.

4. The method according to claim 3, characterized in that: Further determining the sub-band energy thresholds of the multiple target sub-bands included in the frequency domain energy threshold according to the noise information and the preset rule; The determining whether there is a user's voice according to the time domain energy feature, the frequency domain energy feature, the time domain energy threshold, and the frequency domain energy threshold comprises: When the time domain energy feature is less than the time domain energy threshold, determining that there is no user's voice; When the time domain energy feature is greater than the time domain energy threshold and the sub-band energy of each target sub-band is greater than the corresponding sub-band energy threshold, determining that the user's voice exists; In a case where the time domain energy feature is greater than the time domain energy threshold and there is at least one target sub-band whose sub-band energy is less than the corresponding sub-band energy threshold, it is determined that there is no user's voice.

5. The method according to claim 3, characterized in that: The noise information is the noise level; The determining the noise information according to the frequency domain signal comprises: Determine the differential feature and the frequency roll-off coefficient according to the frequency domain signal, and determine the noise level of the audio signal from a plurality of preset noise levels according to the differential feature and the frequency roll-off coefficient; wherein, The differential feature is obtained by subtracting frequency domain signals of two adjacent time units in a target frequency portion, and the target frequency portion is a frequency portion in a frequency signal whose frequency is greater than a frequency threshold.

6. The method according to claim 5, characterized in that The plurality of preset noise levels include a first noise level and a second noise level, and the noise corresponding to the first noise level is smaller than the noise corresponding to the second noise level; Determining the noise level of the audio signal from a plurality of preset noise levels according to the differential characteristics of the frequency domain signal and the frequency roll-off coefficient, comprising: When the differential characteristic is less than or equal to the differential characteristic threshold, or the frequency roll-off coefficient is less than or equal to the frequency roll-off coefficient threshold, determining that the noise level is the first noise level; When the differential characteristic is greater than the differential characteristic threshold and the frequency roll-off coefficient is greater than the frequency roll-off coefficient threshold, the noise level is determined to be the second noise level.

7. The method according to claim 5, characterized in that The plurality of preset noise levels include a first noise level and a second noise level, and the noise corresponding to the first noise level is smaller than the noise corresponding to the second noise level; Adjusting the frequency division points of the plurality of target sub-bands at least according to the noise information includes: determining a spectral centroid based on the frequency domain signal; The frequency division points of the plurality of target sub-bands are adjusted according to the noise level and the spectral centroid; wherein, In response to the first noise level and the spectral centroid being less than a spectral centroid threshold, reducing the frequency value of the frequency division point of the preset sub-frequency band to determine the multiple target sub-frequency bands; In response to the first noise level and the spectral centroid being greater than or equal to the spectral centroid threshold, increasing the frequency value of the frequency division point of the preset sub-frequency band to determine the multiple target sub-frequency bands; In response to the second noise level, the plurality of target sub-frequency bands are maintained as sub-frequency bands determined at a previous moment before the current moment.

8. The method according to any one of claims 1 to 7, characterized in that: The method of playing the accompaniment music of the current song in the presence of the user's voice comprises: In the case where there is a user's voice, and in the case where the playing state of the current song is to play the accompaniment music of the current song and the voice of the original singer, within a first preset time length, the voice of the original singer in the music is gradually reduced to a first preset value, and the accompaniment music of the current song is played; In the absence of the user's voice, playing the accompaniment music and the original singer's voice of the current song includes: In the absence of the user's voice and when the playback state is to play the accompaniment music of the current song, within a second preset time length, the original singer's voice is gradually increased to a second preset value, and the accompaniment music of the current song and the original singer's voice are played, and the first preset value is less than the second preset value.

9. The method according to claim 8, characterized in that The method further comprises: When there is a user's voice, the user's voice is output after noise is removed from the audio signal.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

11. A vehicle, characterized in that: include: The electronic device, sound receiving device and speaker as described in claim 10, wherein the electronic device is communicatively connected with the sound receiving device and the speaker, the sound receiving device is used to collect audio signals and send the audio signals to the electronic device, and the speaker is used to output the accompaniment music of the current song, or to output the accompaniment music of the current song and the sound of the original singer.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.