A vehicle-mounted music recognition method

Abstract

The application provides a vehicle-mounted music recognition method, which comprises the following steps: step 1, determining whether a command for starting music recognition is received; if yes, the next step is performed; otherwise, the current step is repeatedly performed; step 2, starting audio recording, recording current music with a preset time length, generating a recording file after pretreatment and saving the recording file in a cache; step 3, uploading the recording file to a server, receiving recognition information returned by the server and determining whether the current music is free to download according to the recognition information; if yes, step 5 is performed; otherwise, the next step is performed; step 4, determining whether a command for agreeing to pay for downloading is received; if yes, the server is accepted and the next step is performed; otherwise, step 7 is performed; step 5, downloading corresponding music and performing the next step; step 6, playing the downloaded music; and step 7, exiting the music recognition mode. The application improves the intelligent degree of vehicle-mounted music recognition.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and in particular to a method for in-vehicle music recognition. Background Technology

[0002] With the development of in-vehicle electronic technology, in-vehicle infotainment systems have provided drivers with increasingly rich functions, but audio-visual functions have always been one of the most basic and important functions.

[0003] When radio stations play music, they sometimes don't provide information such as song title or artist, and the in-car infotainment system doesn't save any playback history, so you can't get relevant information when you encounter music you like.

[0004] Music recognition functionality is now available on mobile phones. Users can record music clips on their phones, upload them to music servers for searching and recognition, and then send the results back to the user. However, there are safety concerns when this solution is directly applied to in-vehicle environments. Summary of the Invention

[0005] This invention provides a method for recognizing in-vehicle music, aiming to overcome the shortcomings of existing technologies and improve the intelligence level of in-vehicle music recognition.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for recognizing in-vehicle music includes:

[0008] Step 1: Determine if a command to start music recognition has been received. If yes, proceed to the next step; otherwise, repeat this step.

[0009] Step 2: Start audio recording, record the current music for a preset duration, generate a recording file after preprocessing, and save it to the cache;

[0010] Step 3: Upload the recorded file to the server, receive the identification information returned by the server, and determine whether the current music is available for free download based on the identification information. If yes, proceed to step 5; otherwise, proceed to the next step.

[0011] Step 4: Determine if you have received an instruction to agree to pay for the download. If yes, accept the server's push and proceed to the next step; otherwise, proceed to step 7.

[0012] Step 5: Download the corresponding music and proceed to the next step;

[0013] Step 6: Play the downloaded music;

[0014] Step 7: Exit music recognition mode.

[0015] Specifically, step 6 includes:

[0016] Step 601: Obtain the priority of the audio playback thread currently running in the foreground. If the priority of the audio playback thread running in the foreground is lower than the priority of the music player, proceed to the next step; otherwise, repeat this step.

[0017] Step 602: Obtain the current road condition level parameter. If the current road condition level parameter is greater than the predefined road condition level parameter, return to the previous step; otherwise, proceed to the next step.

[0018] Step 603: Obtain the current user status level parameter. If the current user status level parameter is greater than the predefined user status level parameter, return to step 601; otherwise, issue a preset prompt message.

[0019] Step 604: Determine whether a confirmation instruction to play has been received. If yes, play the downloaded music; otherwise, end the process.

[0020] Furthermore, the procedure includes the following steps after step 5 and before step 6:

[0021] Step 51: Compare the downloaded music with the recorded file to determine if they are consistent. If they are, proceed to step 6; otherwise, proceed to the next step.

[0022] Step 52: Report the identification error to the server and determine whether the number of error reports has reached the preset threshold. If yes, proceed to step 7; otherwise, proceed to step 3.

[0023] Specifically, step 51 includes:

[0024] Step 5101: Divide the recorded file into segments of preset duration;

[0025] Step 5102: Randomly select a sub-segment with a preset number of frames in the segment, and establish a corresponding reference histogram based on its zero-crossing point;

[0026] Step 5103: Select a sliding window segment of the same length from the downloaded music and slide it with a preset step size. Each time the sliding window segment slides, a corresponding sliding histogram is built based on its zero-crossing point.

[0027] Step 5104: Calculate the first similarity and take the sliding window segment corresponding to the sliding histogram with the first similarity greater than the first preset threshold as the preliminary matching segment. The first similarity is the similarity between the base histogram and the sliding histogram obtained by calculating the histogram intersection value.

[0028] Step 5105: Obtain the first digital audio fingerprint and the second digital audio fingerprint of the preset number of bits of the sub-segment and the preliminarily matched segment;

[0029] Step 5106: Calculate the second similarity using a preset formula. The second similarity is the similarity between the first digital audio fingerprint and the second digital audio fingerprint.

[0030] Step 5107: If the second similarity is greater than the second preset threshold, then it is determined to be the same; otherwise, it is determined to be different.

[0031] Specifically, the preset formula is:

[0032]

[0033] Where F represents the second similarity, F c (n,m), F e (n, m) represent the m-th bit of the n-th frame audio fingerprint of the sub-segment and the preliminary matching segment, respectively. N represents the preset frame number, M represents the number of bits of the first digital audio fingerprint and the second digital audio fingerprint, and ⊕ represents the XOR operation.

[0034] Furthermore, the procedure includes the following steps after step 1 and before step 2:

[0035] Step 11: Check if the noise inside the vehicle exceeds the preset threshold. If yes, proceed to the next step; otherwise, proceed to step 2.

[0036] Step 12: Check if the car window is open. If yes, close the window and proceed to the next step; otherwise, proceed to step 14.

[0037] Step 13: Check again whether the noise inside the vehicle exceeds the preset threshold. If yes, proceed to the next step; otherwise, proceed to step 2.

[0038] Step 14: Increase the volume of the in-vehicle infotainment system and monitor whether the recording signal-to-noise ratio is higher than the preset signal-to-noise ratio threshold. If so, issue a command to record music; otherwise, repeat this step.

[0039] Furthermore, the procedure includes the following steps after step 2 and before step 3:

[0040] Step 21: Return the window height and infotainment system volume to their original settings.

[0041] Furthermore, the procedure includes the following after step 21 and before step 3:

[0042] Step 5: Monitor whether the current network signal strength is greater than the preset value. If yes, upload the recording file to the server; otherwise, lock the recording file and repeat this step.

[0043] Specifically, the preset duration is 3 to 10 seconds.

[0044] Specifically, the preprocessing includes sampling, quantization, encoding, and compression signal processing steps.

[0045] The beneficial effects of this invention are as follows: This invention records music clips using an in-vehicle audio pickup device, sends them to a server for recognition, and can download them according to the link provided by the server. After downloading, the music clips can be automatically compared with the recorded music clips to determine whether the music recognized by the server is correct, thereby improving the intelligence level of in-vehicle music recognition. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the in-vehicle music recognition method of the present invention. Detailed Implementation

[0047] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The drawings are for reference and illustration only and do not constitute a limitation on the scope of protection of the present invention.

[0048] In the process described in the specification, claims, or drawings of this invention, each step is numbered (e.g., step 10, 20, etc.). These numbers are used only to distinguish the steps and do not represent any execution order. It should be noted that the terms "first," "second," etc., used herein are only for distinguishing the objects being described and do not represent a chronological order, nor do they indicate that "first," "second," etc., are different types.

[0049] Example 1

[0050] like Figure 1 As shown, this embodiment provides a method for recognizing in-vehicle music, including:

[0051] Step 1: Determine if a command to start music recognition has been received. If yes, proceed to the next step; otherwise, repeat this step.

[0052] For example, when a car owner hears music they like but are not familiar with, they can activate music recognition through voice control, button control, or by clicking on the relevant icon buttons pre-set on the interface of the in-vehicle infotainment system's related app.

[0053] Step 2: Start audio recording, record the current music for a preset duration, generate a recording file after preprocessing, and save it to the cache.

[0054] In practice, audio recording can be done using microphones or other sound pickup devices.

[0055] In this embodiment, the preset duration is 3 to 10 seconds, but it can also be set by the user.

[0056] In this embodiment, the preprocessing includes signal processing steps such as sampling, quantization, encoding, and compression.

[0057] For example, when the car owner activates music recognition, the system will automatically turn on the microphone and then collect the currently playing music. Through steps such as sampling, quantization, encoding, and compression, the sound signal is converted into a binary digital signal and stored in the buffer.

[0058] Step 3: Upload the recorded file to the server, receive the identification information returned by the server, and determine whether the current music is available for free download based on the identification information. If yes, proceed to step 5; otherwise, proceed to the next step.

[0059] In practice, the server decompresses the recorded files uploaded by the in-vehicle infotainment system, then identifies them, and sends the identification information (including music name, free download icon and download link, and paid download icon) to the in-vehicle infotainment system.

[0060] In this embodiment, the free download icon indicates that the server has found the corresponding music on the free music server and provides a download link; the paid download icon indicates that the server has not found the corresponding music on the free music server, no link is provided and the music is not visible, and payment is required to download it.

[0061] Step 4: Determine if you have received an instruction to agree to pay for the download. If yes, accept the server's push and proceed to the next step; otherwise, proceed to step 7.

[0062] Step 5: Download the corresponding music and proceed to the next step.

[0063] Step 6: Play the downloaded music.

[0064] In this embodiment, step 6 includes:

[0065] Step 601: Obtain the priority of the audio playback thread currently running in the foreground. If the priority of the audio playback thread running in the foreground is lower than the priority of the music player, proceed to the next step; otherwise, repeat this step.

[0066] Step 602: Obtain the current road condition level parameter. If the current road condition level parameter is greater than the predefined road condition level parameter, return to the previous step; otherwise, proceed to the next step.

[0067] In practice, the current road condition level parameters can be obtained through pre-calibration, and the obtained items include, but are not limited to, the distance to the vehicle in front in the same lane and the relative speed.

[0068] Step 603: Obtain the current user status level parameter. If the current user status level parameter is greater than the predefined user status level parameter, return to step 601; otherwise, issue a preset prompt message.

[0069] In practice, user status level parameters can be obtained through pre-calibration. The items obtained include, but are not limited to, the driver's level of attention and whether they are talking to others (mouth movements, facial expressions).

[0070] In specific implementation, the preset prompt information includes: a pop-up dialog box to confirm playback, and / or a playback prompt voice (such as "The downloaded music that has been identified will be played for you soon").

[0071] Step 604: Determine whether a confirmation instruction to play has been received. If yes, play the downloaded music; otherwise, end the process.

[0072] Step 7: Exit music recognition mode.

[0073] Example 2

[0074] Unlike Example 1, this example includes the following steps after step 5 and before step 6:

[0075] Step 51: Compare the downloaded music with the recorded file to determine if they are consistent. If they are, proceed to step 6; otherwise, proceed to the next step.

[0076] Step 52: Report the identification error to the server and determine whether the number of error reports has reached the preset threshold. If yes, proceed to step 7; otherwise, proceed to step 3.

[0077] In this embodiment, step 51 includes:

[0078] Step 5101: Divide the recorded file into segments of a preset duration.

[0079] In practice, the preset duration can be set according to the actual situation, such as 1 second.

[0080] Step 5102: Randomly select a sub-segment with a preset number of frames in the segment, and establish a corresponding reference histogram based on its zero-crossing point.

[0081] Step 5103: Select a sliding window segment of the same length from the downloaded music and slide it with a preset step size. Each time the sliding window segment slides, a corresponding sliding histogram is built based on its zero-crossing point.

[0082] Step 5104: Calculate the first similarity and take the sliding window segment corresponding to the sliding histogram with the first similarity greater than the first preset threshold as the preliminary matching segment. The first similarity is the similarity between the base histogram and the sliding histogram obtained by calculating the histogram intersection value.

[0083] Step 5105: Obtain the first digital audio fingerprint and the second digital audio fingerprint of the preset number of bits of the sub-segment and the preliminarily matched segment.

[0084] Step 5106: Calculate the second similarity using a preset formula. The second similarity is the similarity between the first digital audio fingerprint and the second digital audio fingerprint.

[0085] In this embodiment, the preset formula is:

[0086]

[0087] Where F represents the second similarity, F c (n,m), F e (n, m) represent the m-th bit of the n-th frame audio fingerprint of the sub-segment and the preliminary matching segment, respectively. N represents the preset frame number, M represents the number of bits of the first digital audio fingerprint and the second digital audio fingerprint, and ⊕ represents the XOR operation.

[0088] Step 5107: If the second similarity is greater than the second preset threshold, then it is determined to be the same; otherwise, it is determined to be different.

[0089] Example 3

[0090] Unlike Example 1, this example includes the following steps after step 1 and before step 2:

[0091] Step 11: Check if the noise inside the vehicle exceeds the preset threshold. If yes, proceed to the next step; otherwise, proceed to step 2.

[0092] Step 12: Check if the car window is open. If it is, close the window and proceed to the next step; otherwise, proceed to step 14.

[0093] Step 13: Check again whether the noise inside the vehicle exceeds the preset threshold. If yes, proceed to the next step; otherwise, proceed to step 2.

[0094] Step 14: Increase the volume of the in-vehicle infotainment system and monitor whether the recording signal-to-noise ratio is higher than the preset signal-to-noise ratio threshold. If so, issue a command to record music; otherwise, repeat this step.

[0095] The process includes the following steps after step 2 and before step 3:

[0096] Step 21: Return the window height and infotainment system volume to their original settings.

[0097] Example 4

[0098] Unlike Example 3, this example includes the following steps after step 21 and before step 3:

[0099] Step 5: Monitor whether the current network signal strength is greater than the preset value. If yes, upload the recording file to the server; otherwise, lock the recording file and repeat this step.

[0100] By locking recorded files in the event of poor network conditions to prevent them from being overwritten, and then uploading them to the server once the network connection is good, the reliability of the system is further improved.

[0101] The above-disclosed embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for identifying music in a vehicle, characterized by, include: Step 1: Determine if a command to start music recognition has been received. If yes, proceed to the next step; otherwise, repeat this step. Step 2: Start audio recording, record the current music for a preset duration, generate a recording file after preprocessing, and save it to the cache; Step 3: Upload the recorded file to the server, receive the identification information returned by the server, and determine whether the current music is available for free download based on the identification information. If yes, proceed to step 5; otherwise, proceed to the next step. Step 4: Determine if you have received an instruction to agree to pay for the download. If yes, accept the server's push and proceed to the next step; otherwise, proceed to step 7. Step 5: Download the corresponding music and proceed to the next step; Step 51: Compare the downloaded music with the recorded file to determine if they are consistent. If they are, proceed to step 6; otherwise, proceed to the next step. Step 52: Report the identification error to the server and determine whether the number of error reports has reached the preset threshold. If yes, proceed to step 7; otherwise, proceed to step 3. Step 6: Play the downloaded music; Step 7: Exit music recognition mode; Step 51 includes: Step 5101: Divide the recorded file into segments of preset duration; Step 5102: Randomly select a sub-segment with a preset number of frames in the segment, and establish a corresponding reference histogram based on its zero-crossing point; Step 5103: Select a sliding window segment of the same length from the downloaded music and slide it with a preset step size. Each time the sliding window segment slides, a corresponding sliding histogram is built based on its zero-crossing point. Step 5104: Calculate the first similarity and take the sliding window segment corresponding to the sliding histogram with the first similarity greater than the first preset threshold as the preliminary matching segment. The first similarity is the similarity between the base histogram and the sliding histogram obtained by calculating the histogram intersection value. Step 5105: Obtain the first digital audio fingerprint and the second digital audio fingerprint of the preset number of bits of the sub-segment and the preliminarily matched segment; Step 5106: Calculate the second similarity using a preset formula. The second similarity is the similarity between the first digital audio fingerprint and the second digital audio fingerprint. Step 5107: If the second similarity is greater than the second preset threshold, then it is determined to be the same; otherwise, it is determined to be different.

2. The in-vehicle music recognition method according to claim 1, characterized by, Step 6 includes: Step 601: Obtain the priority of the audio playback thread currently running in the foreground. If the priority of the audio playback thread running in the foreground is lower than the priority of the music player, proceed to the next step; otherwise, repeat this step. Step 602: Obtain the current road condition level parameter. If the current road condition level parameter is greater than the predefined road condition level parameter, return to the previous step; otherwise, proceed to the next step. Step 603: Obtain the current user status level parameter. If the current user status level parameter is greater than the predefined user status level parameter, return to step 601; otherwise, issue a preset prompt message. Step 604: Determine whether a confirmation instruction to play has been received. If yes, play the downloaded music; otherwise, end the process.

3. The in-vehicle music recognition method according to claim 1, characterized in that, The preset formula is: Where F represents the second similarity, F c (n,m), F e (n, m) represent the m-th bit of the n-th frame audio fingerprint of the sub-segment and the preliminary matching segment, respectively. N represents the preset frame number, M represents the number of bits of the first digital audio fingerprint and the second digital audio fingerprint, and ⊕ represents the XOR operation.

4. The in-vehicle music recognition method of claim 1, characterized by, The steps following step 1 and before step 2 include: Step 11: Check if the noise inside the vehicle exceeds the preset threshold. If yes, proceed to the next step; otherwise, proceed to step 2. Step 12: Check if the car window is open. If yes, close the window and proceed to the next step; otherwise, proceed to step 14. Step 13: Check again whether the noise inside the vehicle exceeds the preset threshold. If yes, proceed to the next step; otherwise, proceed to step 2. Step 14: Increase the volume of the in-vehicle infotainment system and monitor whether the recording signal-to-noise ratio is higher than the preset signal-to-noise ratio threshold. If so, issue a command to record music; otherwise, repeat this step.

5. The in-vehicle music recognition method according to claim 4, characterized in that, The process includes the following steps after step 2 and before step 3: Step 21: Return the window height and infotainment system volume to their original settings.

6. The in-vehicle music recognition method according to claim 5, characterized in that, The steps following step 21 and before step 3 include: Step 5: Monitor whether the current network signal strength is greater than the preset value. If yes, upload the recording file to the server; otherwise, lock the recording file and repeat this step.

7. The in-vehicle music recognition method according to any one of claims 1 to 6, characterized in that, The preset duration is 3~10s.

8. The in-vehicle music recognition method according to any one of claims 1 to 6, characterized by, The preprocessing includes sampling, quantization, encoding, and compression signal processing steps.

Images