Lyrics display method and electronic device

Abstract

A lyrics display method and an electronic device, in the method, the electronic device can determine the display position of each word according to the corresponding frequency in real time, and display the word with high frequency at a higher position relative to the word with low frequency, realize the linkage of the song and the lyrics, enrich the display effect of the lyrics, and improve the user experience.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to methods and electronic devices for displaying lyrics. Background Technology

[0002] With the continuous improvement and development of computer science and technology, electronic devices such as mobile phones, tablets, and laptops are becoming increasingly common in people's daily lives. At the same time, the various software installed on these devices provides increasingly rich services. For example, music players can provide users with functions such as song playback, song recording, and song recognition.

[0003] Currently, when playing a song using a music player, the lyrics display can synchronize with the song's playback progress. For example, when the music player reaches the 10th second of song A, the lyrics display can highlight the line corresponding to that 10th second, while the remaining lyrics, whether played or not, are displayed in gray. However, this method of displaying lyrics is rather limited in its effectiveness. Summary of the Invention

[0004] This application provides a lyrics display method and an electronic device. Through some embodiments of this application, when playing a song, the electronic device can adjust the display position of each word according to its corresponding frequency, thereby enriching the lyrics display effect and improving the user experience.

[0005] In a first aspect, embodiments of this application provide a lyrics display method, the method comprising:

[0006] In response to the operation of playing a song, the electronic device plays a first song and displays a lyrics interface; the lyrics interface includes a first display area for displaying the currently playing lyric.

[0007] When playing the first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment at a first height in the first display area; the first height is determined by the electronic device based on the frequency of the first audio segment;

[0008] When playing the second audio segment of the first song, the electronic device displays the second target word corresponding to the second audio segment at a second height in the first display area; the second height is determined by the electronic device based on the frequency of the second audio segment; the frequency of the second audio segment is greater than the frequency of the first audio segment, and the second height is greater than the first height.

[0009] The first audio segment is an audio segment in the audio data of the first song, and the first target word is a word in the lyrics file of the first song that corresponds to the first audio segment;

[0010] The second audio segment is another audio segment in the audio data of the first song that is different from the first audio segment, and the second target word is a word in the lyrics file of the first song that corresponds to the second audio segment.

[0011] It is understood that electronic devices can respond to various types of song playback operations to play the first song. For example, an electronic device can respond to a song playback operation by playing the first song from the beginning or the middle of the first song.

[0012] It is understood that the first target word and the second target word can be understood as two different words in the lyrics file of the first song. In this embodiment of the application, a target word can be understood as a word in the lyrics file that is defined by a start position and an end position. It is understood that the words defined by the start and end times in the lyrics files may be different in different language environments. For example, in a Chinese environment, the target word can be understood as a single character; in an English environment, the target word can be understood as a single word.

[0013] Furthermore, each word in the lyrics file has a definite start and end time, which can also be understood as each word having a definite time interval. The audio segment in the first song's audio data corresponding to the time interval of the first target word can be understood as the first audio segment, and the audio segment corresponding to the time interval of the second target word can be understood as the second audio segment.

[0014] For example, the first target word in the lyrics file starts at second 1 and ends at second 1.5. Thus, the audio data corresponding to the audio played in the first song from second 1 to second 1.5 can be understood as the first audio segment.

[0015] In this embodiment, the electronic device can determine the display position of each word in real time according to its corresponding frequency, so that words with higher frequency are displayed at a higher position relative to words with lower frequency, thereby realizing the linkage between the song and the lyrics, enriching the display effect of the lyrics, and improving the user experience.

[0016] In conjunction with the first aspect, in some embodiments, when playing a first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment before a first height in the first display area, the method further comprising:

[0017] The electronic device acquires the audio data and lyrics file of the first song;

[0018] The electronic device samples the audio data at a first sampling period to obtain a first sampling result set; the first sampling result set includes the frequency corresponding to each sampling moment;

[0019] The electronic device obtains the frequency of the first audio segment from the first sample result set based on the start and end times of the first target word in the lyrics file.

[0020] Understandably, electronic devices can obtain the audio file of the first song from a local or third-party server, process the audio file, such as parsing the container format and decoding it, to obtain audio data.

[0021] Understandably, electronic devices can obtain the lyrics file of the first song from a local or third-party server, then parse the lyrics file to obtain the start and end times of each word in the lyrics file.

[0022] In some embodiments, taking the Android system as an example, the electronic device can call the callback method onFftDataCapture provided by the Android system during the first sampling period to directly obtain the audio data of the currently playing song and calculate the frequency corresponding to each sampling moment.

[0023] In conjunction with the first aspect, in some embodiments, the electronic device obtains the frequency of the first audio segment from the first sampling result set based on the start and end times of the first target word in the lyrics file, including:

[0024] The electronic device determines a first time interval consisting of the start time and the end time of the first target word;

[0025] The electronic device collects the first sampling results and sums the frequencies corresponding to the sampling times within the first time interval, or sums them and then averages them to obtain the frequency of the first audio segment.

[0026] In conjunction with the first aspect, in some embodiments, when playing a first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment before a first height in the first display area, the method further comprising:

[0027] The electronic device divides the first display area horizontally into K sub-display areas, where K is an integer greater than or equal to 2;

[0028] The electronic device determines the frequency range corresponding to each sub-display area in the K display areas;

[0029] When playing the first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment at a first height in the first display area, including:

[0030] When playing the first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment in the first display area in the first sub-display area. The first sub-display area is one of the K sub-display areas. The frequency of the first audio segment is included in the frequency range corresponding to the first sub-display area.

[0031] For example, the electronic device can horizontally divide the first display area into 5 sub-display areas. From bottom to top, each sub-display area corresponds to a frequency range of 200Hz-300Hz, 300Hz-400Hz, 400Hz-500Hz, 500Hz-600Hz, and 600Hz-700Hz. The word "B" corresponds to a frequency of 284Hz, which falls within the 200Hz-300Hz frequency range. Therefore, the electronic device displays the word "B" in the first sub-display area from bottom to top.

[0032] Understandably, the above method allows the display height of each element to be proportional to its corresponding frequency, enabling the song and lyrics to be displayed in tandem, thus enriching the display effect of the lyrics.

[0033] In conjunction with the first aspect, in some embodiments, the first display area includes a lyrics display reference position, which is a reference position for offsetting the first target word;

[0034] When playing the first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment at a first height in the first display area, including:

[0035] When playing the first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment in the first display area, offset by the first height relative to the lyrics display reference position.

[0036] In conjunction with the first aspect, in some embodiments, before the electronic device displays the first target word corresponding to the first audio segment in the first display area at a height offset from the lyrics display reference position when playing the first audio segment of the first song, the method further includes:

[0037] The electronic device determines a first frequency, a second frequency, and a total offset height; the second frequency is greater than the first frequency.

[0038] The electronic device determines the first height based on the frequency of the first audio band, the first difference, and the total offset height; the first difference is the difference between the second frequency and the first frequency.

[0039] It is understood that the first frequency and the second frequency can be set according to actual conditions. In some embodiments, the first frequency is the lowest sound frequency that the human ear can hear, and the second frequency is the highest sound frequency that the human ear can hear.

[0040] For example, it is generally believed that the range of sound frequencies that the human ear can hear is 20Hz to 20000Hz. The first frequency can be set to the lowest sound frequency that the human ear can hear, 20Hz, and the second frequency can be set to the lowest sound frequency that the human ear can hear, 20000Hz.

[0041] In some embodiments, the first frequency is the lowest frequency of the audio in the first song, and the second frequency is the highest frequency of the audio in the first song.

[0042] In conjunction with the first aspect, in some embodiments, the electronic device determines the first height based on the frequency of the first audio segment, the first difference, and the total offset height, including:

[0043] The electronic device subtracts the first frequency from the frequency of the first audio band to obtain a second difference;

[0044] The electronic device divides the second difference by the first difference to obtain a first ratio;

[0045] The electronic device multiplies the first ratio by the total offset height to obtain the first height.

[0046] It is understood that, in some embodiments, in order to avoid overlapping when displaying lyrics, the first height is 0 when the frequency of the first audio segment is less than or equal to the first frequency; and the first height is the total offset height when the frequency of the first audio segment is greater than or equal to the second frequency.

[0047] In conjunction with the first aspect, in some embodiments, the offset of the first target word includes an upward offset or a downward offset.

[0048] Understandably, when displaying each target word on an electronic device, it can shift upwards or downwards according to the layout to enrich the display effect of the lyrics.

[0049] In conjunction with the first aspect, in some embodiments, the method further includes: in response to a jump playback operation, the electronic device plays the first audio segment of the first song; while playing the first audio segment of the first song, the electronic device displays N words preceding the first target word at different heights in the first display area, wherein the height of each of the N words is determined by the frequency of the audio segment corresponding to each word; the N words belong to the same lyric as the first target word; and N is an integer greater than or equal to 1.

[0050] In conjunction with the first aspect, in some embodiments, the method further includes: in response to a jump playback operation, the electronic device plays the first audio segment of the first song; while playing the first audio segment of the first song, the electronic device displays each of the M words preceding the first target word at a third height in the first display area; the third height is determined by the average frequency of the audio segment corresponding to the M words; the M words and the first target word belong to the same lyric; and M is an integer greater than or equal to 1.

[0051] In a second aspect, embodiments of this application provide an electronic device comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform the method in the first aspect or any possible implementation thereof.

[0052] Thirdly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform the method shown in the first aspect or any possible implementation of the first aspect.

[0053] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method shown in the first aspect or any possible implementation thereof.

[0054] Fifthly, embodiments of this application provide a computer-readable storage medium including instructions, characterized in that, when the instructions are executed on an electronic device, the electronic device performs the method shown in the first aspect or any possible implementation thereof. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0056] Figure 1 These are schematic diagrams illustrating some lyric formats provided in embodiments of this application;

[0057] Figure 2 This is a schematic diagram of the architecture of a music player provided in an embodiment of this application;

[0058] Figure 3This is a schematic diagram of screen coordinates provided in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of an interface for synchronizing songs and lyrics line by line, provided in an embodiment of this application;

[0060] Figure 5 This is a schematic diagram of an interface for synchronizing songs and lyrics word by word, provided in an embodiment of this application;

[0061] Figure 6 This is a schematic diagram illustrating the correspondence between words and times in a lyrics file, as provided in an embodiment of this application.

[0062] Figures 7-9 This is a schematic diagram illustrating the lyrics display effect after playing a song from the beginning position, as provided in an embodiment of this application.

[0063] Figures 10-13 This is a schematic diagram illustrating the lyrics display effect after playing a song from the middle position, as provided in an embodiment of this application.

[0064] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0065] Figure 15 This is a software structure block diagram of an electronic device provided in an embodiment of this application;

[0066] Figure 16 This is a schematic diagram of data flow provided in an embodiment of this application;

[0067] Figure 17 This is a flowchart illustrating another lyrics display method provided in an embodiment of this application;

[0068] Figure 18 This is a schematic diagram illustrating the result of sampling audio data by an electronic device with a sampling period of 0.1 seconds, as provided in an embodiment of this application.

[0069] Figure 19 This is a schematic diagram illustrating the correspondence between words and time in another lyrics file provided in an embodiment of this application;

[0070] Figure 20 This is a schematic diagram illustrating how to segment the sampling results based on the time information of the target word to obtain the frequency of the target word, according to an embodiment of this application.

[0071] Figure 21 This is a schematic diagram illustrating the result of obtaining the frequency corresponding to a target word by summation, provided in an embodiment of this application.

[0072] Figure 22This is a schematic diagram illustrating how the frequency of a target word is obtained by summing and then averaging, according to an embodiment of this application.

[0073] Figure 23 This is a schematic diagram illustrating how the offset height is determined by the frequency corresponding to the target word, as provided in an embodiment of this application.

[0074] Figure 24 This is a schematic diagram of an interface for displaying lyrics and songs in conjunction with an embodiment of this application. Detailed Implementation

[0075] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0076] To more clearly describe the solution provided in this application, the terminology involved in the embodiments of this application will be introduced below.

[0077] 1. Sound

[0078] Understandably, sound is a sound wave produced by the vibration of an object. The frequency of sound can be called pitch, which can be understood as the number of times a sound changes periodically within one second. For example, the range of human hearing is between 20Hz and 20kHz; low-frequency sounds are muffled and heavy, while high-frequency sounds are sharp and piercing.

[0079] 2. Audio files and audio data

[0080] Understandably, sound signals are continuous signals (i.e., analog signals). In order for electronic devices to process sound signals, analog signals need to be converted into digital signals.

[0081] In some embodiments, the conversion of sound signals includes sampling, quantization, and encoding. For example, sound signals can be converted into electrical signals using electronic devices such as microphones, and then the electrical signals can be converted into binary numbers (i.e., digital signals) represented by 1s and 0s using an analog-to-digital converter (ADC).

[0082] Because the file size after converting sound signals into digital signals is large, and only a small portion of the sound signal contains truly useful information that can be perceived by the human ear, audio compression technology can be used when converting sound signals.

[0083] For example, an audio file with the .flac extension can be obtained by using free lossless audio code (FLAC); an audio file with the .ape extension can be obtained by using Monkey's Audio (a software with lossless compression technology); an audio file with the .mp3 extension can be obtained by using Moving Picture Experts Group Audio Layer III (MP3) encoding, and so on.

[0084] In this embodiment of the application, an audio file can be understood as a file used to store digital signals converted from sound signals. For example, the files obtained by audio compression, such as files with the .flac extension, files with the .ape extension, and files with the .mp3 extension, can all be understood as audio files.

[0085] In this embodiment of the application, audio data can be understood as digitized audio data obtained by converting sound signals. For example, digitized data obtained through sampling, quantization, encoding, or compression encoding can be understood as audio data.

[0086] 3. Lyrics file

[0087] Generally, a lyrics file can be understood as a file used to store lyrics data. Typically, a lyrics file includes a header element and the lyrics content. The header element can include information such as the song title, artist, album, and composer. For ease of understanding, the following examples mainly describe the lyrics content in a lyrics file.

[0088] Understandably, to achieve different lyric display effects, lyric files can store lyric data in different formats. For example, common lyric formats include LRC, TRC, KRC, and QRC.

[0089] For example, please refer to Figure 1 , Figure 1 These are schematic diagrams illustrating some lyric formats provided in the embodiments of this application.

[0090] like Figure 1 As shown in (A), this lyrics format can be understood as supporting a line-by-line precise LRC format. The LRC format specifies the start time for each line of lyrics, in the format [minutes:seconds.percentseconds]. For example, the first line of lyrics is "first line of lyrics", and the start time of the first line of lyrics is 23.25 seconds. The start times of the remaining lyrics can be deduced similarly, and will not be elaborated here.

[0091] like Figure 1As shown in (B), this lyrics format can be understood as an LRC format that supports precise word-by-word processing. This means that in addition to supporting precise line-by-line processing, a time tag is added to each word. Specifically, the time tag format for each line of lyrics is [minutes:seconds.percentseconds], and the time tag format for each word is <minutes:seconds.percentseconds>.

[0092] like Figure 1 As shown in (C), this lyrics format can be understood as a TRC format that supports precise word-by-word execution. Building upon the LRC format, which supports precise line-by-line execution, the TRC format adds a time stamp (in milliseconds) before each word in the lyrics text. For example, the first line of lyrics, "When you are leaving," begins at 16.54 milliseconds; the first word, "when," lasts for 250 milliseconds; the second word, "you," lasts for 300 milliseconds, and so on. Further details are omitted here.

[0093] Understandably, the display effect of lyrics is closely related to the information recorded in the lyrics file. The more detailed the lyrics format in the lyrics file, the more accurate the lyrics display can be.

[0094] In this application embodiment, the lyrics file can be understood as a lyrics file that supports precise word-by-word lyrics, and the lyrics data can be understood as the data in the lyrics file. For example, the above... Figure 1 The lyrics files corresponding to (B) and (C) in this application can be understood as the lyrics files in this embodiment. Alternatively, files with the .qrc extension also support word-by-word precision and can also be understood as lyrics files in this embodiment.

[0095] It is understandable that word-by-word precision can be achieved in various ways. This application does not limit the way lyrics are recorded in the lyrics file, as long as the position (start time and duration) of each word can be located by parsing the lyrics file.

[0096] 4. Music Player Architecture

[0097] In this embodiment, a music player can be understood as software or electronic device capable of playing audio files stored in digital signal form. In some embodiments, a music player can be understood as software or electronic device capable of reading, parsing, and rendering audio files stored locally or on a server.

[0098] For example, please refer to Figure 2 , Figure 2 This is a schematic diagram of the architecture of a music player provided in an embodiment of this application. The music player includes a reading module, a parsing module, a decoding module, and a rendering module. Wherein:

[0099] The reading module is used to read audio data from an audio file. For example, the audio file can be stored locally or on a server; therefore, the music player can read the audio data from the local machine or the server via an interface. For example, the reading module can read audio data sequentially from the beginning of the audio file, or it can read audio data based on a specified offset.

[0100] Understandably, the audio data read by the reading module is generally encapsulated audio data. For example, the audio data read by the reading module can be audio data encapsulated in formats such as mp3, ape, and flac.

[0101] The parsing module is used to parse the encapsulation format of the audio data output by the reading module to obtain the encoded audio data.

[0102] The decoding module is used to decode the audio data output by the parsing module to obtain the decoded audio data. For example, if the audio file is packaged in APE format, Monkey's Audio decoder is called for decoding; if the audio file is packaged in FLAC format, FLAC decoder is called; if the audio file is packaged in MP3 format, LAME decoder (an MP3 file decoding tool) is called.

[0103] Understandably, when decoding audio data, it is not always possible to output a frame of audio data every time a frame of audio data is sent to the decoder. Instead, several reference frames need to be buffered before the output can be obtained.

[0104] The rendering module is used to play audio using the audio data output by the decoding module. For example, the decoded audio data can be sent to the sound card and then played using speakers.

[0105] In some embodiments, the music player further includes a monitoring module for monitoring information such as audio playback progress.

[0106] In some embodiments, the music player's reading module is also used to read lyrics data from a lyrics file, and the rendering module is also used to synchronously display the lyrics according to the audio playback progress. It is understood that the reading module's interface obtains lyrics data, rather than directly obtaining an LRC file.

[0107] For example, when a music player displays lyrics, it first parses the lyrics file, such as converting the time in the timestamps to milliseconds and using regular expressions to match text, to obtain the single-line lyrics and the time information of each word. Secondly, during playback, the music player can obtain the playback time through a monitoring module, find the corresponding single-line lyrics and the corresponding single word in the lyrics file based on the playback time, and then draw them on the screen.

[0108] 5. Screen coordinates

[0109] Screen coordinates were introduced to accurately locate the position on the screen of electronic devices.

[0110] Understandably, the smallest unit that makes up an image is a pixel, and for electronic devices, a pixel is the smallest unit that hardware and software can control. Therefore, the screen coordinates of an electronic device are defined in pixels.

[0111] For example, please refer to Figure 3 , Figure 3 This is a schematic diagram of screen coordinates provided in an embodiment of this application.

[0112] like Figure 3 As shown, the top left corner of the electronic device screen is the origin (0, 0), the positive x-axis points to the right, and the positive y-axis points downwards. Assuming the screen resolution of the electronic device is 28*26, the screen is divided into 28 rows horizontally and 26 columns vertically. The area formed by the intersection of rows and columns can be understood as a pixel.

[0113] It's understandable that words displayed on a screen are composed of a series of pixels, and the shape and size of the words are achieved through manipulation of these pixels. For example, the shape of a word can be represented by a dot matrix, such as a 16x16 dot matrix, a 24x24 dot matrix, or a 32x32 dot matrix. Each dot matrix can consist of a 0 or a 1, where 1 indicates that the pixel is lit, and 0 indicates that the pixel is not lit.

[0114] Understandably, the position of each word displayed on the screen of an electronic device is determined by screen coordinates. For example, the screen coordinates of the word "A" are (10, 12), represented by a 4x4 dot matrix. The electronic device can... Figure 3 The position shown (i.e., the area formed by the intersection of column 10 and row 12) is used as the top left, top right, or center element of the dot matrix to draw and display the word "A" based on the information in the dot matrix.

[0115] In some embodiments, the music player can highlight the corresponding line of lyrics at the current playback time of the song, based on the start time of each line in the lyrics file, while displaying the remaining played or unplayed lyrics in gray, thereby achieving line-by-line synchronization between the song and the lyrics. For example, Figure 4 This is a schematic diagram of an interface for synchronizing songs and lyrics line by line, as provided in an embodiment of this application. Figure 4As shown, when the song "Everything Illuminated by Rain To Cloud Power" plays for 5 seconds, the corresponding lyric in the lyrics file is "The rain of suffering, let it flow into a river". The lyrics display interface 41 enlarges and bolds the lyric "The rain of suffering, let it flow into a river" compared to the other lyrics, so as to achieve line-by-line synchronization between the song and the lyrics.

[0116] In other embodiments, the music player can highlight a corresponding word at the current playback time of the song, based on the start time of each line of lyrics and the start time of each word in the lyrics file, while displaying the remaining played or unplayed lyrics in gray, thereby achieving word-by-word synchronization between the song and the lyrics. For example, Figure 5 This is a schematic diagram of an interface for synchronizing songs and lyrics word by word, as provided in an embodiment of this application. Figure 5 As shown, when the song "Everything Illuminated by Rain To Cloud Power" plays for 5 seconds, the corresponding lyric in the lyrics file is "The rain of suffering, let it flow into a river". The word in the corresponding lyric is "it". The lyrics display interface 51 enlarges and bolds the already played and currently playing "The rain of suffering, let it flow into a river" relative to the other lyrics, so as to realize word-by-word synchronization between the song and the lyrics.

[0117] This application provides a lyrics display method that, when playing a song, allows each word to be displayed in real time according to its corresponding frequency, with higher-frequency words displayed at higher positions relative to lower-frequency words. This achieves linkage between the song and the lyrics, enriches the display effect of the lyrics, and improves the user experience.

[0118] For example, please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating the correspondence between words and time in a lyrics file provided in an embodiment of this application. Figure 6 The following is an example of detailed information regarding the first, second, and fifth lines of the song "The Rain Illuminates Everything To Cloud Power". Specifically, the first line, "It's going to rain, that cloud said," begins at second 0 and ends at second 2.3 (2300 milliseconds); the second line, "Lightning flashes crisply in my heart," begins at second 2.3 and ends at second 4.4, which can also be understood as the second line lasting 2100 milliseconds; the fifth line, "The rain of suffering, let it flow into a river," begins at second 12 and ends at second 15.1, which can also be understood as the fifth line lasting 3100 milliseconds. Where:

[0119] The first word "It's going to" in the first line of lyrics "It's going to rain, said that cloud" starts at the 0th second of the song and lasts for 300 milliseconds; the second word "rain" starts at the 300th millisecond of the song and lasts for 400 milliseconds; the third word "rain" starts at the 700th millisecond of the song and lasts for 300 milliseconds. The time correspondence of subsequent lyrics can be inferred by analogy and will not be elaborated here.

[0120] Next, based on the information of the lyrics file shown in Figure 6 the lyrics display method provided by the embodiments of the present application will be introduced by way of example in different scenarios.

[0121] Scenario 1: The electronic device plays the song from the start position of the song.

[0122] Exemplarily, Figures 7 to 9 are schematic diagrams of some user interfaces provided by the embodiments of the present application for Scenario 1.

[0123] As Figure 7 shown in (A) of , the electronic device displays the user interface 71, which is used to display the song information in the playlist. Exemplarily, the electronic device is currently playing the song "Winter Snow", and in response to a user operation acting on the control of the song "All That the Rain Illuminates To Yun次方", such as a click operation, the electronic device starts playing this song.

[0124] In some embodiments, the user operation acting on the control of the song "All That the Rain Illuminates To Yun次方", such as a click operation, can be understood as an operation to play the song; the song "All That the Rain Illuminates To Yun次方" can be understood as the first song.

[0125] Specifically, the electronic device can display the user interface 72, which can be used to display the lyrics of the currently playing song and information such as the playing progress of the song.

[0126] In some embodiments, displaying the user interface 72 can be understood as a lyrics interface, and the area displaying the lyrics "It's going to rain, said that cloud" can be understood as the first display area.

[0127] Exemplarily, the electronic device can start displaying the lyrics from the middle of the screen. The area above the middle of the screen can be used to display the lyrics corresponding to the played part of the song, and the area below the middle of the screen can be used to display the lyrics corresponding to the unplayed part of the song. The present application does not limit this.

[0128] It can be understood that the display of the playing progress of the song in the electronic device is generally in seconds. However, the duration of each word in the lyrics is generally only a few hundred milliseconds. For example Figure 6The duration of the first word "want" in the first sentence of the lyrics is 300 milliseconds. Therefore, there are often multiple words corresponding to the display within 1 second.

[0129] For the sake of easy understanding, it is exemplarily set that the frequencies corresponding to the first five words in the first sentence of the lyrics "It's going to rain. That cloud said" of the song are 120Hz, 150Hz, 115Hz, 210Hz, and 145Hz respectively; the electronic device allows each word to be offset upward by at most 10 pixels based on the lyrics display reference position.

[0130] Next, a specific introduction will be made to the display method of the lyrics after the electronic device starts playing the song "All That the Rain Illuminates To Yun Supernova".

[0131] (1.1) The first word "want" in the first sentence of the lyrics

[0132] It can be understood that the first word "want" in the first sentence of the lyrics starts from the 0th second of the song and lasts for 0.3 seconds. When the song is played to the 0th second, the electronic device obtains the frequency 120Hz corresponding to the word "want", and then obtains its offset height of 2 pixels according to this frequency.

[0133] Optionally, the electronic device can highlight the word "want" to make the synchronization relationship between the lyrics and the song more obvious and enhance the user experience. Exemplarily, the highlighting effect can be achieved by magnifying and / or bolding the word, changing the color, etc., and this application does not limit this.

[0134] Specifically, from the 0th second to the 0.3th second of the song playback, the electronic device offsets the word "want" upward by 2 pixels relative to the lyrics display reference position and highlights it; other unplayed lyrics can be non-highlightedly displayed on their respective lyrics display reference positions, and the display effect is as shown in Figure 7 shown in (B) of

[0135] It can be understood that the duration of each word in the lyrics is generally only a few hundred milliseconds, but the display of the playback progress of the song in the electronic device is generally in seconds. Therefore, when the electronic device displays the word "want", the song playback progress corresponds to the display of "00:00".

[0136] (1.2) The second word "down" in the first sentence of the lyrics

[0137] It can be understood that the second word "down" in the first sentence of the lyrics starts from the 0.3th second of the song and lasts for 0.4 seconds. When the song is played to the 0.3th second, the electronic device obtains the frequency 150Hz corresponding to the word "down", and then obtains its offset height of 5 pixels according to this frequency.

[0138] Therefore, within the time period from 0.3 seconds to 0.7 seconds of the song playback, the electronic device offsets the word "down" 5 pixels upward based on the lyric display reference position and highlights the word "down".

[0139] It can be understood that when the first lyric sentence plays to the second word "down", the first word "want" of the first lyric sentence has already been offset upward according to the frequency. Therefore, the final display effect of the lyrics is that both words are offset upward: the word "want" is offset 2 pixels upward, and the word "down" is offset 5 pixels upward; the other unplayed lyrics can be non - highlighted at their respective lyric display reference positions. The specific display effect is as shown in Figure 8 (A) in, where the song playback progress corresponds to the display of "00:00".

[0140] (1.3) The third word "rain" of the first lyric sentence

[0141] It can be understood that the third word "rain" of the first lyric sentence starts from 0.7 seconds of the song and lasts for 0.3 seconds. At 0.7 seconds of the song playback, the electronic device obtains the frequency 115Hz corresponding to the word "rain", and then obtains its offset height of 2 pixels according to this frequency.

[0142] Similarly, within the time period from 0.7 seconds to 1 second of the song playback, the electronic device offsets the word "rain" 2 pixels upward and highlights the word "rain". The already - played words "want" and "down" respectively maintain the upward offsets of 2 pixels and 5 pixels, and the other unplayed lyrics can be non - highlighted at their respective lyric display reference positions. The specific display effect is as shown in Figure 8 (B) in, where the song playback progress corresponds to the display of "00:00".

[0143] (1.4) The fourth word "over" of the first lyric sentence

[0144] It can be understood that the fourth word "over" of the first lyric sentence starts from 1 second of the song and lasts for 0.4 seconds. At 1 second of the song playback, the electronic device obtains the frequency 210Hz corresponding to the word "over", and then obtains its offset height of 10 pixels according to this frequency.

[0145] Within the time period from 1 second to 1.4 seconds of the song playback, the electronic device offsets the word "over" 10 pixels upward based on the lyric display reference position and highlights the word "rain". The already - played words "want", "down" and "rain" maintain the upward offsets according to their respective offset heights, and the other unplayed lyrics can be non - highlighted at their respective lyric display reference positions. The specific display effect is as shown in Figure 9 (A) in, where since the start time of the word "over" is 1 second of the song, the song playback progress corresponds to the display of "00:01".

[0146] (1.5) The fifth word "that" in the first line of lyrics

[0147] It can be understood that the fifth word "that" in the first line of lyrics starts at the 1.4th second of the song and lasts for 0.2 seconds. At the 1.4th second of the song playback, the electronic device obtains the frequency 145Hz corresponding to the word "that", and then obtains its offset height of 5 pixels based on this frequency.

[0148] Similarly, within the time period from the 1.4th second to the 1.6th second of the song playback, the electronic device offsets the word "that" 5 pixels upward based on the lyrics display reference position and highlights the word "rain". The already played words "will", "fall", "rain", and "soon" respectively maintain their respective display positions, and other unplayed lyrics can be non - highlighted at their respective reference positions.

[0149] Finally, the display effect of the 5 words "will fall rain soon that" in the first line of lyrics after offsetting upward by a certain height according to their respective frequencies is as shown in Figure 9 (B) in.

[0150] From Figure 9 (B) in, it can be seen that when the song is played to the word "that", the offset heights of the 5 words "will fall rain soon that" in the first line of lyrics are proportional to their respective frequencies 120Hz, 150Hz, 115Hz, 210Hz, 145Hz. That is, the words with higher frequencies are displayed at higher positions relative to the words with lower frequencies, realizing the linkage between the song and the lyrics and enriching the display effect of the lyrics.

[0151] Scenario 2: The electronic device plays the song from the middle position of the song.

[0152] Exemplarily, Figures 10 to 12 are schematic diagrams of some user interfaces provided by the embodiments of the present application for Scenario 2.

[0153] As shown in Figure 10 (A) in, the electronic device displays the user interface 101, which is used to display the song information in the playlist and the song playback progress. Exemplarily, the electronic device is currently playing the song "All That the Rain Illuminates To Yun Fang", and in response to a user operation on the song playback progress control, such as a click operation or a swipe operation, the electronic device starts playing the song from the 13th second of the song.

[0154] From Figure 6 the information of the lyrics file shown, it can be known that the 13th second of the song corresponds to the word "rain" in the fifth line of lyrics "The suffering rain, let it flow into a river", so when the electronic device starts playing the song from the 13th second, it correspondingly starts displaying the lyrics from the word "rain" in the fifth line of lyrics.

[0155] For the sake of easy understanding, it is exemplarily set that the frequencies corresponding to the first 6 words in the fifth line of lyrics of the song, "The rain of suffering, let it flow into a river", are 140 Hz, 115 Hz, 155 Hz, 125 Hz, 145 Hz, and 130 Hz respectively.

[0156] Next, a specific introduction is made to the display method of the lyrics after the electronic device starts playing the song from the 13th second of the song "All That the Rain Illuminates To Yun Square".

[0157] In some embodiments, the electronic device starts from the word "rain" in the fifth line of lyrics, "The rain of suffering, let it flow into a river", and performs an offset display of the lyrics.

[0158] (2.1) Jump to the first word "rain" after the 13th second of the song

[0159] It can be understood that the word "rain" starts from the 13th second of the song and lasts for 0.2 seconds. When the song is played to the 13th second, the electronic device obtains the frequency 125 Hz corresponding to the word "rain", and then obtains its offset height of 3 pixels according to this frequency.

[0160] During the period from the 13th second to the 13.2nd second of the song playback, the electronic device offsets the word "rain" 4 pixels upward based on the lyrics display reference position and highlights the word "rain". The played words and other unplayed lyrics can be non - highlighted at their respective lyrics display reference positions, and the display effect is as shown in Figure 10 shown in (B) of

[0161] (2.2) Jump to the second word "ah" after the 13th second of the song

[0162] It can be understood that the word "ah" starts from the 13.2nd second of the song and lasts for 0.4 seconds. When the song is played to the 13.2nd second, the electronic device obtains the frequency 145 Hz corresponding to the word "ah", and then obtains its offset height of 5 pixels according to this frequency.

[0163] Similar to the second word "down" in the first line of lyrics in Scenario 1, during the period from the 13.2nd second to the 13.6th second of the song playback, the electronic device offsets the word "ah" 5 pixels upward based on the lyrics display reference position and highlights the word "ah". The played word "rain" remains offset 3 pixels upward based on the lyrics display reference position. The skipped words and unplayed words can be non - highlighted at their respective lyrics display reference positions. The specific display effect is as shown in Figure 11 shown in (A) of

[0164] (2.3) Jump to the third word "let" after the 13th second of the song

[0165] It is understandable that the word "Ren" starts at the 13.6th second of the song and lasts for 0.5 seconds. At the 13.6th second of the song playing, the electronic device obtains the frequency 130Hz corresponding to the word "Ren", and then obtains its offset height of 3 pixels according to this frequency.

[0166] Similarly, within the time period from the 13.2nd second to the 13.6th second of the song playing, the electronic device offsets the word "Ren" upward by 3 pixels and highlights the word "Ren". The already played words "Yu" and "Ah" respectively maintain an upward offset of 3 pixels and 5 pixels. The words in the skipped part and the unplayed words can be non-highlighted at their respective reference positions. The specific display effect is as shown in Figure 11 (B) in.

[0167] The display method of subsequent lyrics can be deduced by analogy, and will not be elaborated here.

[0168] In some other embodiments, when the song jumps to the 13th second for playing, the electronic device obtains the frequencies corresponding to each skipped word (i.e., the words "Ku", "Nan", "De") in the fifth line of lyrics and the word "Yu" corresponding to the 13th second: 140Hz, 115Hz, 155Hz, 125Hz, and respectively obtains their offset heights: 4 pixels, 2 pixels, 6 pixels, 3 pixels. Then, the lyrics "Ku Nan De Yu" are respectively offset upward by 4 pixels, 2 pixels, 6 pixels, 3 pixels based on the lyrics display reference position. The specific effect is as shown in Figure 12 (A) in.

[0169] After that, within the time period from the 13.2nd second to the 13.6th second of the song playing, the electronic device offsets the word "Ah" upward by 5 pixels, similar to "(2.2)" in the previous embodiment. The specific display effect is as shown in Figure 12 (B) in. Within the time period from the 13.6th second to the 14.1st second of the song playing, the electronic device offsets the word "Ren" upward by 3 pixels, similar to "(2.3)" in the previous embodiment. The specific display effect is as shown in Figure 12 (C) in.

[0170] In some other embodiments, when the song jumps to the 13th second for playing, the electronic device obtains the frequencies corresponding to each skipped word (i.e., the words "Ku", "Nan", "De") in the fifth line of lyrics and the word "Yu" corresponding to the 13th second: 140Hz, 115Hz, 155Hz, 125Hz. The electronic device calculates the average frequency of the lyrics to be 133.75Hz, and obtains its offset height of 4 pixels according to this average frequency.

[0171] When the electronic device displays "Ku Nan De Yu", each word is respectively offset upward by 4 pixels based on the lyrics display reference position. The specific display effect is as shown in Figure 13 shown.

[0172] The above Figure 6 Based on the information in the lyrics file shown, the lyrics display method provided by the embodiments of this application is illustrated in different scenarios. Next, the electronic device involved in the embodiments of this application will be described.

[0173] Please see Figure 14 , Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0174] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0175] In some embodiments, the internal memory 121 can be used to store audio files and lyrics files.

[0176] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0177] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0178] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0179] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0180] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0181] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0182] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0183] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0184] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0185] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0186] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0187] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0188] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0189] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0190] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0191] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0192] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0193] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0194] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0195] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0196] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0197] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0198] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0199] In some embodiments, the display screen 194 may display lyrics from a lyrics file.

[0200] The electronic device 100 can realize the acquisition function through ISP, camera 193, video codec, GPU, display screen 194 and application processor, so as to realize the image acquisition module of HAL layer in this embodiment of the application.

[0201] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into a visible image or video. The ISP can also perform algorithmic optimizations on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 193.

[0202] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element through a lens, generating an optical image. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP (Internet Service Provider) for conversion into a digital image or video signal. The ISP outputs the digital image or video signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image or video signal into image or video signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0203] A digital signal processor (DSP) is used to process digital signals. Besides processing digital images or video signals, it can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0204] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0205] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0206] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0207] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image or video playback, etc.). The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0208] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0209] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0210] The audio module 170 can also be used for encoding and decoding audio signals, and for format parsing and encapsulating audio files. In some embodiments, the audio module 170 can directly obtain audio files from the internal memory 121, or it can obtain audio files through the external memory interface 120.

[0211] In some embodiments, the audio module 170 can also be used to perform Fourier transform on the audio data to obtain the corresponding frequency.

[0212] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. In some embodiments, the electronic device 100 can play a song selected by the user through a first user operation via the speaker 170A.

[0213] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0214] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0215] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0216] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example: when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0217] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0218] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0219] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0220] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0221] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0222] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 may use the proximity sensor 180G to detect when a user holds the electronic device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0223] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0224] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0225] Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs thermal protection by reducing the performance of a processor located near temperature sensor 180J to reduce power consumption. In other embodiments, when the temperature is below another threshold, electronic device 100 heats battery 142 to prevent abnormal shutdown of electronic device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0226] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0227] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

[0228] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0229] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0230] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0231] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0232] In some embodiments, the processor 110 can invoke computer instructions stored in the internal memory 121 to retrieve lyrics files from the internal memory 121 or through the external memory interface 120; parse the lyrics files to obtain the time information of each word in the lyrics files. For example, the start and end times of the second word in the first line of lyrics.

[0233] The processor 110 can call the computer instructions stored in the internal memory 121 to obtain the audio data of the song currently playing in the electronic device from the audio module 170, and obtain the corresponding frequency through Fourier transform.

[0234] The processor 110 can call computer instructions stored in the internal memory 121 to obtain the frequency of each word based on the time information of each word in the lyrics file, and calculate their offset height based on the frequency of each word. The display screen 194 can offset the words relative to the lyrics display reference position according to the offset height of each word.

[0235] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0236] Please see Figure 15 , Figure 15 This is a software structure block diagram of an electronic device 100 provided in an embodiment of this application.

[0237] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0238] The application layer can include a series of application packages.

[0239] like Figure 15 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0240] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0241] like Figure 15 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0242] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0243] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0244] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0245] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0246] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, etc.

[0247] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0248] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0249] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0250] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0251] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0252] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0253] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0254] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0255] A 2D graphics engine is a graphics engine for 2D drawing.

[0256] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0257] In some embodiments, the sensor driver is used to respond to user actions and generate events to trigger a music application in the application layer to play a song.

[0258] In this embodiment of the application, the application framework layer may further include a lyrics display module. The following will combine... Figure 16 Explain the layered architecture of electronic device 100.

[0259] The sensor driver responds to user actions, generating events to trigger the lyrics display module to retrieve an audio file and extract audio data from it; or, to trigger the lyrics display module to retrieve the audio data of the song currently playing in the music application. The lyrics display module also performs Fourier transforms on the audio data.

[0260] The lyrics display module is used to acquire lyrics files and segment each lyric character in the lyrics file to obtain the start and end times of each lyric character.

[0261] The lyrics display module is also used to segment the audio data according to the start and end times of each word, calculate the frequency corresponding to each lyric character, and then calculate their offset height based on the frequency corresponding to each lyric character.

[0262] In some embodiments, the lyrics display module may trigger a two-dimensional graphics engine or a three-dimensional graphics processing library to render the lyrics text, and trigger the kernel layer display driver to display each lyrics text according to the offset height of each lyrics text.

[0263] It is understood that in some embodiments, the lyrics display module may also be located at other layers of the layered architecture, such as the application layer, etc., which is not limited here.

[0264] The electronic devices involved in the embodiments of this application have been described above. The lyrics display method provided in the embodiments of this application will now be explained. It is understood that the lyrics display method provided in the embodiments of this application can be executed by the electronic device 100.

[0265] Please see Figure 17 , Figure 17 This is a flowchart illustrating another lyrics display method provided in an embodiment of this application. Figure 17 As shown, the method for displaying lyrics includes:

[0266] 1701: In response to a first user operation, the electronic device acquires audio data; the audio data is the audio data corresponding to the song selected by the first user operation.

[0267] Understandably, in response to a first user action, the electronic device can play the song selected by the user. The electronic device can respond to various types of user actions; several examples are given below:

[0268] The first method: Responding to user actions on the song track controls, such as a click, the electronic device starts playing the corresponding song, for example. Figure 7 The user action shown in (A) is shown in the image.

[0269] The second method: responding to user actions on the previous or next track control, such as a click, the electronic device starts playing the corresponding song.

[0270] The third method: responding to user actions on the playback control, such as a click, the electronic device starts playing the corresponding song.

[0271] The fourth method: Responding to user actions on the playback progress control, such as a click, the electronic device starts playing the corresponding song, for example. Figure 10 The user action shown in (A) is shown in the image.

[0272] In some embodiments, the user operation can be understood as the operation of playing a song; the song selected by the user through the first user operation can be understood as the first song.

[0273] In some embodiments, the electronic device can obtain audio files from a local or third-party server, process the audio files, such as parsing the container format, decoding, etc. (see Part 4 of the Terminology section above for a description of the music player architecture) to obtain audio data.

[0274] It's understandable that speakers are driven by electrical signals, while audio files store digital signals. Therefore, before an electronic device can play a song through a speaker, it must process the audio file, such as parsing and decoding, to convert the digital signal into an electrical signal that can drive the speaker to play the song.

[0275] Therefore, in some embodiments, the electronic device can directly acquire the audio data during the above processing.

[0276] For example, taking the Android system as an example, the Android system provides a class: Visualizer (hereinafter referred to as the audio visualization class). Through this audio visualization class, audio data can be obtained in real time when playing songs using MediaPlayer (a component for playing audio).

[0277] 1702: The electronic device samples the audio data at the first sampling period to obtain the frequency corresponding to each sampling moment.

[0278] After the electronic device acquires the audio data, it samples the audio data at the first sampling frequency, and the frequency corresponding to each sampling moment can be obtained through Fourier transform.

[0279] In some embodiments, the audio visualization class provides an interface Visualizer.OnDataCaptureListener (hereinafter referred to as the first audio visualization interface), which provides a callback method onFftDataCapture (hereinafter referred to as the first frequency callback method). The second parameter of the first frequency callback method represents the Fourier transform frequency data, and the third parameter represents the sampling rate. The corresponding frequency can be obtained directly through the first frequency callback method.

[0280] For example, an electronic device can call the first frequency callback method at a first sampling period to obtain the frequency corresponding to each sampling time point.

[0281] Please see Figure 18 , Figure 18 An example is given of the results obtained by sampling audio data by an electronic device with a sampling period of 0.1 seconds (i.e., a sampling frequency of 10Hz).

[0282] like Figure 18 As shown, the electronic device samples 23 times, obtaining 23 frequency data points. For easier understanding, a timer can be set at the start of sampling and initialized to 0 to record each sampling moment.

[0283] For example, the sampling time can be represented by t. t = 0.1 seconds represents the first sampling, and the frequency obtained from the first sampling is 120Hz; t = 0.2 seconds represents the second sampling, and the frequency obtained from the second sampling is 130Hz; t = 0.3 seconds represents the third sampling, and the frequency obtained from the third sampling is 155Hz. The meaning of subsequent data can be deduced in the same way, and will not be elaborated here.

[0284] 1703: The electronic device acquires the frequency corresponding to the target word at the first moment, which is the moment when the target word begins to be displayed.

[0285] In this embodiment, the target word can be understood as a word in the lyrics file that has a defined start and end position. Understandably, the word with the defined start and end times may differ in different language environments. For example, in Chinese, the target word can be understood as a single character; in English, the target word can be understood as a single word.

[0286] Understandably, electronic devices play songs and display lyrics synchronously; that is, when a target word is displayed, the electronic device plays a corresponding audio segment for that target word.

[0287] For a better understanding of the target terms in the embodiments of this application, please refer to [link / reference]. Figure 19 , Figure 19 This is a schematic diagram illustrating the correspondence between words and time in another lyrics file provided in this application embodiment.

[0288] Figure 19 The details of the first three lines of the song are shown: the first line is “ABCDEF” starting at millisecond 0 and ending at millisecond 2400; the second line is “GHIJKL” starting at millisecond 2400 and ending at millisecond 4400; and the third line is “MNOPQR” starting at millisecond 4400 and ending at millisecond 6300.

[0289] The first line of lyrics lasts for 2400 milliseconds. The first word "A" starts at millisecond 0 of the song and lasts for 400 milliseconds; the second word "B" starts at millisecond 400 and lasts for 600 milliseconds; the third word "C" starts at millisecond 1000 and lasts for 300 milliseconds. The information for the other words can be deduced in the same way, and will not be repeated here.

[0290] For example, in the embodiments of this application, the start time of word "A" can be understood as the 0th second of the song, and the end time can be understood as the 0.4th second of the song; the start time of word "B" can be understood as the 0.4th second of the song, and the end time can be understood as the 0.6th second of the song.

[0291] For example, any word in the lyrics file, such as "A", "B", "C", etc., can be understood as a target word.

[0292] When the target word is displayed, the electronic device acquires the frequency corresponding to that target word. Specifically, the electronic device can acquire the frequency based on the start and end times of the target word in the lyrics file.

[0293] For ease of understanding, the songs played on electronic devices correspond to... Figure 19 The lyrics file shown is used as an example for illustration. For instance, the sampling result obtained by the electronic device sampling the audio data at a sampling period of 0.1 seconds is as follows: Figure 20 As shown, Figure 20 The frequency data shown are Figure 17 The same applies, so I won't explain further here.

[0294] For example, if the target word is word "A", from Figure 19 It can be seen that the target word "A" starts at second 0 of the song and lasts for 0.4 seconds. When the target word "A" begins to be displayed, as follows... Figure 18 As shown, the electronic device obtains data from t=0 to t=0.4 from the sampling results.

[0295] Understandably, since the sampling period is 0.1 seconds, a total of 4 data points are obtained. For example, the electronic device acquires 120Hz, 130Hz, 155Hz and 195Hz.

[0296] For example, if the target word is word "B", from Figure 19 It can be seen that the target word "B" starts at 0.4 seconds into the song and lasts for 0.6 seconds. When the target word "B" begins to be displayed, as follows... Figure 18 As shown, the electronic device obtains data from the sampling results from t=0.4 seconds to t=1 seconds, and obtains a total of 6 data points: 275Hz, 290Hz, 375Hz, 310Hz, 230Hz and 220Hz.

[0297] Similarly, if the target word is "C", when the target word "C" is first displayed, the electronic device obtains data from the sampling results from t=1 second to t=1.3 seconds, namely 95Hz, 125Hz and 160Hz.

[0298] If the target word is "D", when the target word "D" is first displayed, the electronic device obtains data from the sampling results for t=1.3 seconds to t=1.7 seconds, namely 185Hz, 170Hz, 200Hz and 210Hz.

[0299] If the target word is "E", when the target word "E" is first displayed, the electronic device obtains data from the sampling results from t=1.7 seconds to t=2.1 seconds, namely 325Hz, 330Hz, 360Hz and 290Hz.

[0300] After obtaining the frequency data of the target word, the electronic device calculates the frequency corresponding to the target word by analyzing the frequency data. For example, this embodiment provides two methods for calculating the frequency of a word, wherein:

[0301] (1) Sum the acquired frequency data directly. For an example, please refer to... Figure 21 , Figure 21 This is a schematic diagram illustrating the result of obtaining the frequency corresponding to a target word by summation, as provided in an embodiment of this application.

[0302] For example, when the target word is "A", the electronic device sums the frequency data of 120Hz, 130Hz, 155Hz and 195Hz to get the frequency corresponding to the word "A" as 600Hz.

[0303] For example, when the target word is "B", the electronic device sums the frequency data of 275Hz, 290Hz, 375Hz, 310Hz, 230Hz and 220Hz to get the frequency corresponding to the word "B" as 1700Hz.

[0304] For example, when the target word is "C", the electronic device sums the frequency data of 95Hz, 125Hz and 160Hz to get the frequency corresponding to the word "C" as 380Hz.

[0305] For example, when the target word is "D", the electronic device sums the frequency data of 185Hz, 170Hz, 200Hz and 210Hz to get the frequency corresponding to the word "D" as 765Hz.

[0306] For example, when the target word is "E", the electronic device sums the frequency data of 325Hz, 330Hz, 360Hz and 290Hz to get the frequency corresponding to the word "E" as 1305Hz.

[0307] (2) Sum the acquired frequency data and then average them. For an example, please refer to [link to example]. Figure 22 , Figure 22 This is a schematic diagram illustrating how the frequency of a target word is obtained by summing and then averaging, according to an embodiment of this application. It is understood that, for ease of description, this embodiment explains the rounding of the calculation results.

[0308] For example, when the target word is "A", there are 4 sampling points in the time period corresponding to the word "A". Dividing the total frequency value of 600Hz by the number of sampling points of 4, we get the frequency corresponding to the word "A" as 150Hz.

[0309] For example, when the target word is "B", there are 6 sampling points in the time period corresponding to the word "B". Dividing the total frequency of 1700Hz by the number of sampling points of 6, we get the frequency corresponding to the word "B" as 284Hz.

[0310] For example, when the target word is "C", there are 3 sampling points in the time period corresponding to the word "C". Dividing the total frequency of 380Hz by the number of sampling points of 3, we get the frequency corresponding to the word "C" as 127Hz.

[0311] For example, when the target word is "D", there are 4 sampling points in the time period corresponding to the word "D". Dividing the total frequency of 765Hz by the number of sampling points of 4, we get the frequency corresponding to the word "D" as 191Hz.

[0312] For example, when the target word is "E", there are 4 sampling points in the time period corresponding to the word "E". Dividing the total frequency of 1305Hz by the number of sampling points of 4, we get the frequency corresponding to the word "E" as 326Hz.

[0313] Understandably, considering that in reality, the frequency of the lyrics sung may remain basically unchanged for a period of time, but the singing time is relatively long, the frequency obtained by the second method is more accurate than the frequency obtained by the first method.

[0314] In the embodiments of this application, as described above Figure 19 The word "A" shown can be understood as the first target word, and the word "B" can be understood as the second target word. Correspondingly, the audio data segment corresponding to the word "A" can be understood as the first audio segment, and the audio data segment corresponding to the word "B" can be understood as the second audio segment.

[0315] 1704: Based on the frequency corresponding to the target word, the electronic device determines the offset height of the target word relative to the lyrics display reference position.

[0316] In this embodiment of the application, the lyrics display reference position can be understood as the reference position for offsetting the target word.

[0317] In this embodiment of the application, the target word may be offset upwards or downwards relative to the lyrics display reference position, and this application does not limit this. For example, the target word may be offset upwards or downwards relative to the coordinates (10, 12), where the coordinates (10, 12) can be understood as the lyrics display reference position of the target word.

[0318] To make it easier to understand, the following explanation will take the example of shifting the target word upwards relative to the lyrics display reference position.

[0319] It is understandable that when the target word is shifted upward relative to the lyrics display reference position, the lyrics display reference position can be understood as the lowest display position of the target word, the lyrics display highest position can be understood as the highest display position of the target word, and the height difference between the lyrics display reference position and the lyrics display highest position is the total offset height of the target word.

[0320] To map frequency data to height data, in this embodiment, the lyrics display reference position corresponds to a lowest frequency, which can be understood as a first frequency; the lyrics display highest position corresponds to a highest frequency, which can be understood as a second frequency. The electronic device determines the offset height of the target word relative to the lyrics display reference position based on the highest and lowest frequencies.

[0321] In some embodiments, the first frequency may be set to the lowest sound frequency that the human ear can hear, and the second frequency may be set to the highest sound frequency that the human ear can hear; or, the first frequency may be set to the lowest audio frequency in the song, and the second frequency may be set to the highest audio frequency in the song, and this application does not limit this.

[0322] For ease of understanding, Figure 22 The data shown exemplarily illustrates step 1503. Specifically, the electronic device can obtain the offset height of the target word in the following manner:

[0323] Method 1:

[0324] The electronic device calculates the difference between the frequency corresponding to the target word and the lowest frequency; then it calculates the proportion of this difference in the frequency difference obtained by subtracting the lowest frequency from the highest frequency; finally, it multiplies this proportion by the total offset height to obtain the offset height of the target word.

[0325] For ease of understanding, for example, the lowest frequency corresponding to the lyrics display reference position is set to 100Hz, and the highest frequency corresponding to the lyrics display highest position is set to 300Hz, with a frequency difference of 200Hz between the two; the height difference between the lyrics display reference position and the lyrics display highest position is 10 pixels.

[0326] For example, in the case where the target word is "A", by Figure 22 We know that the frequency corresponding to the word "A" is 150Hz. Subtracting the frequency of 100Hz corresponding to the lyrics display reference position from this frequency gives a difference of 50Hz. The difference of 50Hz accounts for 1 / 4 of the frequency difference of 200Hz. Therefore, the offset height of the word "A" is 10 times 1 / 4.

[0327] Understandably, since the position of the lyrics is determined by pixels, and pixel positions are discrete values ​​for electronic devices, the calculation result can be rounded up. For example, rounding up results in an offset height of 3 pixels for the word "A".

[0328] Similarly, when the target word is "B", the frequency corresponding to the word "B" is 284Hz. Subtracting the frequency of 100Hz corresponding to the lyrics display reference position from this frequency gives 184Hz. The difference of 50Hz accounts for 92 / 100 of the frequency difference of 200Hz. Therefore, the offset height of the word "B" is 10 pixels.

[0329] With the target word "C", the frequency corresponding to "C" is 127Hz. Subtracting the frequency of the lyrics display reference position (100Hz) from this frequency gives 27Hz. The difference of 27Hz accounts for 135 / 1000 of the frequency difference of 200Hz. Therefore, the offset height of the word "C" is 2 pixels.

[0330] With the target word "D", the frequency corresponding to "D" is 191Hz. Subtracting the frequency of the lyrics display reference position (100Hz) from this frequency gives 91Hz. The difference of 91Hz accounts for 455 / 1000 of the frequency difference of 200Hz. Therefore, the offset height of the word "D" is 5 pixels.

[0331] In some embodiments, when the frequency corresponding to the target word is less than or equal to the first frequency, the offset height of the target word is 0; when the frequency corresponding to the target word is greater than or equal to the second frequency, the offset height of the target word is the total offset height.

[0332] When the target word is “E”, the frequency of the word “E” is 326Hz, which is greater than the highest frequency of 300Hz corresponding to the highest position of the lyrics display. Therefore, the offset height of the target word “D” is 10 pixels.

[0333] Method 2:

[0334] For example, please refer to Figure 23 , Figure 23 This is a schematic diagram illustrating how the offset height is determined by the frequency corresponding to the target word, as provided in an embodiment of this application.

[0335] For ease of understanding, the lowest frequency corresponding to the lyrics display reference position is set to 100Hz, and the highest frequency corresponding to the lyrics display highest position is set to 300Hz; the frequency difference between the two is 200Hz, and the height difference between the lyrics display reference position and the lyrics display highest position is 10 pixels.

[0336] like Figure 23As shown, the height between the baseline position and the highest position of the lyrics display can be divided into 10 equal parts, each representing one pixel.

[0337] In other words, if the frequency of the target word is between 100Hz and 120Hz, it will be offset by 1 pixel relative to the lyrics display reference position; if the frequency of the target word is between 120Hz and 140Hz, it will be offset by 2 pixels relative to the lyrics display reference position; if the frequency of the target word is between 140Hz and 160Hz, it will be offset by 3 pixels relative to the lyrics display reference position, and so on. This will not be elaborated further here.

[0338] For example, when the target word is the word "A", the frequency range corresponding to the frequency 150Hz of the word "A" is 140Hz to 160Hz. Therefore, the word "A" is offset by 3 pixels relative to the lyrics display reference position.

[0339] For example, when the target word is the word "B", the frequency range corresponding to the frequency 284Hz of the word "B" is 280Hz to 300Hz. Therefore, the word "B" is offset by 10 pixels relative to the lyrics display reference position.

[0340] For example, when the target word is the word "C", the frequency corresponding to the word "C" is 127Hz, which corresponds to a frequency range of 120Hz to 140Hz. Therefore, the word "C" is offset upward by 2 pixels relative to the lyrics display reference position.

[0341] For example, when the target word is the word "D", the frequency corresponding to the word "D" is 191Hz, which corresponds to a frequency range of 180Hz to 200Hz. Therefore, the word "D" is offset upward by 5 pixels relative to the lyrics display reference position.

[0342] For example, when the target word is "E", the frequency corresponding to "E" is 326Hz, which is greater than the frequency of 300Hz corresponding to the highest position of the lyrics display. Therefore, "E" can be placed in the highest position of the display area. Thus, "E" is offset by 10 pixels relative to the lyrics display reference position.

[0343] Finally, for the lyrics "ABCDE", the offset heights of each word relative to the lyrics display reference position, expressed in pixels, are 3, 10, 2, 5, and 10 respectively. It's understandable that the first and second methods described above are essentially the same, and yield the same results.

[0344] 1705: The electronic device displays the target word based on its offset height.

[0345] As can be understood from the description in step 1704, the electronic device may shift the target word upward or downward relative to the lyrics display reference position according to the actual situation, and this application does not impose any restrictions on this.

[0346] To facilitate understanding, the following example illustrates the upward offset of the target word relative to the lyrics display reference position, using the offset height data shown in step 1504. For example, the y-coordinate of the lyrics display reference position is set to 15.

[0347] Please see Figure 24 , Figure 24 This is a schematic diagram of an interface for displaying lyrics and songs in conjunction with an embodiment of this application. Figure 24 As shown, the title can be used to display the song name. Lyrics display area 1 can be used to display the lyrics for the part of the song that has been played, and lyrics display area 2 can be used to display the lyrics for the part of the song that has not been played. The player control area can be used to control the playback progress of the song, as well as switch songs, etc.

[0348] The area between the baseline lyric display position and the highest lyric display position (hereinafter referred to as the playback display area) can be used to display the lyric corresponding to the current playback time. For example, the target word currently being played can be displayed in this playback display area.

[0349] For example, when the target word is "A", the word "A" is shifted upward by 3 pixels relative to the reference position of the lyrics display area, resulting in a y-coordinate of 12 for the word "A".

[0350] For example, when the target word is "B", the word "B" is shifted upward by 10 pixels relative to the reference position of the lyrics display area, resulting in a y-coordinate of 5 for the word "B".

[0351] For example, when the target word is "C", the word "C" is shifted upward by 2 pixels relative to the reference position of the lyrics display area, resulting in a y-coordinate of 13 for the word "C".

[0352] For example, when the target word is "D", the word "D" is shifted upward by 5 pixels relative to the reference position of the lyrics display area, resulting in a y-coordinate of 10 for the word "D".

[0353] For example, when the target word is "E", the word "E" is shifted upward by 10 pixels relative to the reference position of the lyrics display area, resulting in a y-coordinate of 5 for the word "E".

[0354] Finally, when the target word is "E", the electronic device has already processed the lyrics "ABCD" for display. At this point, the display effect of the electronic device is as follows: Figure 24As shown, the lyrics “ABCDE” are displayed according to their respective display heights.

[0355] In some embodiments, the unplayed word in the lyrics corresponding to the target word can be placed at the lyrics display reference position. For example, the unplayed word "F" in the currently playing "ABCDEF" can be placed at the lyrics display reference position, that is, the y-coordinate of the word "F" is 15.

[0356] In this embodiment of the application, the area used to display the currently playing lyric can be called the lyric display area, which can be understood as the first display area.

[0357] In some embodiments, the electronic device divides the first display area horizontally into K sub-display areas, where K is an integer greater than or equal to 2;

[0358] The electronic device determines the frequency range corresponding to each sub-display area within the K display areas;

[0359] At the first moment, the electronic device displays the target word in the first sub-display area in the first display area, and the first sub-display area is one of the K sub-display areas; the frequency of the first audio segment is included in the frequency range corresponding to the first sub-display area.

[0360] For example, the electronic device can horizontally divide the first display area into 5 sub-display areas. From bottom to top, each sub-display area corresponds to a frequency range of 200Hz-300Hz, 300Hz-400Hz, 400Hz-500Hz, 500Hz-600Hz, and 600Hz-700Hz. The word "B" corresponds to a frequency of 284Hz, which falls within the 200Hz-300Hz frequency range. Therefore, the electronic device displays the word "B" in the first sub-display area from bottom to top.

[0361] Understandably, in the scenario description above, Figures 7-11 The scenario shown can be obtained from steps 1701-1705 above.

[0362] In some embodiments, for example Figure 12 In the corresponding scenario, in response to the jump playback operation, the electronic device starts playing the song from the target word A; at this first moment, the electronic device displays the N words preceding the target word A at different heights in the first display area, wherein the height of each of the N words is determined by the frequency of each word; the N words and the target word A belong to the same lyric.

[0363] For example, in response to a playback jump operation, the electronic device starts playing the song from the position corresponding to the word "E" in the first line of lyrics. When displaying the word "F", the electronic device obtains the frequencies corresponding to the words "A", "B", "C" and "D", as detailed in step 1703; then it calculates the offset height of each word and displays each word with an offset, as detailed in steps 1704 and 1705.

[0364] In some embodiments, for example Figure 13 In the corresponding scenario, in response to the jump playback operation, the electronic device starts playing the song from the target word A; in the first display area, the electronic device displays each of the M words preceding the target word A at a third height; the third height is determined by the average frequency corresponding to the M words; the M words and the target word A belong to the same lyric.

[0365] For example, in response to a playback jump operation, the electronic device starts playing the song from the position corresponding to the word "E" in the first line of lyrics. When displaying the word "F", the electronic device obtains the frequencies corresponding to the words "A", "B", "C" and "D", as detailed in step 1703; then it calculates the average frequency of the four words and displays them at an offset based on the average frequency, as detailed in steps 1704 and 1705.

[0366] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0367] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0368] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for displaying lyrics, characterized in that, The method includes: In response to a playback jump operation, the electronic device plays the first audio segment of a first song and displays a lyrics interface; the lyrics interface includes a first display area, which is used to highlight the currently playing lyric; when playing the first target word of the first audio segment in the first song, the electronic device displays each of the M words preceding the first target word at a third height in the first display area; the third height is determined by the average frequency of the audio segment corresponding to the M words; the M words and the first target word belong to the same lyric; M is an integer greater than or equal to 1; In the lyrics interface, all lyrics except the currently playing line are not highlighted; When the first target word in the first audio segment of the first song is played, the electronic device displays an animation that adjusts the height of the first target word from the reference position to the first height, wherein the first height is determined by the electronic device according to the frequency of the audio corresponding to the first target word; When playing the second target word of the second audio segment in the first song, the electronic device displays an animation effect that adjusts the height of the second target word from the reference position to the second height, and displays the second target word corresponding to the second audio segment at the second height in the first display area; the second height is determined by the electronic device according to the frequency of the audio corresponding to the second target word; the frequency of the audio corresponding to the second target word is greater than the frequency of the audio corresponding to the first target word, and the second height is greater than the first height; The first audio segment is an audio segment in the audio data of the first song, and the first target word is a word in the lyrics file of the first song that corresponds to the first audio segment; The second audio segment is another audio segment in the audio data of the first song that is different from the first audio segment, and the second target word is a word in the lyrics file of the first song that corresponds to the second audio segment.

2. The method according to claim 1, characterized in that, Before the electronic device displays an animation adjusting the height of the first target word from a reference position to a first height while playing the first audio segment of the first song, the method further includes: The electronic device acquires the audio data and lyrics file of the first song; The electronic device samples the audio data at a first sampling period to obtain a first sampling result set; the first sampling result set includes the frequency corresponding to each sampling moment. The electronic device obtains the frequency of the first audio segment from the first sampling result set based on the start and end times of the first target word in the lyrics file.

3. The method according to claim 2, characterized in that, The electronic device obtains the frequency of the first audio segment from the first sampling result set based on the start and end times of the first target word in the lyrics file, including: The electronic device determines a first time interval consisting of the start time and the end time of the first target word; The electronic device collects the first sampling results and sums the frequencies corresponding to the sampling times within the first time interval, or sums them and then averages them to obtain the frequency of the first audio segment.

4. The method according to any one of claims 1-3, characterized in that, Before the electronic device displays an animation adjusting the height of the first target word from a reference position to a first height while playing the first audio segment of the first song, the method further includes: The electronic device divides the first display area horizontally into K sub-display areas, where K is an integer greater than or equal to 2; The electronic device determines the frequency range corresponding to each sub-display area in the K display areas; When playing a first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment at a first height in the first display area, including: When playing the first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment in the first sub-display area in the first display area. The first sub-display area is one of the K sub-display areas. The frequency of the first audio segment is included in the frequency range corresponding to the first sub-display area.

5. The method according to any one of claims 1-3, characterized in that, The first display area includes a lyrics display reference position, which is a reference position for offsetting the first target word; When playing the first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment at a first height in the first display area, including: When playing the first audio segment of the first song, the electronic device displays the first target word corresponding to the first audio segment in the first display area, offset by the first height relative to the lyrics display reference position.

6. The method according to claim 5, characterized in that, Before the electronic device displays an animation adjusting the height of the first target word from a reference position to a first height while playing the first audio segment of the first song, the method further includes: The electronic device determines a first frequency, a second frequency, and a total offset height; the second frequency is greater than the first frequency. The electronic device determines the first height based on the frequency of the first audio segment, the first difference, and the total offset height; the first difference is the difference between the second frequency and the first frequency.

7. The method according to claim 6, characterized in that, The first frequency is the lowest sound frequency that the human ear can hear, and the second frequency is the highest sound frequency that the human ear can hear. Alternatively, the first frequency is the lowest frequency of the audio in the first song, and the second frequency is the highest frequency of the audio in the first song.

8. The method according to claim 6 or 7, characterized in that, The electronic device determines the first height based on the frequency of the first audio band, the first difference, and the total offset height, including: The electronic device subtracts the first frequency from the frequency of the first audio segment to obtain a second difference; The electronic device divides the second difference by the first difference to obtain a first ratio; The electronic device multiplies the first ratio by the total offset height to obtain the first height.

9. The method according to claim 8, characterized in that, The method further includes: When the frequency of the first audio segment is less than or equal to the first frequency, the first height is 0; When the frequency of the first audio segment is greater than or equal to the second frequency, the first height is the total offset height.

10. The method according to any one of claims 5-9, characterized in that, The first target word can be offset by either upward or downward offset.

11. An electronic device, characterized in that, The electronic device includes: one or more processors, memory, and a display screen; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-10.

Images