Display method, electronic equipment and storage medium

Abstract

The invention discloses a display method, electronic equipment and a storage medium, and belongs to the technical field of terminals.The display method comprises the steps that a music cover and audio data of currently played music are acquired; extracting main color elements from the music cover; filling color steps according to the main color elements; and in the process of playing the audio data, controlling the circular display of the color step on a terminal display interface at least according to the time domain wave crest of the audio data in the time domain.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a display method, an electronic device and a storage medium. Background Art

[0002] As smartphones and other terminals have become an indispensable part of daily life, terminal wallpapers, as miniature canvases of personality and emotion, have gradually attracted the attention of many users.

[0003] Although there are many wallpapers for terminals, they are usually colorful static pictures. Even if there are some dynamic wallpapers, they are only fixed loop animations or visual effects, which are relatively simple and cannot meet the growing demand for personalization and interactivity. Summary of the invention

[0004] In a first aspect, an embodiment of the present application provides a display method, comprising: Get the music cover and audio data of the currently playing music; extracting a primary color element from the music cover; Filling the color ladder according to the main color element; and

[0005] During the playing of the audio data, the cyclic display of the color ladder on the terminal display interface is controlled at least according to the time domain peak of the audio data in the time domain.

[0006] In some embodiments, extracting the main color element from the music cover includes: Extracting a plurality of colors from the music cover, and selecting at least one main color from the plurality of colors; The brightness and saturation of the at least one primary color in the HSL color space are adjusted to obtain at least one gradient color corresponding to the at least one primary color, and the at least one gradient color is used as a primary color element.

[0007] In some embodiments, the color ladder includes a plurality of color ladders, and different color ladders are filled with different main color elements.

[0008] In some embodiments, the plurality of color steps are arranged along the vertical direction of the wallpaper, and the coloring of the plurality of color steps gradually deepens upward or downward along the vertical direction.

[0009] In some embodiments, controlling the cyclic display of the color ladder on the terminal display interface according to the time domain peak of the audio data in the time domain includes: During the playing of the audio data, determining that the audio data of the current frame reaches a time domain peak in the time domain, and controlling each of the color steps to move a preset distance along a preset moving direction; Among them, when the first color ladder moves beyond the first boundary area of ​​the terminal display interface, the part of the first color ladder that exceeds the first boundary area is controlled to move to the blank area adjacent to the second boundary area, and the blank area adjacent to the second boundary area is the blank area left after the second color ladder moves a preset distance from the second boundary area along a preset moving direction.

[0010] In some embodiments, the color ladder includes three, and the controlling the cyclic display of the color ladder on the terminal display interface according to the time domain peak of the audio data in the time domain includes: During the playing of the audio data, determining that the audio data of the current frame reaches a time domain peak in the time domain, and controlling the three color ladders to move a preset distance along a preset moving direction; Among them, when the third color step among the three color steps exceeds the first boundary area of ​​the terminal display interface, the part of the third color step that exceeds the first boundary area is controlled to move to the blank area adjacent to the second boundary area, and the blank area adjacent to the second boundary area is the blank area left after the fourth color step among the three color steps moves a preset distance from the second boundary area along a preset moving direction.

[0011] In some embodiments, controlling the dynamic display of the color ladder according to the music rhythm information includes: During the playing of the audio data, determining whether a human voice signal exists in the audio data of the current frame; The maximum frequency domain peak of the human voice signal of the current frame in the frequency domain is determined, and the color depth of the color ladder is controlled according to the amplitude of the maximum frequency domain peak of the current frame.

[0012] In some embodiments, controlling the color depth of the color ladder according to the amplitude of the maximum frequency domain peak of the current frame includes: Determine that the amplitude of the maximum frequency domain peak of the current frame is greater than the amplitude of the maximum frequency domain peak of the previous frame, and control the brightness and saturation of the color ladder in the HSL color space to increase; or

[0013] It is determined that the amplitude of the maximum frequency domain peak of the current frame is smaller than the amplitude of the maximum frequency domain peak of the previous frame, and the brightness and saturation of the color ladder in the HSL color space are controlled to decrease.

[0014] In some embodiments, it also includes: Determining that there is no human voice signal in the audio data of the current frame; The maximum time domain peak of the audio data of the current frame in the time domain is determined, and the color depth of the color ladder is controlled according to the amplitude of the maximum time domain peak of the current frame.

[0015] In some embodiments, controlling the color depth of the color ladder according to the amplitude of the maximum temporal peak of the current frame includes: Determine that the amplitude of the maximum time-domain peak of the current frame is greater than the amplitude of the maximum time-domain peak of the previous frame, and control the brightness and saturation of the color ladder in the HSL color space to increase; or

[0016] It is determined that the amplitude of the maximum time-domain peak of the current frame is smaller than the amplitude of the maximum time-domain peak of the previous frame, and the brightness and saturation of the color ladder in the HSL color space are controlled to decrease.

[0017] In a second aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the methods described above is implemented.

[0018] In a third aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods described above.

[0019] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any of the methods described above when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of the structure of a terminal provided in an embodiment of the present application; Figure 2 A schematic diagram of a flow chart of a display method provided in an embodiment of the present application; Figure 3 One of the scene schematic diagrams of the display method provided in the embodiment of the present application; Figure 4 The second schematic diagram of the scene of the display method provided in the embodiment of the present application; Figure 5 One of the display effect schematic diagrams of the display method provided in the embodiment of the present application; Figure 6 The second schematic diagram of the display effect of the display method provided in the embodiment of the present application; Figure 7 The third schematic diagram of the display effect of the display method provided in the embodiment of the present application; Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0024] In today's era where smartphones and other smart terminals are deeply integrated into daily life, wallpapers, as a subtle medium for personalized expression and emotional transmission, are attracting more and more attention from users. Traditional wallpapers are mostly static, and even some dynamic wallpapers are often just simple animation loops, lacking real-time interaction with user emotions or the external environment, and are relatively simple, making it difficult to meet the growing demand for personalization and interactivity.

[0025] To this end, this application provides a display method that integrates the main color elements of the music cover into the wallpaper design, and adjusts the color ladder in real time according to the time domain peak of the audio data in the time domain. The loop display on the terminal display interface forms a dynamic music wallpaper, which greatly enhances the user's personalized experience. This interactivity allows the user's device interface to not only reflect their music taste, but also change dynamically with the rhythm of the music, increasing the fun and immersion of use.

[0026] The display method provided in the embodiments of the present application can be applied to terminals such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc., and can also be applied to databases, servers, and service response systems based on terminal artificial intelligence. The embodiments of the present application do not impose any restrictions on the specific types of terminals.

[0027] For example, the terminal can be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a computer, a laptop computer, a handheld communication device, a handheld computing device, and / or other devices for communicating on a wireless system and a next-generation communication system, such as a mobile terminal in a 5G network, a mobile terminal in a future evolved Public Land Mobile Network (PLMN), or a mobile terminal in a future evolved Non-terrestrial Network (NTN), etc.

[0028] As an example but not limitation, when the terminal is a wearable device, the wearable device can also be a general term for wearable devices that are intelligently designed and developed by applying wearable technology to daily wear, such as gloves, watches, AR (Augmented Reality) head-mounted display devices, VR (Virtual Reality) head-mounted display devices or MR (Mixed Reality) head-mounted display devices, etc., which are equipped with far-field communication modules and / or near-field communication modules.

[0029] In some embodiments, the terminal may be a Figure 1 The mobile phone 100 of the hardware structure shown in FIG. Figure 1As shown, the mobile phone 100 may specifically include: a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a short-range wireless communication module 170, a processor 180, and a power supply 190. Those skilled in the art will appreciate that Figure 1 The structure of the mobile phone 100 shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0030] Combine the following Figure 1 A detailed introduction to the various components of the mobile phone: The RF circuit 110 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 180 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 110 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, and the wireless communication can include global system for mobile communications (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), new radio (new radio, NR), GNSS, FM, low orbit satellite connection and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS), etc.

[0031] The memory 120 can be used to store software programs and modules. The processor 180 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 120. The memory 120 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as pictures, audio data, phone books, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Specifically, the memory 120 may store pictures taken by an electronic device or downloaded via a wireless network.

[0032] The input unit 130 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone 100. Specifically, the input unit 130 may include a touch panel 131 and other input devices 132. The touch panel 131, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 131 or near the touch panel 131 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 131 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 180, and can receive and execute the command sent by the processor 180. In addition, the touch panel 131 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 131, the input unit 130 may also include other input devices 132. Specifically, other input devices 132 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0033] The display unit 140 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 140 may include a display panel 141. Optionally, the display panel 141 may be configured in the form of a liquid crystal display (LCD), a light emitting diode (LED), an organic light emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), etc. Further, the touch panel 131 may cover the display panel 141. When the touch panel 131 detects a touch operation on or near it, it is transmitted to the processor 180 to determine the type of touch event. Subsequently, the processor 180 provides a corresponding visual output on the display panel 141 according to the type of touch event. Although in Figure 1 In the embodiment, the touch panel 131 and the display panel 141 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 131 and the display panel 141 can be integrated to realize the input and output functions of the mobile phone.

[0034] The mobile phone 100 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 141 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0035] The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the mobile phone. The audio circuit 160 can transmit the received audio data to the speaker 161 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then the audio data is output to the processor 180 for processing, and then sent to another electronic device through the RF circuit 110, or the audio data is output to the memory 120 for further processing.

[0036] Communication technologies such as Wi-Fi, Bluetooth, and Near Field Communication (NFC) belong to short-range wireless transmission technologies. The mobile phone can help users send and receive emails, browse web pages, access streaming media, etc. through the short-range wireless module 170, which provides users with wireless broadband Internet access. The above-mentioned short-range wireless module 170 may include a Wi-Fi chip, a Bluetooth chip, and an NFC chip. The Wi-Fi chip can realize the function of Wi-Fi Direct connection between the mobile phone 100 and other electronic devices, and can also make the mobile phone 100 work in an AP mode (Access Point mode) that can provide wireless access services and allow other wireless devices to access, or work in a STA mode (Station mode) that can connect to an AP but does not accept wireless devices to access, thereby establishing point-to-point communication between the mobile phone 100 and other Wi-Fi devices.

[0037] The processor 180 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 120, and calling data stored in the memory 120, so as to monitor the mobile phone as a whole. Optionally, the processor 180 may include one or more processing units; optionally, the processor 180 may include, for example, an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors.

[0038] The mobile phone 100 also includes a power source 190 (such as a battery) for supplying power to various components. Preferably, the power source can be logically connected to the processor 180 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0039] The mobile phone 100 may also include a camera. Optionally, the position of the camera on the mobile phone may be front-mounted or rear-mounted, which is not limited in the embodiments of the present application. The mobile phone may collect a scene image of the current scene through the camera, and determine the scene information and the scene type by analyzing the scene image.

[0040] Figure 2 This is one of the flow charts of the display method provided in the embodiment of the present application. Figure 2 As shown, a display method is provided, including the following steps: step 210, step 220, step 230, and step 240. The steps of the method flow are only used as a possible implementation of the present application.

[0041] Step 210: Obtain the music cover and audio data of the currently playing music; The music cover here refers to the album cover image of the music played in the current playback interface of a music player (such as a music player pre-installed on the terminal or a music player downloaded from, for example, an application installation platform), which usually includes album information and related images (for example, images generated by a machine learning model based on information such as the singer and lyrics content).

[0042] Specifically, the music cover and audio data can be obtained after the user triggers the music wallpaper function. The music wallpaper function can be triggered by a button on the music player interface, a setting option, or a media card in the notification bar, etc., and there is no limitation on this.

[0043] like Figure 3 As shown, when the user starts the music player of the terminal, the music player will load and parse the local or online music library, prepare the playlist and audio data. At the same time, the terminal creates a music media card, which contains the key information of the currently playing music. The terminal will register and display the music media card to the specified location in the system notification bar. This music media card is the entrance to enable the music wallpaper function. Specifically, when the user clicks on the music media card, the interface will jump from the notification bar interface to the music playback interface. At the same time, the terminal will turn on the music wallpaper function and start to obtain the music cover and audio data to achieve the subsequent generation of dynamic music wallpaper effects based on the music cover and audio data.

[0044] Step 120: extracting main color elements from the music cover; Here, the main color element can be a color element obtained based on the color with higher saturation, brightness and other attributes in the music cover, or a color element obtained based on the color with a larger display area in the music cover, or a color element obtained based on the color that best represents the theme or emotional tone of the music, etc.

[0045] Step 130, filling the color ladder according to the main color element; and

[0046] Here, after extracting the main color element, the specific position, shape and size of the color ladder on the terminal display interface are determined, and then the color element is applied as fill to the interface element of the color ladder on the terminal display interface.

[0047] like Figure 4 As shown, after extracting the main color elements 401a, 402a and 403a from the music cover 400a, the specific positions, shapes and sizes of the three color ladders 401b, 402b and 403b on the terminal display interface 400b are determined. Here, the highest point H of the color ladder 401b to the lowest point L of the color ladder 403b can just be the height of the terminal display interface 400b.

[0048] After extracting the main color elements 401a, 402a and 403a, the main color element 401a is then filled into the interface element applied to the color ladder 401b, the main color element 402a is filled into the interface element applied to the color ladder 402b, and the main color element 403a is filled into the interface element applied to the color ladder 403b, to generate the following: Figure 4 Wallpaper shown on the right.

[0049] It can be understood that a color ladder refers to a graphic that displays color gradient or layering, which usually includes a starting part, a middle transition part, and an ending part, and the color gradually deepens or lightens from the starting part to the ending part. Here, before the color ladder is filled with the main color element, it can exist in a transparent or colorless form, or it can be as Figure 4 As shown, gray, light gray and black are used as placeholders to fill.

[0050] Here, when the main color element is a gradient color, the gradient color can be directly used to fill the color ladder. Specifically, the starting color of the gradient color is mapped to the starting part of the color ladder, the middle transition color of the gradient color is mapped to the middle transition part of the color ladder, and the end color of the gradient color is mapped to the end part of the color ladder.

[0051] When the main color element is a solid color, you can convert this solid color into a gradient color and then fill it on the color ladder. You can also apply this solid color to the initial fill of the color ladder, that is, each part is covered with the same solid color. Then, fine-tune the color properties (brightness, saturation) of each part of the color ladder to create a gradient effect.

[0052] Step 140: During the playback of the audio data, control the cyclic display of the color ladder on the terminal display interface at least according to the time domain peak of the audio data in the time domain.

[0053] In the field of audio processing, PCM (Pulse Code Modulation) format is usually used to generate audio data, which represents the digital representation of audio data in the time domain.

[0054] Here, by detecting the time domain peaks of the audio data in the time domain, during the playback of the audio data, the cyclic display of the color ladder on the terminal display interface is controlled in real time according to the time domain peaks of the audio data of the current frame in the time domain, so as to achieve synchronous interaction with the audio playback.

[0055] In one example, the moving speed of the color ladder can be adjusted according to the number of time domain peaks of the audio data of the current frame in the time domain. For example, the more time domain peaks there are in the current frame, the faster the moving speed of the color ladder is; conversely, the fewer time domain peaks there are in the current frame, the slower the moving speed of the color ladder is.

[0056] Here, when only one color ladder is included in the terminal display interface, the position of the color ladder can be adjusted in real time according to the time domain peak of the audio data of the current frame in the time domain. When the terminal display interface includes multiple color ladders, in addition to adjusting the position of at least one color ladder in real time according to the time domain peak of the audio data of the current frame in the time domain, the arrangement mode, position layout, etc. between the multiple color ladders can also be adjusted according to the time domain peak of the audio data of the current frame in the time domain.

[0057] Continue to refer Figure 4 As shown, three color ladders 401b, 402b and 403b are arranged from top to bottom in the terminal display interface 400b. In the process of playing audio data, the arrangement layout of the three color ladders can be gradually transformed into (402b, 403b, 401b), (403b, 401b, 402b), (401b, 402b, 403b)... in sequence according to the time domain peaks of the audio data in the time domain, thereby forming a circular display effect.

[0058] In the embodiment of the present application, by integrating the main color elements of the music cover and the time domain peaks of the audio data in the time domain, the color filling and the circular display of the color ladder in the terminal display interface are realized, which not only enhances the immersion and personalization of the user experience, but also enhances the emotional expressiveness of the music, and increases interactivity and fun.

[0059] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0060] In some embodiments, extracting the main color element from the music cover includes: Extracting a plurality of colors from the music cover, and selecting at least one main color from the plurality of colors; The brightness value and the saturation value of the at least one primary color in the HSL color space are adjusted to obtain at least one gradient color corresponding to the at least one primary color, and the at least one primary color is used as a primary color element.

[0061] Here, multiple colors can be automatically identified and extracted from the music cover by a color extraction algorithm (such as Palette.Swatch color extraction algorithm). After the multiple colors are extracted, at least one main color can be selected from the multiple colors based on the color area, brightness, saturation or other factors.

[0062] Furthermore, after at least one main color is screened out, the at least one main color may be adjusted to a gradient color to serve as a main color element.

[0063] It should be understood that color in the HSL color space includes three components: hue (H), saturation (S), and lightness (L). Therefore, in this embodiment, the main color can be adjusted to a gradient color by adjusting the lightness value and saturation value of the main color in the HSL color space.

[0064] Specifically, the gradient color corresponding to the main color is divided into three parts: a dark end, an intermediate transition part, and a light end. The dark end can be generated by keeping the hue of the main color unchanged, increasing the saturation and reducing the brightness, and the light end can be generated by keeping the hue of the main color unchanged, reducing the saturation and increasing the brightness.

[0065] After the above adjustments, a gradient color from dark to light can be generated. Gradient color is used as the main color element of the color ladder in the wallpaper to enhance the visual effect and color layering.

[0066] In the embodiment of the present application, by extracting and screening the main colors in the music cover and generating corresponding gradient colors based on the HSL color space, a rich color selection is provided for wallpaper design.

[0067] In some embodiments, the color ladder includes a plurality of color ladders, and different color ladders are filled with different main color elements.

[0068] Here, the wallpaper may include multiple color steps, each color step is an independent color area, and each color area is filled with different main color elements. These main color elements can be similar colors, contrasting colors, or arbitrarily selected colors, but the main color elements in each color step remain consistent.

[0069] It can be understood that the number of main color elements is at least greater than or equal to the number of color steps. For example, if the number of main color elements is greater than the number of color steps, main color elements corresponding to the number of color steps can be randomly selected from multiple main color elements to fill the color steps. In other examples, unfilled main color elements can be randomly selected to replace one or more filled main color elements of the color steps in the cyclic display of the color steps.

[0070] Furthermore, a plurality of the color steps are arranged along the vertical direction of the wallpaper, and the coloring of each of the color steps gradually deepens upward or downward along the vertical direction.

[0071] Here, the coloring of the color ladder may be gradually deepened from top to bottom along the vertical direction, or the coloring of the color ladder may be gradually deepened from bottom to top along the vertical direction.

[0072] refer to Figure 3 As shown, multiple color ladders can be arranged along the vertical direction y of the terminal display interface. The main color elements used in the color ladders along the vertical direction y or the opposite direction of the vertical direction y gradually change in hue, brightness or saturation, so that a visual guiding effect can be formed to guide the user's sight to move along the arrangement direction of the color ladder.

[0073] In some embodiments, controlling the cyclic display of the color ladder on the terminal display interface according to the time domain peak of the audio data in the time domain includes: During the playing of the audio data, determining that the audio data of the current frame reaches a time domain peak in the time domain, and controlling each of the color steps to move a preset distance along a preset moving direction; Among them, when the first color ladder moves beyond the first boundary area of ​​the terminal display interface, the part of the first color ladder that exceeds the first boundary area is controlled to move to the blank area adjacent to the second boundary area, and the blank area adjacent to the second boundary area is the blank area left after the second color ladder moves a preset distance from the second boundary area along a preset moving direction.

[0074] Here, in the process of playing audio data, each time domain peak of the audio data in the time domain is used as a driving point for the movement of the color ladder, that is, when the time domain peak is reached, a movement instruction of the color ladder will be triggered. Each time the movement instruction is triggered, the color ladder will move a preset distance along the preset movement direction.

[0075] It can be understood that in some examples, a movement instruction can be triggered for one color ladder, or a movement instruction can be triggered for multiple color ladders.

[0076] Here, when the preset moving direction is vertically downward along the terminal display interface, the first boundary area may be the lower boundary area, and the second boundary area may be the upper boundary area. When the preset moving direction is vertically upward along the terminal display interface, the first boundary area may be the upper boundary area, and the second boundary area may be the lower boundary area. In addition, the preset moving direction may also be horizontally left or right along the terminal display interface, and the first boundary area and the second boundary area may be the left boundary area or the right boundary area, etc., without limitation.

[0077] like Figure 5 As shown, the terminal display interface includes only one color ladder a. Each time the audio data reaches a time domain peak in the time domain, the color ladder a moves 10 pixel units in the vertical downward direction along the terminal display interface. In the process of playing the audio data, before playing to the current frame (the frame number of the starting frame is 53 and the frame number of the ending frame is 105), the color ladder a is in the upper boundary A area. When the audio data of the current frame reaches the first time domain peak b1 in the time domain, the color ladder a moves from the upper boundary A to the area (A+10), when the audio data of the current frame reaches the second time domain peak b2 in the time domain, the color ladder a moves from the area (A+10) to the area (A+20), and when the audio data of the current frame reaches the third time domain peak b3 in the time domain, the color ladder a moves from the area (A+20) to the area (A+30). In this way, as the audio data continuously reaches the time domain peak in the time domain, the color ladder a continuously moves.

[0078] Furthermore, when the color step a continuously moves vertically downward by 10 pixel units along the terminal display interface, part of the first color step will gradually exceed the upper boundary area of ​​the terminal display interface and will no longer be fully displayed on the terminal display interface. At this time, the part of the first color step that exceeds the lower boundary is moved to the blank area adjacent to the upper boundary area of ​​the terminal display interface.

[0079] In addition, when the terminal display interface includes multiple color steps, each time the audio data reaches a time domain peak in the time domain, a movement instruction can be triggered for the multiple color steps, and the multiple color steps are synchronously moved 10 pixel units in the vertical downward direction along the terminal display interface.

[0080] When the first color ladder continuously moves 10 pixel units vertically downward along the terminal display interface, part of the first color ladder will gradually exceed the first boundary area of ​​the terminal display interface, and will no longer be fully displayed on the terminal display interface. At this time, another edge area corresponding to the first boundary area is the second boundary area. In the second boundary area, the second color ladder originally occupied a part of the space, but when it continuously moved 10 pixel units vertically downward along the terminal display interface, a blank area was left. In this embodiment, in order to maintain the continuity and visual circularity of the interface, the part of the first color ladder that exceeds this part is controlled to move to the blank area adjacent to the second boundary area. In this way, the blank space originally left by the movement of the second color ladder is filled by the corresponding part of the first color ladder, thereby achieving a smooth transition and seamless connection of visual elements on the interface.

[0081] In this embodiment, the time domain peaks of the audio data in the time domain are combined with the cyclic display of the color ladder in the terminal display interface to create an audio-visual synchronization experience. In this way, when the audio is played to a high-energy event (such as a drum beat), the user can not only hear the strong sound impact, but also see the corresponding visual feedback through the dynamic movement of the color ladder.

[0082] In some embodiments, the color ladder includes three, and the controlling the cyclic display of the color ladder on the terminal display interface according to the time domain peak of the audio data in the time domain includes: During the playing of the audio data, determining that the audio data of the current frame reaches a time domain peak in the time domain, and controlling the three color ladders to move a preset distance along a preset moving direction; Among them, when the third color step among the three color steps exceeds the first boundary area of ​​the terminal display interface, the part of the third color step that exceeds the first boundary area is controlled to move to the blank area adjacent to the second boundary area, and the blank area adjacent to the second boundary area is the blank area left after the fourth color step of the three color steps moves a preset distance from the second boundary area along a preset moving direction.

[0083] like Figure 6As shown, the terminal display interface includes three color ladders c, d, and e, and the three color ladders c, d, and e just fill the entire terminal display interface. Each time the time domain peak is reached, the color ladders c, d, and e will move 10 pixel units vertically downward along the terminal display interface. In the process of playing audio data, before playing to the current frame (the frame number of the starting frame is 53 and the frame number of the ending frame is 105), the color ladder c is currently in the first area (i.e., the upper boundary area) of the terminal display interface, the color ladder d is in the second area of ​​the terminal display interface, and the color ladder e is currently in the third area (i.e., the lower boundary area) of the terminal display interface.

[0084] In the process of playing audio data, when the audio data of the current frame reaches the first time domain peak b1 in the time domain, the color ladder moves downward from the first area to 10 pixel units, and the color ladder d moves downward from the second area to 10 pixel units. It should be noted here that when the color ladder e moves downward from the third area by 10 pixel units, part of the color ladder e will gradually exceed the lower boundary area of ​​the terminal display interface, and thus will no longer be fully displayed on the terminal display interface. At this time, another edge area corresponding to the lower boundary area is the upper boundary area. In the upper boundary area, the color ladder c originally occupied a part of the space, but when the color ladder c continuously moved vertically downward by 10 pixel units along the terminal display interface, a blank area was left. Therefore, the part of the color ladder e that exceeds the limit moves to the blank area adjacent to the upper boundary area.

[0085] Next, when the audio data of the current frame reaches the second time domain peak b2 in the time domain, the color steps c, d, and e continue to move downward by 10 pixel units, and the portion of the color step e that exceeds the limit continues to move to the blank area of ​​the upper boundary area. When the audio data of the current frame reaches the second time domain peak b3 in the time domain, the color steps c, d, and e continue to move downward by 10 pixel units. At this time, the color step e just moves completely to the upper boundary area. In this way, as the audio data continues to reach the time domain peak in the time domain, the three color steps c, d, and e continue to move in a downward cycle, forming a cyclic visual effect.

[0086] In some embodiments, it also includes: During the playing of the audio data, determining whether a human voice signal exists in the audio data of the current frame; The maximum frequency domain peak of the human voice signal of the current frame in the frequency domain is determined, and the color depth of the color ladder is controlled according to the amplitude of the maximum frequency domain peak of the current frame.

[0087] It should be understood that the audio data may contain multiple sound components, such as human voice, music, background noise, etc. Here, the human voice signal in the audio data is used to control the dynamic display of the color depth of the color ladder.

[0088] In some examples, before playing each frame of audio data, the next frame of audio data can be separated from the human voice by separation techniques (such as FFT filter, deep neural network DNN or convolutional neural network CNN). For example, the deep neural network (DNN) can be combined with the ratio ideal floating value masking (IRM) technology to construct a complex mapping relationship from mixed signals to time-frequency masking to separate the target human voice from complex background noise.

[0089] If it is determined based on the result of voice separation that there is a voice signal in the audio data of the next frame, the amplitude of the maximum frequency domain peak of the voice signal in the frequency domain in the next frame is further analyzed. Subsequently, during the playback of the current frame of the audio data, based on the preset mapping mechanism, the depth of the color ladder is controlled based on the amplitude of the maximum frequency domain peak of the current frame. For example, if the amplitude of the maximum frequency domain peak of the current frame is higher, indicating that the energy of the voice signal is stronger, the coloring of the color ladder will become darker accordingly; conversely, if the amplitude is lower, the coloring will become lighter.

[0090] In some embodiments, controlling the color depth of the color ladder according to the amplitude of the maximum frequency domain peak of the current frame includes: Determine that the amplitude of the maximum frequency domain peak of the current frame is greater than the amplitude of the maximum frequency domain peak of the previous frame, and control the brightness and saturation of the color ladder in the HSL color space to increase; or,

[0091] It is determined that the amplitude of the maximum frequency domain peak of the current frame is smaller than the amplitude of the maximum frequency domain peak of the previous frame, and the brightness and saturation of the color ladder in the HSL color space are controlled to decrease.

[0092] refer to Figure 7 As shown, from left to right are the frequency spectrograms of data frames A1, A2 and A3, and the amplitude a2 of the maximum frequency domain peak of data frame A2 is greater than the amplitude a1 of the maximum frequency domain peak of data frame A1, and greater than the amplitude a3 of the maximum frequency domain peak of data frame A3.

[0093] During the playback process of audio data, when playing from data frame A1 to data frame A2, since the amplitude a2 of the maximum frequency domain peak of data frame A2 is greater than the amplitude a1 of the maximum frequency domain peak of data frame A1, the adjustment of color deepening is triggered. Specifically, the brightness and saturation of the colors in the color ladder h in the HSL color space are reduced to deepen the overall coloring.

[0094] Continue to refer Figure 7 As shown, when playing from data frame A2 to data frame A1, since the amplitude a2 of the maximum frequency domain peak of data frame A2 is greater than the amplitude a1 of the maximum frequency domain peak of data frame A1, the adjustment of lightening of color is triggered. Specifically, the brightness and saturation of the colors in the color ladder h in the HSL color space are reduced to make the overall coloring lighter.

[0095] In some embodiments, it also includes: Determining that there is no human voice signal in the audio data of the current frame; The maximum time domain peak of the audio data of the current frame in the time domain is determined, and the color depth of the color ladder is controlled according to the amplitude of the maximum time domain peak of the current frame.

[0096] Here, when it can be determined based on the result of voice separation that there is no voice signal in the audio data of the next frame (that is, there is only a pure music signal), the amplitude of the maximum time domain peak of the time domain signal in the next frame can be further analyzed. Subsequently, during the playback of the audio data, based on a preset mapping mechanism, the depth of the color ladder is controlled based on the amplitude of the maximum time domain peak of the current frame. For example, if the amplitude of the maximum time domain peak of the current frame is higher, indicating that the energy of the pure music signal is stronger, the coloring of the color ladder will become darker accordingly; conversely, if the amplitude is lower, the coloring will become lighter.

[0097] In some embodiments, controlling the color depth of the color ladder according to the amplitude of the maximum temporal peak of the current frame includes: Determine that the amplitude of the maximum time-domain peak of the current frame is greater than the amplitude of the maximum time-domain peak of the previous frame, and control the brightness and saturation of the color ladder in the HSL color space to increase; or

[0098] It is determined that the amplitude of the maximum time-domain peak of the current frame is smaller than the amplitude of the maximum time-domain peak of the previous frame, and the brightness and saturation of the color ladder in the HSL color space are controlled to decrease.

[0099] Here, if the amplitude of the maximum temporal peak of the current frame is greater than that of the maximum temporal peak of the previous frame, the adjustment of color deepening is triggered, specifically, the brightness and saturation of the colors in the color ladder in the HSL color space are increased to make the overall coloring darker. Conversely, if the amplitude of the maximum temporal peak of the current frame is smaller than that of the maximum temporal peak of the previous frame, the adjustment of color lightening is triggered, specifically, the brightness and saturation of the colors in the color ladder in the HSL color space are reduced to make the overall coloring lighter.

[0100] In this embodiment, by dynamically adjusting the color depth in the color ladder according to the change in the amplitude of the maximum time domain peak or the amplitude of the maximum frequency domain peak in the audio signal, it is possible to synchronously adjust the color depth in the color ladder when the pitch of the timbre changes, thereby enhancing the immersion and personalization of the user experience, and also improving the emotional expressiveness of the music and increasing interactivity and fun.

[0101] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the display method, which includes: Get the music cover and audio data of the currently playing music; extracting a primary color element from the music cover; Filling the color ladder according to the main color element; and

[0102] During the playing of the audio data, the cyclic display of the color ladder on the terminal display interface is controlled at least according to the time domain peak of the audio data in the time domain.

[0103] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0104] On the other hand, the present application further provides a computer program product, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the display method provided by the above-mentioned method embodiments, the method comprising: Get the music cover and audio data of the currently playing music; extracting a primary color element from the music cover; Filling the color ladder according to the main color element; and

[0105] During the playing of the audio data, the cyclic display of the color ladder on the terminal display interface is controlled at least according to the time domain peak of the audio data in the time domain.

[0106] In another aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the display method provided by the above-mentioned method embodiments is implemented, and the method includes: Get the music cover and audio data of the currently playing music; extracting a primary color element from the music cover; Filling the color ladder according to the main color element; and

[0107] During the playing of the audio data, the cyclic display of the color ladder on the terminal display interface is controlled at least according to the time domain peak of the audio data in the time domain.

[0108] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including multiple instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A display method, comprising: Get the music cover and audio data of the currently playing music; extracting a primary color element from the music cover; Filling the color ladder according to the main color element; as well as During the playing of the audio data, the cyclic display of the color ladder on the terminal display interface is controlled at least according to the time domain peak of the audio data in the time domain.

2. The display method according to claim 1, wherein: Extracting main color elements from the music cover, including: Extracting a plurality of colors from the music cover, and selecting at least one main color from the plurality of colors; The brightness and saturation of the at least one primary color in the HSL color space are adjusted to obtain at least one gradient color corresponding to the at least one primary color, and the at least one gradient color is used as a primary color element.

3. The display method according to claim 1, wherein: The color ladder includes a plurality of color ladders, and different color ladders are filled with different main color elements.

4. The display method according to claim 3, wherein: The plurality of color steps are arranged along the vertical direction of the wallpaper, and the coloring of each color step gradually deepens upward or downward along the vertical direction.

5. The display method according to claim 1, wherein: Controlling the cyclic display of the color ladder on the terminal display interface according to the time domain peak of the audio data in the time domain includes: During the playing of the audio data, determining that the audio data of the current frame reaches a time domain peak in the time domain, and controlling each of the color steps to move a preset distance along a preset moving direction; Among them, when the first color ladder moves beyond the first boundary area of ​​the terminal display interface, the part of the first color ladder that exceeds the first boundary area is controlled to move to the blank area adjacent to the second boundary area, and the blank area adjacent to the second boundary area is the blank area left after the second color ladder moves a preset distance from the second boundary area along a preset moving direction.

6. The display method according to claim 1, wherein: The color ladder includes three, and the cyclic display of the color ladder on the terminal display interface is controlled according to the time domain peak of the audio data in the time domain, including: During the playing of the audio data, determining that the audio data of the current frame reaches a time domain peak in the time domain, and controlling the three color ladders to move a preset distance along a preset moving direction; Among them, when the third color step among the three color steps exceeds the first boundary area of ​​the terminal display interface, the part of the third color step that exceeds the first boundary area is controlled to move to the blank area adjacent to the second boundary area, and the blank area adjacent to the second boundary area is the blank area left after the fourth color step among the three color steps moves a preset distance from the second boundary area along a preset moving direction.

7. The display method according to claim 1, further comprising: During the playing of the audio data, determining whether a human voice signal exists in the audio data of the current frame; The maximum frequency domain peak of the human voice signal of the current frame in the frequency domain is determined, and the color depth of the color ladder is controlled according to the amplitude of the maximum frequency domain peak of the current frame.

8. The display method according to claim 7, wherein: Controlling the color depth of the color ladder according to the amplitude of the maximum frequency domain peak of the current frame includes: Determine that the amplitude of the maximum frequency domain peak of the current frame is greater than the amplitude of the maximum frequency domain peak of the previous frame, and control the brightness and saturation of the color ladder in the HSL color space to increase; or It is determined that the amplitude of the maximum frequency domain peak of the current frame is smaller than the amplitude of the maximum frequency domain peak of the previous frame, and the brightness and saturation of the color ladder in the HSL color space are controlled to decrease.

9. The display method according to claim 1, further comprising: Determining that there is no human voice signal in the audio data of the current frame; The maximum time domain peak of the audio data of the current frame in the time domain is determined, and the color depth of the color ladder is controlled according to the amplitude of the maximum time domain peak of the current frame.

10. The display method according to claim 9, wherein: Controlling the color depth of the color ladder according to the amplitude of the maximum temporal peak of the current frame includes: Determine that the amplitude of the maximum time-domain peak of the current frame is greater than the amplitude of the maximum time-domain peak of the previous frame, and control the brightness and saturation of the color ladder in the HSL color space to increase; or It is determined that the amplitude of the maximum time-domain peak of the current frame is smaller than the amplitude of the maximum time-domain peak of the previous frame, and the brightness and saturation of the color ladder in the HSL color space are controlled to decrease.

11. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the display method according to any one of claims 1 to 10 when executing the program.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the display method according to any one of claims 1 to 10.

Images