Image display method and device, storage medium and program product

Abstract

The invention discloses an image display method and device, a storage medium and a program product, and relates to the technical field of computers. The method comprises the following steps: executing beat extraction on a first audio to obtain beat information of the first audio; according to the beat information of the first audio, determining a motion parameter corresponding to a first image, the first image being an image displayed in the playing process of the first audio; and in the playing process of the first audio, driving the image elements in the first image to move according to the motion parameters corresponding to the first image, and displaying the first image of which the image elements dynamically change. In the playing process of the first audio, the image elements in the first image are driven to dynamically change along with the rhythm change of the first audio, music rhythm and visual elements are combined, the rhythm change of the first audio is displayed in a visual mode, the display content of the first image is enriched, and the user experience is improved. And the display effect of the music playing interface is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to an image display method and device, a storage medium, and a program product. BACKGROUND

[0002] Playing music by using a music player is a common way for people to entertain themselves.

[0003] In related technologies, in the process in which a user selects to play a piece of music, an image related to the piece of music is displayed in a music playing interface of the piece of music, which is usually an album cover image of an album where the music is located or a static display image of a singer.

[0004] However, the image displayed in the above method is monotonous, which affects the display effect of the image. SUMMARY

[0005] Embodiments of the present application provide an image display method, device, storage medium, and program product. The technical solutions provided by the embodiments of the present application are as follows.

[0006] According to an aspect of an embodiment of the present application, an image display method is provided, and the method comprises:

[0007] performing beat extraction on a first audio to obtain beat information of the first audio;

[0008] determining motion parameters corresponding to a first image according to the beat information of the first audio, the first image being an image displayed in a playing process of the first audio;

[0009] driving image elements in the first image to move according to the motion parameters corresponding to the first image in the playing process of the first audio, and displaying the first image in which the image elements dynamically change.

[0010] According to an aspect of an embodiment of the present application, an image display device is provided, and the device comprises:

[0011] a beat extraction module configured to perform beat extraction on a first audio to obtain beat information of the first audio;

[0012] a parameter determination module configured to determine motion parameters corresponding to a first image according to the beat information of the first audio, the first image being an image displayed in a playing process of the first audio;

[0013] an image driving module configured to drive image elements in the first image to move according to the motion parameters corresponding to the first image in the playing process of the first audio, and display the first image in which the image elements dynamically change.

[0014] According to one aspect of the embodiments of this application, a terminal device is provided, the terminal device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described image display method.

[0015] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-described image display method.

[0016] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program, the computer program being loaded and executed by a processor to implement the above-described image display method.

[0017] The technical solution provided in this application can bring the following beneficial effects:

[0018] By determining the motion parameters corresponding to the first image based on the beat information of the first audio, the image elements in the first image can be driven to move according to the motion parameters corresponding to the first image during the playback of the first audio. This allows the image elements in the first image to dynamically change in accordance with the rhythm of the first audio. Compared with the method of directly displaying the first image in related technologies, the technical solution provided in this application displays a first image with dynamically changing image elements during the playback of the first audio. It interprets the beat information of the first audio into a visual driving signal, combines the musical rhythm with visual elements, and displays the rhythm changes of the first audio in a visual way. This enriches the display content of the first image, enhances the display interest of the first image, and improves the display effect of the music playback interface. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of a computer system provided in one embodiment of this application;

[0020] Figure 2 This is a flowchart of an image display method provided in one embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the system architecture of an image display method provided in one embodiment of this application;

[0022] Figure 4 This is a block diagram of an image display device provided in one embodiment of this application;

[0023] Figure 5 This is a structural block diagram of a terminal device provided in one embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0025] Please refer to Figure 1 This illustration shows a schematic diagram of a computer system provided in one embodiment of this application. The computer system may include: a terminal device 10 and a server 20.

[0026] There may be one or more terminal devices 10. Terminal devices 10 may be electronic devices such as mobile phones, tablets, laptops, desktop computers, game consoles, e-book readers, multimedia playback devices, wearable devices, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc.

[0027] The terminal device 10 may have a music player client installed. Optionally, the music player may be an application that needs to be downloaded and installed, or it may be an application that can be used instantly. This application does not limit this.

[0028] Server 20 provides background services for the music playback program installed and running on terminal device 10. For example, server 20 can be the background server of the aforementioned music playback program. Server 20 can be a standalone physical server, a server cluster consisting of multiple servers, or a cloud computing service center. Optionally, server 20 can simultaneously provide background services for music playback programs on multiple terminal devices 10. Terminal devices 10 and server 20 can communicate with each other via a network.

[0029] In this embodiment, after the user selects to play the first audio, a first image is displayed in the music playback interface of the music player program. Beat extraction is performed on the first audio to obtain its beat information. Based on the beat information of the first audio, motion parameters corresponding to the first image are determined. During the playback of the first audio, the image elements in the first image are driven to move according to the motion parameters corresponding to the first image, and the dynamically changing first image is displayed in the music playback interface.

[0030] Please refer to Figure 2 This document illustrates a flowchart of an image display method according to an embodiment of this application. The execution entity for each step of this method can be a terminal device. The method may include at least one of the following steps 210 to 230:

[0031] Step 210: Perform beat extraction on the first audio to obtain the beat information of the first audio.

[0032] The first audio file can be of any type, and this application makes no limitation on it. For example, the first audio file includes, but is not limited to, song audio, instrumental music audio, opera audio, audiobook audio, and other audio types. The first audio file can be locally stored audio or audio stored in network data, meaning that after the user selects to play the first audio file, they can directly obtain the first audio file from their local device for playback, or they can download the playback resource of the first audio file from the server for playback.

[0033] The first audio is segmented into frames, resulting in multiple audio frames. The beat information of the first audio includes the beat information corresponding to each of the multiple audio frames. Beat detection is performed on each audio frame to identify the position of the beat point in each audio frame, thus obtaining the number of beats contained in each audio frame. Based on the number of beats contained in the audio frame and the corresponding audio duration, the beat information of the audio frame is obtained. The beat information of the audio frame is used to indicate the number of beats contained in a unit time period corresponding to the audio frame.

[0034] When a user selects the first audio file to play in a music player, the music playback interface for that first audio file is displayed on the client's screen. It's important to note that the step of extracting the beat information of the first audio file can be performed after the user selects it, meaning the beat information is extracted in real-time, capturing the beat information of the current audio frame during playback. Alternatively, the step can be performed pre-emptively, in which case the beat information is obtained directly after the user selects it.

[0035] A first image is displayed in the music playback interface of the first audio. Optionally, the first image can be the cover image of the first audio, the background image of the music playback interface of the first audio, or a decorative image in the music playback interface of the first audio, etc.

[0036] Optionally, the first image can be a pre-configured image that matches the first audio, or it can be a user-uploaded image. The first image is loaded after the user selects to play the first audio.

[0037] In some embodiments, the process of determining the first image is as follows: obtaining the audio features of the first audio based on the audio information of the first audio; calculating the similarity between the audio features of the first audio and the image features corresponding to at least one image in the image library to obtain at least one similarity; and determining the image corresponding to the maximum value of the at least one similarity as the first image.

[0038] The audio information of the first audio file includes, but is not limited to, the singer's name, lyrics, audio style, and rhythm. Feature extraction is performed on the audio information of the first audio file to obtain its audio features. Similarly, feature extraction is performed on at least one image from the image database to obtain its corresponding image features. Similarity algorithms such as cosine similarity and Euclidean distance can be used to calculate the similarity between the audio features of the first audio file and the image features of the image, thereby obtaining the first image.

[0039] Optionally, the image library can be an image library for all audio files in the music player program, meaning the image library includes images of any type, and an image corresponding to any audio file can be matched from the image library. Optionally, the image library can also be an image library specific to the first audio file, meaning the image library includes images related to the first audio file, and only the first image corresponding to the first audio file can be matched from the image library. For example, the image library may include at least one image of the singer corresponding to the first audio file, at least one album image corresponding to the album to which the first audio file is located, at least one image corresponding to the audio style of the first audio file, and so on. For example, if the first audio file is a rock-style audio file, then the first image corresponding to the first audio file may be an image of a singer with a unique appearance.

[0040] By obtaining the image that best matches the first audio from the image library, compared to images with a fixed configuration, the auditory experience of the first audio can be enhanced during playback, increasing the user's sense of immersion in the first audio.

[0041] Step 220: Determine the motion parameters corresponding to the first image based on the beat information of the first audio. The first image is the image displayed during the playback of the first audio.

[0042] The motion parameters corresponding to the first image refer to the motion parameters corresponding to the image elements in the first image. They are used to indicate the motion trajectory of the image elements in the first image during the playback of the first audio. They can control the image elements in the first image to move according to the motion parameters corresponding to the first image.

[0043] The image elements in the first image include at least one of the following elements: human figures, animal figures, plant figures, landscape figures, still life figures, etc. For example, if the first image is a human figure image of the singer in the first audio, then the image elements in the first image include human figures of the singer in the first audio and environmental elements in which the singer is located. Environmental elements include, but are not limited to, animal figures, plant figures, landscape figures, still life figures, etc.

[0044] The motion parameters corresponding to the image elements in the first image include the motion parameters corresponding to at least one image element in the first image. Based on the beat information of the first audio and at least one image element in the first image, the motion parameters corresponding to at least one image element in the first image are obtained. The motion parameters corresponding to each image element include at least one of the following: the motion amplitude of the image element, the motion speed of the image element, the motion frequency of the image element, etc. The specific motion parameters corresponding to each image element are related to the element type of the image element; for details, please refer to the following embodiments.

[0045] The motion parameters corresponding to the first image are matched with the beat information of the first audio. The motion rhythm of the image elements in the first image changes with the beat information of the first audio. Generally, the faster the beat information of the first audio, the faster the motion rhythm of the image elements in the first image. For example, the motion amplitude of the image elements in the first image may be larger, or the motion speed of the image elements in the first image may be faster, or the motion frequency of the image elements in the first image may be faster. The motion parameters corresponding to at least one image element in the first image are matched with the beat information of the first audio. The motion rhythm of different image elements following the change in the beat information of the first audio may be the same or different, as can be seen in the following embodiments.

[0046] Since the beat information of the first audio includes the beat information corresponding to multiple audio frames in the first audio, the motion parameters corresponding to each audio frame can be determined based on the beat information corresponding to each audio frame. The motion parameters corresponding to each audio frame include the motion parameters of at least one image element in the first image within the playback time period corresponding to the audio frame. When different audio frames have different beat information, different audio frames have different motion parameters. That is, when the first audio is played to different time periods, the motion parameters corresponding to the current audio frame are determined based on the beat information of the audio frame at the current moment.

[0047] The specific process for determining the motion parameters corresponding to the first image can be found in the following embodiments, which will not be described here.

[0048] Step 230: During the playback of the first audio, the image elements in the first image are driven to move according to the motion parameters corresponding to the first image, and the first image with dynamically changing image elements is displayed.

[0049] During the playback of the first audio, the image elements in the first image are driven to move according to the motion parameters corresponding to the current audio frame, so that the image elements in the first image change with the rhythm information of the first audio, causing the image elements in the first image to undergo positional or morphological changes, resulting in a dynamically changing first image. Thus, during the playback period corresponding to the current audio frame, the dynamically changing first image of the image elements corresponding to the current audio frame is displayed, and during the playback period corresponding to the next audio frame, the dynamically changing first image of the image elements corresponding to the next audio frame is displayed.

[0050] The specific process for determining the first image with dynamically changing image elements can be found in the following embodiments, and will not be described here.

[0051] The technical solution provided in this application determines the motion parameters corresponding to the first image based on the beat information of the first audio. This allows the image elements in the first image to move according to the motion parameters corresponding to the first image during the playback of the first audio. The image elements in the first image dynamically change with the rhythm of the first audio. Compared with the method of directly displaying the first image in related technologies, the technical solution provided in this application displays a first image with dynamically changing image elements during the playback of the first audio. It interprets the beat information of the first audio into a visual driving signal, combining musical rhythm with visual elements to visually display the rhythm changes of the first audio. This enriches the display content of the first image, enhances the display interest of the first image, and improves the display effect of the music playback interface.

[0052] In some embodiments, step 220 includes at least one of sub-steps 221 to 223.

[0053] Sub-step 221: Perform grid subdivision on the image elements in the first image to obtain multiple grid points corresponding to the image elements.

[0054] Optionally, the image elements in the first image can be triangularly meshed to obtain multiple mesh points corresponding to the image elements, where each mesh point refers to a vertex of the triangular mesh. Alternatively, the image elements in the first image can be squarely meshed to obtain multiple mesh points corresponding to the image elements, where each mesh point refers to a vertex of the square mesh.

[0055] In some embodiments, the first image is meshed to obtain multiple mesh points corresponding to the first image. Based on the display position of the image element in the first image, the mesh points located outside the image element are removed from the multiple mesh points corresponding to the first image to obtain the mesh points corresponding to the image element.

[0056] Sub-step 222: For each audio frame among multiple audio frames, determine the motion parameters corresponding to the audio frame based on the beat information corresponding to the audio frame. The motion parameters corresponding to the audio frame include the motion parameters of multiple grid points within the playback period corresponding to the audio frame.

[0057] Motion parameters are bound to multiple grid points corresponding to image elements. Based on the beat information corresponding to the audio frame, the motion parameters corresponding to each grid point are obtained. The motion parameters corresponding to each grid point refer to the motion parameters of each grid point within the playback period corresponding to the audio frame. The motion parameters corresponding to the audio frame include the motion parameters corresponding to multiple grid points.

[0058] Each grid point is bound to at least one motion parameter, and the parameter types of the motion parameters bound to each grid point can be the same or different. The motion parameters of each grid point within the playback period corresponding to different audio frames are determined by the beat information corresponding to the audio frames. When the beat information corresponding to the audio frames is different, the motion parameters of the grid points within the playback period corresponding to different audio frames are usually different, but there may also be cases where they are the same.

[0059] Sub-step 223: Based on the motion parameters corresponding to the multiple audio frames, obtain the motion parameters corresponding to the first image.

[0060] The motion parameters corresponding to the first image include the motion parameters corresponding to multiple audio frames, specifically the motion parameters of multiple grid points within the playback time periods corresponding to the multiple audio frames.

[0061] By dividing the image elements in the first image into grids, multiple grid points corresponding to the image elements are obtained. Motion parameters can be bound to the grid points, and by driving multiple grid points to move according to the motion parameters corresponding to each grid point, dynamic changes of the image elements in the first image are realized, improving the operability and feasibility of dynamic changes of image elements.

[0062] In some embodiments, sub-step 222 includes at least one of sub-steps 2221 to 2223.

[0063] Sub-step 2221: The first image is layered to obtain at least one image layer corresponding to the first image.

[0064] Based on the layering granularity of the first image, at least one image element corresponding to each image layer is obtained. The image element corresponding to each image layer refers to the image element displayed in that image layer, and each image layer includes at least one image element from the first image. The smaller the layering granularity of the first image, the fewer image elements are included in each image layer.

[0065] For example, if the first image is a portrait of the singer in the first audio recording, the first image is layered according to a first layering granularity. The resulting first image corresponds to at least one image layer, including a person layer and a background layer. The image elements corresponding to the person layer include all elements of the singer, including but not limited to head elements, limb elements, and torso elements. The image elements corresponding to the background layer include all elements of the singer's environment, including but not limited to scene elements, animal elements, plant elements, and landscape elements.

[0066] For example, the first image is layered according to a second layering granularity, resulting in at least one image layer corresponding to the first image, including image layers such as a person's head layer, a person's limbs layer, a person's torso layer, a scene layer, an animal layer, a plant layer, and a landscape layer. The second layering granularity is smaller than the first layering granularity. Layering the first image according to the second layering granularity involves further layering at least one image layer obtained from the first layering granularity. For example, the person's head layer, person's limbs layer, and person's torso layer are image layers obtained by layering the person layer, while the scene layer, animal layer, plant layer, and landscape layer are image layers obtained by layering the background layer.

[0067] For example, the first image is layered according to a third layering granularity, resulting in at least one image layer corresponding to the first image, including image layers such as a person's hair layer, a person's eyes layer, a person's mouth layer, and a person's ears layer. The third layering granularity is smaller than the second layering granularity. Layering the first image according to the third layering granularity is a further layering of at least one image layer obtained from the second layering granularity. For example, the person's hair layer, person's eyes layer, person's mouth layer, and person's ears layer are image layers obtained by layering the person's head layer.

[0068] Sub-step 2222: For each image layer in at least one image layer, obtain the motion parameters corresponding to the image layer based on the beat information corresponding to the audio frame. The motion parameters corresponding to the image layer include the motion parameters of at least one grid point on the image layer during the playback period corresponding to the audio frame.

[0069] Based on multiple grid points corresponding to image elements in the first image and image elements in the image layer, at least one grid point on the image layer is obtained. Based on the beat information corresponding to the audio frame, motion parameters corresponding to each of the at least one grid point on the image layer are obtained. The motion parameters corresponding to each grid point on the image layer refer to the motion parameters of each grid point on the image layer within the playback period corresponding to the audio frame. The motion parameters corresponding to the image layer include the motion parameters corresponding to each of the at least one grid point on the image layer.

[0070] The parameter types of motion parameters corresponding to an image layer are related to the element types of the image elements in that image layer. At least one motion parameter may be of the same type across different image layers, or all motion parameters across different image layers may have different parameter types.

[0071] In some embodiments, when the image layer includes a person layer, the motion parameters corresponding to the person layer include at least one of the following: local scaling parameters, local translation parameters, local rotation parameters, and transparency flickering parameters.

[0072] Local scaling parameters indicate the contraction magnitude and speed of image elements in the person layer, or the expansion magnitude and speed of image elements in the person layer. Local translation parameters indicate the displacement distance and direction of image elements in the person layer. Local rotation parameters indicate the rotation direction, rotation angle, and rotation speed of image elements in the person layer. Transparency flicker parameters indicate the flicker duration, flicker frequency, and flicker amplitude of image elements in the person layer. The flicker duration refers to the duration of each flicker; a longer flicker duration results in a slower and smoother flicker effect. The flicker frequency controls the speed of the flicker to achieve different visual flicker effects. The flicker amplitude indicates the intensity or magnitude of the flicker; a larger flicker amplitude results in a more pronounced flicker effect.

[0073] If the person layer is further subdivided, the motion parameters corresponding to at least one image layer are a subset of the motion parameters extracted from the motion parameters corresponding to the person layer. For example, if the person layer includes a head layer, the motion parameters corresponding to the head layer include at least one of the following: head scaling parameters, head translation parameters, head rotation parameters, and head transparency flickering parameters. If the head layer is further subdivided to obtain an eye layer, the motion parameters corresponding to the eye layer include at least one of the following: eye scaling parameters, eye rotation parameters, and eye transparency flickering parameters.

[0074] In some embodiments, when the image layer includes a background layer, the motion parameters corresponding to the background layer include at least one of the following: global scaling parameters, global translation parameters, blur gradient parameters, hue cycle frequency, and hue cycle order.

[0075] Global scaling parameters indicate the shrinkage magnitude and speed of image elements in the background layer, or the expansion magnitude and speed of image elements in the background layer. Global translation parameters indicate the displacement distance and direction of image elements in the background layer. Blur gradient parameters indicate the blur gradient direction, blur gradient speed, and blur gradient magnitude of image elements in the background layer. The blur gradient direction can be from left to right, from right to left, from top to bottom, or from bottom to top; the blur gradient speed controls how quickly the blur of the image elements changes; a slower blur gradient speed results in a more pronounced blur gradient effect; the blur gradient magnitude indicates whether the blur gradually increases or decreases, and the magnitude of the increase or decrease in blur. Hue cycle frequency indicates the frequency of color changes for each color in the background layer as it cycles through various colors on the color wheel, starting from a certain color and returning to the initial color, with the frequency of each color change gradually increasing. The hue cycle order is used to indicate the order in which colors change on the color wheel. For example, the hue cycle order can start with red, then proceed through orange, yellow, green, and blue, and finally return to red.

[0076] If the background layer is further divided into layers, the motion parameters corresponding to at least one background layer are a subset of the motion parameters extracted from the motion parameters corresponding to the background layer. For example, if the background layer includes an ocean scene layer, the motion parameters corresponding to the ocean scene layer include at least one of the following: the scaling parameter of the ocean scene, the translation parameter of the ocean scene, and the blur gradient parameter of the ocean scene.

[0077] By configuring different motion parameters for different image layers, the motion parameters corresponding to the image elements in each image layer can be matched with the element type of the image elements. This allows the dynamic changes of the image elements to conform to the element characteristics of the image elements, improving the naturalness of the changes of the image elements in the first image and thus enhancing the dynamic change effect of the first image.

[0078] Sub-step 2223: Obtain the motion parameters corresponding to the audio frame based on the motion parameters corresponding to at least one image layer.

[0079] The motion parameters corresponding to the audio frame include motion parameters corresponding to at least one image layer, specifically including the motion parameters of at least one grid point configured on at least one image layer during the playback period corresponding to the audio frame.

[0080] By dividing the first image into layers, grid points on each image layer can be obtained, allowing at least one grid point on each image layer to be driven to move according to the motion parameters corresponding to each image layer. This refines the motion parameters of the grid points on each image layer, avoids confusion with the motion of image elements in other image layers, and helps improve the motion accuracy of image elements in the first image, thereby enhancing the dynamic change effect of the first image.

[0081] The specific determination process for the motion parameters corresponding to the image layer in sub-step 2222 above is as follows:

[0082] In some embodiments, the motion parameters corresponding to the image layer are obtained based on the beat information corresponding to the audio frame and the mapping relationship between the beat information and motion parameters.

[0083] Optionally, the mapping relationship between beat information and motion parameters can be obtained based on a fixed-configuration mapping table between beat information and motion parameters, or it can be obtained based on a mapping function between beat information and motion parameters.

[0084] The motion parameters corresponding to the image layer include the motion parameters corresponding to at least one grid point on the image layer. Each grid point corresponds to at least one motion parameter, meaning that each image layer corresponds to at least one motion parameter. Therefore, based on the beat information corresponding to the audio frame and the mapping relationship between the beat information and the first motion parameter, the first motion parameter corresponding to the image layer is obtained. The first motion parameter is any one of the at least one motion parameter corresponding to each grid point.

[0085] The mapping relationship between beat information and different motion parameters can be obtained based on the same mapping method or different mapping methods. For example, if the mapping relationship between beat information and the first motion parameter is obtained based on a fixed-configuration mapping table between beat information and the first motion parameter, then the mapping relationship between beat information and the second motion parameter can be obtained based on a fixed-configuration mapping table between beat information and the second motion parameter, or it can be obtained based on a mapping function between beat information and the second motion parameter.

[0086] By mapping the beat information to motion parameters, the motion parameters corresponding to the image layer are obtained, making the acquisition process of the motion parameters of the image layer orderly and accurate, and ensuring the operability and feasibility of the acquisition process.

[0087] In some embodiments, a mapping function corresponding to the first motion parameter on the image layer is obtained; the first motion parameter corresponding to the image layer is obtained based on the beat information corresponding to the audio frame and the mapping function corresponding to the first motion parameter.

[0088] For example, if the first motion parameter is the scaling parameter of a person's eyes, then the mapping function corresponding to the first motion parameter can be: the first motion parameter corresponding to the image layer = 0.1 × the beat information corresponding to the audio frame. Then, based on the beat information corresponding to the audio frame and the mapping function corresponding to the first motion parameter, the first motion parameter corresponding to at least one grid point on the image layer can be obtained.

[0089] For example, if the first motion parameter is the translation parameter of a person's head, then the mapping function corresponding to the first motion parameter can be a random perturbation function, generating a smooth random direction based on a noise algorithm (such as Perlin noise). The control logic of this random perturbation function can be: input grid point coordinates + timestamp, output pseudo-random direction vector. Therefore, the implementation of the random perturbation function does not need to refer to the beat information corresponding to the audio frame. An example formula can be: direction = (PerlinNoise(x, y, t), PerlinNoise(x+100, y, t)), thereby obtaining the first motion parameter corresponding to at least one grid point on the image layer according to the mapping function corresponding to the first motion parameter.

[0090] By determining the first motion parameter corresponding to the image layer based on the mapping function corresponding to the first motion parameter, the accuracy of the first motion parameter can be improved, thereby driving at least one grid point on the image layer to move according to the first motion parameter. This can improve the naturalness of the motion of image elements on the image layer, avoid the mechanical feeling caused by inaccurate first motion parameters, enhance the motion details of image elements on the image layer, and help improve the dynamic change effect of the first image.

[0091] In some embodiments, when the beat information corresponding to the audio frame is in the i-th beat interval, the i-th mapping function corresponding to the first motion parameter is obtained from at least one mapping function corresponding to the first motion parameter. Different mapping functions correspond to different beat intervals, and i is a positive integer. Based on the beat information corresponding to the audio frame and the i-th mapping function corresponding to the first motion parameter, the first motion parameter corresponding to the image layer is obtained.

[0092] There are N pre-configured beat intervals, where N is a positive integer. Each beat interval includes at least one beat information, meaning that at least one beat interval may contain only one beat information. For example, the N beat intervals can be, in sequence, the first beat interval with beat information less than 160, the second beat interval with beat information equal to 160, and the third beat interval with beat information greater than 160.

[0093] The N beat intervals correspond to the N mapping functions corresponding to the first motion parameter. When the beat information corresponding to the audio frame is in the i-th beat interval, the i-th mapping function corresponding to the i-th beat interval is obtained from the N mapping functions corresponding to the first motion parameter. For example, the first mapping function corresponding to the first motion parameter can be: the first motion parameter corresponding to the image layer = 0.1 × the beat information corresponding to the audio frame; the second mapping function corresponding to the first motion parameter can be: the first motion parameter corresponding to the image layer = 0.2 × the beat information corresponding to the audio frame; and the third mapping function corresponding to the first motion parameter can be: the first motion parameter corresponding to the image layer = 0.3 × the beat information corresponding to the audio frame.

[0094] In some embodiments, when the beat information corresponding to the audio frame is in the i-th beat interval, the first motion parameter corresponding to the i-th beat interval is obtained from the mapping table between the beat information and motion parameters according to the beat information corresponding to the audio frame, which is the first motion parameter corresponding to the image layer.

[0095] For example, the first motion parameter is the amplitude of the shaking of the character's hair. If the beat information corresponding to the audio frame is less than 160, the amplitude of the shaking of the character's hair is 30%. If the beat information corresponding to the audio frame is equal to 160, the amplitude of the shaking of the character's hair is 50%. If the beat information corresponding to the audio frame is greater than 160, the amplitude of the shaking of the character's hair is 80%.

[0096] For example, the first motion parameter is the contraction speed of the human eye. If the beat information corresponding to the audio frame is less than 160, the contraction speed of the human eye is 3 seconds per cycle. If the beat information corresponding to the audio frame is equal to 160, the contraction speed of the human eye is 2 seconds per cycle. If the beat information corresponding to the audio frame is greater than 160, the contraction speed of the human eye is 1 second per cycle.

[0097] By setting multiple beat intervals for the beat information corresponding to the audio frames, and setting different beat intervals to correspond to different mapping functions, the dynamic changes of image elements on the image layer following the beat information of the audio frames are made more detailed and accurate. This can more accurately reflect the influence of different beats on the movement of image elements, improve the matching degree between music rhythm and visual elements, and help enhance the user's sense of immersion when listening to music.

[0098] In some embodiments, step 230 includes at least one of sub-steps 231 to 233.

[0099] Sub-step 231: During the playback of the first audio, the motion trajectory corresponding to the image element in the first image is obtained according to the motion parameters corresponding to the first image.

[0100] The first image is divided into at least one image layer, and the first image contains multiple grid points, with each image layer containing at least one grid point. During the playback of the first audio, the motion trajectories corresponding to the image elements in each of the at least one image layer of the first image are extracted, thereby driving the image elements in each image layer to move according to the motion trajectories corresponding to the image elements in the image layer, thus achieving accurate driving of each image element in the first image.

[0101] For each image layer in at least one image layer, during the playback of the first audio, motion parameters corresponding to the audio frame are obtained based on the beat information corresponding to the audio frame. This means obtaining the motion parameters of at least one grid point on the image layer within the playback time period corresponding to the audio frame. Based on the motion parameters corresponding to the audio frame, the motion trajectory corresponding to the audio frame is obtained. The motion trajectory includes the motion trajectories of at least one grid point on the image layer within the playback time period corresponding to the audio frame. The motion trajectory indicates the movement pattern of the grid points.

[0102] Sub-step 232: Perform smoothing processing on the motion trajectory corresponding to the image element in the first image to obtain the smoothed motion trajectory.

[0103] For example, a spring-damped system is used to smooth the motion trajectory corresponding to the audio frame, resulting in a smoothed motion trajectory. During the smoothing process, the smoothed motion trajectories of at least one audio frame preceding the current audio frame need to be considered, allowing the smoothed motion trajectory of the current audio frame to be integrated with the previous motion trajectories.

[0104] Sub-step 233: Drive the image elements in the first image to move according to the smoothed motion trajectory, and display the first image with dynamically changing image elements.

[0105] During the playback of the first audio, based on the motion parameters of the audio frames corresponding to at least one image layer, at least one grid point in each image layer is driven to move according to the motion parameters of the audio frames corresponding to that image layer, thus displaying a first image with dynamically changing image elements. The first image with dynamically changing image elements is a first image generated by the image elements in at least one image layer moving according to their respective motion trajectories.

[0106] By performing smoothing processing on the motion trajectories corresponding to the image elements in the first image, the dynamic changes of the image elements in the first image can be transitioned more naturally and smoothly during the playback of the first audio, improving the smoothness and naturalness of the dynamic changes of the image elements and enhancing the image expressiveness of the first image.

[0107] In some embodiments, the method further includes step 240, which includes at least one of sub-steps 241 to 242.

[0108] Sub-step 241: Based on the beat information of the first audio, obtain the effect parameters corresponding to the target effect.

[0109] Target effects are effects added to the first image. Target effects include, but are not limited to, particle effects, light streamers, light bands, and gradient transparency effects. The parameter types of the effect parameters corresponding to the target effect are related to the effect element. For example, if the target effect is a particle effect, the corresponding effect parameters are related to the particle element; if the target effect is a light streamer effect, the corresponding effect parameters are related to the light streamer element.

[0110] The special effect parameters corresponding to the target special effect include at least one of the following: the movement speed of the special effect element, the movement direction of the special effect element, the transparency and blinking parameters of the special effect element, the blur gradient parameters of the special effect element, the hue cycle frequency of the special effect element, and the hue cycle order of the special effect element.

[0111] For example, the effect parameters corresponding to the target effect are obtained based on the beat information corresponding to the audio frame.

[0112] In some embodiments, the effect parameters corresponding to the target effect are obtained based on the beat information corresponding to the audio frame and the mapping relationship between the beat information and the effect parameters. Optionally, the mapping relationship between the beat information and the effect parameters can be obtained based on a fixed-configuration mapping table between beat information and effect parameters, or it can be obtained based on a mapping function between beat information and effect parameters.

[0113] The mapping relationship between beat information and different special effect parameters can be obtained based on the same mapping method or different mapping methods. For example, if the mapping relationship between beat information and the first special effect parameter is obtained based on a fixed-configuration mapping table between beat information and the first special effect parameter, then the mapping relationship between beat information and the second special effect parameter can be obtained based on a fixed-configuration mapping table between beat information and the second special effect parameter, or it can be obtained based on a mapping function between beat information and the second special effect parameter.

[0114] In some embodiments, the mapping function corresponding to the first effect parameter is obtained, and the effect parameter corresponding to the target effect is obtained based on the beat information corresponding to the audio frame and the mapping function corresponding to the first effect parameter.

[0115] For example, if the target effect is a particle effect and the first effect parameter is the movement speed of the particle element, then the mapping function corresponding to the first effect parameter can be: v = (0, -BPM × 0.5). Based on the beat information corresponding to the audio frame and the mapping function corresponding to the first effect parameter, the horizontal and vertical movement speeds of the particle element in the first image can be obtained.

[0116] For example, if the target effect is a transparency gradient effect, and the first effect parameter is transparency, then the mapping function corresponding to the first effect parameter can be: Alpha = 0.5 + 0.5 × sin(time × π / BPM). If the second effect element is the scaling ratio, then the mapping function corresponding to the second effect element can be: Scale = 1.0 + 0.05 × cos(time × 2π / slow cycle).

[0117] In some embodiments, when the beat information corresponding to the audio frame is within the j-th beat interval, the j-th mapping function corresponding to the effect parameter is obtained from at least one mapping function corresponding to the effect parameter. Different mapping functions correspond to different beat intervals, and j is a positive integer. Based on the beat information corresponding to the audio frame and the j-th mapping function corresponding to the effect parameter, the effect parameter corresponding to the target effect is obtained.

[0118] M beat intervals are pre-configured, where M is a positive integer. Each beat interval includes at least one beat information, meaning that at least one beat interval may contain only one beat information. For example, the M beat intervals can be, in sequence, the first beat interval with beat information less than 160, the second beat interval with beat information equal to 160, and the third beat interval with beat information greater than 160.

[0119] The M beat intervals correspond to the M mapping functions corresponding to the effect parameters. When the beat information corresponding to the audio frame is in the j-th beat interval, the j-th mapping function corresponding to the j-th beat interval is obtained from the M mapping functions corresponding to the effect parameters. For example, if the target effect is a particle effect and the effect parameter is the movement speed of the particle element, then the first mapping function corresponding to the effect parameter can be: v = (0, -BPM × 0.4), the second mapping function corresponding to the first motion parameter can be: v = (0, -BPM × 0.5), and the third mapping function corresponding to the first motion parameter can be: v = (0, -BPM × 0.6).

[0120] In some embodiments, when the beat information corresponding to the audio frame is in the j-th beat interval, the effect parameters corresponding to the j-th beat interval are obtained from the mapping table between the beat information and the effect parameters according to the beat information corresponding to the audio frame. These are the effect parameters corresponding to the target effect.

[0121] For example, if the target effect is a particle effect and the effect parameter is the movement speed of the particle element, then if the beat information corresponding to the audio frame is less than 160, the movement speed of the particle element is v = (0, -40); if the beat information corresponding to the audio frame is equal to 160, the movement speed of the particle element is v = (0, -50); and if the beat information corresponding to the audio frame is greater than 160, the movement speed of the particle element is v = (0, -60).

[0122] By setting multiple beat intervals for the beat information corresponding to the audio frames, and setting different beat intervals to correspond to different mapping functions, the dynamic changes of the special effects elements corresponding to the target special effects are made more detailed and precise. This can more accurately reflect the influence of different beats on the movement of the target special effects, improve the matching degree between music rhythm and visual elements, and help enhance the user's sense of immersion when listening to music.

[0123] Sub-step 242: During the playback of the first audio, the target effect is superimposed on the first image according to the effect parameters corresponding to the target effect.

[0124] For example, an effects layer is overlaid on the first image, and during the playback of the first audio, the target effects are rendered and generated on the effects layer according to the effects parameters corresponding to the target effects.

[0125] By overlaying target effects onto the first image, the presentation of the first image is enriched. Furthermore, during the playback of the first audio, the target effects dynamically change in accordance with the rhythm of the first audio, further enriching the visual content of the first image and enhancing its dynamic change effect.

[0126] In some embodiments, the target effect includes a first effect and a second effect, where the rhythm of the first effect is greater than that of the second effect. Then, if the beat information corresponding to the audio frame is greater than or equal to a first threshold, the first effect is superimposed on the first image; if the beat information corresponding to the audio frame is less than the first threshold, the second effect is superimposed on the first image.

[0127] For example, the first effect can be a particle effect, and the second effect can be a transparency gradient effect. The rhythm of change of particle elements in the particle effect is greater than the rhythm of change of transparency in the transparency gradient effect, and the effect of the first effect is stronger than that of the second effect. When the beat information corresponding to the audio frame is greater than or equal to a first threshold, the particle effect is superimposed on the first image; when the beat information corresponding to the audio frame is less than the first threshold, the transparency gradient effect is superimposed on the first image.

[0128] This application does not limit the specific value of the first threshold. For example, if the first threshold is 100, then when the beat information corresponding to the audio frame is greater than or equal to 100, particle effects are superimposed on the first image; when the beat information corresponding to the audio frame is less than 100, a transparency gradient effect is superimposed on the first image.

[0129] By determining the target effect that matches the beat information corresponding to the audio frame, the target effect superimposed on the first image can better match the rhythm changes of the first audio. The first effect is selected when the rhythm changes of the first audio are stronger, and the second effect is selected when the rhythm changes of the first audio are more gentle. This makes the user more immersive while listening to the audio and enhances the appeal and display effect of the first image.

[0130] Figure 3 A schematic diagram of the system architecture of the image display method is shown. The system architecture of the image display method includes an image processing module, an audio processing module, and a dynamic rendering module. The image processing module is used to acquire a first image corresponding to a first audio, the audio processing module is used to extract the beat information of the first audio, the dynamic rendering module is used to determine the motion parameters corresponding to the first image, drive the image elements in the first image to move according to the motion parameters corresponding to the first image, and overlay target effects on the first image.

[0131] The image processing module includes an image recommendation and matching unit, an image caching and processing unit, an image customization unit, and an image layering and processing unit. The image recommendation and matching unit retrieves the first image with the highest matching degree from the image library based on the audio information of the first audio. The image caching and processing unit preloads / caches the first image, allowing for the downloading or reading of local image resources to reduce latency caused by real-time rendering. The image customization unit allows users to upload custom images as the first image to match the first audio. The image layering and processing unit layers the first image; for example, if the first image is a singer's portrait, a segmentation model can be used to identify the singer's outline and key features in the first image, resulting in a layered mask, where each image element can be stored as an independent texture.

[0132] The audio processing module includes an audio input interface unit, a beat extractor, and a beat detector. The audio input interface unit is used to acquire the first audio selected by the user in the music playback program, the beat extractor is used to extract the beat information of the first audio, and the beat detector is used to detect the position of the beat point in the first audio.

[0133] The dynamic rendering module includes a parameter mapper, a mesh deformation controller, and an effects overlay unit. The parameter mapper maps the beat information of the first audio track to the motion parameters corresponding to the first image. The mesh deformation controller achieves deformation by driving the movement of mesh points. Specifically, each image layer is delaunay triangulated, and when a beat point is triggered, the vertices move in a pre-defined direction, such as a vertical bounce. A spring-damping system smooths the movement trajectory of the mesh points, ensuring that during the playback of the first audio track, the image elements in the first image move according to the smoothed trajectory corresponding to the first image. The effects overlay unit overlays target effects onto the first image, such as particle effects or flowing light effects. Effects are activated based on a beat threshold; for example, when the BPM is greater than 140, particle effects are enabled. The particle system's emission position is bound to the character's joints, and its speed changes with increasing BPM. Finally, the dynamic changes in image elements and the overlaid effects are displayed on the terminal device.

[0134] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0135] Please refer to Figure 4 This diagram illustrates a block diagram of an image display apparatus according to an embodiment of this application. The apparatus has the function of implementing the image display method described above; this function can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the terminal device described above, or it can be installed within a terminal device. For example... Figure 4 As shown, the device 400 may include: a beat extraction module 410, a parameter determination module 420, and an image driving module 430.

[0136] The beat extraction module 410 is used to extract the beat from the first audio and obtain the beat information of the first audio.

[0137] The parameter determination module 420 is used to determine the motion parameters corresponding to the first image based on the beat information of the first audio, wherein the first image is an image displayed during the playback of the first audio.

[0138] The image driving module 430 is used to drive the image elements in the first image to move according to the motion parameters corresponding to the first image during the playback of the first audio, and to display the first image in which the image elements dynamically change.

[0139] In some embodiments, the beat information of the first audio includes: beat information corresponding to multiple audio frames in the first audio; the parameter determination module 420 is used for:

[0140] The image elements in the first image are divided into grids to obtain multiple grid points corresponding to the image elements;

[0141] For each of the plurality of audio frames, the motion parameters corresponding to the audio frame are determined according to the beat information corresponding to the audio frame. The motion parameters corresponding to the audio frame include the motion parameters of the plurality of grid points respectively within the playback period corresponding to the audio frame.

[0142] The motion parameters corresponding to the first image are obtained based on the motion parameters corresponding to the multiple audio frames.

[0143] In some embodiments, the parameter determination module 420 is configured to:

[0144] The first image is layered to obtain at least one image layer corresponding to the first image;

[0145] For each of the at least one image layer, motion parameters corresponding to the image layer are obtained based on the beat information corresponding to the audio frame. The motion parameters corresponding to the image layer include the motion parameters of at least one grid point on the image layer during the playback period corresponding to the audio frame.

[0146] The motion parameters corresponding to the audio frame are obtained based on the motion parameters corresponding to the at least one image layer.

[0147] In some embodiments, the parameter determination module 420 is configured to:

[0148] The motion parameters corresponding to the image layer are obtained based on the beat information corresponding to the audio frame and the mapping relationship between the beat information and the motion parameters.

[0149] In some embodiments, the parameter determination module 420 is configured to:

[0150] Obtain the mapping function corresponding to the first motion parameter on the image layer;

[0151] The first motion parameter corresponding to the image layer is obtained based on the beat information corresponding to the audio frame and the mapping function corresponding to the first motion parameter.

[0152] In some embodiments, the parameter determination module 420 is configured to:

[0153] When the beat information corresponding to the audio frame is in the i-th beat interval, the i-th mapping function corresponding to the first motion parameter is obtained from at least one mapping function corresponding to the first motion parameter. Different mapping functions correspond to different beat intervals, and i is a positive integer.

[0154] The first motion parameter corresponding to the image layer is obtained based on the beat information corresponding to the audio frame and the i-th mapping function corresponding to the first motion parameter.

[0155] In some embodiments, when the image layer includes a person layer, the motion parameters corresponding to the person layer include at least one of the following: local scaling parameters, local translation parameters, local rotation parameters, and transparency flickering parameters;

[0156] When the image layer includes a background layer, the motion parameters corresponding to the background layer include at least one of the following: global scaling parameter, global translation parameter, blur gradient parameter, hue cycle frequency, and hue cycle order.

[0157] In some embodiments, the image driving module 430 is configured to:

[0158] During the playback of the first audio, the motion trajectory corresponding to the image element in the first image is obtained based on the motion parameters corresponding to the first image.

[0159] Smoothing is performed on the motion trajectories corresponding to the image elements in the first image to obtain the smoothed motion trajectories.

[0160] The image elements in the first image are driven to move according to the smoothed motion trajectory, and the first image with the image elements dynamically changing is displayed.

[0161] In some embodiments, the apparatus further includes a special effects display module, the special effects display module being configured to:

[0162] Based on the beat information of the first audio, the special effect parameters corresponding to the target special effect are obtained;

[0163] During the playback of the first audio, the target effect is superimposed on the first image according to the effect parameters corresponding to the target effect.

[0164] In some embodiments, the beat information of the first audio includes: beat information corresponding to multiple audio frames in the first audio; the special effects display module is used for:

[0165] When the beat information corresponding to the audio frame is in the j-th beat interval, the j-th mapping function corresponding to the effect parameter is obtained from at least one mapping function corresponding to the effect parameter. Different mapping functions correspond to different beat intervals, and j is a positive integer.

[0166] Based on the beat information corresponding to the audio frame and the j-th mapping function corresponding to the effect parameters, the effect parameters corresponding to the target effect are obtained.

[0167] In some embodiments, the target special effect includes a first special effect and a second special effect, wherein the rhythm of change of the first special effect is greater than the rhythm of change of the second special effect; the special effect display module is used for:

[0168] If the beat information corresponding to the audio frame is greater than or equal to the first threshold, the first effect is superimposed on the first image;

[0169] If the beat information corresponding to the audio frame is less than the first threshold, the second effect is superimposed on the first image.

[0170] In some embodiments, the parameter determination module 420 is configured to:

[0171] Based on the audio information of the first audio, the audio features of the first audio are obtained;

[0172] Calculate the similarity between the audio features of the first audio and the image features corresponding to at least one image in the image library, and obtain at least one similarity.

[0173] The image corresponding to the maximum value among the at least one similarity scores is determined as the first image.

[0174] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0175] Please refer to Figure 5 This diagram illustrates a structural block diagram of a terminal device 500 provided in one embodiment of this application. The terminal device 500 can be any electronic device capable of data calculation, processing, and storage. The terminal device 500 can be used to implement the image display method provided in the above embodiments.

[0176] Typically, terminal device 500 includes a processor 501 and a memory 502.

[0177] Processor 501 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0178] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 502 are used to store a computer program configured to be executed by one or more processors to implement the above-described image display method.

[0179] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the terminal device 500, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0180] In an illustrative embodiment, a computer-readable storage medium is also provided, wherein a computer program is stored in the storage medium, and the computer program implements the above-described image display method when executed by the processor of a terminal device. Optionally, the above-described computer-readable storage medium may be ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage device, etc.

[0181] In an exemplary embodiment, a computer program product is also provided, comprising a computer program stored in a computer-readable storage medium. A processor of a terminal device reads the computer program from the computer-readable storage medium and executes the computer program, causing the terminal device to perform the image display method described above.

[0182] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user data. These prompt interfaces, pop-ups, or voice prompts are used to inform users that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without receiving confirmation from the user), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected by this application is processed strictly in accordance with the requirements of relevant national laws and regulations. The informed consent or separate consent of the data subject is obtained only with the user's consent and authorization. Subsequent data use and processing are conducted within the scope of laws, regulations, and the data subject's authorization, and the collection, use, and processing of relevant user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0183] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0184] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image display method, characterized in that, The method includes: Perform beat extraction on the first audio to obtain the beat information of the first audio; Based on the beat information of the first audio, the motion parameters corresponding to the first image are determined. The first image is an image displayed during the playback of the first audio. During the playback of the first audio, the image elements in the first image are driven to move according to the motion parameters corresponding to the first image, and the first image with the image elements dynamically changing is displayed.

2. The method according to claim 1, characterized in that, The beat information of the first audio includes: the beat information corresponding to each of the multiple audio frames in the first audio; The step of determining the motion parameters corresponding to the first image based on the beat information of the first audio includes: The image elements in the first image are divided into grids to obtain multiple grid points corresponding to the image elements; For each of the plurality of audio frames, the motion parameters corresponding to the audio frame are determined according to the beat information corresponding to the audio frame. The motion parameters corresponding to the audio frame include the motion parameters of the plurality of grid points respectively within the playback period corresponding to the audio frame. The motion parameters corresponding to the first image are obtained based on the motion parameters corresponding to the multiple audio frames.

3. The method according to claim 2, characterized in that, The step of determining the motion parameters corresponding to the audio frame based on the beat information corresponding to the audio frame includes: The first image is layered to obtain at least one image layer corresponding to the first image; For each of the at least one image layer, motion parameters corresponding to the image layer are obtained based on the beat information corresponding to the audio frame. The motion parameters corresponding to the image layer include the motion parameters of at least one grid point on the image layer during the playback period corresponding to the audio frame. The motion parameters corresponding to the audio frame are obtained based on the motion parameters corresponding to the at least one image layer.

4. The method according to claim 3, characterized in that, The step of obtaining the motion parameters corresponding to the image layer based on the beat information corresponding to the audio frame includes: The motion parameters corresponding to the image layer are obtained based on the beat information corresponding to the audio frame and the mapping relationship between the beat information and the motion parameters.

5. The method according to claim 4, characterized in that, The step of obtaining the motion parameters corresponding to the image layer based on the beat information corresponding to the audio frame and the mapping relationship between the beat information and the motion parameters includes: Obtain the mapping function corresponding to the first motion parameter on the image layer; The first motion parameter corresponding to the image layer is obtained based on the beat information corresponding to the audio frame and the mapping function corresponding to the first motion parameter.

6. The method according to claim 5, characterized in that, The step of obtaining the first motion parameter corresponding to the image layer based on the beat information corresponding to the audio frame and the mapping function corresponding to the first motion parameter includes: When the beat information corresponding to the audio frame is in the i-th beat interval, the i-th mapping function corresponding to the first motion parameter is obtained from at least one mapping function corresponding to the first motion parameter. Different mapping functions correspond to different beat intervals, and i is a positive integer. The first motion parameter corresponding to the image layer is obtained based on the beat information corresponding to the audio frame and the i-th mapping function corresponding to the first motion parameter.

7. The method according to any one of claims 3 to 6, characterized in that, When the image layer includes a person layer, the motion parameters corresponding to the person layer include at least one of the following: local scaling parameters, local translation parameters, local rotation parameters, and transparency flickering parameters; When the image layer includes a background layer, the motion parameters corresponding to the background layer include at least one of the following: global scaling parameter, global translation parameter, blur gradient parameter, hue cycle frequency, and hue cycle order.

8. The method according to any one of claims 1 to 7, characterized in that, During the playback of the first audio, the image elements in the first image are driven to move according to the motion parameters corresponding to the first image, and the first image with dynamically changing image elements is displayed, including: During the playback of the first audio, the motion trajectory corresponding to the image element in the first image is obtained based on the motion parameters corresponding to the first image. Smoothing is performed on the motion trajectories corresponding to the image elements in the first image to obtain the smoothed motion trajectories. The image elements in the first image are driven to move according to the smoothed motion trajectory, and the first image with the image elements dynamically changing is displayed.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Based on the beat information of the first audio, the special effect parameters corresponding to the target special effect are obtained; During the playback of the first audio, the target effect is superimposed on the first image according to the effect parameters corresponding to the target effect.

10. The method according to claim 9, characterized in that, The beat information of the first audio includes: the beat information corresponding to each of the multiple audio frames in the first audio; The step of obtaining the effect parameters corresponding to the target effect based on the beat information of the first audio includes: When the beat information corresponding to the audio frame is in the j-th beat interval, the j-th mapping function corresponding to the effect parameter is obtained from at least one mapping function corresponding to the effect parameter. Different mapping functions correspond to different beat intervals, and j is a positive integer. Based on the beat information corresponding to the audio frame and the j-th mapping function corresponding to the effect parameters, the effect parameters corresponding to the target effect are obtained.

11. The method according to claim 10, characterized in that, The target special effect includes a first special effect and a second special effect, wherein the rhythm of change of the first special effect is greater than the rhythm of change of the second special effect; the step of superimposing the target special effect on the first image includes: If the beat information corresponding to the audio frame is greater than or equal to the first threshold, the first effect is superimposed on the first image; If the beat information corresponding to the audio frame is less than the first threshold, the second effect is superimposed on the first image.

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: Based on the audio information of the first audio, the audio features of the first audio are obtained; Calculate the similarity between the audio features of the first audio and the image features corresponding to at least one image in the image library, and obtain at least one similarity. The image corresponding to the maximum value among the at least one similarity scores is determined as the first image.

13. A terminal device, characterized in that, The terminal device includes a processor and a memory, the memory storing a computer program, which is loaded and executed by the processor to implement the image display method as described in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the image display method as described in any one of claims 1 to 12.

15. A computer program product, characterized in that, The computer program product includes a computer program that is loaded and executed by a processor to implement the image display method as described in any one of claims 1 to 12.

Images