A photograph display method, a photograph display apparatus, a storage medium, and a program product

Abstract

A photo display method, a photo display device, a storage medium and a program product, the method comprising: acquiring a shooting timestamp, a positioning latitude and longitude, and a pixel color value of each photo in a photo set to be displayed, and a rhythm intensity and a segmented time interval of each music segment in music to be played; dividing the photos into multiple theme clusters according to the shooting timestamp and the positioning latitude and longitude; determining a color feature vector of the theme cluster according to the pixel color value of all photos in the theme cluster, to calculate the inter-cluster switching intensity between the theme clusters; determining the inter-segment switching intensity from a previous music segment to a next music segment according to the rhythm intensity of the music segment; establishing a corresponding relationship between the music segment and the theme cluster through the inter-segment switching intensity and the inter-cluster switching intensity, to match each theme cluster to the segmented time interval of the corresponding music segment; and according to the music segment length and the number of photos in the theme cluster, allocating a display start and end time of each photo, to generate a photo display sequence synchronized with the rhythm of the music to be played.

Description

Technical Field

[0001] This application relates to the field of multimedia information processing, and in particular to a photo display method, photo display device, storage medium, and program product. Background Technology

[0002] With the development of digital photography technology, a large number of digital photos have accumulated on user terminals. How to efficiently manage and automatically display these photos has become an important research direction in the fields of digital image processing and smart devices. Currently, various photo display devices and photo album applications are widely used in personal and family settings to meet the needs of photo browsing, review, and sharing.

[0003] In related photo display technologies, a linear sorting display scheme based on a single-dimensional attribute is typically adopted. In practice, the system first reads the metadata information of the photos and extracts the shooting time or geographical coordinates of each photo. Then, the system performs a global linear sorting of all photos based on chronological order, or simply groups photos with the same geographical coordinates and arranges them in order. In the final display stage, the system plays each photo sequentially at fixed time intervals according to the generated one-dimensional linear sequence, thereby completing the automated carousel display of the album.

[0004] However, the presentation process of this technology is completely independent of audio information, failing to organically combine photo content with musical elements, and making it difficult to match and synchronize photos with the melody and rhythm of background music. This lack of audio-visual synergy makes the photo album slideshow process monotonous and dull, unable to leverage the emotional rendering power of music to enhance the immersiveness and narrative of photo review, thus failing to meet users' higher demands for personalized and intelligent photo browsing experiences. Summary of the Invention

[0005] This application provides a photo display method, photo display device, storage medium, and program product for achieving matching and synchronization between photos and background music.

[0006] In a first aspect, this application provides a photo display method applied to a photo display device. The method includes: obtaining the shooting timestamp, location latitude and longitude, and pixel color value of each photo in a set of photos to be displayed, and the rhythm intensity and segmented time interval of each music segment in a music to be played; dividing the photos into multiple theme clusters according to the shooting timestamp and location latitude and longitude; the time interval between the shooting timestamps of any two adjacent photos within a theme cluster is less than a preset time threshold, and the distance between the location latitude and longitude is less than a preset distance threshold; determining the color feature vector of the theme cluster based on the pixel color values ​​of all photos within the theme cluster, so as to calculate the inter-cluster switching intensity between any two theme clusters; determining the segment switching intensity from one music segment to the next music segment based on the rhythm intensity of the music segment; establishing a correspondence between music segments and theme clusters through the segment switching intensity and the cluster switching intensity, so as to match each theme cluster to the segmented time interval of the corresponding music segment; allocating the display start and end time of each photo according to the music segment duration and the number of photos in the theme cluster, so as to generate a photo display sequence synchronized with the rhythm of the music to be played.

[0007] By adopting the above technical solutions, the limitations of traditional photo displays, which are only linearly sorted in a single dimension and completely disconnected from the music, are overcome. This achieves deep synergy between photo content and background music in terms of rhythm and segmentation, making photo carousels no longer monotonous. By leveraging the emotional rendering of music, the immersive and narrative experience of browsing is greatly enhanced, accurately meeting users' core needs for personalized and intelligent audio-visual synchronized photo displays.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the color feature vector of the theme cluster is determined based on the pixel color values ​​of all photos within the theme cluster. Specifically, this includes: extracting the dominant hue value and color richness of each photo within the theme cluster; calculating the gradient of the change in dominant hue value between adjacent photos according to the shooting timestamp order of the photos within the theme cluster to obtain the dynamic color evolution characteristics of the theme cluster; determining the static basic color characteristics of the theme cluster based on the statistical distribution results of the color richness of all photos within the theme cluster; and concatenating the dynamic color evolution characteristics and the static basic color characteristics to generate the color feature vector of the theme cluster.

[0009] By adopting the above technical solution, compared with the method of extracting only a single color parameter, the color attributes of each theme cluster can be more comprehensively and accurately depicted. It not only preserves the dynamic color context of the photo over time, but also solidifies the basic color style of the theme cluster as a whole. This provides reliable and complete color data support for the subsequent accurate calculation of the switching intensity between clusters, ensuring the rationality and visual coherence of the music and photo switching.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, calculating the inter-cluster switching intensity between any two theme clusters specifically includes: determining the visual inertia weight of a first theme cluster based on the static basic color features of the first theme cluster, wherein the first theme cluster is any one of a plurality of theme clusters; calculating the feature difference degree between the dynamic color evolution features of the first theme cluster and the static basic color features of a second theme cluster, wherein the second theme cluster is any one of a plurality of theme clusters other than the first theme cluster; and weighting the feature difference degree with the visual inertia weight to obtain the inter-cluster switching intensity of the transition from the first theme cluster to the second theme cluster.

[0011] By adopting the above technical solution, the calculation of the switching intensity between clusters is made more in line with the laws of human visual perception, so that the matching of subsequent music segments and theme clusters is more in line with the dual experience of vision and hearing, avoiding the sense of disharmony caused by abrupt color switching, and further improving the smoothness and viewing experience when audio and video are displayed in sync.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, a correspondence between music segments and theme clusters is established through inter-segment switching intensity and inter-cluster switching intensity. Specifically, this includes: calculating the mean of the shooting timestamps of all photos within a theme cluster to generate a time centroid; generating a theme cluster sequence according to the chronological order of the time centroids; determining a music segment sequence based on the music to be played; constructing a matching matrix with theme clusters as the first dimension and music segments as the second dimension based on the theme cluster sequence and the music segment sequence; calculating the numerical difference between the inter-cluster switching intensity and the inter-segment switching intensity corresponding to each node in the matching matrix, and determining the numerical difference as the matching cost of the corresponding node; under the constraint of maintaining the sequential increase of the first and second dimensions, solving for the connected path with the minimum cumulative matching cost from the starting node to the ending node in the matching matrix; and establishing a mapping correspondence between music segments and theme clusters based on the coordinates of the nodes traversed by the connected path.

[0013] By adopting the above technical solution, under the premise of strictly following the narrative logic of photo time and the playback order of music segments, the global optimal matching of music segments and theme clusters is achieved, avoiding the misalignment of audio and video rhythm caused by local matching deviations. Overall, it ensures that the photo display is highly consistent with the melody and rhythm changes of the music, and makes the narrative rhythm of the album slideshow completely unified with the emotional rhythm of the music, significantly improving the overall coordination and immersiveness of the intelligent photo display.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, a correspondence between music segments and theme clusters is established through inter-segment switching intensity and inter-cluster switching intensity. Specifically, this includes: establishing a correspondence between the first theme cluster in a pre-set time sequence and the first music segment of the music to be played, and taking the first theme cluster as the current theme cluster and the first music segment as the current music segment; obtaining the inter-segment switching intensity of the transition from the current music segment to the next music segment in the time sequence, as the target switching intensity; calculating the inter-cluster switching intensity between the current theme cluster and each candidate theme cluster in multiple candidate theme clusters after the current theme cluster; selecting the candidate theme cluster corresponding to the inter-cluster switching intensity with the smallest difference from the target switching intensity value, as the target theme cluster; establishing a correspondence between the target theme cluster and the next music segment, updating the target theme cluster to the current theme cluster, updating the next music segment to the current music segment, and repeating the matching steps until the correspondence between all music segments and theme clusters is established.

[0015] By adopting the above technical solution, there is no need to construct complex matrices and global path calculations. The matching logic is simple, efficient, and computationally inefficient, enabling rapid dynamic adaptation of music segments and theme clusters. While ensuring accurate alignment of audio-visual switching intensity, it improves the generation efficiency of photo display sequences, adapts to the real-time processing needs of various photo display devices, and balances matching accuracy and smooth operation.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, the start and end times for displaying each photo are allocated based on the duration of the music segment and the number of photos within the theme cluster. Specifically, this includes: obtaining a target music segment that has a mapping relationship with the target theme cluster and extracting micro-beat points within the target music segment; dividing the target music segment into multiple candidate display intervals of varying lengths based on the micro-beat points as the time segmentation benchmark; calculating the visual complexity of each photo based on the pixel color values ​​of each photo within the target theme cluster; and sequentially allocating the photos within the target theme cluster to the corresponding candidate display intervals according to a matching principle of visual complexity from high to low and candidate display duration from long to short, thereby determining the start and end times for displaying each photo.

[0017] By adopting the above technical solutions, the switching of photos can be precisely matched with the micro-beats of the music, achieving deep synchronization of sound and image in detail rhythm. At the same time, photos with richer visual information can be displayed for a longer time to ensure the viewing effect, while photos with simple visuals can be adapted to shorter display times to match the rhythm density of the music. This balances the constraints of music rhythm with the visual viewing needs of photos, making the overall display rhythm well-paced and greatly improving the user's browsing comfort and experience.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of allocating the start and end times of display for each photo based on the duration of the music segment and the number of photos within the theme cluster, the method further includes: extracting the amplitude energy of micro-beat points at the intersection of adjacent preceding and subsequent photos in the candidate display interval; calculating the numerical difference in visual complexity between preceding and subsequent photos to obtain the visual jump degree between adjacent images; performing a positive mapping between the amplitude energy of the micro-beat points and the visual jump degree between adjacent images to determine the transition animation type and transition duration when switching from preceding to subsequent photos; and inserting a transition rendering frame between the end time of display of the preceding photo and the start time of display of the subsequent photo based on the transition animation type and transition duration.

[0019] By adopting the above technical solution, the transition effect can be dynamically adjusted according to the intensity of the music beat and the visual jump of the screen, avoiding the abruptness and incongruity caused by fixed transitions. This makes the photo switching action highly coordinated with the music rhythm and visual changes of the screen, further optimizing the smoothness of the visual transition while ensuring audio-visual synchronization. This makes the photo carousel more visually appealing and professional, perfectly achieving the invention's purpose of personalized and intelligent audio-visual collaborative display.

[0020] In a second aspect, embodiments of this application provide a photo display device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the photo display device to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a photo display device, cause the photo display device to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a photo display device, cause the photo display device to perform the method described in the first aspect and any possible implementation thereof.

[0023] Understandably, the photo display device provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0025] 1. By adopting the above technical solution, the limitations of traditional photo display, which is only linearly sorted in a single dimension and completely disconnected from the music, are broken. The photo content and background music are deeply coordinated in terms of rhythm and segmentation, making the photo carousel no longer monotonous. With the help of the emotional rendering of the music, the immersive and narrative feeling of browsing is greatly enhanced, and the core needs of users for personalized and intelligent audio-visual synchronized photo display are accurately met.

[0026] 2. By adopting the above technical solution, compared with the method of extracting only a single color parameter, the color attributes of each theme cluster can be more comprehensively and accurately depicted. It not only preserves the dynamic color context of the photo over time, but also solidifies the basic color style of the theme cluster as a whole. This provides reliable and complete color data support for the subsequent accurate calculation of the switching intensity between clusters, ensuring the rationality and visual continuity of the music and photo switching.

[0027] 3. By adopting the above technical solutions, under the premise of strictly following the narrative logic of photo time and the playback order of music segments, the global optimal matching of music segments and theme clusters is achieved, avoiding the misalignment of audio and video rhythm caused by local matching deviations. Overall, it ensures that the photo display is highly consistent with the melody and rhythm changes of the music, and makes the narrative rhythm of the album slideshow completely unified with the emotional rhythm of the music, which significantly improves the overall coordination and immersiveness of the intelligent photo display. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a photo display method in an embodiment of this application;

[0029] Figure 2 This is another flowchart illustrating the photo display method in this application embodiment;

[0030] Figure 3 This is a schematic diagram of the physical device structure of a photo display device in the embodiments of this application. Detailed Implementation

[0031] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0032] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0033] The following describes the process of the method provided in this implementation. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating a photo display method in an embodiment of this application.

[0034] S101. Obtain the shooting timestamp, location latitude and longitude, and pixel color value of each photo in the set of photos to be displayed, as well as the rhythm intensity and segment time interval of each music segment in the music to be played.

[0035] The set of photos to be displayed refers to a collection of photos selected by the user or automatically filtered by the photo display device to be shown, such as several photos selected by the user in their album. Each photo represents a specific digital image within the set. The shooting timestamp indicates the exact time the photo was taken, usually accurate to the second, such as "2023-10-01 14:30:00". The location coordinates (latitude and longitude) are the geographical coordinates of the photo's shooting location on the Earth's surface, composed of longitude and latitude, used to determine the photo's spatial location. The pixel color value represents the color information of each pixel in the photo image, typically composed of the values ​​of the red (R), green (G), and blue (B) color channels, used to reflect the overall visual color characteristics of the photo.

[0036] The music to be played refers to the pre-loaded, selected audio file prepared for synchronous playback during the photo display. It refers to complete audio data containing melody, beat, and rhythmic variations, which can originate from local storage, cloud downloads, or real-time user uploads. A music segment refers to a continuous audio fragment obtained by the photo display device after dividing the music to be played according to audio waveform, beat, melody, or emotional characteristics. It can also be an attribute configuration of the music to be played. A music segment is the basic unit constituting a complete piece of music to be played, used to represent the rhythm and emotional state within different time intervals. For example, a piece of music can be divided into an intro, verse, chorus, interlude, and outro. Rhythm intensity is used to represent the tempo, drum density, volume amplitude, and rhythmic strength of a single music segment. It is a numerical indicator that quantifies the rhythmic characteristics of a music segment. A higher value indicates a more intense and rapid rhythm, while a lower value indicates a more relaxed and gentle rhythm. For example, the rhythm intensity of the chorus is usually higher than that of the verse. The segmented time interval refers to the start and end time range defined for each music segment. It is used to clarify the time position and duration of a single music segment within the complete music to be played. It is usually marked with milliseconds or seconds as the time unit. For example, the segmented time interval of a music segment is 00:00:05 to 00:00:18.

[0037] Specifically, when a user triggers the photo display function (e.g., by clicking the "Play Memories" button), the photo display device will traverse the set of photos to be displayed, read the metadata information such as EXIF ​​(Exchangeable Image File Format) of each photo, and extract the shooting timestamp and location latitude and longitude. At the same time, the photo display device will also analyze the image content of each photo and extract the color values ​​of all pixels.

[0038] The photo display device reads the complete audio waveform and timeline information of the music to be played through its built-in audio analysis module. Then, based on audio signal processing algorithms, it identifies beat points, melodic transitions, and emotional shifts in the music, automatically dividing it into multiple continuous and non-overlapping segments according to preset rules. Each segment is then uniquely labeled with its designated time interval. Subsequently, the device performs frequency and time domain analysis on the audio signal of each segment, extracting feature data such as drum frequency, peak amplitude, and beat intervals. Through quantization calculations, it generates the rhythm intensity of the corresponding music segment. Finally, the device stores the rhythm intensity of each music segment in a one-to-one correspondence with its designated time interval.

[0039] S102. Divide the photos into multiple theme clusters according to the shooting timestamp and location latitude and longitude; the time interval between the shooting timestamps of any two adjacent photos in the theme cluster is less than a preset time threshold and the distance between the location latitude and longitude is less than a preset distance threshold.

[0040] A theme cluster refers to a set of photos that are close in time and adjacent in space. It represents a relatively independent but coherent scene or event segment, such as "the process of visiting a certain scenic spot." The preset time threshold represents the maximum allowed time interval for determining whether two photos belong to the same time period, for example, set to 10 minutes. The preset distance threshold represents the maximum allowed physical distance for determining whether two photos belong to the same location, for example, set to 500 meters. Adjacent photos are two photos that are immediately next to each other after being sorted by time or space.

[0041] Specifically, to organize scattered photos into meaningful scene fragments, the photo display device performs clustering. The device sets preset time and distance thresholds, and then performs spatiotemporal joint clustering based on the shooting timestamp and location latitude and longitude of each photo. If two adjacent photos, ordered by time, have a time interval (calculated based on shooting timestamps) less than the preset time threshold, and their physical distance (calculated based on location latitude and longitude) is also less than the preset distance threshold, then these two photos will be grouped into the same thematic cluster. In this way, the originally chaotic mass of photos is divided into several thematic clusters with clear scene meaning.

[0042] S103. Based on the pixel color values ​​of all photos within a subject cluster, determine the color feature vector of the subject cluster in order to calculate the inter-cluster switching intensity between any two subject clusters.

[0043] The color feature vector is used to mathematically quantify the visual features of a subject cluster. The inter-cluster switching intensity represents a numerical value that quantifies the degree of color difference between two subject clusters.

[0044] Specifically, after clustering the photos, the photo display device needs to extract representative indicators that represent the overall characteristics of each theme cluster. For any target theme cluster, the device aggregates the pixel color values ​​of all photos within that cluster, generating a global color histogram (a graph showing the pixel distribution across different color ranges for all photos within a theme cluster, reflecting the overall color tone of the cluster) to determine the overall hue of the scene (e.g., cool or warm tones). Subsequently, the device uses a specific feature extraction algorithm to transform this global color histogram into a multi-dimensional color feature vector. After determining the color feature vector, the device calculates the Euclidean distance between the color feature vectors of any two theme clusters to obtain the inter-cluster switching intensity between each pair of theme clusters.

[0045] Optionally, in general, the color feature vector of a subject cluster can be determined based on the pixel color values ​​of all photos within the subject cluster in the following ways, without limitation: extract the dominant hue value and color richness of each photo within the subject cluster; calculate the gradient of the change in dominant hue value between adjacent photos according to the shooting timestamp order of the photos within the subject cluster to obtain the dynamic color evolution characteristics of the subject cluster; determine the static basic color characteristics of the subject cluster based on the statistical distribution results of the color richness of all photos within the subject cluster; and concatenate the dynamic color evolution characteristics and the static basic color characteristics to generate the color feature vector of the subject cluster.

[0046] Among them, the dominant color value represents the most prevalent and representative color value in the photo, referring to the core color parameter obtained through pixel color value statistics. For example, the dominant color value of a landscape photo is sky blue and green. Color richness represents the number of colors contained in the photo and the uniformity of color distribution. It is a numerical indicator that quantifies the color diversity of the photo. For example, a single-color ID photo has low color richness, while a colorful oil painting photo has high color richness. The shooting timestamp order indicates the order in which the photos are arranged from morning to night. It refers to the photo sorting rule based on the shooting timestamp. For example, a photo taken at 10:00 AM is placed before a photo taken at 11:00 AM. Adjacent photos refer to two photos that are immediately adjacent in the shooting timestamp order. For example, the 3rd and 4th photos after sorting by time are adjacent photos. The dominant color value change gradient represents the magnitude and direction of the change in the dominant color value of adjacent photos. The numerical values ​​quantify the smoothness of color transitions between two photos, for example, a small gradient from light blue to dark blue and a large gradient from red to green; dynamic color evolution features represent the color change patterns of photos within a subject cluster over time, referring to a set of features that reflect the dynamic changes in photo colors; color richness statistical distribution results represent the numerical distribution of color richness across all photos within a subject cluster, referring to color richness features obtained through statistics such as mean, variance, and percentage; static basic color features represent the core color attributes of a subject cluster that remain unchanged over time; combining dynamic color evolution features and static basic color features represents combining the two types of feature data in a fixed format, referring to a processing method that generates a unified color feature vector; color feature vectors are used to mathematically quantify all color attributes of a subject cluster, referring to a multi-dimensional numerical vector containing both dynamic and static color information.

[0047] Specifically, the photo display device first performs image analysis on all photos within each theme cluster, extracting the dominant color value and color richness of each photo. Then, it traverses the photos within the theme cluster according to the order of their shooting timestamps, calculating the difference and direction of change of the dominant color value of adjacent photos group by group, integrating all the change gradients to form dynamic color evolution features, and completely recording the color change trajectory of the theme cluster over time. At the same time, it performs statistical analysis on the color richness of all photos within the cluster, calculating the average, median, and dispersion of color richness to determine the static basic color features of the theme cluster as a whole. Finally, according to the preset vector splicing rules, it places the dynamic color evolution features in the first half of the vector and the static basic color features in the second half of the vector, combining them to generate a unique color feature vector representing the theme cluster.

[0048] Optionally, in general, the inter-cluster switching intensity between any two subject clusters can be calculated in the following way, without limitation: Based on the static basic color characteristics of the first subject cluster, determine the visual inertia weight of the first subject cluster, where the first subject cluster is any one of multiple subject clusters; calculate the feature difference between the dynamic color evolution characteristics of the first subject cluster and the static basic color characteristics of the second subject cluster, where the second subject cluster is any one of multiple subject clusters other than the first subject cluster; and calculate the inter-cluster switching intensity of the transition from the first subject cluster to the second subject cluster by weighting the feature difference with the visual inertia weight.

[0049] Here, the first theme cluster refers to any one of the multiple theme clusters selected as the starting point for calculation. It is the initial subject for calculating the inter-cluster switching intensity. For example, when switching from theme cluster A to theme cluster B, theme cluster A is the first theme cluster. The static basic color features (as described above) are used to represent the fixed core color attributes of the first theme cluster. The visual inertia weight represents the degree of visual persistence and adaptation of the human eye to the colors of the first theme cluster. It refers to the weight value that quantifies visual continuity. For example, the visual inertia weight is high for theme clusters with stable colors and low for theme clusters with chaotic colors. The second theme cluster refers to any one of the multiple theme clusters other than the first theme cluster. It refers to the inter-cluster switching intensity. The target object of the calculation is, for example, when switching from theme cluster A to theme cluster B, theme cluster B is the second theme cluster; the dynamic color evolution feature (as described above) is used to represent the dynamic color change pattern of the first theme cluster; the feature difference degree represents the numerical difference between the dynamic color feature of the first theme cluster and the static color feature of the second theme cluster, which is an indicator that quantifies the color contrast between the two theme clusters; the weighted calculation represents the sum of the feature difference degree and the visual inertia weight in a proportional manner, which is a mathematical operation method for calculating the final switching intensity; the inter-cluster switching intensity of the transition from the first theme cluster to the second theme cluster is used to represent the degree of visual jump when switching between the two theme clusters, which is the final value that quantifies the intensity of the visual switching.

[0050] Specifically, the photo display device first traverses all theme clusters, designating each theme cluster as the first theme cluster. Based on the stability and uniformity of the static basic color features of the first theme cluster, a visual inertia weight is calculated; the more stable and uniform the colors, the higher the visual inertia weight. Then, one of the remaining theme clusters is selected as the second theme cluster. The dynamic color evolution features of the first theme cluster and the static basic color features of the second theme cluster are extracted, and the feature difference between the two is calculated using the Euclidean distance algorithm; the greater the difference, the higher the feature difference value. Finally, the calculated feature difference value is multiplied by the visual inertia weight to obtain the inter-cluster switching intensity from the first theme cluster to the second theme cluster. This process is repeated until the switching intensity calculation between all pairs of theme clusters is completed.

[0051] S104. Determine the transition intensity between the previous and next musical segments based on the rhythmic intensity of the musical segments.

[0052] In this context, the previous musical segment refers to the audio clip that, in the playback order, precedes the current reference position on the timeline of the music to be played. The next musical segment refers to the audio clip that immediately follows the previous musical segment on the timeline of the music to be played. The inter-segment transition intensity is a numerical value that quantifies the drastic change in rhythm when transitioning from one musical segment to the next. It is used to represent the auditory difference and sense of jump between two adjacent musical segments. For example, the inter-segment transition intensity value is relatively high when transitioning from a soothing verse to an exciting chorus.

[0053] Specifically, the photo display device iterates through adjacent music segments (the previous and next segments) sequentially according to the time order of the music to be played. It extracts the rhythm intensity values ​​of the previous and next segments and calculates the rhythmic difference between them by determining the difference, rate of change, or using a specific smoothing transition algorithm. This rhythmic difference is defined as the inter-segment transition intensity. Through this calculation, the photo display device can accurately capture the emotional and rhythmic fluctuations of the music during playback, thus transforming abstract musical melodic transitions into quantifiable data that can be processed by a computer.

[0054] S105. By using the inter-segment switching intensity and inter-cluster switching intensity, establish the correspondence between music segments and theme clusters, so as to match each theme cluster to the segmented time interval of the corresponding music segment.

[0055] Among them, the inter-segment transition intensity refers to a numerical value that quantifies the degree of rhythmic change and emotional shift between two adjacent musical segments, reflecting the transitional characteristics of the music's auditory perception. The inter-cluster transition intensity refers to a numerical value that quantifies the degree of difference in color and visual features between two adjacent thematic clusters, representing the visually jarring effect of the photograph; for example, the inter-cluster transition intensity is high between two thematic clusters with extremely contrasting colors. The correspondence between musical segments and thematic clusters refers to the association state in which specific thematic clusters are bound and mapped to specific musical segments during the photograph's display, ensuring synchronization between the visual scene and the auditory segments.

[0056] Specifically, after calculating the inter-cluster switching intensity reflecting visual transitions and the inter-segment switching intensity reflecting auditory transitions, the photo display device enters the core matching stage of audio-visual coordination, aiming to perfectly match the switching rhythm of photo images with the rise and fall of musical melodies.

[0057] Optionally, in general, establishing the correspondence between music segments and theme clusters through inter-segment switching intensity and inter-cluster switching intensity can be achieved in the following ways, without limitation: Calculate the mean of the shooting timestamps of all photos within the theme cluster to generate a time centroid; generate a theme cluster sequence according to the chronological order of the time centroids; determine the music segment sequence based on the music to be played; construct a matching matrix with theme clusters as the first dimension and music segments as the second dimension based on the theme cluster sequence and music segment sequence; calculate the numerical difference between the inter-cluster switching intensity and the inter-segment switching intensity corresponding to each node in the matching matrix, and determine the numerical difference as the matching cost of the corresponding node; under the constraint of maintaining the sequential increase of the first and second dimensions, solve for the connected path with the minimum cumulative matching cost from the starting node to the ending node in the matching matrix; establish the mapping correspondence between music segments and theme clusters based on the coordinates of the nodes traversed by the connected path.

[0058] The mean of the shooting timestamps represents the arithmetic mean of the shooting timestamp values ​​of all photos within a subject cluster. It is a centralized time value used to represent the overall shooting time of the cluster. For example, if photos in a cluster were shot at 10:00, 10:02, and 10:04, the mean is 10:02. The time centroid represents the core time point generated from the mean of the shooting timestamps, representing the overall time position of the subject cluster. It is the benchmark used to sort the subject clusters by time. The order of the time centroids indicates the order in which they are arranged from earliest to latest, representing the generation rule of the subject cluster sequence. The subject cluster sequence indicates the order in which the subject clusters are arranged by time centroids. The ordered queue of topic clusters after prioritization refers to the order of the photo topic sets participating in the matching; the music segment sequence represents the ordered queue of music segments to be played, formed by breaking down the music into rhythms and segments and then playing them in order, referring to the order of the audio segments participating in the matching; the first dimension represents the vertical dimension representing topic clusters in the matching matrix, referring to the indexing basis in the matrix's row direction; the second dimension represents the horizontal dimension representing music segments in the matching matrix, referring to the indexing basis in the matrix's column direction; the matching matrix is ​​a two-dimensional numerical matrix constructed with topic clusters as rows and music segments as columns, serving as the carrier for calculating matching costs and finding the optimal path; the matching matrix node table... The intersection of a row and column in the matching matrix represents the candidate position corresponding to "a certain theme cluster matching a certain music segment"; the numerical difference represents the absolute or relative difference between the switching intensity between clusters and the switching intensity between segments at the same node, which is a numerical measure of matching fit; the matching cost represents the cost index determined by the numerical difference, which represents the quality of the matching, and the smaller the value, the better the matching; the sequential increasing constraint means that theme clusters can only be matched in chronological order and music segments can only be matched in playback order, and cannot be matched in reverse or out of order, which is a rule that path search must follow; the starting node represents the position of the first theme cluster corresponding to the first music segment in the matching matrix, which is the path The starting point; the ending node represents the position of the last theme cluster corresponding to the last music segment in the matching matrix, which is the end point of the path; the cumulative matching cost represents the sum of the costs of all nodes traversed by the path, which is the basis for judging the overall matching quality; the connected path represents the continuous node route from the start to the end that satisfies the order constraints, which is the carrier of the optimal matching scheme; the node coordinates represent the position identifier composed of row and column numbers in the matching matrix, which is the basis for determining the correspondence between theme clusters and music segments; the mapping correspondence represents the final determined binding relationship of which theme cluster matches which music segment, which is the core basis for macroscopic audio-visual synchronization.

[0059] Specifically, the photo display device first calculates the average of the timestamps of all photos taken within each theme cluster, using this average as the time centroid of that theme cluster. Then, the clusters are arranged from earliest to latest according to their time centroids, forming an ordered sequence. Simultaneously, the music to be played is organized into a music segment sequence based on segmented time intervals and playback order. Next, the device constructs a two-dimensional matching matrix with the theme cluster sequence as the first dimension (rows) and the music segment sequence as the second dimension (columns). Each node in the matrix represents a candidate solution for "matching the theme cluster in that row with the music segment in that column." Then, for each node in the matrix, the device extracts the inter-cluster switching intensity and inter-segment switching intensity corresponding to that node, calculates the difference between them, and sets this difference as the matching cost of that node. The smaller the difference, the more compatible the switching rhythm and the lower the cost. Finally, under the strict constraints that the order of theme clusters and music segments can only increase, the device searches the matching matrix for a connected path from the starting node to the ending node, requiring the cumulative matching cost of all nodes on this path to be minimized. Finally, the photo display device establishes a mapping relationship between music segments and theme clusters one by one based on the row coordinates (theme clusters) and column coordinates (music segments) of each node traversed by this optimal path, thus completing the globally optimal audio-visual segment matching.

[0060] Optionally, under normal circumstances, establishing the correspondence between music segments and theme clusters through inter-segment switching intensity and inter-cluster switching intensity can be achieved in the following ways, without limitation: Establish a correspondence between the first theme cluster in the pre-set time sequence and the first music segment to be played, and take the first theme cluster as the current theme cluster and the first music segment as the current music segment; obtain the inter-segment switching intensity of the transition from the current music segment to the next music segment in the time sequence, as the target switching intensity; calculate the inter-cluster switching intensity between the current theme cluster and each candidate theme cluster among multiple candidate theme clusters after the current theme cluster; select the candidate theme cluster corresponding to the inter-cluster switching intensity with the smallest difference from the target switching intensity value, as the target theme cluster; establish a correspondence between the target theme cluster and the next music segment, update the target theme cluster to the current theme cluster, update the next music segment to the current music segment, and repeat the matching steps until the correspondence between all music segments and theme clusters is established.

[0061] The first theme cluster represents the theme cluster that appears first in chronological order, referring to the initial set of photos in the matching process. This is pre-set and can be either the first in chronological order or a user preference setting. The first music segment represents the music segment played first in the music playback order, referring to the initial audio segment in the matching process. The correspondence relationship indicates the binding and pairing relationship between theme clusters and music segments, used to determine which set of photos matches which music segment. The current theme cluster represents the theme cluster being processed and used as the benchmark in this round of matching, referring to the current benchmark for iterative matching. The current music segment represents the music segment being processed and used as the benchmark in this round of matching, referring to the audio benchmark for iterative matching. The next music segment represents the segment immediately following the current music segment in the music playback order, referring to the next audio segment to be matched. The inter-segment transition intensity indicates the drastic change in rhythm between the current music segment and the next music segment, used to measure... The text describes various aspects of matching, including: the transition intensity of music segments; the target transition intensity; the candidate theme cluster; the inter-cluster transition intensity; the numerical difference; the target theme cluster; the target theme cluster; the final mapping relationship; and the mapping correspondence. It also describes the complete pairing of music segments and theme clusters.

[0062] Specifically, the photo display device first initializes by directly binding the first theme cluster in the chronological order to the first music segment in the music playback order, and setting these two objects as the current theme cluster and the current music segment, respectively. Next, it obtains the inter-segment transition intensity from the current music segment to the next music segment and sets it as the target transition intensity, serving as the standard for selecting theme clusters in this round. Then, among all candidate theme clusters after the current theme cluster, it calculates the inter-cluster transition intensity from the current theme cluster to each candidate theme cluster. Afterward, it calculates the numerical difference between each inter-cluster transition intensity and the target transition intensity, selecting the candidate theme cluster with the smallest difference as the target theme cluster, representing the visual transition rhythm that best matches the music transition rhythm. Then, it establishes a correspondence between the target theme cluster and the next music segment, updating the target theme cluster to the new current theme cluster and the next music segment to the new current music segment. Finally, it repeats the above matching steps until all music segments are matched one-to-one with theme clusters, ultimately forming a complete music segment-theme cluster mapping relationship.

[0063] S106. Based on the duration of the music segment and the number of photos in the theme cluster, allocate the start and end times for displaying each photo to generate a photo display sequence synchronized with the rhythm of the music to be played.

[0064] The music segment duration refers to the length of a single music segment from beginning to end, calculated from its segmented time intervals, and is used to limit the total display time of photos matched to that music segment. The number of photos within a theme cluster refers to the total number of photos grouped into the same theme cluster, reflecting the density of visual information in that scene. The display start and end times refer to the times when each specific photo appears and disappears during the final carousel display, used to precisely control the display duration of each photo. The music rhythm refers to the tempo, volume, and alternation of the music during playback, used to guide the micro-timing of photo transitions. The photo display sequence is the final playlist generated by arranging all photos in the set of photos to be displayed in an ordered manner according to the assigned display start and end times, used to directly drive the photo display device for automated, synchronized audio-visual carousel playback.

[0065] Specifically, after establishing the macroscopic mapping relationship between theme clusters and music segments, the photo display device needs to further determine the specific playback details of each photo at the microscopic level to generate the final display effect. For any target music segment with an established mapping relationship and its corresponding target theme cluster, the photo display device first obtains the duration of the music segment and counts the number of photos within the theme cluster. Based on the music segment duration and the number of photos within the theme cluster, the average playback duration of each photo in the target theme cluster is determined, thereby determining the start and end times for displaying each photo.

[0066] Optionally, in general, the start and end times for each photo's display can be assigned based on the duration of the music segment and the number of photos within the theme cluster, without any limitation: Obtain the target music segment that has a mapping relationship with the target theme cluster, and extract the micro-beat points within the target music segment; using the micro-beat points as the time segmentation benchmark, divide the target music segment into multiple candidate display intervals of varying lengths; calculate the visual complexity of each photo based on its pixel color values ​​within the target theme cluster; and, according to the matching principle of visual complexity from high to low and candidate display duration from long to short, sequentially assign the photos within the target theme cluster to the corresponding candidate display intervals to determine the start and end times for each photo's display.

[0067] By adopting the above technical solutions, the limitations of traditional photo displays, which are only linearly sorted in a single dimension and completely disconnected from the music, are overcome. This achieves deep synergy between photo content and background music in terms of rhythm and segmentation, making photo carousels no longer monotonous. By leveraging the emotional rendering of music, the immersive and narrative experience of browsing is greatly enhanced, accurately meeting users' core needs for personalized and intelligent audio-visual synchronized photo displays.

[0068] The following provides a more detailed description of the process of the method provided in this implementation. Please refer to [link / reference]. Figure 2 This is another flowchart illustrating the photo display method in this application embodiment.

[0069] After step S106, the following steps may also be performed, which are not limited here:

[0070] S201. Extract the amplitude energy of the micro-beat point at the intersection of adjacent preceding and subsequent photos in the candidate display interval.

[0071] In this context, "time sequence" refers to the linear timeline in which photos are played sequentially in the final generated display sequence. A preceding photo is the digital image currently at its reference position in the time sequence, such as the first photo being displayed in a carousel. A subsequent photo is the digital image that immediately follows the preceding photo in the time sequence, such as the second photo about to appear. The boundary between candidate display intervals refers to the critical point between the end of the preceding photo display and the beginning of the subsequent photo display, indicating the specific time gap between the two photos. A micro-beat point refers to a more refined, instantaneous rhythmic beat or melodic accent extracted from the music to be played, used to precisely locate the instantaneous burst of musical energy. Amplitude energy is a numerical value that quantifies the audio signal intensity at a specific moment in time, used to represent the loudness and impact of the music at that instant; for example, the amplitude energy value is higher at a heavy bass beat.

[0072] Specifically, the photo display device iterates through adjacent preceding and subsequent photos in chronological order, precisely locating the timestamps at the boundary between these two photos and the candidate display interval. Then, the device invokes the audio processing module to perform high-precision audio signal analysis on the music segment to be played corresponding to that timestamp, extracting the microscopic beat points that are precisely located or extremely close to that boundary. By calculating the sound wave waveform of these microscopic beat points, the device obtains their instantaneous amplitude energy, thus transforming the abstract impact of musical drumbeats into concrete quantitative indicators that can be used for subsequent transition calculations.

[0073] S202. Calculate the numerical difference in visual complexity between the preceding and subsequent photos to obtain the visual jump between adjacent images.

[0074] Visual complexity refers to a quantitative indicator calculated based on the pixel color values, texture details, or content richness of a photograph. It represents the degree of visual complexity of a single photograph; for example, a colorful landscape photograph with numerous elements has high visual complexity. Numerical difference refers to the mathematical difference in visual complexity between preceding and subsequent photographs, used to quantify the difference in the intrinsic visual features of the two photographs. Adjacent-image visual jump is a comprehensive numerical value that reflects the degree of visual contrast and abrupt change when switching from one photograph to another, used to assess the visual impact that viewers may experience at the moment of image transition.

[0075] Specifically, the photo display device reads the image feature data of the preceding and subsequent photos, and uses image processing algorithms to accurately calculate the visual complexity of each photo. Next, the device subtracts or compares the visual complexity of the preceding and subsequent photos to obtain the numerical difference. This difference, after standardization, is defined as the visual jump between adjacent frames. This visual jump objectively reflects the drastic change in visual information during frame transitions; a larger jump indicates a stronger visual contrast between the two photos, requiring a specific transition to smooth the transition.

[0076] S203. The amplitude energy of the micro-beat point is positively mapped to the visual jump of adjacent images to determine the transition motion effect type and transition duration when switching from the previous photo to the subsequent photo.

[0077] Among them, forward mapping refers to a mathematical association rule that ensures that an increase or decrease in the input variable will lead to an increase or decrease in the output variable in the same direction, thus ensuring a positive correlation between audiovisual features and transition effects. Transition animation type refers to the specific visual transition animation style used during photo switching, used to enrich the expressiveness of the image, such as fade-in / fade-out, flash white, push-pull, or dissolve effects. Transition duration refers to the length of time the above transition animation takes to execute, used to control the speed and rhythm of the image transition, for example, set to 0.5 seconds or 1 second.

[0078] Specifically, the photo display device uses the amplitude energy of the extracted micro-beat points and the calculated visual jump between adjacent images as input parameters, and performs comprehensive calculations using a preset forward mapping algorithm. When the amplitude energy is large and the visual jump is high, a more visually impactful and faster-paced transition effect (such as "flash white" or "rapid push-pull") is matched, with a shorter and crisp transition duration. Conversely, when the amplitude energy is weak and the visual jump is low, a softer transition effect (such as "slow fade in and fade out") is matched, with a longer transition duration. Through this dynamic mapping mechanism, the photo display device achieves intelligent and personalized customization of transition effects.

[0079] S204. Based on the transition animation type and transition duration, insert a transition rendering frame between the end of the display of the preceding photo and the start of the display of the subsequent photo.

[0080] The transition effect type refers to a specific animated transition effect used to connect two photos. The transition duration refers to the time span required for the animated transition effect to complete from start to finish. The end time of the display refers to the precise point in time when the preceding photo should have disappeared from the screen without the transition. The start time of the display refers to the precise point in time when the subsequent photo should have begun to appear on the screen. Transition rendering frames refer to a series of intermediate images generated in real-time using computer graphics algorithms based on the selected transition effect type, used to fill the visual gap between two still photos; for example, a fade-in / fade-out process may include multiple gradient images with different transparency levels of the two photos.

[0081] Specifically, the photo display device precisely locates the end time of the preceding photo's display and the start time of the subsequent photo's display. Using these two times as a benchmark, and combining them with the calculated transition duration, a dedicated transition time window is created on the timeline. Within this transition time window, the photo display device invokes the graphics rendering engine, strictly following the algorithm logic of the defined transition animation type, to calculate and generate transition rendering frames frame by frame that integrate the pixel features of the preceding and subsequent photos. These transition rendering frames are smoothly inserted between the preceding and subsequent photos, thus eliminating abrupt scene transitions and making the final output photo display sequence visually coherent and natural.

[0082] The photo display device in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference]. Figure 3 This is a schematic diagram of the physical device structure of a photo display device in the embodiments of this application.

[0083] It should be noted that, Figure 3 The structure of the photo display device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0084] like Figure 3 As shown, the photo display device includes a CPU 301, which can perform various appropriate actions and processes according to a program stored in the read-only memory ROM 302 or a program loaded from the storage section 308 into the random access memory RAM 303, such as performing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0085] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0086] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by CPU 301, it performs the various functions defined in the present invention.

[0087] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0089] Specifically, the photo display device in this embodiment includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, it implements the photo display method provided in the above embodiment.

[0090] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the photo display device described in the above embodiments; or it may exist independently and not assembled into the photo display device. The storage medium carries one or more computer programs that, when executed by a processor of the photo display device, cause the photo display device to implement the photo display method provided in the above embodiments.

[0091] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0092] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0093] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for displaying photos, characterized in that, Applied to a photo display device, the method includes: Obtain the shooting timestamp, location latitude and longitude, and pixel color value of each photo in the set of photos to be displayed, as well as the rhythm intensity and segment time interval of each music segment in the music to be played; The photos are divided into multiple theme clusters according to the shooting timestamp and the location latitude and longitude; the time interval between the shooting timestamps of any two adjacent photos in the theme cluster is less than a preset time threshold and the distance between the location latitude and longitude is less than a preset distance threshold. Based on the pixel color values ​​of all photos within the subject cluster, the color feature vector of the subject cluster is determined to calculate the inter-cluster switching intensity between each pair of subject clusters. Based on the rhythmic intensity of the musical segment, determine the inter-segment transition intensity from the previous musical segment to the next musical segment; By using the inter-segment switching intensity and the inter-cluster switching intensity, a correspondence between music segments and theme clusters is established, so as to match each theme cluster to the segmented time interval of the corresponding music segment; Based on the duration of the music segment and the number of photos within the theme cluster, the start and end times for displaying each photo are assigned to generate a photo display sequence synchronized with the rhythm of the music to be played.

2. The method according to claim 1, characterized in that, The step of determining the color feature vector of the theme cluster based on the pixel color values ​​of all photos within the theme cluster specifically includes: Extract the dominant color tone and color richness of each photo within the theme cluster; Based on the shooting timestamp order of the photos within the theme cluster, the gradient of the change in the dominant color value between adjacent photos is calculated to obtain the dynamic color evolution characteristics of the theme cluster. Based on the statistical distribution of color richness of all photos within the subject cluster, the static basic color characteristics of the subject cluster are determined. The dynamic color evolution features and the static basic color features are concatenated to generate the color feature vector of the theme cluster.

3. The method according to claim 2, characterized in that, The calculation of inter-cluster switching strength between any two topic clusters specifically includes: Based on the static basic color characteristics of the first theme cluster, the visual inertia weight of the first theme cluster is determined, wherein the first theme cluster is any one of the plurality of theme clusters; Calculate the feature difference degree between the dynamic color evolution features of the first theme cluster and the static basic color features of the second theme cluster, wherein the second theme cluster is any one of the plurality of theme clusters other than the first theme cluster; The inter-cluster switching intensity of the transition from the first topic cluster to the second topic cluster is obtained by weighting the feature difference degree with the visual inertia weight.

4. The method according to claim 1, characterized in that, The process of establishing the correspondence between music segments and theme clusters through the inter-segment switching intensity and the inter-cluster switching intensity specifically includes: The average of the shooting timestamps of all photos within the subject cluster is calculated to generate a time centroid, and a subject cluster sequence is generated according to the chronological order of the time centroids. Determine the music segment sequence based on the music to be played; Based on the theme cluster sequence and the music segment sequence, a matching matrix is ​​constructed with the theme cluster as the first dimension and the music segment as the second dimension; Calculate the numerical difference between the inter-cluster switching intensity and the inter-segment switching intensity corresponding to each node in the matching matrix, and determine the numerical difference as the matching cost of the corresponding node; Under the constraint of maintaining the first and second dimensions in an increasing order, find the connected path in the matching matrix that minimizes the cumulative matching cost from the starting node to the ending node; Based on the coordinates of the nodes traversed by the connected path, a mapping relationship is established between the music segment and the theme cluster.

5. The method according to claim 1, characterized in that, The process of establishing the correspondence between music segments and theme clusters through the inter-segment switching intensity and the inter-cluster switching intensity specifically includes: Establish a correspondence between the first theme cluster in the pre-set time sequence and the first music segment of the music to be played, and take the first theme cluster as the current theme cluster and the first music segment as the current music segment; The inter-segment switching intensity of the current music segment transitioning to the next music segment in the time sequence is obtained and used as the target switching intensity. In the multiple candidate topic clusters following the current topic cluster, the inter-cluster switching intensity between the current topic cluster and each candidate topic cluster is calculated respectively; The candidate topic cluster corresponding to the inter-cluster handover intensity with the smallest difference from the target handover intensity value is selected as the target topic cluster; Establish a correspondence between the target theme cluster and the next music segment, update the target theme cluster to the current theme cluster, update the next music segment to the current music segment, and repeat the matching steps until the correspondence between all music segments and theme clusters is established.

6. The method according to claim 1, characterized in that, The process of allocating the start and end times for displaying each photo based on the duration of the music segment and the number of photos within the theme cluster specifically includes: Obtain the target music segment that has a mapping relationship with the target theme cluster, and extract the micro beat points within the target music segment; Using the micro-beat points as the time segmentation benchmark, the target music segment is divided into multiple candidate display intervals of varying lengths; The visual complexity of each photo is calculated based on the pixel color values ​​of each photo within the target subject cluster. Based on the matching principle of visual complexity from high to low and candidate display duration from long to short, the photos within the target theme cluster are sequentially assigned to the corresponding candidate display intervals to determine the start and end times of each photo's display.

7. The method according to claim 6, characterized in that, After the step of allocating the start and end times for displaying each photo based on the duration of the music segment and the number of photos within the theme cluster, the method further includes: Extract the amplitude energy of the micro-beat point at the boundary of the candidate display interval between adjacent preceding and subsequent photos in chronological order; The visual complexity difference between the preceding and subsequent photos is calculated to obtain the visual jump between adjacent images. The amplitude energy of the micro-beat point is positively mapped to the visual jump degree of the adjacent image to determine the transition motion effect type and transition duration when switching from the preceding photo to the subsequent photo; Based on the transition animation type and the transition duration, a transition rendering frame is inserted between the end of the display of the preceding photo and the start of the display of the subsequent photo.

8. A photo display device, characterized in that, The photo display device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the photo display device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the photo display device, the photo display device performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on the photo display device, the photo display device performs the method as described in any one of claims 1-7.

Images