Photo album recommendation processing method and device based on film and television content and terminal

Abstract

The invention discloses a photo album recommendation processing method and device based on movie and television content and a terminal, and the method comprises the steps: analyzing the current movie and television content, recognizing and extracting key elements including scene features and place information, and storing the key elements as movie and television content label data; traversing and searching a user photo album of the intelligent terminal, analyzing photos and videos in the user photo album, identifying and extracting content elements including scene features and shooting place information in the photos and the videos, and converting the content elements into photo album content label data; calculating the correlation degree between the film and television content label data and the photo album content label data, and screening out photo album photos and videos with the correlation degree reaching a preset value for recommendation and display. New functions are added to the intelligent terminal, the function of effectively associating and recommending information such as movie and television content and the personal photo album can be achieved, multi-dimensional association recommendation is achieved, and photo album browsing experience with more individuation and situation relevance is provided for a user.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent terminals, and particularly to an album recommendation processing method, device, intelligent terminal and storage medium based on video content. Background Art

[0002] With the development of technology and the continuous improvement of people's living standards, the use of various intelligent terminals such as smart phones and smart tablets has become more and more popular. At present, many intelligent terminals have an album recommendation function.

[0003] The album recommendation technology of existing intelligent terminals is relatively single, mainly based on time - dimension recommendations such as "Today in the Past Year", lacking in - depth mining of photo content and correlation analysis with other relevant elements. This single - dimension recommendation method cannot meet users' needs for personalized and context - relevant album recommendations, cannot effectively arouse users' emotional resonance and memories, resulting in users easily feeling bored during the album browsing process, and it is difficult to improve the usage frequency of the album and user activity.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an album recommendation processing method, device, intelligent terminal and storage medium based on video content in view of the problems and defects of the above - mentioned existing technology. The present invention can effectively associate and recommend materials such as video content with personal albums, realizing multi - dimensional associated recommendations, providing users with a more personalized and context - relevant album browsing experience, and facilitating users' use.

[0006] The technical solution adopted by the present invention to solve the problem is as follows: An album recommendation processing method based on video content, which includes: Obtain the currently played video content, parse the video content, identify and extract key elements including scene features and location information, and save the key elements as video content tag data; Traverse and search the user album of the intelligent terminal, parse the photos and videos in the user album, identify and extract content elements including scene features and shooting location information in the photos and videos, and convert the content elements into album content tag data; Compare the video content tag data with the album content tag data, calculate the correlation degree between the video content tag data and the album content tag data, and screen out album photos and videos with a correlation degree reaching a predetermined value; Recommend and display the screened - out album photos and videos with a correlation degree reaching a predetermined value.

[0007] The described photo album recommendation processing method based on video content, wherein the steps of obtaining the currently playing video content, parsing the video content, identifying and extracting key elements including scene features and location information, and saving the key elements as video content tag data include: Detecting the playing of video content and obtaining the currently playing video content; Using image recognition and scene analysis technologies to parse the video content, identifying and extracting key elements including scene features and location information in the video content, wherein the scene features include fireworks features, competition features, and / or concert features; the location information includes geographical coordinates and / or landmark buildings; Saving the key elements including scene features and location information extracted from the video content as tag data of the video content.

[0008] The described photo album recommendation processing method based on video content, wherein the steps of obtaining the currently playing video content, parsing the video content, identifying and extracting key elements including scene features and location information, and saving the key elements as video content tag data include: When detecting the playing of video content, extracting the key frames in the video content; Parsing the key frames extracted from the video content, parsing the scene information in the key frames, and parsing the location information in the key frames; Performing data cleaning on the parsed scene information and location information in the key frames and saving them as video content tag data, which is stored in the video content database.

[0009] The described photo album recommendation processing method based on video content, wherein the steps of traversing and searching the user album of the smart terminal, parsing the photos and videos in the user album, identifying and extracting content elements including scene features and shooting location information in the photos and videos, and converting the content elements into album content tag data include: Detecting whether the smart terminal allows authorized access to the album function; When detecting that the smart terminal allows authorized access to the album function, traversing and searching the user album of the smart terminal; Reading the photos in the user album; Parsing the photos in the user album, parsing the scene information in the photos, and parsing the location information in the photos; Performing data cleaning on the parsed scene information and location information in the photos, and extracting content elements including scene features and shooting location information of the photos; Convert the content elements into album content tag data and store them in the album content database.

[0010] The above-mentioned album recommendation processing method based on video content, wherein the step of traversing and searching the user album of the smart terminal, parsing the photos and videos in the user album, identifying and extracting content elements including scene features and shooting location information in the photos and videos, and converting the content elements into album content tag data includes: Detect whether the smart terminal allows authorized access to the album function; When it is detected that the smart terminal allows authorized access to the album function, traverse and search the user album of the smart terminal; Read the videos in the user album; Parse the videos in the user album, extract key frames from the videos, parse the scene information in the key frames, and parse the location information in the key frames; Clean the data of the scene information and location information in the parsed key frames, and extract content elements including the scene features and shooting location information of the videos; Convert the content elements into album content tag data and store them in the album content database. The above-mentioned album recommendation processing method based on video content, wherein the step of recommending and displaying the album photos and videos with the correlation degree reaching a predetermined value includes: For the album photos and videos with the correlation degree reaching a predetermined value screened out, use a collaborative filtering algorithm based on user behavior data or a recommendation algorithm based on content semantic understanding to further optimize the recommendation results.

[0011] The above-mentioned album recommendation processing method based on video content, wherein after the step of recommending and displaying the album photos and videos with the correlation degree reaching a predetermined value, it further includes: Control the display of the recommended album content, and at the same time receive user feedback and interactive operation data; Based on the received user feedback and interactive operation data, further optimize the recommendation algorithm.

[0012] An album recommendation processing device based on video content, wherein the device includes: A video content parsing module, configured to obtain the currently played video content, parse the video content, identify and extract key elements including scene features and location information, and save the key elements as video content tag data; An album content parsing module, which is used to traverse and search the user album of the smart terminal, parse the photos and videos in the user album, identify and extract content elements including scene features and shooting location information in the photos and videos, and convert the content elements into album content tag data; An association comparison module, which is used to compare the film and television content tag data with the album content tag data, calculate the association degree between the film and television content tag data and the album content tag data, and screen out the album photos and videos with the association degree reaching a predetermined value; A recommendation engine module, which is used to recommend and display the album photos and videos with the association degree reaching a predetermined value screened out.

[0013] A smart terminal, which includes a memory, and one or more programs, wherein one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include methods for executing any one of the above.

[0014] A computer-readable storage medium, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any one of the above methods.

[0015] Advantages of the present invention: The present invention provides a method, device, smart terminal and storage medium for album recommendation processing based on film and television content. Aiming at the problem of single existing album recommendation method, the present invention provides an album recommendation method based on film and television content. By parsing and extracting elements such as scenes and locations of film and television content, and performing the same analysis on user album photos, album photos related to the film and television are found through comparison and matching and recommended to users. The present invention adds a new function to the smart terminal: it can realize the function of effectively associating and recommending materials such as film and television content with personal albums, realizes multi-dimensional association recommendation, and provides a more personalized and contextually relevant album browsing experience for users.

[0016] The present invention solves the problem of single existing album recommendation source. By combining multi-dimensional features of film and television content watched by users and album photos for matching recommendation, a richer, more personalized and contextual album usage experience is provided for users. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments recorded in the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 is a schematic flowchart of the album recommendation processing method based on video content provided in Embodiment 1 of the present invention.

[0019] Figure 2 is a schematic diagram of video content analysis of the album recommendation processing method based on video content provided in Embodiment 2 of the present invention.

[0020] Figure 3 is a schematic flowchart of album content analysis of the album recommendation processing method based on video content provided in Embodiment 2 of the present invention.

[0021] Figure 4 Principle block diagram of the album recommendation processing device embodiment based on video content provided by the present invention.

[0022] Figure 5 is a schematic diagram of the internal structure principle of the intelligent terminal provided in the embodiment of the present invention. Detailed implementation manners

[0023] To make the objectives, technical solutions and advantages of the present invention clearer and more explicit, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0024] It should be noted that if there are directional indications (such as up, down, left, right, front, back,...) involved in the embodiments of the present invention, such directional indications are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0025] Regarding the album recommendation technology of existing intelligent terminals, it is relatively single, mainly based on time - dimension - based recommendations such as "Today in History", lacking in - depth mining of photo content and correlation analysis with other relevant elements. This single - dimension recommendation method cannot meet users' needs for personalized and context - relevant album recommendations, cannot effectively evoke users' emotional resonance and memories, resulting in users easily feeling bored during the album browsing process, and it is difficult to improve the usage frequency of the album and user activity.

[0026] Therefore, the present invention provides an album recommendation processing method based on video content. The present invention extracts elements such as scenes and locations from video content through analysis, and performs the same analysis on users' album photos. Through comparison and matching, album photos related to the video are found and recommended to users, realizing multi - dimensional associated recommendations and providing users with a more personalized and context - relevant album browsing experience.

[0027] Such as Figure 1As shown in the figure, a method for processing album recommendations based on video content in Embodiment 1 of the present invention includes the following steps: Step S100: Obtain the currently played video content, parse the video content, identify and extract key elements including scene features and location information, and save the key elements as video content tag data; In an embodiment of the present invention, when a user watches a video using a mobile terminal, the currently played video content is obtained; the first step in this process is to obtain in real time the video content currently watched by the user, including movies, TV series, short videos, etc. This can be achieved through an API (Application Programming Interface) integrated in the playback device or application.

[0028] For example, when the user is watching the movie "A Dream Space", this step will capture information such as the name of the movie, the playing time, and the background music.

[0029] Then, the video content is parsed. In an embodiment of the present invention, parsing is a process of analyzing and processing the obtained video content. Important information such as scene features and location information is extracted from the video data for understanding the scene information of the video content.

[0030] Then, in an embodiment of the present invention, key elements are identified and extracted, that is, on the basis of parsing, the present invention further identifies and extracts key elements in the content, including scene features (such as rainy days, cities, forests, etc.), location information (such as "City A", "City B"), and important characters and actions, etc.

[0031] For example, for "A Dream Space", the extracted key elements include: Scene features: night, dream, real world; Location information: Paris, Los Angeles, hotel, etc.; Character information: A Dominic Cobb (protagonist), B Ariadne (architect), etc.

[0032] Then, the key elements are saved as video content tag data. In an embodiment of the present invention, finally, the above-extracted key elements are sorted out and saved as tag data for subsequent correlation comparison of data features.

[0033] It can be seen that in an embodiment of the present invention, after the video feature data is tagged, the matching efficiency is improved.

[0034] In a further embodiment of the present invention, the step S100 specifically includes: S101: Detect that the video content is being played, and obtain the currently played video content; S102. Analyze the film and television content using image recognition and scene analysis techniques, identify and extract key elements including scene features and location information in the film and television content, where the scene features include, but are not limited to, fireworks features, competition features, and / or concert features; the location information includes geographical coordinates and / or landmark buildings; This process uses image recognition and scene analysis techniques to deeply analyze the film and television content. Image recognition can identify objects, colors, actions, etc. in the scene, while scene analysis helps to understand the relationships between these objects and the overall environment. In this way, certain key information can be extracted, including scene features (such as features of fireworks, competitions, concerts, etc.) and location information (such as specific geographical coordinates and landmark buildings).

[0035] Example: Take a movie containing a music performance as an example: Scene features: Through image recognition, the system can detect fireworks effects, stage layouts, and audience emotions that appear in a specific scene.

[0036] Location information: Similarly, the system can analyze that the scene may be shot at a well-known venue, extract landmark buildings such as "Central Park" or "Red Square", and their geographical coordinates (for example: latitude and longitude).

[0037] S103. Save the key elements including scene features and location information in the extracted film and television content as label data of the film and television content.

[0038] After extracting the key elements in the film and television content, the system organizes and saves this information as label data. These label data facilitate subsequent retrieval, recommendation, and content management. Tagging the extracted information helps to enhance the user experience and the discoverability of the content.

[0039] Step S200. Traverse and search the user album of the smart terminal, analyze the photos and videos in the user album, identify and extract content elements including scene features and shooting location information in the photos and videos, and convert the content elements into album content label data; In the embodiment of this step, the user album of the smart terminal will be traversed and searched. By accessing the album in the user smart terminal, the stored photos and videos can be obtained. Specifically, when implementing, the user needs to be asked to enable the corresponding access permission to ensure the user's privacy and data security.

[0040] In the embodiment of the present invention, the photos and videos in the user album will be analyzed. The specific analysis is to analyze the obtained photos and videos to identify their internal structures and features. Computer vision and image processing algorithms can be used to extract the details and information of the images.

[0041] Based on the analysis, the present invention identifies and extracts important content elements, including scene features (such as sunny day, dusk), shooting location information (such as "Maldives Beach", "Huangshan Mountain"), etc. This step usually relies on machine learning models to ensure accuracy and comprehensiveness.

[0042] For example, for photos taken during a trip, the extracted content elements may include: Scene features: sunny day, sunrise, beach.

[0043] Shooting location information: Maldives, mountain climbing point, a famous scenic spot, etc.

[0044] Then, the embodiment of the present invention will convert the content elements into album content tag data, that is, organize the identified and extracted content elements into tag data for convenient feature correlation comparison in the subsequent steps.

[0045] As can be seen from the above, in the embodiment of the present invention, based on the tag data, it is convenient for the subsequent steps to recommend photos and videos of similar scenes or themes to the user, improving the user experience. By tagging the scenes and locations, the user can more easily recall the situation and story at the time of shooting, increasing the emotional connection to the photos.

[0046] Step S300: Compare the video content tag data with the album content tag data, calculate the correlation degree between the video content tag data and the album content tag data, and filter out the album photos and videos whose correlation degree reaches a predetermined value; In the embodiment of the present invention, the video content tag data will be compared with the album content tag data. Specifically, the tags in the video works (such as scene features, location information, etc.) will be matched with the tags of the photos and videos in the user's album to find the similarity between the two. The present invention can use algorithms to identify and calculate the similarity of these two sets of data.

[0047] For example, when the tag data of the video content is as follows: json { "title": "Dialogue", "scenes": ["dusk", "city street"], "locations": ["New York Tower Bridge", "Central Park"], "characters": ["John", "Mary"] } ; And the tag data of some photos in the user's album may be: json { "album_title": "Journey to New York", "elements": { "scenes": ["Dusk", "Night view"], "locations": ["New York Tower Bridge", "Times Square"], "other_tags": ["Travel", "City"] } }; In this process, the embodiments of the present invention will compare the similarity of information such as "Dusk" and "New York Tower Bridge" in the film and television tags and album tags; to calculate the correlation between the film and television content tag data and the album content tag data; after comparison, calculate the correlation of the two groups of tags. The correlation can be obtained through different algorithms (such as cosine similarity, Jaccard index, etc.). This metric reflects the correlation between the content in the album and the film and television content.

[0048] For example, in the previous example, if two identical tags "Dusk" and "New York Tower Bridge" are found, the calculated correlation may be a score, for example: the correlation is 0.8, indicating a relatively high correlation between the two.

[0049] Then the embodiments of the present invention will screen out the album photos and videos with the correlation reaching the predetermined value; that is, the present invention, based on the set correlation threshold (for example, >0.7), screens out the eligible photos and videos from the user's album. This process ensures that users can find the media related to the selected film and television content.

[0050] For example, when the correlation reaches the predetermined value, that is, the set threshold is 0.7, then after calculation, the album content (photo of dusk in Central Park) with a correlation of 0.8 will be retained for the user to view.

[0051] As can be seen from the above, the embodiments of the present invention, by connecting the film and television content with the photos and videos in the user's album, can provide more personalized recommendations, helping users recall the experiences and memories related to a certain film and television work. And it improves the intelligent management of the user's album; users can organize and classify their photos and videos through the associated film and television content, making the album more interesting and rich. And extracting the photos and videos related to the film and television works can help users better connect to their memories. Seeing personal photos related to the scenes in a certain movie or program may trigger wonderful memories of travel or events.

[0052] Step S400: Recommend and display the album photos and videos with the correlation reaching the predetermined value.

[0053] In the embodiments of the present invention, the album photos and videos with a correlation degree reaching a predetermined value are screened out and recommended for display. Specifically, after the correlation degree calculation is completed in the previous steps, the photos and videos screened out and related to the film and television content that the user is currently interested in (for example, the movie they are watching) will be integrated and presented to the user in the form of a recommendation. Such a recommendation can appear in the form of a pop-up window, a sidebar, or a recommendation page when the user uses the album or the streaming media platform.

[0054] For example, when the user is watching the movie "Night View of New York", through the previous steps, album photos with a high correlation degree with this movie are found, such as the photo of "New York Tower Bridge" taken at dusk and the video recorded at "Times Square". In this step, in the embodiments of the present invention, these contents can be sorted out and displayed as recommended contents, perhaps presented in the form of a slide show on the side of the screen, accompanied by short explanatory texts, such as "Memories Related to 'Night View of New York'".

[0055] Specifically, there can be various display methods for the recommendation in the embodiments of the present invention. For example: Recommendation card: Display the relevant photos and videos in the form of a card, with a movie poster, a thumbnail, and a simple description.

[0056] Carousel: Display a swipable carousel on the home page of the application, which contains relevant album media and can be quickly browsed.

[0057] Notification or pop-up window: Automatically pop up a window with recommended contents when the user views the movie, so that the user can also see the relevant album contents while watching the movie.

[0058] As can be seen from the above, the recommendation of relevant photos and videos in the album in the embodiments of the present invention not only increases the interactivity of the user, but also provides a richer experience for the user, making them more willing to participate in content exploration. And when the user sees the photos and videos related to the movie they are watching, they can evoke good memories and further deepen the emotional connection with the album contents. Moreover, the recommendation mechanism helps the user quickly obtain the content related to their interests and improves user satisfaction. In the vast amount of content, personalized recommendation is particularly important.

[0059] In other embodiments of the present invention, the step S100 specifically further includes: S111. When detecting the playing of film and television content, extract the key frames in the film and television content; In the embodiments of this step, specific pictures (i.e., key frames) will be detected and extracted from the playing film and television content. A key frame refers to a picture that can represent an important scene or a turning point of the plot.

[0060] For example, in a love movie, important key frames can be the frames of the protagonist in certain tourist scenes, or close-ups of certain city landmark buildings. These pictures can effectively identify the photo-taking scenes.

[0061] S112. Analyze the key frames in the extracted video content, analyze the scene information in the key frames, and analyze the location information in the key frames; In the embodiment of this step, in-depth analysis will be performed on the extracted key frames to extract scene information (such as action scenes, emotional atmospheres, etc.) and location information (such as cities, buildings, etc.).

[0062] For example, following the above example, if the key frames show frames of certain tourist scenes or close-ups of certain city landmark buildings, the present invention can analyze the scene information as "a certain tourist scene - green mountains" and the location information as "a certain city".

[0063] Through analysis in this step, the scene information and location information of the video content can be understood more accurately, promoting the generation of richer metadata, which is very useful for subsequent analysis of associated data.

[0064] S113. Clean the scene information and location information in the parsed key frames and save them as video content tag data, which is stored in the video content database.

[0065] In the embodiment of this step, the parsed scene and location information will be cleaned to remove redundant and duplicate information, and the final result will be generated as tag data, which is then stored in the video content database.

[0066] For example, if the same scene information already exists in the database, the present invention will automatically clean the duplicate information and only retain the unique tag data. Data cleaning can improve the effectiveness of information storage, ensure that the data in the database is the latest and most accurate, and optimize the data retrieval speed.

[0067] In a further embodiment of the present invention, the step S200 specifically includes: S201. Detect whether the smart terminal allows authorized access to the album function; That is, in the embodiment of the present invention, to perform album association, the function of allowing authorized access to the album needs to be turned on; S202. When it is detected that the smart terminal allows authorized access to the album function, traverse and search the user album of the smart terminal; In the present invention, the user album will be searched in real time.

[0068] S203. Read the photos in the user album; In this step, the basic element information of each photo is read from the user's photo album. These elements usually include metadata such as the photo file name, shooting time, scene information, and location information.

[0069] Example: If the user has a photo named "2023-01-01_Holiday.jpg", the element information of this photo will be read, such as the shooting time is January 1, 2023, the resolution is 1920x1080, the format is JPEG, the scene information is a fireworks scene, and the location information is a tourist attraction in a certain city.

[0070] The element information read in this step lays the foundation for subsequent analysis and processing, enabling the system to accurately identify and process different photos. It also provides an effective basis for photo sorting and organization.

[0071] S204. Analyze the photos in the user's photo album, analyze the scene information in the photos, and analyze the location information in the photos; In the embodiment of this step, the read photos will be deeply analyzed to extract the scene information (such as people, actions, situations, etc.) and location information (such as cities, buildings, etc.) in the photos.

[0072] Example: Taking a user's photo as an example, if the photo shows a family on a beach vacation, the present invention can analyze the scene information as "family gathering" and "beach", while the location information may be shown as "Hawaii".

[0073] In this way, the information analyzed by the present invention can not only help users better organize and recall the photo background, but also provide support for subsequent generation of personalized tags and memories.

[0074] S205. Clean the data of the scene information and location information in the analyzed photos, and extract content elements including the scene features of the photos and the shooting location information; In the embodiment of this step, the data of the analyzed scene and location information will be cleaned to delete redundant and duplicate information, and the core features of the photos and the shooting location information will be extracted.

[0075] For example, if during the analysis process, the scene information repeatedly involves "family gathering", the present invention will identify it as duplicate and retain the unique label. The final extracted content may be the scene features "family gathering", "beach", and the shooting location "Hawaii".

[0076] This step ensures that the information stored in the database is accurate and concise through data cleaning, avoids the appearance of duplicate tags, and thus improves the efficiency of search and retrieval.

[0077] S206. Convert the content elements into album content tag data and store them in the album content database.

[0078] In the embodiment of this step, the content elements obtained through cleaning and extraction are converted into tag data of the album content and stored in the album content database. For example, the above-extracted content features and location information are converted into tags: such as "Family Gathering", "Beach", "Hawaii", and these tags are stored in association with the corresponding photos.

[0079] In this way, the album has stronger searchability and manageability. Users can quickly find specific photos through tags, which improves the user experience and also facilitates association in subsequent steps, enhancing the association efficiency.

[0080] In another embodiment of the present invention, step S200 specifically further includes: S211. Detect whether the smart terminal allows authorized access to the album function; In the embodiment of this step, it is checked whether the smart terminal (such as a smartphone or a tablet) allows the application to access the user's album function, as this involves the permission settings of user privacy.

[0081] For example, in the specific implementation of the present invention, before accessing the album, it is queried whether the user has selected to allow the album recommendation function to access the album in the settings. If the user has not authorized, the user will be prompted to authorize access to the album.

[0082] In this way, it can ensure that the user's privacy is respected and protected, and the user can decide which applications can access their personal data.

[0083] S212. When it is detected that the smart terminal allows authorized access to the album function, traverse and search the user album of the smart terminal; In the embodiment of this step, when user authorization is detected, the user's album in the smart terminal is searched, and the video files therein are obtained.

[0084] S213. Read the videos in the user album; In the embodiment of this step, the basic element information of each video is read from the user album, including file name, duration, resolution, format, scene information, location information, etc.

[0085] For example: for the video "2023_Hawaii_Trip.mp4", in the embodiment of the present invention, its information will be read, such as the duration is "2 minutes and 30 seconds", the resolution is "1920x1080", the format is "MP4", the scene information such as "Beach", "Sunset", and the location information is "Hawaii".

[0086] The advantage of this step is that it can ensure that the system accurately records the basic metadata of the video, providing the necessary information for subsequent analysis and classification.

[0087] S214. Analyze the videos in the user's album, extract the key frames in the videos, analyze the scene information in the key frames, and analyze the location information in the key frames; In the embodiment of the present invention, the videos read from the user's album are analyzed, key frames are extracted, and the scene information and location information in the key frames are analyzed.

[0088] Example: When analyzing "2023_Hawaii_Trip.mp4", the key frames that can be extracted in the embodiment of the present invention include a beach, a sunset, etc. These key frames can be parsed into scene information such as "beach", "sunset", and the location information is "Hawaii".

[0089] In the embodiment of this step, through detailed analysis, important scenes and information in the video can be captured, which helps the subsequent steps for correlation comparison, making it easier for users to recall and find relevant videos.

[0090] S215. Clean the data of the scene information and location information in the parsed key frames, and extract content elements including the scene features of the video and the shooting location information; In the embodiment of this step, the key frame information parsed out will be data-cleaned, redundant information will be removed, and the scene features and shooting location information of the video will be extracted.

[0091] For example: In the embodiment of this step, if it is detected that multiple key frames are marked with the same feature of "Hawaii", it will leave a unique label and extract "beach" and "sunset" as scene features.

[0092] Data cleaning in this step can ensure that the information in the database is concise and efficient, while avoiding confusion caused by duplicate data and improving the efficiency of subsequent retrieval.

[0093] S216. Convert the content elements into album content label data and store them in the album content database. In the embodiment of the present invention, in addition to the photo elements in the album, the content elements extracted from the videos in the photos are also converted into album content label data and stored in the album content database.

[0094] For example, convert the above - cleaned information into tags, such as "beach", "sunset", "Hawaii", and store it in association with the video "2023_Hawaii_Trip.mp4". In the embodiment of this step, this enables users to quickly find specific videos through tags, improving the operability of the photo album and the user experience. When users want to review videos of specific scenes or locations, they can more easily find them through relevant tags.

[0095] In a further embodiment of the present invention, the step S400 specifically further includes: S401. For the selected album photos and videos with the degree of association reaching a predetermined value, use a collaborative filtering algorithm based on user behavior data or a recommendation algorithm based on content semantic understanding to further optimize the recommendation results.

[0096] In the embodiment of the present invention, for the selected album photos and videos with the degree of relevance to the user's needs reaching a predetermined threshold, two main methods are also used to further optimize the recommendation results: one is a collaborative filtering algorithm based on user behavior data, and the other is a recommendation algorithm based on content semantic understanding.

[0097] Example: When users often search for and view photos and videos about "winter skiing". In this step, the present invention can use a collaborative filtering algorithm to analyze the behaviors of other similar users, find that they also like albums related to "skiing", and recommend this content to the current user. At the same time, the algorithm of the present invention based on content semantic understanding will analyze the key features of the image content, such as the snow mountains and skiers in the ski resort, so as to recommend skiing videos of similar scenes.

[0098] In the embodiment of this step, by combining user behavior and content understanding, the system can provide more personalized and accurate recommendations, improve user satisfaction and participation, and avoid the occurrence of ineffective recommendations.

[0099] In a further embodiment of the present invention, after the step S400, it further includes: S501. Control the display of the recommended album content, and at the same time receive user feedback and interactive operation data; In the embodiment of this step, the optimized recommended album content is displayed to the user, and at the same time, user feedback and interactive operation data are received. This may include user clicks on the recommended content, marking of preferences, sharing behaviors, etc.

[0100] For example, when the present invention displays multiple photos and videos related to "skiing". When the user browses and sees some content of interest, such as an exciting skiing video, the user clicks on it or marks this video with the "like" button. Through the real-time feedback of the user, this step can directly obtain the preferences of the user for the recommended content, which provides an important basis for subsequent recommendations and helps the system better understand the user's preferences and needs.

[0101] S502. Further optimize the recommendation algorithm based on the received user feedback and interaction operation data.

[0102] In the embodiment of this step, based on the received user feedback and interaction operation data, the recommendation algorithm will be further optimized to improve the accuracy and relevance of future recommendations. The embodiment of the present invention will adjust its model to learn the latest preferences of the user.

[0103] For example, if it is found from the user's feedback that they are more inclined to watch a specific skiing video rather than other content, then the present invention will adjust its recommendation algorithm, increase the recommendation ratio of videos related to "winter sports", and at the same time reduce the content recommendation of "summer activities".

[0104] The advantage of this step is that by continuously learning and optimizing, the ability to predict the user's needs can be gradually improved. Over time, the user experience will become more and more personalized, thereby enhancing user stickiness and satisfaction.

[0105] The following further describes the present invention in detail through another specific application embodiment: Such as Figure 2 and Figure 3 shown, a method for processing album recommendations based on video content in a specific application embodiment 3 of the present invention, the steps include: a video content parsing step S11 - S16 for the user to watch the video content, and an album content parsing step S21 - S31: Among them, as Figure 2 shown, the video content parsing steps S11 - S16 include: S11. Start parsing the video content and enter step S12; S12. Extract the key frames of the video and then enter step S13; S13. Parse the scene information in the key frames and enter step S14; S14. Parse the location information in the key frames and enter step S15.

[0106] S15. Clean the obtained information and enter step S16; S16. Store it in the video content database.

[0107] In the above steps, the film and television resources in the media asset library are parsed, key elements are extracted and stored in the film and television content database.

[0108] Further, as Figure 3 shown, in the embodiment of the present invention, the album content parsing step includes: S21. Parse the album content and enter step S22; S22. Read the EXIF information (location, time, etc.) of the photos and videos; and enter S23; Among them, EXIF (Exchangeable Image File Format) information is a metadata format embedded in image files.

[0109] S23. Determine whether the content type read is a photo or a video. When it is a photo, enter step S24; when it is a video, enter S26; S24. Parse the scene information in the photo and enter step S25; S25. Parse the location information in the photo and enter step S29; S26. Extract the key frames of the video and enter step S27; S27. Parse the scene information in the key frame and enter step S28; S28. Parse the location information in the key frame and enter step S29; S29. Clean the obtained information and enter step S30; S30. Store it in the album content database and enter step S31; In the above steps, the photos and videos in the user's album are parsed, key elements are extracted and stored in the album content database.

[0110] S31. Compare the film and television content tag data with the album content tag data, calculate the correlation degree between the film and television content tag data and the album content tag data, and screen out the album photos and videos whose correlation degree reaches a predetermined value; recommend and display the screened album photos and videos whose correlation degree reaches a predetermined value, and then end the process.

[0111] As can be seen from the above, in the specific embodiments of the present invention, first, image recognition and scene analysis technologies (such as the CLIP model) are used to parse the film and television content, identify and extract the scene features (such as fireworks, competitions, concerts, etc.), location information (such as geographical coordinates, landmark buildings, etc.) and other key elements therein, and save this information as the tag data of the film and television content. Then, the same algorithm is used to parse the photos and videos in the user's album, including the scenes in the photos, the shooting location information (geographical location tags in EXIF metadata or inferred through image recognition), etc., and also convert them into tag data.

[0112] After that, through data comparison and matching algorithms, the feature data of the film and television content is accurately compared with the feature data of the album photos, the correlation degree between the two is calculated, and the album photos and videos with a higher correlation degree are selected. Finally, these photos and videos are recommended to the user. The present invention adds a new function to the intelligent terminal: it can realize the function of effectively associating and recommending materials such as film and television content with the personal album, realizing multi-dimensional associative recommendation, and providing a more personalized and contextually relevant album browsing experience for the user.

[0113] In the embodiments of the present invention, the film and television content watched by the user is used as the basis for album recommendation, breaking through the limitations of traditional album recommendation only based on dimensions such as the time and location of the photos and videos taken by the user. Through feature extraction and matching of film and television and album content, multi-dimensional associative recommendation is realized.

[0114] Moreover, the present invention can also display the recommended album content to the user, and at the same time provide user feedback and interaction functions. For example, the user can like, comment on the recommendation result or mark it as uninteresting, so that the system can further optimize the recommendation algorithm.

[0115] The present invention solves the problem of the single source of existing album recommendation. By combining the multi-dimensional features of the film and television content watched by the user and the album photos for matching recommendation, a richer, more personalized and contextual album usage experience is provided for the user.

[0116] Exemplary device As Figure 4 shown, the embodiments of the present invention provide an album recommendation processing device based on film and television content, and the device includes: A film and television content parsing module 310, configured to obtain the currently played film and television content, parse the film and television content, identify and extract key elements including scene features and location information, and save the key elements as film and television content tag data; The album content parsing module 320 is used to traverse and search the user album of the smart terminal, parse the photos and videos in the user album, identify and extract content elements including scene features and shooting location information in the photos and videos, and convert the content elements into album content tag data; The association comparison module 330 is used to compare the film and television content label data with the album content label data, calculate the association between the film and television content label data and the album content label data, and filter out the album photos and videos whose association reaches a predetermined value; The recommendation engine module 340 is used to recommend and display the selected album photos and videos whose relevance reaches a predetermined value, as described above.

[0117] Based on the above embodiments, the present invention further provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 5 As shown. The intelligent terminal includes a processor, a memory, a network interface, a display screen, and a database connected via a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for recommending an album based on film and television content is implemented. The database of the intelligent terminal is used to store an album recommendation processing program based on film and television content.

[0118] Those skilled in the art will understand that Figure 5 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the smart terminal to which the scheme of the present invention is applied. The specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0119] In one embodiment, a smart terminal is provided, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include instructions for performing the following operations: Acquire the currently played film and television content, parse the film and television content, identify and extract key elements including scene features and location information, and save the key elements as film and television content tag data; Traverse and search the user album of the smart terminal, parse the photos and videos in the user album, identify and extract content elements including scene features and shooting location information in the photos and videos, and convert the content elements into album content tag data; Compare the film and television content tag data with the album content tag data, calculate the correlation degree between the film and television content tag data and the album content tag data, and filter out the album photos and videos whose correlation degree reaches a predetermined value; Recommend and display the filtered album photos and videos whose correlation degree reaches a predetermined value, as described above.

[0120] Among them, the steps of obtaining the currently played film and television content, parsing the film and television content, identifying and extracting key elements including scene features and location information, and saving the key elements as film and television content tag data include: Detect the playing of film and television content and obtain the currently played film and television content; Use image recognition and scene analysis technology to parse the film and television content, identify and extract key elements including scene features and location information in the film and television content, where the scene features include fireworks features, competition features, and / or concert features; the location information includes geographical coordinates and / or landmark buildings; Save the key elements including scene features and location information in the extracted film and television content as the tag data of the film and television content.

[0121] Among them, the steps of obtaining the currently played film and television content, parsing the film and television content, identifying and extracting key elements including scene features and location information, and saving the key elements as film and television content tag data include: When detecting the playing of film and television content, extract the key frames in the film and television content; Parse the extracted key frames in the film and television content, parse the scene information in the key frames, and parse the location information in the key frames; Clean the data of the parsed scene information and location information in the key frames and save them as film and television content tag data, which is stored in the film and television content database.

[0122] Among them, the steps of traversing and searching the user album of the smart terminal, parsing the photos and videos in the user album, identifying and extracting content elements including scene features and shooting location information in the photos and videos, and converting the content elements into album content tag data include: Detect whether the smart terminal allows authorized access to the album function; When it is detected that the smart terminal allows authorization to access the album function, traverse and search the user album of the smart terminal; Read the photos in the user album; Analyze the photos in the user album, analyze the scene information in the photos, and analyze the location information in the photos; Perform data cleaning on the analyzed scene information and location information in the photos, and extract content elements including the scene features of the photos and the shooting location information; Convert the content elements into album content label data and store them in the album content database.

[0123] Among them, the steps of traversing and searching the user album of the smart terminal, analyzing the photos and videos in the user album, identifying and extracting content elements including the scene features and shooting location information in the photos and videos, and converting the content elements into album content label data include: Detect whether the smart terminal allows authorization to access the album function; When it is detected that the smart terminal allows authorization to access the album function, traverse and search the user album of the smart terminal; Read the videos in the user album; Analyze the videos in the user album, extract the key frames in the videos, analyze the scene information in the key frames, and analyze the location information in the key frames; Perform data cleaning on the analyzed scene information and location information in the key frames, and extract content elements including the scene features of the videos and the shooting location information; Convert the content elements into album content label data and store them in the album content database, as specifically described above.

[0124] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0125] In summary, the present invention provides a method, apparatus, intelligent terminal, and storage medium for album recommendation processing based on video content. The present invention adds a new function to the intelligent terminal: it can realize the function of effectively associating and recommending materials such as video content with personal albums, achieving multi-dimensional association recommendation, and providing users with a more personalized and contextually relevant album browsing experience. The present invention solves the problem of the single source of existing album recommendations. By combining the multi-dimensional features of the video content watched by the user and the album photos for matching and recommendation, a richer, more personalized, and contextual album usage experience is provided for the user.

Claims

1. A method for recommending albums based on film and television content, characterized in that: include: Parsing the film and television content, identifying and extracting key elements including scene features and location information, and saving the key elements as film and television content tag data; Traverse and search the user album of the smart terminal, parse the photos and videos in the user album, identify and extract content elements including scene features and shooting location information in the photos and videos, and convert the content elements into album content tag data; Comparing the film and television content label data with the album content label data, calculating the correlation between the film and television content label data and the album content label data, and screening out the album photos and videos whose correlation reaches a predetermined value; The selected album photos and videos whose correlation reaches a predetermined value are recommended and displayed.

2. The method for recommending albums based on film and television content according to claim 1, characterized in that: The steps of obtaining the currently played film and television content, parsing the film and television content, identifying and extracting key elements including scene features and location information, and saving the key elements as film and television content tag data include: Detect the playing of video content and obtain the currently playing video content; The film and television content is analyzed by using image recognition and scene analysis technology to identify and extract key elements including scene features and location information in the film and television content, wherein the scene features include fireworks features, competition features and / or concert features; the location information includes geographic coordinates and / or landmark buildings; The extracted key elements of the film and television content, including scene features and location information, are saved as label data of the film and television content.

3. The method for recommending albums based on film and television content according to claim 1, characterized in that: The steps of obtaining the currently played film and television content, parsing the film and television content, identifying and extracting key elements including scene features and location information, and saving the key elements as film and television content tag data include: When it is detected that the video content is being played, key frames in the video content are extracted; Parsing the extracted key frames in the film and television content, parsing the scene information in the key frames, and parsing the location information in the key frames; The scene information in the key frame and the location information in the key frame are cleaned and saved as film and television content tag data, and stored in a film and television content database.

4. The method for recommending albums based on film and television content according to claim 1, characterized in that: The step of traversing and searching the user album of the smart terminal, parsing the photos and videos in the user album, identifying and extracting content elements including scene features and shooting location information in the photos and videos, and converting the content elements into album content tag data includes: Detect whether the smart terminal allows authorized access to the photo album function; When it is detected that the smart terminal allows authorized access to the photo album function, the user photo album of the smart terminal is searched; Read the photos in the user's photo album; Analyze the photos in the user's photo album, analyze the scene information in the photos, and analyze the location information in the photos; Performing data cleaning on the scene information and location information in the analyzed photos, and extracting content elements including scene features of the photos and shooting location information; The content elements are converted into album content tag data and stored in the album content database.

5. The method for processing album recommendations based on film and television content according to claim 1, characterized in that: The step of traversing and searching the user album of the smart terminal, parsing the photos and videos in the user album, identifying and extracting content elements including scene features and shooting location information in the photos and videos, and converting the content elements into album content tag data includes: Detect whether the smart terminal allows authorized access to the photo album function; When it is detected that the smart terminal allows authorized access to the photo album function, the user photo album of the smart terminal is searched; Read the video in the user's photo album; Parsing the video in the user's album, extracting key frames in the video, parsing scene information in the key frames, and parsing location information in the key frames; Performing data cleaning on the scene information and location information in the parsed key frames, and extracting content elements including scene features of the video and shooting location information; The content elements are converted into album content tag data and stored in the album content database.

6. The method for processing album recommendations based on film and television content according to claim 1, characterized in that: The step of recommending and displaying the selected album photos and videos whose relevance reaches a predetermined value comprises: For the album photos and videos that have been screened out and whose relevance reaches a predetermined value, a collaborative filtering algorithm based on user behavior data or a recommendation algorithm based on content semantic understanding is used to further optimize the recommendation results.

7. The method for processing album recommendations based on film and television content according to claim 1, characterized in that: After the step of recommending and displaying the selected album photos and videos whose relevance reaches a predetermined value, the following step is further performed: Control the display of recommended album content and receive user feedback and interactive operation data; Based on the received user feedback and interaction operation data, the recommendation algorithm is further optimized.

8. A device for processing album recommendations based on film and television content, characterized in that: The device comprises: A film and television content analysis module is used to obtain the currently played film and television content, analyze the film and television content, identify and extract key elements including scene features and location information, and save the key elements as film and television content tag data; The album content parsing module is used to traverse and search the user album of the smart terminal, parse the photos and videos in the user album, identify and extract content elements including scene features and shooting location information in the photos and videos, and convert the content elements into album content tag data; An association comparison module, used to compare the film and television content label data with the album content label data, calculate the association between the film and television content label data and the album content label data, and filter out the album photos and videos whose association reaches a predetermined value; The recommendation engine module is used to recommend and display the selected album photos and videos whose relevance reaches a predetermined value.

9. An intelligent terminal, characterized in that: The device comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include being used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as described in any one of claims 1 to 7.

Images