Video classification and interaction method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of smart televisions, in particular to a video classification and interaction method and device, computer equipment and a storage medium, video elements are obtained by obtaining a television video to be analyzed, and the video elements comprise picture elements, sound elements and character elements; respectively inputting the picture elements, the sound elements and the character elements into a preset trained analysis model to obtain element content classifications of the video elements, obtaining content types of the television videos according to the element content classifications of the video elements, marking the television videos corresponding to the content types, and displaying the marked television videos. And storing the marked video in a resource library of the television equipment. A television video is split into video elements, element content classification is carried out on the video elements, the video elements are matched according to the element content classification, the content type of the television video is obtained, and the television video corresponding to the content type is stored in a resource library. Therefore, the television video can be comprehensively analyzed, and the video type can be accurately confirmed.

Description

Technical Field

[0001] The present invention relates to the field of smart television technology, and in particular to a video classification and interaction method, device, computer equipment and storage medium. Background Art

[0002] With the development of technology, the number of video resources available for television has increased significantly. Traditional television programs rely on a limited number of channels, leaving viewers with a relatively narrow range of choices. Video management on television is primarily based on simple channel divisions, such as news, sports, and entertainment. However, with the development of internet technology, the connection between television terminals and the Internet has become increasingly close, introducing a large amount of video resources to television platforms, including various online videos, homemade programs, and user-uploaded content.

[0003] Currently, most video classification methods have limitations. Most video classification methods rely on single features, such as relying on human experience to determine the video type, or simply classifying the video based on tags. This simplistic classification method fails to fully and accurately reflect the overall content of the video, resulting in inaccurate classification and large errors, leading to a discrepancy between the final video results and user expectations.

[0004] Therefore, how to comprehensively analyze television videos and accurately identify the video type has become an urgent problem to be solved. Summary of the Invention

[0005] The embodiments of the present invention provide a video classification and interaction method, apparatus, computer equipment and storage medium to solve the problem of how to comprehensively analyze television videos and thus accurately determine the video type.

[0006] In a first aspect, an embodiment of the present invention provides a video classification method, which is applied to a television terminal and includes: Acquire a television video to be analyzed, and split the television video to be analyzed to obtain video elements, where the video elements include picture elements, sound elements, and text elements; Inputting the picture elements, the sound elements, and the text elements into a preset trained analysis model for analysis to obtain element content classification of the video elements; Matching the video elements according to the element content classification to obtain the content type of the television video; The television video corresponding to the content type is marked to obtain a marked video, and the marked video is stored in a resource library of a television device.

[0007] In a second aspect, an embodiment of the present invention provides an interactive method, comprising: Acquire a television video played in real time by a user using a television device from the resource library, wherein the resource library stores labeled videos classified by content types of the television videos determined by the video classification method; The interactive animation of the television video is set according to the content type, and if the real-time television video meets the preset interactive conditions, the interactive animation is displayed on the television device.

[0008] In a third aspect, an embodiment of the present invention provides a video classification device, which is applied to a television terminal and includes: A video splitting module is used to obtain a television video to be analyzed, split the television video to be analyzed, and obtain video elements, wherein the video elements include picture elements, sound elements, and text elements; An element classification module, configured to input the picture elements, the sound elements, and the text elements into a preset trained analysis model for analysis, thereby obtaining an element content classification of the video elements; A video content classification module, configured to match the video elements according to the element content classification to obtain the content type of the television video; The video storage module is used to mark the television video corresponding to the content type to obtain a marked video, and store the marked video in a resource library of the television device.

[0009] In a fourth aspect, an embodiment of the present invention provides an interactive device, comprising: a video acquisition module, configured to acquire television videos played in real time from the resource library when a user uses a television device, wherein the resource library stores labeled videos classified by content types of television videos determined by the video classification method; The animation display module is used to set the interactive animation of the TV video according to the content type, and if the real-time TV video meets the preset interactive conditions, the interactive animation is displayed on the TV device.

[0010] In a fifth aspect, an embodiment of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned video classification method or the above-mentioned interactive method when executing the computer program.

[0011] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned video classification method or the above-mentioned interaction method.

[0012] Compared with the prior art, the present invention has the following beneficial effects: by obtaining a television video to be analyzed, the television video to be analyzed is split to obtain video elements, which include picture elements, sound elements, and text elements. The picture elements, sound elements, and text elements are respectively input into a preset trained analysis model for analysis to obtain element content classification of the video elements. According to the element content classification of the video elements, the video elements are matched to obtain the content type of the television video. The television video corresponding to the content type is marked to obtain a marked video, and the marked video is stored in a resource library of the television device. By splitting the television video into video elements, the video elements are classified by element content, and the video elements are matched according to the element content classification to obtain the content type of the television video, and the television video corresponding to the content type is stored in a resource library. In this way, the television video is comprehensively analyzed, thereby accurately confirming the video type. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0014] Figure 1 This is a schematic diagram of an application environment of a video classification method provided by the first embodiment of the present invention; Figure 2 This is a flow chart of a video classification method provided in the second embodiment of the present invention; Figure 3 This is a flow chart of a video classification method provided in the third embodiment of the present invention; Figure 4 1 is a flow chart of a video classification method provided in a fourth embodiment of the present invention; Figure 5 This is a flowchart of an interactive method provided in Example 5 of the present invention; Figure 6 This is a structural diagram of a video classification device provided by Example 6 of the present invention; Figure 7 This is a structural diagram of an interactive device provided by Embodiment 7 of the present invention; Figure 8 This is a structural diagram of a computer device provided in Example 8 of the present invention. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0016] like Figure 1 As shown, it is a schematic diagram of the application environment of a video classification method provided by the first embodiment of the present invention, wherein the client and the server are connected for communication, and the user can provide the conditions, requirements and operation instructions for querying information to the server by operating the client, and the server is used to execute the query results according to the relevant content sent by the client, and return the results to the client for video classification, and present the video classification results to the user through the client, the video classification method of the present invention. Among them, the client includes but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices and other computer devices. The computer device corresponding to the server can be implemented with an independent server or a server cluster composed of multiple servers.

[0017] like Figure 2 FIG. 1 is a flow chart of a video classification method provided in Embodiment 2 of the present invention, wherein the video classification method is applied to a television terminal and may include the following steps: Step S201: Obtain a television video to be analyzed, split the television video to be analyzed, and obtain video elements, where the video elements include picture elements, sound elements, and text elements.

[0018] The TV videos that need to be analyzed and processed are obtained through specific channels and methods. These methods include real-time recording of TV broadcast signals, reading saved TV video files from local storage devices (such as hard drives and USB flash drives), or downloading relevant TV videos from online video platforms.

[0019] The captured complete TV video is decomposed into picture elements, sound elements, and text elements. Picture elements represent the visual aspects of the video. During this decomposition process, each frame of the video must be extracted and analyzed. Video processing techniques can be used to segment the video into individual frames at regular intervals. These frames constitute the basic units of picture elements. Picture elements can include information such as people, objects, scenes, colors, and composition. For example, in a TV series, picture elements may include images of actors, buildings, and natural scenery.

[0020] Sound elements are the auditory components of a video. To extract sound elements, you need to separate the audio signal from the video file. You can use an audio extraction tool to extract the sound from the video as a separate audio file. Sound elements include dialogue, background music, and sound effects. For example, in an action movie video, the sound elements might include dialogue between actors, exciting background music, and various sound effects during combat.

[0021] Text elements are information presented in text form within a video. They can appear within the video itself, such as subtitles, logos, and prompts, or they can be hidden within the audio content (which can be converted to text using speech recognition technology). For example, a news video might have scrolling subtitles at the bottom of the screen, displaying relevant news content, and a movie video might begin with text describing the film's title, production company, and other information.

[0022] In step S202 , the picture elements, the sound elements, and the text elements are respectively input into a preset trained analysis model for analysis to obtain element content classification of the video elements.

[0023] The analysis model can utilize deep learning algorithms such as convolutional neural networks and recurrent neural networks. It can identify features such as objects, scenes, and actions in an image and categorize them into different categories. Alternatively, it can be used to identify the type of sound (such as human voice, music, or sound effects), the content of the speech (if it's human voice), and the style of the music. Furthermore, it can be used to identify the semantics of text and determine its theme and emotional orientation.

[0024] The image elements (single-frame image or continuous frame sequence) extracted in step S201 are input into the analysis model, which extracts and analyzes various features of the image, such as color distribution, texture, shape, object outline, etc.

[0025] For example, if the picture shows a beautiful beach, blue sea and golden sunshine, the model may classify it into the "Landscape - Beach" category based on these features. If the picture shows a group of people playing basketball, the model will recognize the people's actions and the scene and classify it into the "Sports, Basketball" category.

[0026] The sound element (audio file) is input into the analysis model. The analysis model analyzes the sound's frequency, timbre, rhythm and other characteristics to classify and identify the sound.

[0027] For example, for a piece of audio, if it detects exciting drum beats and dynamic melodies, the model may determine that it is a "music-rock style" sound; if someone is clearly narrating a news event in the audio, the model will classify it as "voice-news broadcast".

[0028] The text elements are input into the analysis model, which performs word segmentation, part-of-speech tagging, syntactic analysis, and semantic understanding on the text.

[0029] For example, if the text content is "A company launched a new smartphone with super camera capabilities", the model will classify it into the "Technology, Electronic Product Promotion" category based on semantics.

[0030] Through the analysis model, each video element is assigned one or more classification labels, which clearly define the type of content contained in the element. For example, if the visual element is classified as "food," the audio element is the exclamations of people enjoying delicious food (classified as "Life, Food Experience Sound Effects"), and the text element mentions the cooking method and characteristics of a particular dish (classified as "Food, Recipe Introduction"), the combined classification of these elements can more accurately determine that the content of the entire video is related to food.

[0031] Step S203: Match the video elements according to the element content classification to obtain the content type of the television video.

[0032] When the classifications of different elements are clearly consistent, the content type of the video is directly determined based on the consistency.

[0033] For example, the visual element is categorized as "Sports, Football Match," the audio element is the commentary and cheers of the audience (categorized as "Sports, Football Match Sound Effects"), and the text element mentions the names of the two football teams and the match score (categorized as "Sports, Football Match Report"). The categorization of these three elements is highly consistent, so the content category of the TV video can be clearly determined as "Sports, Football Match."

[0034] The classification of different elements complement each other and can only be combined to fully describe the content of the video.

[0035] For example, if the visual element shows a group of people in period costumes in a setting reminiscent of ancient architecture, it would be classified as "Film and Television - Period Costume Scene." The audio element includes melodious classical music and conversations, but the subject of the conversation is unclear. The text element mentions the name of a historical story. Combining these three elements—the visuals provide context, the text provides background, and the audio creates the atmosphere—the video content can ultimately be classified as "Film and Television, Period Costume Historical Drama."

[0036] Step S204: Mark the television video corresponding to the content type to obtain a marked video, and store the marked video in a resource library of the television device.

[0037] Tagging is the process of assigning a label to a TV video that corresponds to its content type. This label acts as an identifier, summarizing the core content of the video in a concise and clear manner. For example, if the video content type is "Sports, Basketball Game," the video will be tagged with the label "Sports, Basketball Game."

[0038] Tagging videos facilitates subsequent video management and search. Using tags, TVs can quickly identify the content attributes of videos, allowing users to accurately filter videos based on tags when searching, greatly improving search efficiency. For example, if a user wants to watch a basketball game, they can simply enter "sports, basketball game" in the search box, and the TV will quickly locate videos with that tag.

[0039] Once the tagging process is complete, an ordinary TV video becomes a tagged video with a specific content identifier. This process essentially adds information about the content type to the video's metadata. Metadata is like a video's "manual," recording various video attributes, and tagging information is a crucial part of this.

[0040] The data structure of the tagged video closely associates the video file itself with the tag information, thus providing better service to users in the system of the television device.

[0041] In an embodiment of the present application, a television video to be analyzed is obtained, the television video to be analyzed is split to obtain video elements, which include picture elements, sound elements, and text elements. The picture elements, sound elements, and text elements are respectively input into a preset trained analysis model for analysis to obtain element content classification of the video elements. According to the element content classification of the video elements, the video elements are matched to obtain the content type of the television video. The television video corresponding to the content type is marked to obtain a marked video, and the marked video is stored in a resource library of the television device. By splitting the television video into video elements, the video elements are classified by element content, and the video elements are matched according to the element content classification, the content type of the television video is obtained, and the television video corresponding to the content type is stored in a resource library. In this way, the television video is comprehensively analyzed, thereby accurately confirming the video type.

[0042] like Figure 3 FIG. 2 is a flow chart of a video classification method provided in Embodiment 3 of the present invention. In step S202, the training process of the analysis model may include the following steps: Step S301 : obtaining a training set, wherein the training set includes video information of at least one television video, wherein the video information includes picture elements, sound elements, text elements, and annotation results of element content classification of the video elements.

[0043] In step S302, the picture elements, the sound elements and the text elements in each TV video are respectively input into a preset analysis model for feature extraction to obtain a first feature corresponding to the picture element, a second feature corresponding to the sound element and a third feature corresponding to the text element.

[0044] Step S303: Input the first feature, the second feature, and the third feature into the preset analysis model to perform feature fusion, obtain a first fusion feature, and calculate the fusion loss.

[0045] In step S304, the first fusion feature is input into a preset deep learning model to classify the video content type to obtain a first classification result, and the classification task loss is calculated based on the first classification result and the labeling result.

[0046] Step S305 : updating the parameters in the preset analysis model according to the fusion loss and the classification task loss to obtain an updated analysis model.

[0047] Step S306, using the updated analysis model as the preset analysis model, returning to the step of inputting the picture elements, the sound elements, and the text elements in each TV video into the preset analysis model for feature fusion, until the total loss meets the preset conditions, and the updated analysis model is obtained as the trained analysis model.

[0048] Among them, the training set is a data set used to train the analysis model, which contains video information of at least one TV video. By learning from these data, the model can grasp the relationship between different video elements and element content classification.

[0049] Video information content can include visual elements, audio elements, and text elements. Visual elements include visual information such as images, scenes, characters, and objects in a video. For example, in a sports video, visual elements might include athletes' movements and the playing field. Audio elements include auditory information such as voice, music, and sound effects. Examples include the anchor's voice and background music in a news video. Text elements include subtitles, logo text, and text content obtained through speech recognition. For example, subtitles in a movie video might be dialogue.

[0050] The element content classification results are the result of manual pre-classification and labeling of video elements, which is the goal of model learning. For example, elements containing food scenes are labeled as "food".

[0051] The image, sound, and text elements in each TV video are input into a pre-set analysis model. The pre-set analysis model contains sub-modules specifically designed to process different types of elements. These sub-modules extract features from the input elements.

[0052] The first feature is extracted from the image elements. It is a digital representation of the image elements and contains information such as the color, texture, shape, and spatial relationship of objects in the image. For example, for a landscape image, the first feature may reflect the color distribution of the image, the outline of the scenery, etc. The second feature is extracted from the sound elements and reflects the sound attributes such as frequency, timbre, rhythm, and pitch change. For example, the melody characteristics and rhythm pattern of a piece of music. The third feature is the feature obtained after processing the text elements and reflects the semantics, grammatical structure, theme, and other information of the text. For example, in a news text, the third feature may reflect the theme of the news, emotional tendency, etc.

[0053] The extracted first, second, and third features are then fed into the fusion module of the pre-set analysis model for feature fusion. This is because the features of a single element may not be sufficient to fully describe the content of a video. By fusing the features of different elements, richer and more comprehensive information can be obtained. The fused first feature combines the characteristics of the visual, audio, and text elements.

[0054] Fusion loss measures the effectiveness of feature fusion. It is calculated by comparing the difference between the fused features and the expected fused features. If the difference between the fused features and the expected features is large, it indicates poor fusion and a high fusion loss. Conversely, a low fusion loss will be low. Calculating fusion loss helps the model adjust its feature fusion method to improve fusion performance.

[0055] The first fused feature is input into a pre-set deep learning model. The deep learning model then classifies the video's content type based on the fused feature, producing a first classification result. This classification process is like assigning an appropriate label to the video, such as determining whether it belongs to a category like sports, entertainment, or news. The first classification result is compared with the pre-labeled result to calculate the classification task loss. This classification task loss reflects the difference between the model's classification result and the true classification. If the classification result is inconsistent with the labeled result, the model's classification is inaccurate, resulting in a higher classification task loss. If the classification result is consistent with the labeled result, the classification task loss is lower.

[0056] Based on the fusion loss and classification task loss, the parameters of the pre-set analysis model are updated. Model parameters determine its behavior and performance. By adjusting these parameters, the model can be better adapted to the training data. During the update process, an optimization algorithm (such as stochastic gradient descent) is used to find the parameter values ​​that minimize the total loss (the sum of the fusion loss and the classification task loss). After the parameter update, the result is an updated analysis model that is more accurate than the previous model in processing video elements and performing classification.

[0057] The updated analysis model is used as the new preset analysis model, and the operations of steps S302-S305 are repeated to continuously perform feature extraction, fusion, classification and parameter update. Each iteration will improve the performance of the model to a certain extent. The iterative process will continue until the total loss meets the preset conditions. The preset conditions are usually that the total loss reaches a small threshold, or the total loss no longer decreases significantly within a certain number of iterations. When the termination conditions are met, the updated analysis model obtained is the trained analysis model, which can be used to classify the element content of new TV videos.

[0058] In this embodiment, the model is continuously made to learn the data in the training set and the model parameters are adjusted so that the model can accurately classify the element contents of the television video.

[0059] like Figure 4 FIG. 2 is a flow chart of a video classification method provided by a fourth embodiment of the present invention. Step S203 of classifying the video elements according to the element content to obtain the content type of the television video may include the following steps: Step S401 : screening the content types of the video elements according to the element content classification to obtain video elements of the same type.

[0060] Step S402: Combining the video elements of the same type to obtain video segments, and determining the content type of the television video based on the video segments.

[0061] In step S202, the content of video elements (visual elements, audio elements, and text elements) has been classified, and each element has been assigned to a specific category. Based on these existing element content categories, all video elements are filtered by content type. The purpose of filtering is to group elements with the same content type. For example, all visual elements classified as "natural scenery," the related audio elements "natural sound effects," and text elements mentioning natural scenery are filtered out; these elements belong to the same type of video element.

[0062] By filtering video elements of the same type, previously scattered, heterogeneous elements can be consolidated according to content, providing a more organized data foundation for subsequent determination of video content types. This prevents interference between elements of different content types, making subsequent analysis more focused and accurate.

[0063] When analyzing video clips, we need to consider aspects such as the video's theme, plot, and emotion. For example, for a natural scenery video clip, if the footage shows mountains, rivers, flowers, and trees, the sounds are the gurgling of water and birdsong, and the text describes the local geography and ecological environment, then the theme of the video clip can be determined to be natural scenery appreciation.

[0064] Based on the analysis results, the content category of the TV video is determined. Content categories can be categorized according to various criteria, such as news, entertainment, education, and sports. Each category can be further subdivided into subcategories. For the aforementioned natural scenery video clip, its content category could be determined as "Education - Natural Science" or "Entertainment - Scenery Appreciation." The specific categorization depends on the video's specific focus and target audience.

[0065] In this embodiment, by screening and combining video elements, originally scattered information is integrated into meaningful video clips, thereby accurately determining the content type of the TV video, providing an important basis for subsequent video management and recommendation.

[0066] like Figure 5 FIG. 1 is a flow chart of an interactive method provided in Embodiment 5 of the present invention. The interactive method may include the following steps: Step S501: obtaining a TV video played in real time from the resource library when the user uses the TV device. The resource library stores marked videos classified by content types of the TV videos determined by a video classification method.

[0067] Step S502 : setting an interactive animation for the television video according to the content type, and displaying the interactive animation on the television device if the real-time television video meets a preset interactive condition.

[0068] The resource library is the fundamental data storage location for the entire process. It stores videos categorized by content using video classification methods. These categorized videos have been processed through previous steps and assigned specific content category tags, such as "Sports, Basketball" or "Entertainment, Music Variety Show." The resource library acts as an organized video warehouse, facilitating video management and retrieval on television devices.

[0069] When users watch videos on their TVs, they need to obtain the current video being played in real time. This is because subsequent interactive operations are based on the current video content. By obtaining the real-time video, the system can understand the type of content the user is watching, providing a basis for setting up further interactions. For example, if the user is watching a basketball game video, after obtaining this information, the system can design relevant interactions based on the content type of basketball games.

[0070] Different types of TV videos require different interactive animations because they have different characteristics and audience needs. For example, for sports videos, interactive animations could include virtual player celebrations or live score updates. For children's educational videos, interactive animations might feature cute cartoon characters guiding children through questions. By tailoring interactive animations to the content type, user engagement and interactive experience can be enhanced.

[0071] When designing interactive animations, it's important to choose appropriate animation elements that match the content, style, and atmosphere of the video. For example, in a science fiction video, interactive animations can incorporate futuristic special effects and technological elements; in a period drama video, interactive animations can incorporate props and scenic elements from ancient times.

[0072] Preset interaction conditions determine when to display interactive animations. These conditions are typically based on factors such as the video's content, time, and plot. For example, in a sports video, the system considers an interaction condition satisfied when a goal is scored (a content- and plot-based condition) or when the game reaches a specific time period (a time-based condition).

[0073] Interaction conditions can be a single condition or a combination of multiple conditions. For example, in a TV series, an interaction condition is considered met only when the plot develops to a key conflict and the audience's viewing behavior within a certain period of time reaches a certain threshold (such as the number of continuous viewing and pauses).

[0074] When a live TV video meets the preset interactive conditions, the system displays a pre-set interactive animation on the TV device. This interactive animation can appear in specific areas of the TV screen, such as corners or the center. The specific location and display method are determined based on the user experience and the layout of the video. The interactive animation can attract users' attention and inspire them to interact with the TV video, thereby enhancing their viewing experience and sense of engagement.

[0075] Optionally, setting the interactive animation of the television video according to the content type in step S502 may include the following steps: The segmented videos are marked with interaction timestamps to obtain segmented point videos.

[0076] The interactive animation is set at the interactive timestamp of the segmented point video to obtain the interactive point animation.

[0077] Interaction timestamps refer to specific time points identified in segmented videos, which are critical moments for interactive operations. The purpose of marking interaction timestamps is to clarify at which specific moment in the video the interactive animation can be inserted, so that the interaction can be closely integrated with the video content and enhance the user's interactive experience. For example, in a concert video, when the lead singer begins to sing the high notes, this is a wonderful moment for interaction, and this time point can be marked as an interaction timestamp. After determining the interaction timestamp, it is necessary to select the appropriate interactive animation for each interaction timestamp based on the content type of the video and the purpose of the interaction. The interactive animation should match the video content to enhance the naturalness and fun of the interaction.

[0078] Optionally, after setting the interactive animation of the television video according to the content type in step S502, the following steps may be further included: Acquiring an operation behavior of a user using the television device, and determining a behavior animation of the television video according to the operation behavior; If the real-time played TV video meets the preset behavior condition, the behavior animation is displayed on the TV device.

[0079] TV devices can obtain user operations in a variety of ways, including button operations performed on a remote control (such as play, pause, fast forward, and change channels), commands issued through voice control (such as "play next episode" and "turn up the volume"), and operations performed by interacting with the TV using smart devices (such as screen casting and video sharing via a mobile phone).

[0080] The TV system records and analyzes these operational behaviors in real time. It records information such as the time, frequency, and type of the operation, allowing for the subsequent determination of appropriate behavioral animations based on this data. Different operational behaviors correspond to different behavioral animations. This mapping is based on user experience and operational logic. The determination of behavioral animations can also take into account the user's individual needs and usage habits. For users who frequently perform specific operations, customized behavioral animations can be created specifically for them. Furthermore, the style and presentation of the animations must be consistent with the content and overall style of the TV video. For example, in science fiction videos, behavioral animations can use high-tech special effects, while in children's animations, they can be more adorable and lively.

[0081] Preset action conditions determine when to display action animations. These conditions are typically related to factors such as the video content and the timing of user actions. For example, in some videos, action animations are triggered only when the user takes action at specific plot points. For example, at a crucial moment in a suspense drama, if the user pauses the video and the preset action conditions are met, the corresponding action animation will be displayed.

[0082] A behavior condition can be a single condition or a combination of multiple conditions. For example, a behavior condition can be considered satisfied only when the user performs a certain action continuously within a certain timeframe, or when the video reaches a certain point and the user performs a specified action.

[0083] When the live TV video matches the preset behavior conditions, the TV device will display the corresponding behavior animation on the screen. The timing of the display should be just right, neither too early to interrupt the user's viewing experience nor too late to cause the user to feel the feedback. The behavior animation should be displayed clearly and smoothly, able to attract the user's attention and provide intuitive operation feedback.

[0084] By displaying behavioral animations, users can enhance their sense of interaction and immersion with their TV devices and video content. Users can receive immediate feedback from the animations, making the viewing experience more enjoyable. For example, displaying an encouraging behavioral animation (such as a flashing star or a like icon) after a user performs a correct interactive action can increase user engagement.

[0085] In this embodiment, behavioral animations are determined by obtaining user operation behaviors and displayed when preset conditions are met, thereby further enriching the interactive experience of TV videos and enabling the TV device to provide more personalized and vivid feedback based on user behaviors.

[0086] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0087] like Figure 6 FIG. 1 is a schematic diagram of a video classification device provided in Example 6 of the present invention. The video classification device corresponds one-to-one with the video classification method in the above-mentioned embodiment. The video classification device is applied to a television terminal and includes a video splitting module 61, an element classification module 62, a video content classification module 63, and a video storage module 64. The functional modules are described in detail as follows: The video splitting module 61 is used to obtain a television video to be analyzed, split the television video to be analyzed, and obtain video elements, wherein the video elements include picture elements, sound elements, and text elements; An element classification module 62 is configured to input the image elements, the sound elements, and the text elements into a preset trained analysis model for analysis to obtain element content classification of the video elements; A video content classification module 63 is configured to match the video elements according to the element content classification to obtain the content type of the television video; The video storage module 64 is configured to mark the television video corresponding to the content type to obtain a marked video, and store the marked video in a resource library of the television device.

[0088] Optionally, the element classification module 62 includes: a training acquisition unit, configured to acquire a training set, wherein the training set includes video information of at least one television video, wherein the video information includes picture elements, sound elements, text elements, and annotation results of element content classification of the video elements; a feature extraction unit, configured to input the picture element, the sound element, and the text element in each television video into a preset analysis model for feature extraction, thereby obtaining a first feature corresponding to the picture element, a second feature corresponding to the sound element, and a third feature corresponding to the text element; a fusion loss calculation unit, configured to input the first feature, the second feature, and the third feature into the preset analysis model to perform feature fusion, obtain a first fusion feature, and calculate a fusion loss; a classification loss calculation unit, configured to input the first fusion feature into a preset deep learning model to classify the video content type, obtain a first classification result, and calculate a classification task loss based on the first classification result and the labeling result; A model updating unit, configured to update parameters in a preset analysis model according to the fusion loss and the classification task loss to obtain an updated analysis model; The loop unit is used to use the updated analysis model as the preset analysis model, return to execute the step of inputting the picture elements, the sound elements and the text elements in each TV video into the preset analysis model for feature fusion, until the total loss meets the preset conditions, and the updated analysis model is obtained as the trained analysis model.

[0089] Optionally, the video content classification module 63 includes: a video element screening unit, configured to screen the content types of the video elements according to the element content classification to obtain video elements of the same type; The combining unit is configured to combine the video elements of the same type to obtain video segments, and determine the content type of the television video according to the video segments.

[0090] like Figure 7 As shown, an interactive device provided by the seventh embodiment of the present invention is provided. The video classification device corresponds one-to-one with the video classification method in the above embodiment. The interactive device includes a video acquisition module 71 and an animation display module 72. The functional modules are described in detail as follows: The video acquisition module 71 is used to acquire the TV video played in real time when the user uses the TV device from the resource library, wherein the resource library stores the marked videos classified by the content type of the TV video determined by the video classification method; The animation display module 72 is used to set the interactive animation of the TV video according to the content type, and if the real-time TV video meets the preset interactive conditions, the interactive animation is displayed on the TV device.

[0091] Optionally, the animation display module 72 includes: A segmentation unit, configured to mark the interaction timestamps on the segmented videos to obtain segmented point videos; The animation setting unit is used to set the interactive animation at the interactive timestamp of the segmented point video to obtain the interactive point animation.

[0092] Optionally, the animation display module 72 includes: a behavior animation determining unit, configured to, after setting the interactive animation of the television video according to the content type, obtain an operation behavior of a user using the television device, and determine the behavior animation of the television video according to the operation behavior; The animation triggering unit is used to display the behavior animation on the television device if the real-time television video meets the preset behavior condition.

[0093] For the specific limitations of the video classification device, please refer to the limitations of the video classification method above. For the specific limitations of the interactive device, please refer to the limitations of the interactive method above, which will not be repeated here. The various modules in the above-mentioned video classification device and interactive device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0094] like Figure 8The figure shows a schematic diagram of the structure of a computer device provided in Example 8 of the present invention. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a video classification method is implemented.

[0095] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the video classification method in the above embodiment is implemented, for example Figures 2 to 4 As shown, or implement the interactive method in the above embodiment, for example Figure 5 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the video classification device or the interactive device are realized, for example Figure 6 The video splitting module 61, the element classification module 62, the video content classification module 63, and the video storage module 64 are shown. Figure 7 The functions of the video acquisition module 71 and the animation display module 72 are not described here in detail to avoid repetition.

[0096] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the video classification method in the above embodiment is implemented. Figures 2 to 4 As shown, the interactive method in the above embodiment is implemented, such as Figure 5 Alternatively, when the computer program is executed by a processor, the functions of each module / unit in the embodiment of the video classification device are realized, for example Figure 6 The video splitting module 61, the element classification module 62, the video content classification module 63, and the video storage module 64 are shown. Figure 7 To avoid repetition, the functions of the video acquisition module 71 and the animation display module 72 are not described here in detail. The computer-readable storage medium may be non-volatile or volatile.

[0097] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0098] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0099] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A video classification method, characterized in that: The video classification method is applied to a television terminal, and the video classification method includes: Acquire a television video to be analyzed, and split the television video to be analyzed to obtain video elements, where the video elements include picture elements, sound elements, and text elements; Inputting the picture elements, the sound elements, and the text elements into a preset trained analysis model for analysis, respectively, to obtain element content classification of the video elements; Matching the video elements according to the element content classification to obtain the content type of the television video; The television video corresponding to the content type is marked to obtain a marked video, and the marked video is stored in a resource library of a television device.

2. The video classification method according to claim 1, characterized in that The training process of the analysis model includes: Acquire a training set, the training set including video information of at least one television video, the video information including picture elements, sound elements, text elements, and annotation results of element content classification of the video elements; Inputting the picture element, the sound element, and the text element in each television video into a preset analysis model for feature extraction, thereby obtaining a first feature corresponding to the picture element, a second feature corresponding to the sound element, and a third feature corresponding to the text element; Inputting the first feature, the second feature, and the third feature into the preset analysis model to perform feature fusion to obtain a first fused feature, and calculating a fusion loss; Inputting the first fusion feature into a preset deep learning model to classify the video content type to obtain a first classification result, and calculating the classification task loss based on the first classification result and the labeling result; updating the parameters of the preset analysis model according to the fusion loss and the classification task loss to obtain an updated analysis model; The updated analysis model is used as the preset analysis model, and the step of inputting the picture elements, the sound elements, and the text elements in each TV video into the preset analysis model for feature fusion is returned to the step until the total loss meets the preset conditions, so that the updated analysis model is the trained analysis model.

3. The video classification method according to claim 1, characterized in that The matching of the video elements according to the element content classification to obtain the content type of the television video includes: According to the element content classification, the content type of the video element is screened to obtain video elements of the same type; The video elements of the same type are combined to obtain video segments, and the content type of the television video is determined based on the video segments.

4. An interactive method, characterized in that include: Acquire a television video played in real time by a user using a television device from the resource library, wherein the resource library stores labeled videos classified by content type of the television video based on the video classification method according to any one of claims 1 to 3; The interactive animation of the television video is set according to the content type, and if the real-time television video meets the preset interactive conditions, the interactive animation is displayed on the television device.

5. The interactive method according to claim 4, characterized in that: The step of setting the interactive animation of the television video according to the content type includes: Marking the segmented videos with interaction timestamps to obtain segmented point videos; The interactive animation is set at the interactive timestamp of the segmented point video to obtain the interactive point animation.

6. The interactive method according to claim 4, characterized in that: After setting the interactive animation of the television video according to the content type, the method further includes: Acquiring an operation behavior of a user using the television device, and determining a behavior animation of the television video according to the operation behavior; If the real-time played TV video meets the preset behavior condition, the behavior animation is displayed on the TV device.

7. A video classification device, characterized in that: The video classification device is applied to a television terminal, and the video classification device includes: A video splitting module is used to obtain a television video to be analyzed, split the television video to be analyzed, and obtain video elements, wherein the video elements include picture elements, sound elements, and text elements; An element classification module, configured to input the picture elements, the sound elements, and the text elements into a preset trained analysis model for analysis, thereby obtaining an element content classification of the video elements; A video content classification module, configured to match the video elements according to the element content classification to obtain the content type of the television video; The video storage module is used to mark the television video corresponding to the content type to obtain a marked video, and store the marked video in a resource library of the television device.

8. An interactive device, characterized in that: include: a video acquisition module, configured to acquire television videos played in real time by a user using a television device from the resource library, wherein the resource library stores labeled videos classified by content type of the television videos based on the video classification method according to any one of claims 1 to 3; The animation display module is used to set the interactive animation of the TV video according to the content type, and if the real-time TV video meets the preset interactive conditions, the interactive animation is displayed on the TV device.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the video classification method according to any one of claims 1 to 3 or the interactive method according to any one of claims 4 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the video classification method according to any one of claims 1 to 3 or the interactive method according to any one of claims 4 to 6 is implemented.

Images