Interaction method, device, terminal device and computer readable storage medium

Abstract

The application discloses an interactive method and device, a terminal device and a computer readable storage medium. The method comprises: acquiring video stream data played by a user; identifying the video stream data and extracting a key scene; when the key scene is not a predetermined scene, sending scene video data corresponding to the key scene to a cloud end, and interacting with the user based on first interactive content data returned by the cloud end, wherein the cloud end is used to generate the first interactive content data based on the scene video data. The method of the application can reduce the computing pressure of the client through the functional division and collaborative cooperation of the client and the cloud end, improve the accuracy of scene identification and the relevance of interactive content, realize efficient and intelligent interaction in a complex video playing scene, and improve the user experience.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an interactive method, apparatus, terminal device, and computer-readable storage medium. Background Technology

[0002] With the improvement of mobile terminal performance and the development of streaming media technology, online video playback on the client side has become a mainstream application. Current video playback systems typically implement video decoding and playback control on the client side, triggering advertisements or interactive prompts at specific time points or preset locations. Existing technologies have shortcomings in terms of the accuracy of video scene recognition, system resource utilization efficiency, and real-time interactivity, making it difficult to provide a good user experience while ensuring system performance. Summary of the Invention

[0003] This application provides an interaction method, device, terminal equipment, and computer-readable storage medium. By dividing the functions and cooperating with the cloud, the computational burden on the client can be reduced, while improving the accuracy of scene recognition and the relevance of interactive content. This enables efficient and intelligent interaction in complex video playback scenarios, thereby enhancing the user experience.

[0004] The technical solution adopted by this invention to solve the problem is as follows: On the one hand, this application provides an interaction method applied to a client, the interaction method including: Get the video stream data played by the user; Identify and extract key scenes from video stream data; When the key scene is not the predetermined scene, the scene video data corresponding to the key scene is sent to the cloud, and the user is interacted with based on the first interactive content data returned by the cloud. The cloud is used to generate the first interactive content data based on the scene video data.

[0005] In some implementations of this application, user interaction is based on first interactive content data returned from the cloud, including: Generate the first interaction option based on the first interaction content data; Display the first interactive option; In response to the user's operation signal on the first interaction option, the first interaction content data is displayed.

[0006] In some embodiments of this application, the method further includes: When the key scenario is a predetermined scenario, determine the second interaction option corresponding to the key scenario; Display a second interaction option and a third interaction option, wherein the third interaction option is used to ask the user whether they need deeper interaction; In response to the user's operation signal for the third interactive option, the scene video data corresponding to the key scene is sent to the cloud, and the user is interacted with based on the second interactive content data returned by the cloud. The cloud is used to generate the second interactive content data based on the scene video data. Secondly, this application provides an interaction method applied in the cloud, the interaction method comprising: The system receives scene video data corresponding to key scenes sent by the client. The client is used to identify the video stream data played by the user, extract key scenes, and send the scene video data to the cloud when the key scene is not a predetermined scene. First interactive content data is generated based on scene video data; The first interaction content data is returned to the client, which is also used to interact with the user based on the first interaction content.

[0007] In some embodiments of this application, first interactive content data is generated based on scene video data, including: Identify scene video data to determine scene type; Generate retrieval tasks corresponding to the scene type; The retrieval task is input into the intelligent model, which then executes the retrieval task and outputs the first interactive content data.

[0008] In some implementation schemes of this application, retrieval tasks corresponding to scene types are generated, including: Obtain user preference information; A search task is generated based on user preference information, scene type, and scene video data.

[0009] Thirdly, embodiments of the present invention also provide an interactive device applied to a client, comprising: The acquisition module is used to acquire the video stream data played by the user; The recognition module is used to identify video stream data and extract key scenes; The interaction module is used to send the scene video data corresponding to the key scene to the cloud when the key scene is not the predetermined scene, and to interact with the user based on the first interactive content data returned by the cloud. The cloud is used to generate the first interactive content data based on the scene video data.

[0010] Fourthly, embodiments of the present invention also provide an interactive device applied in the cloud, comprising: The receiving module is used to receive scene video data corresponding to key scenes sent by the client. The client is used to identify the video stream data played by the user, extract key scenes, and send the scene video data to the cloud when the key scene is not a predetermined scene. The generation module is used to generate the first interactive content data based on the scene video data; The sending module is used to return the first interaction content data to the client, whereby the client is also used to interact with the user based on the first interaction content.

[0011] Fifthly, this application also provides a terminal device, which includes: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor to implement the interaction method of either the first aspect or the second aspect.

[0012] In a sixth aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform steps in the interaction method of either the first or second aspect.

[0013] The beneficial effects of this invention are as follows: By coordinating client and cloud interactions with users, the client performs real-time identification of the video stream data played by the user, extracts key scenes, and when it is determined that the key scene is not a predetermined scene, sends the corresponding scene video data to the cloud for in-depth analysis. The cloud generates first interactive content data and returns it to the client for user interaction, reducing the computational burden on the client while improving the accuracy of scene identification and the relevance of interactive content. This enables efficient and intelligent interaction in complex video playback scenarios, thereby enhancing the user experience. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of a scenario of the interactive system provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating one embodiment of the interaction method provided in this invention. Figure 3 This is a flowchart illustrating another embodiment of the interaction method provided in this invention. Figure 4 This is a flowchart illustrating a specific embodiment of the interaction method provided in this invention. Figure 5This is a schematic block diagram of one embodiment of the interactive device provided in this invention; Figure 6 This is a schematic block diagram of another embodiment of the interactive device provided in this invention; Figure 7 This is a schematic diagram of an embodiment of the terminal device provided in this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of the stated features.

[0018] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0019] It should be noted that since the method in this application embodiment is executed in a terminal device, the processing objects of each terminal device exist in the form of data or information, such as time, which is essentially time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the terminal device can process them. Specific details will not be elaborated here.

[0020] This application provides an interactive method, apparatus, terminal device, and computer-readable storage medium, which will be described in detail below.

[0021] Please see Figure 1 , Figure 1This is a schematic diagram of an interactive system provided in an embodiment of this application. The interactive system may include a terminal device 100, which integrates an interactive device, such as... Figure 1 Terminal devices in the process.

[0022] In this embodiment, the terminal device 100 is mainly used to acquire video stream data played by the user; identify the video stream data and extract key scenes; when the key scene is not a predetermined scene, send the scene video data corresponding to the key scene to the cloud, and interact with the user based on the first interactive content data returned by the cloud. The cloud is used to generate the first interactive content data based on the scene video data. Through the division of labor and cooperation between the client and the cloud, the computational pressure on the client can be reduced, while improving the accuracy of scene recognition and the relevance of interactive content. This enables efficient and intelligent interaction in complex video playback scenarios and improves the user experience.

[0023] In this embodiment, the terminal device 100 can be an independent server, a server network, or a server cluster. For example, the terminal device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0024] It is understood that the terminal device 100 used in the embodiments of this application can be a device that includes both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the terminal device 100 may be a desktop terminal or a mobile terminal, and the terminal device 100 may also be one of a mobile phone, tablet computer, laptop computer, etc.

[0025] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of more or fewer terminal devices shown, for example Figure 1 Only one terminal device is shown in the diagram. It is understood that the interactive system may also include one or more other services, which are not specified here.

[0026] In addition, such as Figure 1As shown, the interactive system may also include a memory 200 for storing data, such as video stream data, predetermined scenes, scene video data, first interactive content data, etc.

[0027] It should be noted that, Figure 1 The schematic diagram of the interactive system shown is merely an example. The interactive system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of interactive systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0028] First, this application provides an interaction method. The execution subject of the interaction method is an interaction device, which is applied to a terminal device. The interaction method includes acquiring video stream data played by the user; identifying the video stream data and extracting key scenes; when the key scene is not a predetermined scene, sending the scene video data corresponding to the key scene to the cloud, and interacting with the user based on the first interaction content data returned by the cloud, wherein the cloud is used to generate the first interaction content data based on the scene video data.

[0029] like Figure 2 The diagram shown is a flowchart of an embodiment of the interaction method in this application. This interaction method is applied to a client and may include the following steps S201 to S203, as detailed below: Step S201: Obtain the video stream data played by the user.

[0030] In this step, video stream data refers to continuous video data that is received and decoded during playback on the client side. This includes video frame data, optional audio data, and relevant timestamp information; essentially, it is a dynamic media data stream organized chronologically. By synchronously acquiring the video stream during the playback process, the client can obtain the raw data foundation for analysis as playback occurs, ensuring that the object to be identified remains consistent with the content the user is currently viewing, thereby achieving real-time interaction.

[0031] Step S202: Identify the video stream data and extract key scenes.

[0032] In this step, the client can analyze the features of the video frame using image processing, pattern matching, or machine learning algorithms to determine the content type or semantic state of the current frame. Key scenarios refer to segments during video playback that have specific semantic meaning or business triggering value, such as frame segments with phased characteristics like game pauses, commercial breaks, or plot transitions.

[0033] This step is used to analyze image features (such as changes in image content, subtitle features, audio features, or recognition results from pre-trained models) to divide continuous media data into different semantic units, and to filter out segments with processing value from the continuous video stream, rather than performing subsequent processing on all video content.

[0034] Step S203: When the key scene is not the predetermined scene, send the scene video data corresponding to the key scene to the cloud, and interact with the user based on the first interactive content data returned by the cloud. The cloud is used to generate the first interactive content data based on the scene video data.

[0035] This step establishes a collaborative processing mechanism between the client and the cloud. The client handles real-time identification and initial screening, while the cloud handles complex semantic analysis and content generation. The client and cloud then collaborate to interact with the user. The predetermined scenario refers to a specific scenario type pre-defined in the client that requires no further processing, such as the end of a sports program or the completion of an episode of a TV series. The first interactive content data refers to information generated by the cloud based on scenario video data analysis, used for user interaction, such as recommendations, supplementary explanations, or related resource links. Scenario video data refers to continuous video segments corresponding to key scenarios, which can be structured data segments formed by extracting, organizing, and encapsulating data around the identified key scenarios from continuous video stream data. Specifically, after the client completes key scenario identification, the time range corresponding to the key scenario can be determined based on the identification results, i.e., the start and end times of the key scenario in the video stream. This time range can be determined in various ways, such as based on scene boundary frames output by the identification model, switching points based on continuous frame feature change detection, or a fixed-length interval based on a sliding time window. After determining the time interval, the client extracts and processes the original video frame data within that interval to form continuous video segments, thus obtaining the scene video data.

[0036] This step is used to differentiate the processing of different types of key scenarios, only calling cloud resources for in-depth analysis when necessary, thereby achieving reasonable resource allocation. When a key scenario is determined to be a non-pre-defined scenario, the client uploads the relevant segments to the cloud. The cloud then uses its stronger computing power and richer data resources to perform deep retrieval or model inference, generating interactive content that semantically matches the current scenario, and then returning it to the client for presentation and interaction. Through this collaborative mechanism, the client's computational burden can be reduced while ensuring real-time response, unnecessary data transmission can be minimized, and the matching degree between interactive content and the current playback scenario can be improved, thereby achieving a more intelligent and precise human-computer interaction effect.

[0037] In one specific implementation, interacting with the user based on the first interactive content data returned from the cloud includes: generating a first interactive option based on the first interactive content data; displaying the first interactive option; and displaying the first interactive content data in response to the user's operation signal on the first interactive option.

[0038] In this embodiment, the first interactive content data is structured content data generated by the cloud based on scene video data analysis. The first interactive content data may include various data fields such as prompts, prediction results, recommended content, comment summaries, and remaining time parameters. For example, when the cloud-based agent identifies the current scene as a "halftime" type in a sports event, it can calculate the remaining break time based on the match type, current match progress, and event rule model, and generate prompt text containing variable parameters; when identified as a "end of a period" type, it can generate information such as win rate prediction results, real-time data analysis conclusions, or popular comment summaries. The agent can be an analysis module or algorithm model deployed in the cloud, possessing the ability to perform semantic understanding and business reasoning on scene information.

[0039] Building upon this foundation, the client constructs user-selectable interface interaction entry points—the first interaction options—based on the initial interaction content data. These typically refer to buttons, prompt cards, or clickable controls within the interface, such as "Play the match playlist," "View win rate predictions," or "Browse popular comments." This allows users to actively participate in the interaction, rather than passively receiving information pushes. In this way, the client establishes a mapping relationship between the presentation layer and the content layer: the cloud is responsible for generating semantic content, and the client is responsible for transforming it into operable interface elements. When the client detects that the user has interacted with an option, it further displays the specific content data generated by the cloud, such as showing the complete match playlist playback interface, presenting a win rate prediction curve, or displaying a list of popular comments.

[0040] In one specific implementation, the method further includes: when the key scenario is a predetermined scenario, determining a second interaction option corresponding to the key scenario; displaying the second interaction option and a third interaction option, wherein the third interaction option is used to ask the user whether they need in-depth interaction; in response to the user's operation signal on the third interaction option, sending the scene video data corresponding to the key scenario to the cloud, and interacting with the user based on the second interaction content data returned by the cloud, wherein the cloud is used to generate the second interaction content data based on the scene video data.

[0041] In this embodiment, the second interactive option refers to an interactive entry point that the client can directly generate and display based on local logic after recognizing a predetermined scenario, such as "watch replay highlights," "play the next episode," or "change channel," which can be formed without relying on real-time cloud analysis. The third interactive option is a control entry point used to ask the user whether they need to obtain further in-depth information or perform extended operations, such as "view more analysis" or "recommend similar content," which is a confirmatory or guiding option.

[0042] For predetermined scenarios, the client can already determine that the current content unit has ended based on the playback progress or scene recognition results. At this point, the user's focus often shifts from the current content itself to subsequent selections or extended information. Therefore, the client first generates and displays a second interactive option to provide immediate, lightweight operation suggestions. Simultaneously, a third interactive option prompts the user if they need more in-depth information, thus giving the user control over triggering in-depth analysis and avoiding automatic execution of complex analysis or loading of large amounts of information immediately after the content ends. Only when the user signals the third interactive option does the client send the scene video data corresponding to the key scene to the cloud. The cloud then performs deeper semantic analysis based on this scene video data to generate second interactive content data, such as complete match statistics, in-depth tactical analysis, player performance evaluation, or extended information based on film and television content, behind-the-scenes information, and similar program recommendations. The cloud can pre-execute some analysis tasks and pre-store rich background content in the background, thus quickly returning results after receiving user confirmation. The client then displays the second interactive content data.

[0043] By introducing a pre-defined scenario recognition and user confirmation mechanism, the system enables interactive logic that better aligns with user expectations at the end of content sessions. On one hand, users receive clear prompts for subsequent choices as content naturally concludes, enhancing the coherence and guidance of the interaction. On the other hand, in-depth analysis is only triggered after the user explicitly expresses their needs, reducing unnecessary data transmission and computational resource consumption, thus improving overall system efficiency. Simultaneously, because the cloud enables more complex data processing and content aggregation, users receive richer, more structured information output when choosing in-depth interactions, thereby enhancing the overall intelligence and personalization of the experience.

[0044] like Figure 3 The diagram shown is a flowchart of another embodiment of the interaction method in this application. This interaction method is applied to the cloud and may include the following steps S301 to S303, as detailed below: Step S301: Receive scene video data corresponding to key scenes sent by the client. The client is used to identify the video stream data played by the user, extract key scenes, and send the scene video data to the cloud when the key scene is not a predetermined scene.

[0045] In this step, scene video data refers to specific video segment data extracted and encapsulated from the video stream by the client when it identifies a key scene and determines that it is not a predetermined scene. This typically includes consecutive frames within a certain time range before and after the key scene, along with related metadata. The cloud server receives the scene video data from the client via a network interface and stores or caches it.

[0046] This step provides the input data foundation for subsequent cloud-based analysis. The cloud does not continuously analyze the entire video stream; instead, after initial screening on the client side, it processes only key scene segments with business value. This step effectively reduces redundant data processing in the cloud, improves the efficiency of computing resource utilization, and ensures that the analyzed data has clear semantic boundaries.

[0047] Step S302: Generate the first interactive content data based on the scene video data.

[0048] In this step, the cloud can use algorithm models, rule engines, or retrieval systems to perform semantic analysis, content recognition, and information reasoning on the received scene video data to generate first interactive content data that the client can use to interact with the user, such as prompts, statistics, prediction results, comment summaries, or recommended content.

[0049] This step leverages the relatively abundant computing resources and model capabilities of the cloud to perform more complex feature extraction and semantic understanding of video footage. For example, it can identify the type of match, determine if the current segment is halftime or a break, extract score information, or analyze the plot of a film or television scene. Based on this, rule matching or content generation models are used to transform the recognition results into structured data that meets interactive requirements.

[0050] Step S303: Return the first interactive content data to the client, wherein the client is also used to interact with the user based on the first interactive content.

[0051] In this step, the cloud sends the generated first interactive content data to the corresponding client device through the network interface to complete the closed loop of end-to-cloud collaboration, so that the analysis results of the cloud can be called by the client in real time and transformed into an interactive interface visible to the user.

[0052] By analyzing key scenarios in unplanned environments on the cloud side, the system avoids continuous processing of the entire video stream while ensuring depth and accuracy of analysis, thereby optimizing resource allocation and improving response efficiency. Furthermore, because the cloud can continuously update models and knowledge bases, the generated initial interactive content data has strong scalability and adaptability, contributing to improved intelligence and user experience across the entire video playback system.

[0053] In another embodiment, after the client identifies the key scenario as a predetermined scenario (e.g., the complete end of a sporting event or the completion of an episode of a TV series), the client first provides the user with basic interaction options based on local logic, such as "play the next episode," "watch replay highlights," and "recommend similar content," while simultaneously displaying prompts for the user to confirm whether deeper interaction is needed. When the user chooses to engage in deeper interaction, the client sends the scene video data corresponding to the key scenario or the identification information related to that scenario to the cloud, where the cloud executes subsequent processing. Upon receiving the scene video data from the client, the cloud can perform semantic analysis and context reconstruction of the scenario. For example, in the case of a sporting event that has completely ended, the cloud can identify the event category, the two opposing teams, and the final score; in the case of an episode of a TV series that has completed, it can identify the episode number, the series it belongs to, and the user's viewing progress. After confirming the scenario, the cloud can perform a deep analysis task. That is, compared to the basic judgment on the client side, the cloud utilizes more complex algorithm models, recommendation engines, and content retrieval systems to perform extended processing on the current scenario to generate second interactive content data, which may include text descriptions, data metrics, recommendation lists, and executable instruction information. For example, in the case of a match ending, the cloud can generate complete match statistics, player performance analysis, win rate review curves, and related match information recommendations; in the case of a movie or TV series finishing airing, it can generate a list of similar themes, suggestions for the next episode, recommendations for similar programs, or behind-the-scenes information. The cloud then returns the second interactive content data to the client. The client then instantly displays or performs corresponding operations based on the received data, such as displaying a detailed statistics page, popping up a related recommendation interface, or executing a channel switching command.

[0054] The above solution ensures efficient use of system resources while providing users with a more intelligent and hierarchical interactive experience at the end of content delivery. In this embodiment, the cloud platform undertakes the core responsibilities of deep semantic analysis and extended content generation, while the client platform is responsible for user interface presentation and operation response. The two work together to achieve a progressive interactive processing mechanism from basic prompts to in-depth services.

[0055] In one specific implementation, generating first interactive content data based on scene video data includes: identifying the scene video data to determine the scene type; generating a retrieval task corresponding to the scene type; inputting the retrieval task into an intelligent model, which then executes the retrieval task and outputs the first interactive content data.

[0056] In this embodiment, after receiving the scene video clip uploaded by the client, the cloud can perform semantic analysis on the video footage and any accompanying audio information using image recognition models, temporal analysis models, or multimodal models to determine the specific business category to which the clip belongs. Scene type is a classification result with clear semantic meaning, such as "halftime," "end of period," or "game pause" in sports events, or a stage in other programs. In addition to scene stages, the recognition process can further determine contextual data such as information about the two teams, the event category, and specific match identifiers.

[0057] After determining the scenario type, the cloud constructs task instructions matching that type based on the recognition results. A retrieval task refers to a set of query and generation instructions targeting data resources or knowledge bases. It is a structured task description dynamically generated based on the current scenario type and contextual information, and can include retrieving real-time match data, calling statistical analysis modules, querying historical match records, generating win rate predictions, and summarizing popular comments. For example, when the scenario type is "halftime," the retrieval task could include calculating the remaining break time, querying the current score and technical statistics, and generating interactive prompts; when the scenario type is "end of the period," the retrieval task could include calling a prediction model to output win rate trends or retrieving frequently used comments from social media platforms.

[0058] Intelligent models can be constructed using a system architecture capable of task planning, execution, and evaluation. Specifically, this can be a relatively complete intelligent agent invoked through a third-party interface, or an intelligent model trained autonomously. During this process, the intelligent model can not only retrieve structured data but also generate natural language prompts or interactive suggestions based on a language generation model. The final output is the first interactive content data.

[0059] In one specific implementation, generating a retrieval task corresponding to a scene type includes: obtaining user preference information; and generating a retrieval task based on the user preference information, scene type, and scene video data.

[0060] In this embodiment, user preference information refers to a data set reflecting a user's interests, behavioral habits, or historical viewing records, such as preferred teams, followed players, frequently viewed event types, preferred program themes, frequency of interactive behaviors, and past click records. Before generating a search task, the cloud can retrieve relevant feature data from the user account system or profile database as input conditions for task generation.

[0061] When constructing search instructions, three dimensions are considered simultaneously: the current business stage of the scenario, the specific content reflected in the scenario video, and the user's personal preferences. For example, when the scenario type is "halftime," the scenario video data indicates that the two sides in the match are specific teams, and the user's preference information shows that they have long followed one of these teams or a certain player, the generated search task can focus on querying the team's historical data, the player's performance statistics, or related comments. If the user's preference leans towards tactical analysis, the task can lean towards generating technical statistical charts or win rate prediction curves. If the user's preference is for entertainment and interaction, the task can focus on generating interesting comments or related playlist recommendations. The task type is no longer a fixed template, but rather, given the user's preferences and scenario information, it dynamically analyzes the user's possible optional operation paths and constructs deeper interactive content generation goals. As a result, the intelligent agent will focus on the user's interests when executing tasks, rather than outputting uniform content, thereby achieving customization of the content generation path.

[0062] In another embodiment, after generating detailed interactive content, the cloud can cache and store this content, associating it with scene identifiers and user preferences as tags. When other users watching the same video and exhibiting similar preferences trigger the same scene, the cloud can directly access the generated and cached detailed interactive content without repeating the entire deep analysis process. By classifying or clustering users, users with similar interests can be grouped into similar categories, thereby enabling the reuse and push of interactive content among these users.

[0063] As a specific embodiment, such as Figure 4As shown, this solution adopts a client-cloud collaborative processing architecture. The left side represents the client processing flow, the right side represents the cloud processing flow, and data interaction is conducted via the network in the middle, forming a complete end-to-cloud closed loop. When a user plays a video, the client first acquires the video stream data, which consists of continuously decoded and output frame data and their related time information during playback. The client performs real-time analysis of this video stream data during the playback process, using image recognition or multimodal recognition technology to detect the presence of critical scenarios. Critical scenarios refer to phased nodes with specific business significance, such as the interruption of a sporting event for commercial breaks, the end of a segment, the end of the event, or the completion of an episode of a TV series. When a critical scenario is detected, the client enters a judgment phase to determine whether the current critical scenario belongs to a predetermined scenario. Predetermined scenarios are typically pre-set scenarios in the system that signify termination or phase completion, such as the complete end of a sporting event or the end of a TV series.

[0064] If the judgment result is "no," meaning the current key scene does not belong to the predetermined scene, the client sends the scene video data corresponding to the key scene to the cloud. The scene video data consists of video clips extracted around the key scene, containing necessary contextual information. Upon receiving the scene video data, the cloud's internal intelligent agent processes it. Following a "task planning—task execution—result evaluation" process, the agent first performs semantic recognition based on the scene video data to determine the scene type and related context; then it generates and executes the corresponding retrieval or analysis task; finally, it performs consistency verification and quality evaluation on the generated results. After processing, the cloud outputs detailed information about the key scene or the first interactive content data and returns it to the client. Based on the detailed information returned by the cloud, the client displays interactive options or specific content to the user, thereby achieving real-time interaction highly relevant to the current video scene.

[0065] If the judgment result is "yes," meaning the current key scenario belongs to the predetermined scenario, the client first determines the corresponding interaction option locally and displays that option to the user. For example, at the end of an event, similar content might be recommended or a prompt might appear asking if the next episode is played. The client then asks the user if they want further interaction. If the user chooses not to engage in deeper interaction, the process ends; if the user chooses further interaction, the client sends the scene video data corresponding to the key scenario to the cloud, where a cloud-based intelligent agent performs a deep analysis process. The cloud also generates richer detailed information or secondary interaction content data through task planning, execution, and evaluation mechanisms and returns it to the client. Based on this detailed information, the client displays further interaction options to the user or performs corresponding operations.

[0066] To better implement the first interaction method in this application embodiment, based on this interaction method, this application embodiment also provides an interaction device applied to a client, such as... Figure 5As shown, the interactive device includes: The acquisition module 510 is used to acquire the video stream data played by the user; The recognition module 520 is used to identify video stream data and extract key scenes; The interaction module 530 is used to send the scene video data corresponding to the key scene to the cloud when the key scene is not the predetermined scene, and to interact with the user based on the first interactive content data returned by the cloud. The cloud is used to generate the first interactive content data based on the scene video data.

[0067] In this embodiment, the client and cloud collaborate to interact with the user. The client performs real-time identification of the video stream data played by the user, extracts key scenes, and when it is determined that the key scene is not a predetermined scene, the corresponding scene video data is sent to the cloud for in-depth analysis. The cloud generates first interactive content data and returns it to the client for interaction with the user. Thus, while ensuring real-time performance, accurate identification and targeted processing of video semantic scenes are achieved.

[0068] In some embodiments of this application, the interaction module 530 interacts with the user based on the first interaction content data returned from the cloud, including: Generate the first interaction option based on the first interaction content data; Display the first interactive option; In response to the user's operation signal on the first interaction option, the first interaction content data is displayed.

[0069] In some embodiments of this application, the interaction module 530 is further configured to: When the key scenario is a predetermined scenario, determine the second interaction option corresponding to the key scenario; Display a second interaction option and a third interaction option, wherein the third interaction option is used to ask the user whether they need deeper interaction; In response to the user's operation signal for the third interactive option, the scene video data corresponding to the key scene is sent to the cloud, and the user is interacted with based on the second interactive content data returned by the cloud. The cloud is used to generate the second interactive content data based on the scene video data.

[0070] To better implement the second interaction method in this application embodiment, based on the interaction method, this application embodiment also provides an interaction device applied in the cloud, such as... Figure 6 As shown, the interactive device includes: The receiving module 610 is used to receive scene video data corresponding to key scenes sent by the client. The client is used to identify the video stream data played by the user, extract key scenes, and send the scene video data to the cloud when the key scene is not a predetermined scene. The generation module 620 is used to generate the first interactive content data based on the scene video data; The sending module 630 is used to return the first interaction content data to the client, wherein the client is also used to interact with the user based on the first interaction content.

[0071] In this embodiment, the client and cloud collaborate to interact with the user. The client performs real-time identification of the video stream data played by the user, extracts key scenes, and when it is determined that the key scene is not a predetermined scene, the corresponding scene video data is sent to the cloud for in-depth analysis. The cloud generates first interactive content data and returns it to the client for interaction with the user. Thus, while ensuring real-time performance, accurate identification and targeted processing of video semantic scenes are achieved.

[0072] In some embodiments of this application, the generation module 620 generates first interactive content data based on scene video data, including: Identify scene video data to determine scene type; Generate retrieval tasks corresponding to the scene type; The retrieval task is input into the intelligent model, which then executes the retrieval task and outputs the first interactive content data. In some embodiments of this application, the generation module 620 generates a retrieval task corresponding to the scene type, including: Obtain user preference information; A search task is generated based on user preference information, scene type, and scene video data.

[0073] This application also provides a terminal device that integrates any of the interactive devices provided in this application. The terminal device includes: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor from the steps of the interaction method in any of the embodiments described above.

[0074] This application also provides a terminal device that integrates any of the interactive devices provided in this application. For example... Figure 7 As shown, it illustrates a structural schematic diagram of the terminal device involved in the embodiments of this application. Specifically: The terminal device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will understand that... Figure 7The terminal device structure shown does not constitute a limitation on the terminal device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 701 is the control center of the terminal device. It connects various parts of the terminal device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 702, and by calling data stored in the memory 702, it performs various functions and processes data of the terminal device, thereby providing overall monitoring of the terminal device. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 701.

[0075] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0076] The terminal device also includes a power supply 703 that supplies power to the various components. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 703 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0077] The terminal device may also include an input unit 704, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0078] Although not shown, the terminal device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the terminal device loads the executable files corresponding to the processes of one or more applications into the memory 702 according to the following instructions, and the processor 701 runs the applications stored in the memory 702 to realize various functions, as follows: Get the video stream data played by the user; Identify and extract key scenes from video stream data; When the key scene is not the predetermined scene, the scene video data corresponding to the key scene is sent to the cloud, and the user is interacted with based on the first interactive content data returned by the cloud. The cloud is used to generate the first interactive content data based on the scene video data.

[0079] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0080] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the interactive methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps: Get the video stream data played by the user; Identify and extract key scenes from video stream data; When the key scene is not the predetermined scene, the scene video data corresponding to the key scene is sent to the cloud, and the user is interacted with based on the first interactive content data returned by the cloud. The cloud is used to generate the first interactive content data based on the scene video data.

[0081] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0082] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0083] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0084] The foregoing has provided a detailed description of an interactive method, apparatus, terminal device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An interaction method, characterized in that, Applied to the client side, including: Get the video stream data played by the user; The video stream data is identified, and key scenes are extracted; When the key scene is not a predetermined scene, the scene video data corresponding to the key scene is sent to the cloud, and the user is interacted with based on the first interactive content data returned by the cloud, wherein the cloud is used to generate the first interactive content data based on the scene video data.

2. The interaction method according to claim 1, characterized in that, The interaction with the user based on the first interactive content data returned from the cloud includes: Based on the first interaction content data, generate the first interaction option; Display the first interactive option; In response to the user's operation signal on the first interactive option, the first interactive content data is displayed.

3. The interaction method according to claim 1, characterized in that, Also includes: When the key scenario is the predetermined scenario, a second interaction option corresponding to the key scenario is determined; The second interaction option and the third interaction option are displayed, wherein the third interaction option is used to ask the user whether they need deeper interaction; In response to the user's operation signal for the third interactive option, the scene video data corresponding to the key scene is sent to the cloud, and the user is interacted with based on the second interactive content data returned by the cloud, wherein the cloud is used to generate the second interactive content data based on the scene video data.

4. An interaction method, characterized in that, Applied to the cloud, including: The client receives scene video data corresponding to key scenes sent by the client. The client is used to identify the video stream data played by the user, extract the key scenes, and send the scene video data to the cloud when the key scenes are not predetermined scenes. First interactive content data is generated based on the scene video data; The first interactive content data is returned to the client, wherein the client is further configured to interact with the user based on the first interactive content.

5. The interaction method according to claim 4, characterized in that, The generation of the first interactive content data based on the scene video data includes: The scene video data is identified to determine the scene type; Generate the retrieval task corresponding to the scenario type; The search task is input into the intelligent model, which executes the search task and outputs the first interactive content data.

6. The interaction method according to claim 5, characterized in that, The process of generating the retrieval task corresponding to the scenario type includes: Obtain the user's user preference information; The retrieval task is generated based on the user preference information, the scene type, and the scene video data.

7. An interactive device, characterized in that, Applied to the client side, including: The acquisition module is used to acquire the video stream data played by the user; The recognition module is used to identify the video stream data and extract key scenes; An interaction module is used to send scene video data corresponding to the key scene to the cloud when the key scene is not the predetermined scene, and to interact with the user based on the first interactive content data returned by the cloud, wherein the cloud is used to generate the first interactive content data based on the scene video data.

8. An interactive device, characterized in that, Applied to the cloud, including: The receiving module is used to receive scene video data corresponding to key scenes sent by the client. The client is used to identify the video stream data played by the user, extract the key scenes, and send the scene video data to the cloud when the key scenes are not predetermined scenes. The generation module is used to generate first interactive content data based on the scene video data; The sending module is used to return the first interactive content data to the client, wherein the client is also used to interact with the user based on the first interactive content.

9. A terminal device, characterized in that, The terminal device includes: one or more processors, a memory, and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the interaction method of any one of claims 1 to 3, and to implement the interaction method of any one of claims 4 to 6.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps of the interaction method according to any one of claims 1 to 3, and to perform the steps of the interaction method according to any one of claims 4 to 6.

Images