Multi-mode interaction method and device based on television video information and screen device

Abstract

The application provides a multi-mode interactive method and device based on television video information and a screen device, the method is applied to an interactive television terminal or a digital television terminal, and is used for controlling at least one second screen through a television, the method comprises the following steps: receiving interactive video information, the interactive video information comprises standard video information and interactive information; determining a video interactive frame and corresponding extended information of the video interactive frame based on at least the interactive video information; receiving an interactive request of a target audience to the video interactive frame, extracting the extended information corresponding to the video interactive frame in the interactive information; and controlling a second screen corresponding to the target audience to play the extended information, wherein the second screen comprises a mobile terminal or a wearable device. The application enables the target audience to actively interact with the television terminal without affecting normal linear playing of the television terminal, and enables the target audience to obtain interactive content from the second screen, so that the target audience's television watching experience is not easily interrupted.

Description

Technical Field

[0001] This invention relates to the field of interactive television and Internet television service technology, specifically to a multi-mode interactive method, apparatus, and screen-equipped device based on television video information. Background Technology

[0002] Existing television programs are typically presented in a linear format, with viewers acting as passive recipients, unable to adjust or explore the content in real time. While this one-way transmission model satisfies basic entertainment needs, it suffers from significant shortcomings in the depth, relevance, and interactivity of information acquisition. For example, when viewers have questions about characters' backgrounds, plot developments, sports tactics, or scoring rules, they often need to pause watching and search for relevant information through external channels (such as search engines, social media, or specialized websites). However, television programs are broadcast continuously, and viewers may miss important plot points or exciting moments while searching for information or reflecting, leading to an interrupted and incomplete viewing experience. Furthermore, even if viewers choose to continue watching, their questions may persist, affecting their understanding and appreciation of the program content.

[0003] Interactive systems based on smart TV platforms can activate floating information windows via voice commands, allowing real-time access to encyclopedic data, historical clips, or tactical analysis videos. However, the relevance of the window's content to the currently playing TV program depends on the accuracy of the user's question, rather than the program's content itself; therefore, the correlation between the floating information window's content and the TV program cannot be effectively guaranteed. Furthermore, the appearance of these floating information windows obscures part of the TV screen, and their voice content can easily disrupt the viewing experience of other viewers.

[0004] Video content on mobile devices (such as smartphones or tablets) has gained increasing acceptance due to its ability to be paused at any time and its diverse and rich interactive information (such as bullet comments). How to leverage the unique live / rebroadcast rights of television programs, along with the growing intelligence of interactive / digital television, to address bottlenecks in television manufacturing, sales, and television program production while providing viewers with a richer viewing experience has become a pressing issue for the television industry. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a multi-mode interactive method, device, and screen-equipped device based on television video information. This solves the problem that existing technologies suffer from the shortcomings of linear playback of television programs during interactive / digital television broadcasts, which result in insufficient depth, relevance, and interactivity in information acquisition, leading to easily interrupted viewing experiences and a lack of understanding and appreciation of program content.

[0006] The technical solution of this invention is implemented as follows: a multi-mode interactive method based on television video information, applied to an interactive television terminal or a digital television terminal, for controlling at least one second screen via a television, the method comprising:

[0007] Receive interactive video information, which includes standard video information and interactive information;

[0008] At least based on the interactive video information, determine the video interaction frame and the extended information corresponding to the video interaction frame;

[0009] Receive the interaction request from the target audience to the video interaction frame, and extract the extended information corresponding to the video interaction frame from the interaction information;

[0010] The extended information is played on a second screen corresponding to the target audience, the second screen including a mobile terminal or wearable device.

[0011] This invention, through the execution and control of a second screen on an interactive TV / digital TV terminal, allows the following steps: When the interactive TV / digital TV terminal receives interactive video information including standard video information and interactive information, the digital TV terminal executes and controls the second screen to determine, at least based on the interactive video information, a video interaction frame and corresponding extended information; it receives an interaction request from a target viewer for the video interaction frame, extracts the extended information corresponding to the video interaction frame from the interactive information, and controls the second screen corresponding to the target viewer to play the extended information. This allows viewers to actively interact with the TV terminal without affecting normal linear playback, and to obtain interesting depth and interactive information (such as character backgrounds, plot developments, sports tactics, or scoring rules in TV programs) beyond the standard video information on the second screen. The extended information (such as the aforementioned depth and interactive information) can be activated on the second screen according to the viewer's needs, making the viewer's TV viewing experience less likely to be interrupted; when the viewer does not want to be disturbed, the extended information can be pushed later, improving the viewer's TV viewing experience. Meanwhile, when the same interactive TV / digital TV terminal corresponds to multiple viewers, the interactive TV / digital TV terminal can also push personalized extended information corresponding to different viewers based on the characteristic information of different viewers or the customized services in Internet TV, so as to meet the TV viewing needs of different viewers without affecting the viewing experience of other viewers.

[0012] In one embodiment, addressing the technical shortcomings of existing television video information (standard video information) in one or more of the above-mentioned technical solutions, which cannot provide interactive content or closely associate interactive content with standard video information down to specific video interaction frames, this embodiment also provides an improvement solution. Specifically, the method for generating the interactive video information includes:

[0013] Obtain feature information from standard video information;

[0014] Obtain the interactive extended model or copyright extended information corresponding to this feature information;

[0015] Based on the interactive extension model or copyright extension information, the standard video information is parsed to generate extended sub-information and the first anchor point information corresponding to the extended sub-information.

[0016] At least based on the extended sub-information and the first anchor point information corresponding to the extended sub-information, extended information and interactive information are generated;

[0017] Interactive video information is generated based on interactive information and standard video information.

[0018] The steps described above in this embodiment of the invention typically occur before acquiring standard video information. However, it is readily understood that these steps can also occur while the viewer is watching television video. This involves acquiring feature information of the standard video information; acquiring the interactive extension model or copyright extension information corresponding to the feature information; parsing the standard video information based on the interactive extension model or copyright extension information to generate extended sub-information and first anchor information corresponding to the extended sub-information; enabling real-time or pre-generation of extended sub-information and assigning first anchor information corresponding to the standard video information to the extended sub-information. Then, at least based on the extended sub-information and the first anchor information corresponding to the extended sub-information, extended information and interactive information are generated; and interactive video information is generated based on the interactive information and standard video information. This embodiment of the invention, by generating extended information and interactive information before or while watching television video, achieves the simultaneous generation of interactive video information for both live and rebroadcast / replay signals. This facilitates a close association between interactive content and standard video information and makes it easier to subsequently refine the extended information to specific video interaction frames, further enhancing the viewer's television viewing experience.

[0019] In one embodiment, addressing the drawback of the above-mentioned technical solutions where the playback time of extended information may overlap with the exciting content on the television screen, potentially causing viewers to miss the exciting content while reading interactive information due to distraction from the second screen, thus affecting their television viewing experience, this embodiment also provides an improvement. Specifically, the step of determining the video interaction frame and the corresponding extended information based at least on the interactive video information includes:

[0020] Determine the reading length of each extended sub-information;

[0021] Based on the reading length of the extended sub-information and the first anchor point information corresponding to the extended sub-information, generate first distribution state information based on the extended information;

[0022] Obtain the video segments of interest based on standard video information, as well as the second anchor point information corresponding to each video segment of interest;

[0023] Based on the video length of the video segment of interest and the second anchor point information corresponding to each video segment of interest, a second distribution state information based on the video segment of interest is generated.

[0024] Obtain audience characteristic information and setting information from the second screen;

[0025] Based on the first distribution state information, the second distribution state information, and the audience feature information and setting information of the second screen, determine the vector offset information corresponding to the first anchor point information;

[0026] The video interaction frame and the corresponding extended sub-information are determined based on the first anchor point information and vector offset information.

[0027] This embodiment generates first distribution state information based on the extended sub-information and the first anchor point information corresponding to the extended sub-information, and generates second distribution state information based on the video length of the video segments of interest and the second anchor point information corresponding to each video segment of interest. Then, based on the first distribution state information, the second distribution state information, and the audience characteristic information and settings information of the second screen, vector offset information corresponding to the first anchor point information is determined. This allows the vector offset information, which serves as the playback offset time for the extended sub-information, to be clearly, efficiently, and personalized based on multiple pieces of information, including the first distribution state information, the second distribution state information, and the audience characteristic information and settings information of the second screen. This avoids overlap between the playback time of the extended information and the exciting content on the TV, preventing viewers from missing exciting content on the TV while reading interactive information due to distraction from the second screen, thus improving the viewer's TV viewing experience. The vector offset information is used to determine the video interaction frame and the corresponding extended sub-information based on the first anchor point information.

[0028] In one embodiment, addressing the issue that the interaction between the television and the audience is rather rigid in one or more of the above technical solutions—for example, when the audience temporarily leaves, chats with others, or plays on their phone and misses part of the video content on the television, the television cannot adjust the recommendation priority of extended information in real time based on the missed video content, or the audience cannot identify the more relevant extended information from a large amount of extended information, resulting in a lack of correspondence between the content of the extended information (such as causal explanations of plot points)—this embodiment also provides an improvement solution. Specifically, the audience characteristic information includes audience preference information, audience age information, and playback environment information; the setting information includes playback setting information for extended information; the step of generating second distribution state information based on the video length of the video segment of interest and the second anchor point information corresponding to each video segment of interest includes:

[0029] Based on the playback environment information and the usage information of the second screen, determine the audience's focus information;

[0030] Based on the first video segment of interest that viewers have focused on after the video has been played, and viewer preference information, identify the second video segment of interest that is highly relevant and has not been played.

[0031] Based on the video length of the second video segment of interest and the second anchor point information corresponding to each second video segment of interest, generate second distribution state information based on the second video segment of interest;

[0032] Prioritize each extended sub-information based on the second video segment of interest;

[0033] Update the first distribution state information based on the reading length of the priority-sorted extended sub-informations and the corresponding first anchor point information;

[0034] The step of determining the vector information corresponding to the anchor point information based on the first distribution state information, the second distribution state information, and the audience feature information and setting information of the second screen includes:

[0035] At least the audience's age and the playback environment should be considered to determine the audience's reading efficiency;

[0036] Based on the updated first distribution state information, second distribution state information, reading efficiency, and playback settings information, the vector offset information corresponding to the first anchor point information is determined, and the playback settings information includes playback format and playback efficiency.

[0037] This invention identifies the first video segment of interest already played based on viewer focus information. Then, combining viewer preference information (such as favorite actors, characters, sports, and plot scenes), it identifies an unplayed second video segment of interest that is highly relevant to the plot / scene of the first video segment of interest. The extended sub-information is prioritized based on the second video segment of interest, and the first distribution state information is updated based on the reading length of the prioritized extended sub-information and the corresponding first anchor point information. This allows for the push of more and higher-priority extended information when the viewer is interested in or relatively focused on the currently playing standard video information, and fewer extended information when the viewer is relatively uninterested or distracted. Simultaneously, it ensures a high degree of continuity between the first and second video segments of interest previously viewed by the viewer and their corresponding extended sub-information, while the above adjustments are highly real-time, guaranteeing a consistent viewing experience across various behavioral modes.

[0038] Meanwhile, this embodiment determines the viewer's reading efficiency based at least on the viewer's age information and playback environment information; then, based on the updated first distribution state information, second distribution state information, reading efficiency, and playback settings information, it determines the vector offset information corresponding to the first anchor point information. This makes the determination of vector offset information more accurate, reliable, and real-time, and highly correlated with the viewer's behavioral patterns.

[0039] In the previous embodiment, as an improvement to the technical solution of determining the video interaction frame based on the television end, the step of receiving the target viewer's interaction request for the video interaction frame and extracting the extended information corresponding to the video interaction frame from the interaction information includes:

[0040] Based on the priority-sorted extended sub-information, a priority identification identifier is added to the standard video information, and the priority identification identifier corresponds to the video interaction frame.

[0041] When the video plays to the video interaction frame corresponding to the extended information, a priority identification flag corresponding to the current video interaction frame is displayed.

[0042] Based on the target audience's selection operation of the priority identification identifier in the video interaction frame, the extended sub-information corresponding to the video interaction frame is extracted from the interaction information.

[0043] The step of controlling a second screen corresponding to the target audience to play the extended information, wherein the second screen includes a mobile terminal or wearable device, includes:

[0044] The second screen corresponding to the target audience is controlled to play extended sub-information corresponding to the video interaction frame during the video interaction frame.

[0045] This invention prioritizes extended sub-information by using priority identification markers, allowing viewers to select higher-priority extended sub-information to play on the second screen, thus avoiding missing exciting content on the television and further enhancing the viewer's television viewing experience.

[0046] In one embodiment, addressing the issue of the rise of the short video industry and the relative decrease in television video practitioners in one or more of the above technical solutions, this embodiment also provides a method for obtaining copyright extension information. This method aims to provide television video practitioners with stable and reliable copyright revenue, ensuring the sustainability of the television industry. Specifically, the step of obtaining copyright extension information corresponding to the feature information includes:

[0047] When it is determined that the feature information includes replay information, the keyword information in the feature information is determined, and the keyword information includes video name information, video category information and video date information;

[0048] The interactive information resource database of the server is retrieved based on the keyword information, and the interactive information resource database includes copyright author information and corresponding copyright extended information.

[0049] Viewer customized service information is obtained based on viewer accounts, including the resource usage period of copyrighted authors;

[0050] When the resource usage period has not expired, obtain the copyright extension information corresponding to the standard video information from the copyrighted author.

[0051] This embodiment determines keyword information within the feature information of standard video information when it is determined that the feature information includes replay information. Then, it retrieves the interactive information resource library of the server based on the keyword information. Next, it obtains viewer-customized service information based on the viewer's account and acquires the copyright extension information corresponding to the standard video information from the copyrighted authors. This allows television video professionals (including but not limited to sports commentators, film critics, or plot analysts) to extend the standard video information according to their personal understanding, thereby obtaining extended information with different personal styles, which can be used to generate interactive video information later. Viewers can subscribe to customized services from different copyrighted authors according to their preferences, thereby obtaining different personalized copyright extension information. After parsing, the copyright extension information yields extended sub-information and the first anchor point information corresponding to the extended sub-information for subsequent use. This embodiment of the invention protects the intellectual property rights and revenue of copyrighted authors. Furthermore, since the copyrighted extended information of the copyrighted author is only sent to the television terminal simultaneously with the standard video information when the customized service information matches, it avoids the possibility of copyrighted authors infringing on the standard video information, giving copyrighted authors a natural advantage in their production activities on television terminals compared to short video platforms on mobile terminals.

[0052] The present invention also provides a multi-mode interactive device based on television video information, applied to an interactive television terminal or a digital television terminal, for controlling at least one second screen via a television, the device comprising:

[0053] A receiving module is used to receive interactive video information, which includes standard video information and interactive information;

[0054] The determining module is used to determine, at least based on the interactive video information, the video interaction frame and the extended information corresponding to the video interaction frame;

[0055] An extraction module is used to receive an interaction request from a target viewer to the video interaction frame and extract the extended information corresponding to the video interaction frame from the interaction information.

[0056] A control module is used to control a second screen corresponding to the target audience to play the extended information. The second screen includes a mobile terminal or a wearable device.

[0057] The present invention also provides a television platform for implementing the multi-mode interaction method described in any of the above claims.

[0058] The present invention also provides a device with a screen for interacting with a television terminal as a second screen, wherein the television terminal executes a multi-mode interaction method based on television video information as described in any one of the above claims.

[0059] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned multi-mode interactive method based on television video information.

[0060] The present invention also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned multi-mode interactive method based on television video information. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart of a multi-mode interaction method based on television video information according to the first embodiment of the present invention;

[0063] Figure 2 This is a flowchart of a multi-mode interaction method based on television video information according to a second embodiment of the present invention;

[0064] Figure 3 This is a detailed flowchart of S202 of the second embodiment of the present invention;

[0065] Figure 4 This is another detailed flowchart of S202 of the second embodiment of the present invention;

[0066] Figure 5 This is a detailed flowchart of S209 of the second embodiment of the present invention;

[0067] Figure 6 This is a detailed flowchart of S211 of the second embodiment of the present invention;

[0068] Figure 7 This is a detailed flowchart of S213 of the second embodiment of the present invention.

[0069] Figure 8 This is a schematic diagram of the structure of a multi-mode interactive device based on television video information according to a third embodiment of the present invention;

[0070] Figure 9 This is a schematic diagram of the internal structure of a computer according to another embodiment of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Well-known modules, units, and their connections, links, communications, or operations are not shown or described in detail. Furthermore, the described features, architectures, or functions can be combined in any way in one or more embodiments. Those skilled in the art should understand that the various embodiments described below are only for illustrative purposes and not for limiting the scope of protection of the present invention. It is also readily understood that the modules, units, or processing methods in the various embodiments described herein and shown in the accompanying drawings can be combined and designed in various different configurations. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] The definitions of various terms or methods used in the following embodiments are, except where logically impossible, generally defined as broad concepts that can be implemented under the premise of the content disclosed in the embodiments. Under this understanding, all specific subordinate limitations of the terms or methods should be considered as part of the invention, and should not be narrowly interpreted or biased simply because the specification does not disclose such a specific limitation. For example, when the present invention refers to a cloud platform, it includes not only virtual network servers but also real physical devices, which not only have data storage capabilities but also data processing, intelligent analysis, and reasoning capabilities. Similarly, provided logically feasible, the order of steps in the method is flexible and varied, and all specific subordinate limitations within the broad concepts of various terms or methods fall within the scope of protection of this invention.

[0073] First embodiment:

[0074] Please refer to Figure 1 As shown, this invention discloses a multi-mode interaction method based on television video information, applied to an interactive television terminal or a digital television terminal, for controlling at least one second screen via a television. The method includes:

[0075] S11, Receive interactive video information, which includes standard video information and interactive information.

[0076] The interactive TV terminal or digital TV terminal described in this embodiment includes large-screen TV terminals such as floor-standing TVs, wall-mounted TVs, and projectors. The floor-standing TVs and wall-mounted TVs can be LCD TVs, LED TVs, QLED TVs, MiniLED TVs, or OLED TVs, and the projectors can be LCD projectors, DLP projectors, or LCoS projectors.

[0077] Interactive video information is typically sent from internet servers, television stations, or cloud platforms. The standard video information in this embodiment is usually traditional live / rebroadcast / recorded television video signals, including traditional advertising information, news information, documentaries, educational films, TV dramas, and movies continuously played on CCTV, local stations, on-demand channels, or special channels. Correspondingly, the interactive information mentioned above is interactive information that is not part of the standard video information but is deeply related to its content.

[0078] As a specific solution and not a limitation, the aforementioned interactive information may include extended information, haptic information, or smart home control information. The haptic information includes, but is not limited to, vibration information, olfactory information, and light and shadow information, used to control the corresponding haptic terminal to operate and to achieve interaction with the television. The smart home control information is used to control the operation of other smart home terminals in the network. The extended information is played on the second screen and is mainly used to provide in-depth and interactive information (such as background information on characters in television programs, plot development, tactical actions in sports competitions, or scoring rules, etc.). In this embodiment, tactical actions include both individual technical actions and team tactical actions. In some other embodiments, the extended information may also include advertising information.

[0079] S12, determine the video interaction frame and the extended information corresponding to the video interaction frame based at least on the interactive video information.

[0080] The interactive video information in this embodiment includes interactive information, which includes video interactive frames and extended information corresponding to the video interactive frames. The video interactive frames are used to describe the reference time frame of the corresponding extended information in the standard video information. It should be noted that the video interactive frame can be a single frame, but usually it can be dozens or hundreds of frames in a continuous video, so that the prompt of the extended information is for a period of time.

[0081] S13, receive the target audience's interaction request for the video interaction frame, and extract the extended information corresponding to the video interaction frame from the interaction information.

[0082] In this embodiment, when a viewer is watching the video corresponding to the standard video information in real time and the video reaches the corresponding interactive frame, a corresponding reminder will be displayed on the second screen or the TV screen to inform the viewer that extended information corresponding to the current standard video information can be obtained. The reminder may include, but is not limited to, visual or vibration prompts on the second screen, and indicator markers pushed onto the TV screen. Viewers can activate an interaction request for the video interactive frame via remote control buttons, voice commands, or touch operations on the second screen. At this time, the TV extracts the extended information corresponding to the current video interactive frame from the interactive information.

[0083] In this embodiment, there can be multiple viewers and second screens. Therefore, the target viewer in this embodiment refers to the viewer who sends an interaction request to the video interaction frame, used to distinguish other viewers who have not sent an interaction request. Taking remote control buttons as an example, the idle buttons on the remote control during standard video information playback are obtained and assigned to different viewers, with prompts displayed on the TV screen / each second screen. When a target viewer presses a remote control button corresponding to their identity, their personalized identity is determined. Personalized identity includes, but is not limited to, the target viewer's age, education level, or level of knowledge about the category of the currently playing standard video information. Similarly, voice commands can distinguish different viewers by recognizing their timbre, voiceprint, and other voice characteristics, thereby determining the target viewer. Additionally, account information or hardware address information from different second screens can also be used to distinguish different viewers, thereby determining the target viewer.

[0084] S14, control the second screen corresponding to the target audience to play the extended information, the second screen including a mobile terminal or wearable device.

[0085] The playback timing described above can be in real-time in response to viewer interaction requests, or it can be delayed; this embodiment does not impose any limitations. Mobile terminals in this embodiment include, but are not limited to, mobile phones, tablets, or laptops; wearable devices include, but are not limited to, AR / VR / MR glasses, AR / VR / MR helmets, or smartwatches / bracelets.

[0086] The connection methods between the second screen and the TV include, but are not limited to, Wi-Fi or Bluetooth pairing, QR code scanning, NFC (Near Field Communication), and multi-device shared app implementation. It should be noted that there can be multiple second screens to simultaneously push extended information to different viewers, or to push customized extended information to different viewers at corresponding video interaction frames.

[0087] Once the personalized identity of the target audience is determined, the second screen corresponding to the target audience is determined through the matching information of "second screen-audience". The method of obtaining the matching information of "second screen-audience" can be manual or automatic, and this embodiment does not impose any restrictions.

[0088] The control of the second screen referred to in this invention includes, but is not limited to, controlling the second screen to play extended information, controlling the second screen to provide prompts based on the timing of video interaction frames, and controlling the second screen to store, analyze, classify, and establish associations for the extended information. For example, when the second screen plays the second related extended information B2, the first related extended information B1 and the third related extended information B3 associated with the second related extended information B2 are associated and stored in the second screen's memory, allowing viewers to review them specifically. The control referred to in this invention includes both directly establishing a connection with the second screen to achieve direct control of the second screen from the television terminal, and indirectly establishing a connection with the second screen through a server to achieve indirect control of the second screen from the television terminal via the server.

[0089] As a preferred option rather than a limitation, corresponding to the above-described implementation scheme of obtaining the personalized identity of the target audience in S13, this step S14 obtains the target audience's age, education level, or knowledge accumulation level of the category described in the currently played standard video information, and further adjusts the extended information by adding or subtracting content and adjusting the knowledge depth, thereby obtaining updated extended information corresponding to the target audience's age, education level, or knowledge accumulation level of the category described in the currently played standard video information, and controls the second screen corresponding to the target audience to play the video.

[0090] This invention, through the execution and control of a second screen on an interactive TV / digital TV terminal, allows the following steps: When the interactive TV / digital TV terminal receives interactive video information including standard video information and interactive information, the digital TV terminal executes and controls the second screen to determine, at least based on the interactive video information, a video interaction frame and corresponding extended information; it receives an interaction request from a target viewer for the video interaction frame, extracts the extended information corresponding to the video interaction frame from the interactive information, and controls the second screen corresponding to the target viewer to play the extended information. This allows viewers to actively interact with the TV terminal without affecting normal linear playback, and to obtain interesting depth and interactive information (such as character backgrounds, plot developments, sports tactics, or scoring rules in TV programs) beyond the standard video information on the second screen. The extended information (such as the aforementioned depth and interactive information) can be activated on the second screen according to the viewer's needs, making the viewer's TV viewing experience less likely to be interrupted; when the viewer does not want to be disturbed, the extended information can be pushed later, improving the viewer's TV viewing experience. Meanwhile, when the same interactive TV / digital TV terminal corresponds to multiple viewers, the interactive TV / digital TV terminal can also push personalized extended information corresponding to different viewers based on the characteristic information of different viewers or the customized services in Internet TV, so as to meet the TV viewing needs of different viewers without affecting the viewing experience of other viewers.

[0091] Second embodiment:

[0092] Given the technical shortcomings of the existing television video information (standard video information) in the above embodiments, which cannot provide interactive content and cannot closely associate interactive content with standard video information down to specific video interaction frames, and also considering the potential overlap between the playback time of extended information and the exciting content on the television screen, viewers may miss the exciting content on the television screen due to distraction while reading the interactive information, thus affecting their television viewing experience. Please refer to... Figures 2 to 7 As shown, this embodiment also provides a multi-mode interaction method based on television video information, including S201-S214, wherein:

[0093] S201, Obtain feature information from standard video information.

[0094] The standard video information in this embodiment is the same as that in the first embodiment described above, and will not be repeated here. The feature information of the standard video information includes whether the standard video information is live / replay information, the keywords of the standard video information, or program category information, etc.

[0095] S202, Obtain the interactive extended model or copyright extended information corresponding to this feature information.

[0096] In this embodiment, the interaction extension model is mainly used to parse standard video information related to sports and fitness live streams. The step of obtaining the interaction extension model corresponding to the feature information includes S2021-S2024, wherein:

[0097] S2021, Construct and train interactive extension models corresponding to different program categories;

[0098] S2022, when it is determined that the feature information includes live broadcast information, the program category information in the feature information is determined, and the program category information includes major category information and minor category information;

[0099] S2023, Determine the corresponding interactive extension model based on the program category information;

[0100] S2024, Based on the interactive extension model, the structured feature information in the standard video information is analyzed in real time, and the technical and tactical action annotation information corresponding to the structured feature information is obtained. The technical and tactical action annotation information includes at least the basic information of the technical and tactical action, the technical and tactical action analysis information, and the technical and tactical action evaluation information.

[0101] Steps S2021-S2024 above construct interactive extension models for different program categories (such as ball games, chess, or track and field). Then, based on program category information, including major and minor category information, the corresponding interactive extension model is intelligently determined. Based on the interactive extension model, the structured feature information in the standard video information is analyzed in real time, and the corresponding technical and tactical action annotation information is obtained. Based on the basic information of technical and tactical actions, real-time identification and basic explanation of the technical and tactical actions in the live standard video information are achieved. Based on the technical and tactical action analysis information, real-time analysis of the advantages and disadvantages of the live standard video information is achieved. Based on the technical and tactical action evaluation information, real-time analysis of the standardity of the technical and tactical actions in the live standard video information is achieved. The above interactive extension models typically employ dynamic frame sampling technology, fuse spatiotemporal features through a dual-stream convolutional network (TSN), extract motion features through a 3D residual network (3D-ResNet50), output skeletal coordinates through pose keypoint detection (HRNet-W48), model action sequences through a graph convolutional network (ST-GCN), fuse multimodal features through an attention mechanism, and store the technical and tactical action database through a knowledge graph (Neo4j).

[0102] In this embodiment, the copyright extension information is created and uploaded by the copyright author of the network server based on the existing playback content. The step of obtaining the copyright extension information corresponding to the feature information includes S2025-S2028, wherein:

[0103] S2025, when it is determined that the feature information includes replay information, the keyword information in the feature information is determined, and the keyword information includes video name information, video category information and video date information;

[0104] S2026, retrieve the interactive information resource library of the server based on the keyword information, wherein the interactive information resource library includes copyright author information and corresponding copyright extended information;

[0105] S2027, Obtain customized service information for viewers based on their accounts, including the resource usage period of copyrighted authors;

[0106] S2028, when the resource usage period has not expired, obtain the copyright extension information corresponding to the standard video information from the copyrighted author.

[0107] Steps S2025-S2028 above determine keyword information in the feature information when judging that the feature information of the standard video information includes replay information, and then retrieves the interactive information resource library of the server based on the keyword information; then, based on the viewer's account, they obtain viewer-customized service information and acquire the copyright extension information of the purchased copyrighted authors corresponding to the standard video information. This allows television video practitioners (including but not limited to sports commentators, film critics, or plot analysts) to extend the standard video information according to their personal understanding, thereby obtaining extension information with different personal styles, which can be used to generate interactive video information later. Viewers can subscribe to customized services of different copyrighted authors according to their own preferences, thereby obtaining different personalized copyright extension information. After parsing, the copyright extension information yields extended sub-information and the first anchor information corresponding to the extended sub-information for subsequent use. This embodiment of the invention not only protects the intellectual property rights and income of copyrighted authors, but also, since the copyrighted author's copyright extension information is only sent to the television terminal simultaneously with the standard video information when the customized service information matches, it avoids the possibility of copyrighted authors infringing on the standard video information, giving copyrighted authors a natural advantage in their production activities on the television terminal compared to short video platforms on mobile terminals.

[0108] An embodiment of the present invention provides a multi-modal interactive method based on television video information, and also provides a method for copyright authors to extend copyright information based on existing TV series / movies, including S20209-S20219, wherein:

[0109] S20209, the first set of target video segments for obtaining standard video information based on a single video storyline;

[0110] S20210, A second set of target video clips for which standard video information is obtained based on visual / narrative scenes;

[0111] S20211, Obtain a third target video clip set corresponding to different preference tags based on the audience preference tags. The first target video clip set, the second target video clip set, and the third target video clip set may contain the same video clip.

[0112] S20212, Generate basic correlation coefficients for each first target video segment in the first target video segment set based on plot relevance;

[0113] S20213, Perform deduplication operation based on the first target video segment set, the second target video segment set, and the third target video segment set to remove duplicate video segments;

[0114] S20214, Based on different preference tags, combine the first target video segment set, the second target video segment set after deduplication, and the third target video segment corresponding to the preference tags to obtain two or more personalized video segment sets, and define the first target video segment set, the second target video segment set, and the third target video segment in the same personalized video segment set as personalized video segments.

[0115] S20215, Based on the basic correlation coefficient, generate personalized correlation coefficients for each personalized video segment in each personalized video segment set;

[0116] S20216, When the copyright author creates extended sub-information for a personalized video clip, one or more personalized video clips with a personalized correlation coefficient higher than a preset value are selected from the corresponding set of personalized video clips, so that the copyright author can determine the video clip of interest corresponding to the preference tag.

[0117] S20217, in response to the copyright author's confirmation of the plot supplement of the extended sub-information, push the video clip of interest corresponding to the current extended sub-information, so that the copyright author can make the plot supplement information corresponding to the current extended sub-information;

[0118] S20218, in response to the copyright author's operation of determining the first anchor information of the extended sub-information, determine the reading length of the corresponding extended sub-information, and determine the first anchor information between two video segments of interest based on the reading length;

[0119] S20219, Based at least on the extended sub-information and the corresponding first anchor point information, determine the copyright extension information, which includes plot supplement information.

[0120] Steps S20209-S20219 above provide a method for copyright authors to create copyright extension information based on existing TV series / movies. This provides a solution for copyright authors to quickly create extended sub-information and generate first anchor information, allowing viewers to access the copyright author's extended copyright information from a second screen while watching television. This ensures that during continuous playback, viewers won't miss exciting plot points of videos they're interested in, and can interact with the second screen to obtain corresponding commentary from the copyright author. Simultaneously, steps S2029-S20211 above, by generating a personalized correlation coefficient, allow copyright authors to add personalized commentary based on their understanding and the viewer's preferences (determined by the aforementioned preference tags). For example, the extended information corresponding to the personalized commentary can focus on a character from the show, greatly enhancing the viewer's immersion. Finally, by pushing video clips of interest corresponding to the current extended sub-information, the copyright author can create supplementary plot information for the current extended sub-information. This allows viewers to quickly fill in the plot chain when they miss some video clips of interest, and further enhances the viewing experience. The supplementary plot information can include corresponding video, image, and text information.

[0121] It should be noted that, by reading the above content, those skilled in the art can also obtain methods for creating copyright extension information based on existing competition videos, etc. All improvements and evolutions based on the concept of this invention fall within the protection scope of this invention.

[0122] S203, based on the interactive extension model or copyright extension information, the standard video information is parsed to generate extended sub-information and the first anchor point information corresponding to the extended sub-information.

[0123] Corresponding to the above interactive extension model, the extended sub-information usually includes video, image and text information corresponding to the identification, explanation, advantages and disadvantages and standardization of technical and tactical movements. The first anchor point information is usually determined by the interactive extension model and is the insertion time information of the standard video information corresponding to each extended sub-information. The first anchor point information can be located in the athlete's preparation time or movement interval time in the live broadcast (such as the competition pause time or rest time).

[0124] Corresponding to the aforementioned copyright extension information, the extension sub-information and the first anchor point information are usually created and determined by the copyright author, and may also include corresponding video, image and text information.

[0125] S204, at least based on the extended sub-information and the first anchor point information corresponding to the extended sub-information, generate extended information and interactive information.

[0126] The extended information is a collection of all extended sub-information and the first anchor point information. The interactive information also includes the haptic information or smart home control information described in the first embodiment above, which will not be repeated here.

[0127] S205 generates interactive video information based on interactive information and standard video information.

[0128] S206, determine the reading length of each extended sub-information.

[0129] As a quantitative indicator, the reading length in this step includes the number of text and images in the extended sub-information, as well as the duration of the video content.

[0130] S207, Based on the reading length of the extended sub-information and the first anchor point information corresponding to the extended sub-information, generate first distribution state information based on the extended information.

[0131] The first distribution state information includes the reading length of the extended sub-information and the first anchor point information corresponding to the extended sub-information, which are used to characterize the distribution of the audience's usage time on the second screen.

[0132] S208: Obtain the video segments of interest and the second anchor point information corresponding to each video segment of interest based on the standard video information.

[0133] In this embodiment, the video clip of interest can be the main storyline / exciting plot of a movie / TV series, or a segment of a sports event where viewers are focused on an athlete's appearance / competition. It typically corresponds to interactive video information in a broadcast or replay. The second anchor point information in this embodiment is similar to the first anchor point information; it can be determined by the interactive extension model or by the copyright holder, and it corresponds to the playback start time information on the television screen for the standard video information.

[0134] S209, Generate second distribution state information based on the video segments of interest and the second anchor point information corresponding to each video segment of interest.

[0135] The second distribution state information includes the video length of the video segments of interest and the second anchor point information corresponding to each video segment of interest, which is used to characterize the distribution of viewers' viewing time on the television.

[0136] S210, acquires audience characteristic information and setting information for the second screen.

[0137] As an example and not a limitation, the aforementioned audience characteristic information includes audience preference information, audience age information, and playback environment information. The settings information includes playback settings information as an extension. Audience preference information and audience age information can be obtained through audience accounts. Audience preference information includes audience favorite actors, roles, favorite sports, and plot scenes. The aforementioned playback environment information can be obtained through the camera / microphone on the second screen or the television. Playback environment information includes, but is not limited to, audience departure information, audience dialogue status information, and current time information.

[0138] It should be noted that when there are multiple second screens, it is necessary to determine the audience characteristics and settings of each audience member.

[0139] The preceding step S209 further includes S2091-S2095, wherein:

[0140] S2091, determine the audience focus information based on the playback environment information and the usage information of the second screen.

[0141] In step S2091, the audience's departure information, dialogue status information, and current time information are obtained through the camera / microphone on the second screen or the TV. Combined with the usage information of the second screen, the time period during which the audience is focused on the interactive video information is determined, thereby obtaining the audience focus information.

[0142] It should be noted that when there are multiple second screens, it is necessary to determine the focus information of each audience member.

[0143] S2092, Based on the first video segment of interest that viewers have focused on and the viewer's preference information, determine the second video segment of interest that is highly relevant to the video segment that has not been played;

[0144] This step S2092 further filters video segments of interest during the time period when viewers are focused on standard video information on the television screen, obtaining the first video segment of interest that viewers are focused on. Combined with viewer preference information, it determines the second video segment of interest that is highly relevant but not yet played. This method has extremely high real-time performance, allowing the second video segment of interest to be adjusted in real time according to the viewer's personalized needs. For example, the high relevance referred to in this embodiment can mean that the plot or tactical actions of the second video segment of interest are highly related to those of the first video segment of interest.

[0145] It should be noted that when there are multiple second screens, it is necessary to determine the unplayed second video segments of interest for each viewer based on their primary focus on the video segment and their individual preferences.

[0146] S2093, Generate second distribution state information based on the second video segment of interest, according to the video length of the second video segment of interest and the second anchor point information corresponding to each second video segment of interest.

[0147] Compared to S209 above, step S2093 effectively reduces the distribution range of the second distribution state information by extracting the second video segment of interest and its corresponding second anchor point information. This reduces the number of second video segments of interest when the viewer is relatively uninterested or distracted by the standard video information currently being played on television, thereby reducing the viewer's energy expenditure on the television.

[0148] It should be noted that when there are multiple second screens, it is necessary to generate second distribution state information corresponding to different viewers.

[0149] S2094, Prioritize the corresponding extended sub-information according to the second video segment of interest.

[0150] This step prioritizes each extended sub-information based on the time period during which viewers focus on the standard video information on the TV screen and the second most interesting video clips. It should be noted that the main storyline / exciting plot points have a relatively high weighting value during the priority ranking to prevent viewers from missing them.

[0151] For example, when prioritizing each extended sub-information, priority labels are assigned to each second video segment of interest, such as "main plot", "special effects", "XX character / star / athlete", "XX tactical action", etc., and different colors are assigned to the priority labels according to their priorities.

[0152] S2095, update the first distribution state information according to the reading length of the extended sub-information after priority sorting and the corresponding first anchor point information.

[0153] Compared to S207 above, step S2095, by prioritizing the reading length of the extended sub-information and its corresponding first anchor point information, preferably further filters out higher-priority extended sub-information. Based on the filtered higher-priority extended sub-information, its corresponding reading length, and the first anchor point information, the first distribution state information is updated. This effectively reduces the distribution range of the first distribution state information, thus reducing the viewer's attention expenditure on the second screen when they are relatively uninterested or distracted by the standard video information currently being played on television. Simultaneously, step S2095 can quickly filter out extended sub-information that is of high viewer interest from a large number of extended sub-information items, reducing the overlap between the first and second distribution state information. This enhances the viewer's viewing experience.

[0154] It should be noted that when there are multiple second screens, it is necessary to generate initial distribution state information corresponding to different viewers. This ensures that different vector offset information is generated subsequently, and personalized extended sub-information is pushed to different viewers.

[0155] Steps S2091-S2095 above determine the first video segment of interest already played based on viewer focus information. Then, combining viewer preference information (such as favorite actors, characters, sports, and plot scenes), they determine the second video segment of interest that is highly related to the plot / sport of the first video segment of interest but has not yet been played. The extended sub-information is then prioritized based on the second video segment of interest. The first distribution state information is updated based on the reading length of the prioritized extended sub-information and the corresponding first anchor point information. This allows for the push of more and higher-priority extended information when the viewer is interested in or relatively focused on the currently played standard video information, and fewer and higher-priority extended information when the viewer is relatively uninterested or distracted. Simultaneously, it ensures a high degree of continuity between the first and second video segments of interest previously seen by the viewer and their corresponding extended sub-information. Furthermore, the above adjustments are highly real-time, ensuring a consistent viewing experience across various behavioral patterns.

[0156] S211, based on the first distribution state information, the second distribution state information, and the audience feature information and setting information of the second screen, determine the vector offset information corresponding to the first anchor point information.

[0157] When there is an intersection between the first distribution state information and the second distribution state information, the current extended sub-information can be played on the second screen using delayed playback or segmented playback.

[0158] As a preferred option and not a limitation, step S211 further includes S2111-S2112, wherein:

[0159] S2111, at least based on audience age information and playback environment information, determine the audience's reading efficiency.

[0160] In this step, the audience's current state is determined by identifying audience departure information, audience dialogue status information, and current time information from the playback environment information. At the same time, the audience's reading efficiency is determined by combining the audience's age information.

[0161] S2112, based on the updated first distribution state information, second distribution state information, reading efficiency, and playback setting information, determine the vector offset information corresponding to the first anchor point information, wherein the playback setting information includes playback format and playback efficiency.

[0162] In this embodiment, the vector offset information is the offset time amount corresponding to the first anchor point information. It can offset forward or backward, and the optimal standard is that the first distribution state information and the second distribution state information do not intersect. The function of the vector offset information is twofold: firstly, to ensure that the first distribution state information and the second distribution state information do not intersect; and secondly, to personalize the position of the video interactive frames according to the viewer's reading efficiency, so as to avoid affecting the viewer's viewing experience.

[0163] S212, determine the video interaction frame and the extended sub-information corresponding to the video interaction frame based on the first anchor point information and vector offset information.

[0164] S213, receive the interaction request from the target audience for the video interaction frame, and extract the extended sub-information corresponding to the video interaction frame from the interaction information.

[0165] Please refer to this as a preferred option rather than a limitation. Figure 7 As shown, this step S213 can specifically include S2131-S2133, wherein:

[0166] S2131, add a priority identification identifier to the standard video information according to the priority sorted extended sub-information, wherein the priority identification identifier corresponds to the video interaction frame;

[0167] S2132, When the video plays to the video interaction frame corresponding to the extended information, the priority identification flag corresponding to the current video interaction frame is displayed;

[0168] Corresponding to the second paragraph of the explanation of S2094 above, step S2132 can display a priority identification mark at the video interaction frame corresponding to the standard video information. The text content of the priority identification mark can be "main plot", "special effects", "XX character / star / athlete", "XX tactical action", etc., and different colors can be used to mark the priority identification mark according to different priorities. As a preferred solution rather than a limitation, for the case of multiple viewers and multiple second screens, step S2132 can also indicate "viewer 1 / viewer 2 / viewer 3" etc. at the text content of the priority identification mark to indicate that the current video interaction frame is indicating extended information for one or more viewers.

[0169] S2133, based on the target audience's selection operation of the priority identification mark in the video interaction frame, extract the extended sub-information corresponding to the video interaction frame from the interaction information.

[0170] Viewers can select the priority identification mark in the video interaction frame through remote control buttons, voice commands, or touch operations on the second screen. At this time, the TV terminal extracts the extended sub-information corresponding to the current video interaction frame from the interaction information.

[0171] S214, control the second screen corresponding to the target audience to play extended sub-information corresponding to the video interaction frame in the video interaction frame.

[0172] This embodiment acquires feature information from standard video information; acquires interactive extension models or copyright extension information corresponding to the feature information; parses the standard video information based on the interactive extension models or copyright extension information to generate extended sub-information and first anchor information corresponding to the extended sub-information; it can realize the generation of extended sub-information in real time or in advance, and assign first anchor information corresponding to the standard video information to the extended sub-information. Then, at least based on the extended sub-information and the first anchor information corresponding to the extended sub-information, extended information and interactive information are generated; interactive video information is generated based on the interactive information and standard video information. This embodiment of the invention, by generating extended information and interactive information before or while watching television video, realizes the simultaneous generation of interactive video information for live signals and rebroadcast / replay signals, which is beneficial for the close association between interactive content and standard video information, and also facilitates the subsequent refinement of extended information to specific video interactive frames, further enhancing the viewer's television viewing experience. Meanwhile, this embodiment generates first distribution state information based on the extended sub-information according to the reading length of the extended sub-information and the first anchor point information corresponding to the extended sub-information, and generates second distribution state information based on the video segments of interest according to the video length of the video segments of interest and the second anchor point information corresponding to each video segment of interest. Then, based on the first distribution state information, the second distribution state information, and the audience feature information and setting information of the second screen, the vector offset information corresponding to the first anchor point information is determined. This allows the vector offset information, which serves as the playback offset time of the extended sub-information, to be determined clearly, efficiently, and personally based on multiple pieces of information, including the first distribution state information, the second distribution state information, and the audience feature information and setting information of the second screen. This avoids the playback time of the extended information overlapping with the exciting content on the TV, preventing viewers from missing exciting content on the TV while reading interactive information due to distraction from the second screen, thus improving the viewer's TV viewing experience. The vector offset information is used to determine the video interaction frame and the extended sub-information corresponding to the video interaction frame based on the first anchor point information.

[0173] Third embodiment:

[0174] Please refer to Figure 8As shown, the present invention also provides a multi-mode interactive device 100 based on television video information, applied to an interactive television terminal or a digital television terminal, for controlling at least one second screen via a television. The device includes:

[0175] The receiving module 110 is used to receive interactive video information, which includes standard video information and interactive information;

[0176] The determining module 120 is used to determine, at least based on the interactive video information, the video interaction frame and the extended information corresponding to the video interaction frame;

[0177] The extraction module 130 is used to receive the interaction request from the target audience to the video interaction frame and extract the extended information corresponding to the video interaction frame from the interaction information;

[0178] The control module 140 is used to control the second screen corresponding to the target audience to play the extended information. The second screen includes a mobile terminal or a wearable device.

[0179] The modules in this embodiment correspond one-to-one with the steps in the first embodiment described above, and will not be repeated here.

[0180] All modules / units in this embodiment are the same as the corresponding steps in one or more of the above method embodiments, and their logical relationships and working principles are also the same, so they will not be repeated here. Those skilled in the art can learn the corresponding virtual modules or units from the above method embodiments to make them correspond to the steps of the above method embodiments. Virtual modules / units not disclosed in this embodiment should also be regarded as the part of the content disclosed in this invention.

[0181] This invention, through the execution and control of a second screen on an interactive TV / digital TV terminal, allows the following steps: When the interactive TV / digital TV terminal receives interactive video information including standard video information and interactive information, the digital TV terminal executes and controls the second screen to determine, at least based on the interactive video information, a video interaction frame and corresponding extended information; it receives an interaction request from a target viewer for the video interaction frame, extracts the extended information corresponding to the video interaction frame from the interactive information, and controls the second screen corresponding to the target viewer to play the extended information. This allows viewers to actively interact with the TV terminal without affecting normal linear playback, and to obtain interesting depth and interactive information (such as character backgrounds, plot developments, sports tactics, or scoring rules in TV programs) beyond the standard video information on the second screen. The extended information (such as the aforementioned depth and interactive information) can be activated on the second screen according to the viewer's needs, making the viewer's TV viewing experience less likely to be interrupted; when the viewer does not want to be disturbed, the extended information can be pushed later, improving the viewer's TV viewing experience. Meanwhile, when the same interactive TV / digital TV terminal corresponds to multiple viewers, the interactive TV / digital TV terminal can also push personalized extended information corresponding to different viewers based on the characteristic information of different viewers or the customized services in Internet TV, so as to meet the TV viewing needs of different viewers without affecting the viewing experience of other viewers.

[0182] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0183] This invention also provides a television platform for implementing the multi-mode interaction method described in the first or second embodiment.

[0184] This invention also provides a device with a screen for interacting with a television as a second screen, wherein the television executes a multi-mode interaction method based on television video information as described in the first or second embodiment above.

[0185] This invention also provides a computer storage medium storing a computer program that, when executed by a processor, implements a multi-mode interactive method based on television video information as described in the above embodiments.

[0186] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of each of the above embodiments of a multi-mode interaction method based on television video information. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0187] Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, RAM, ROM, magnetic disks, or optical disks.

[0188] Corresponding to the computer storage medium described above, one embodiment also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a multi-mode interaction method based on television video information as described in the above embodiments.

[0189] This computer device can be a terminal, and its internal structure diagram can be as follows: Figure 9As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a multi-modal interactive method based on television video information. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0190] This invention, through the execution and control of a second screen on an interactive TV / digital TV terminal, allows the following steps: When the interactive TV / digital TV terminal receives interactive video information including standard video information and interactive information, the digital TV terminal executes and controls the second screen to determine, at least based on the interactive video information, a video interaction frame and corresponding extended information; it receives an interaction request from a target viewer for the video interaction frame, extracts the extended information corresponding to the video interaction frame from the interactive information, and controls the second screen corresponding to the target viewer to play the extended information. This allows viewers to actively interact with the TV terminal without affecting normal linear playback, and to obtain interesting depth and interactive information (such as character backgrounds, plot developments, sports tactics, or scoring rules in TV programs) beyond the standard video information on the second screen. The extended information (such as the aforementioned depth and interactive information) can be activated on the second screen according to the viewer's needs, making the viewer's TV viewing experience less likely to be interrupted; when the viewer does not want to be disturbed, the extended information can be pushed later, improving the viewer's TV viewing experience. Meanwhile, when the same interactive TV / digital TV terminal corresponds to multiple viewers, the interactive TV / digital TV terminal can also push personalized extended information corresponding to different viewers based on the characteristic information of different viewers or the customized services in Internet TV, so as to meet the TV viewing needs of different viewers without affecting the viewing experience of other viewers.

[0191] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0192] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A multi-mode interactive method based on television video information, characterized in that, The application is applied to an interactive television terminal or a digital television terminal, and is used for controlling at least one second screen through a television, and the method comprises the following steps: receiving interactive video information, wherein the interactive video information comprises standard video information and interactive information; determining a video interactive frame and corresponding extended information of the video interactive frame based on at least the interactive video information; receiving an interactive request of a target audience to the video interactive frame, and extracting the extended information corresponding to the video interactive frame in the interactive information; controlling a second screen corresponding to the target audience to play the extended information, wherein the second screen comprises a mobile terminal or a wearable device; a method for generating the interactive video information, comprising the following steps: obtaining feature information of the standard video information; obtaining an interactive extension model or copyright extension information corresponding to the feature information; analyzing the standard video information based on the interactive extension model or the copyright extension information, generating extended sub-information and first anchor point information corresponding to the extended sub-information; generating the extended information and the interactive information based on at least the extended sub-information and the first anchor point information corresponding to the extended sub-information; generating the interactive video information based on the interactive information and the standard video information; the step of determining the video interactive frame and the corresponding extended information of the video interactive frame based on at least the interactive video information, comprising the following steps: determining a reading length of each extended sub-information; generating first distribution state information based on the extended information according to the reading length of the extended sub-information and the first anchor point information corresponding to the extended sub-information; obtaining a video segment of interest and second anchor point information corresponding to each video segment of interest according to the standard video information; generating second distribution state information based on the video segment of interest according to a video length of the video segment of interest and the second anchor point information corresponding to each video segment of interest; obtaining audience feature information and setting information of the second screen; determining vector offset information corresponding to the first anchor point information based on the first distribution state information, the second distribution state information, and the audience feature information and the setting information of the second screen; determining the video interactive frame and the corresponding extended sub-information of the video interactive frame according to the first anchor point information and the vector offset information.

2. The method of claim 1, wherein, The audience feature information comprises audience preference information, audience age information and playing environment information, and the setting information comprises playing setting information of the extended information; the step of generating the second distribution state information based on the video segment of interest according to the video length of the video segment of interest and the second anchor point information corresponding to each video segment of interest, comprising the following steps: determining audience focus information according to the playing environment information and the use information of the second screen; determining a second video segment of interest which has not been played and has high correlation according to the first video segment of interest which has been played and the audience preference information; generating second distribution state information based on the second video segment of interest according to a video length of the second video segment of interest and the second anchor point information corresponding to each second video segment of interest; performing priority sorting on each extended sub-information according to the second video segment of interest. According to the reading length of the extended sub-information after the priority sorting and the corresponding first anchor point information, the first distribution state information is updated; The step of determining the vector information corresponding to the anchor point information based on the first distribution state information, the second distribution state information, and the audience characteristic information and setting information of the second screen comprises: Determining the reading efficiency of the audience according to at least the audience age information and the playing environment information; According to the updated first distribution state information, the second distribution state information, the reading efficiency, and the playing setting information, the vector offset information corresponding to the first anchor point information is determined, and the playing setting information comprises a playing form and a playing efficiency.

3. The method of claim 2, wherein, The step of receiving the interactive request of the target audience on the video interactive frame and extracting the extended information corresponding to the video interactive frame in the interactive information comprises: According to the priority-ordered extended sub-information, a priority identification mark is added to the standard video information, and the priority identification mark corresponds to the video interactive frame; When the video is played to the video interactive frame corresponding to the extended information, the priority identification mark corresponding to the current video interactive frame is displayed; Based on the selection operation of the target audience on the priority identification mark in the video interactive frame, the extended sub-information corresponding to the video interactive frame in the interactive information is extracted.

4. The method of claim 3, wherein, The step of controlling the second screen corresponding to the target audience to play the extended information, wherein the second screen comprises a mobile terminal or a wearable device, comprises: Controlling the second screen corresponding to the target audience to play the extended sub-information corresponding to the video interactive frame in the video interactive frame.

5. The method of claim 4, wherein, The step of obtaining the copyright extension information corresponding to the characteristic information comprises: When it is judged that the characteristic information comprises a replay information, keyword information in the characteristic information is determined, and the keyword information comprises video name information, video category information, and video date information; According to the keyword information, an interactive information resource library of a server is searched, and the interactive information resource library comprises copyright author information and corresponding copyright extension information; Based on the audience account, audience customized service information is obtained, and the customized service information comprises a resource use period of a purchased copyright author; When the resource use period is not up, the copyright extension information corresponding to the standard video information of the purchased copyright author is obtained.

6. A multi-mode interactive apparatus based on television video information, characterized by The device is applied to an interactive television terminal or a digital television terminal, and is used for controlling at least one second screen through a television, and the device comprises: A receiving module is configured to receive interactive video information, wherein the interactive video information comprises standard video information and interactive information; A determining module is configured to determine a video interactive frame and extended information corresponding to the video interactive frame based on at least the interactive video information; An extracting module is configured to receive an interactive request of a target audience on the video interactive frame and extract the extended information corresponding to the video interactive frame in the interactive information; A control module is configured to control a second screen corresponding to the target audience to play the extended information, wherein the second screen comprises a mobile terminal or a wearable device. The method for generating the interactive video information comprises: Obtaining characteristic information of the standard video information; Obtaining an interactive extension model or copyright extension information corresponding to the characteristic information; Based on the interactive extension model or copyright extension information, the standard video information is parsed to generate extended sub-information and the first anchor point information corresponding to the extended sub-information. At least based on the extended sub-information and the first anchor point information corresponding to the extended sub-information, extended information and interactive information are generated; Generate interactive video information based on interactive information and standard video information; The module is specifically used for: Determine the reading length of each extended sub-information; Based on the reading length of the extended sub-information and the first anchor point information corresponding to the extended sub-information, generate first distribution state information based on the extended information; Obtain the video segments of interest based on standard video information, as well as the second anchor point information corresponding to each video segment of interest; Based on the video length of the video segment of interest and the second anchor point information corresponding to each video segment of interest, a second distribution state information based on the video segment of interest is generated. Obtain audience characteristic information and setting information from the second screen; Based on the first distribution state information, the second distribution state information, and the audience feature information and setting information of the second screen, determine the vector offset information corresponding to the first anchor point information; The video interaction frame and the corresponding extended sub-information are determined based on the first anchor point information and vector offset information.

7. A band device, comprising: Used as a second screen for interaction with a television terminal, wherein the television terminal executes a multi-mode interaction method based on television video information as described in any one of claims 1 to 5.

8. A computer storage medium having stored thereon a computer program, characterized in that When the program is executed by the processor, it implements a multi-modal interaction method based on television video information as described in any one of claims 1 to 5.

Images