Interaction method and device, electronic equipment and storage medium

Abstract

The invention provides an interaction method and device, electronic equipment and a storage medium, relates to the field of artificial intelligence, in particular to the technical fields of NLP, speech synthesis, large language models, deep learning, man-machine interaction and the like, and can be used in the application fields of generative search, document intelligent editing, intelligent assistants, intelligent e-commerce and the like. The specific implementation scheme is as follows: playing a target video on a video playing page; wherein the target video is generated based on the plot of the long text, and the audio stream of the target video is obtained by dubbing the long text; under the condition that the first interaction condition is met, entering a screen clearing mode so as to switch from a video playing page to a text browsing page; continuously broadcasting the audio stream on the text browsing page, and synchronously displaying the long text in a rolling manner; wherein a first text segment matched with the current broadcast content of the audio stream in the long text is highlighted.

Description

Technical Field

[0001] The present disclosure relates to the field of AI (Artificial Intelligence), specifically to technical fields such as NLP (Natural Language Processing), speech synthesis, LLM (Large Language Model), deep learning, and human-computer interaction. It can be used in application fields such as generative search, intelligent document editing, intelligent assistants, and intelligent e-commerce, and especially to interactive methods, devices, electronic devices, and storage media. Background Art

[0002] With the rapid development of mobile Internet technology and the gradual popularization of smart devices, short videos, as a highly concentrated, intuitive and easy-to-consume content format, have quickly attracted a large number of users. Through immersive interactive designs such as card slot sliding, short video platforms can improve the efficiency and experience of users browsing content, allowing users to easily obtain a large amount of information or entertainment content in a short period of time. The popularity of this interactive mode reflects users' high pursuit of efficiency in online content consumption. Summary of the invention

[0003] The present disclosure provides an interaction method, an apparatus, an electronic device, and a storage medium.

[0004] According to one aspect of the present disclosure, there is provided an interaction method, comprising:

[0005] Play the target video on the video playback page; wherein the target video is generated based on the plot of the long text, and the audio stream of the target video is obtained by dubbing the long text;

[0006] When the first interaction condition is met, entering a clear screen mode to switch from the video playback page to a text browsing page;

[0007] The audio stream continues to be broadcasted on the text browsing page, and the long text is synchronously scrolled and displayed; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0008] According to another aspect of the present disclosure, there is provided an interaction device, comprising:

[0009] A video playback module, used to play a target video on a video playback page; wherein the target video is generated based on the plot of a long text, and the audio stream of the target video is obtained by dubbing the long text;

[0010] A switching module, configured to enter a clear screen mode to switch from the video playback page to the text browsing page when the first interaction condition is met;

[0011] The broadcast display module is used to continue broadcasting the audio stream on the text browsing page and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the interaction method proposed in the above aspect of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium of computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the interaction method proposed in the above aspect of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the interaction method proposed in the above aspect of the present disclosure.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 A flowchart of an interactive method provided in Embodiment 1 of the present disclosure;

[0021] Figure 2 A flowchart of the interaction method provided in the second embodiment of the present disclosure;

[0022] Figure 3 A flowchart of the interactive method provided in the third embodiment of the present disclosure;

[0023] Figure 4 A flowchart of an interactive method provided in Embodiment 4 of the present disclosure;

[0024] Figure 5 A schematic diagram of the interface of the collection playback page provided in the embodiment of the present disclosure;

[0025] Figure 6 A schematic diagram of the interface of a single episode playback page provided in an embodiment of the present disclosure;

[0026] Figure 7 The interface diagram of the text browsing page provided by the embodiment of the present disclosure is as follows: Figure 1 ;

[0027] Figure 8 A schematic diagram of a directory panel provided in an embodiment of the present disclosure;

[0028] Fig. 9 The interface diagram of the video playback page provided by the embodiment of the present disclosure is shown as follows: Figure 1 ;

[0029] Fig.10 The interface diagram of the text browsing page provided by the embodiment of the present disclosure is as follows: Figure 2 ;

[0030] Fig.11 The interface diagram of the text browsing page provided by the embodiment of the present disclosure is as follows: Figure 3 ;

[0031] Fig.12 A flowchart of an interactive method provided in Embodiment 5 of the present disclosure;

[0032] Fig.13 The interface diagram of the video playback page provided by the embodiment of the present disclosure is shown as follows: Figure 2 ;

[0033] Fig.14 A flowchart of an interactive method provided in Embodiment 6 of the present disclosure;

[0034] Fig.15 A flowchart of an interactive method provided in Embodiment 7 of the present disclosure;

[0035] Fig.16 A schematic diagram of the structure of an interactive device provided in Embodiment 8 of the present disclosure;

[0036] Fig.17 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0038] Although traditional list-based filtering interaction provides relatively detailed content classification and filtering functions, it is unable to cope with the growing demand for fast-paced consumption by users. Users need to spend more time and energy on filtering and searching, which is not in line with the current fast-paced lifestyle and information consumption habits.

[0039] Immersive interactive designs such as card slot sliding can greatly improve the efficiency and experience of users browsing content, allowing users to easily obtain a large amount of information or entertainment content in a short period of time. The popularity of this interactive mode reflects users' high pursuit of efficiency in online content consumption.

[0040] However, while immersive interaction improves consumption efficiency, it also exposes its shortcomings in processing long text content. Since one screen area needs to take into account both content display and gesture operations (such as sliding), long text content often cannot be fully displayed and can only be presented in the form of titles, summaries or previews, which greatly reduces the readability and attractiveness of the content. For users who are accustomed to reading long text content, this experience is undoubtedly discounted.

[0041] In addition, for high-quality creators on short video platforms, if their works (such as illustrated articles, novels, etc.) cannot be presented to users in a complete form, it may dampen their creative enthusiasm, thus affecting the content diversity and richness of short video platforms. In the long run, this is not conducive to the healthy development of short video platforms.

[0042] That is to say, due to the special nature of immersive interaction, before this, most short video platforms focused on "image-heavy, text-light" content, such as videos, photo albums, etc., and usually did not introduce a large amount of long text content. Even if introduced, the text is usually folded to display a small part, and users are required to click to evoke a half-screen panel or jump to an independent landing page to browse the text. After consumption, they manually return to the original immersive scene. The user operation path is long, there are many consumption scenarios, and the interaction cost is high. In addition, the value of video images and sounds is usually low, and only text provides consumption value, which is not a native experience for immersive interaction.

[0043] In summary, existing immersive interactions have at least the following shortcomings:

[0044] (1) Long consumption path: Usually, a small part of the text is displayed in a folded line, and users need to click to call up a half-screen panel or jump to an independent landing page to browse the text. After consumption, they need to manually return to the original immersive scene, and the user operation path is long;

[0045] (2) Non-native experience: For long text content, the value of video images and sounds is usually low, and only text provides consumption value. For immersive interaction, the experience is not native, and the user consumption experience is fragmented and abrupt;

[0046] (3) Low consumption efficiency: Long text and graphic content is lengthy, and users are unable to capture the key points and highlights of the content in a timely manner. It is difficult to judge the attractiveness of the content, and users need to browse before making a decision, which is inefficient.

[0047] Therefore, in response to at least one of the above-mentioned problems, the present disclosure proposes an interaction method, device, electronic device and storage medium.

[0048] The following describes the interactive method, device, electronic device and storage medium of the embodiments of the present disclosure with reference to the accompanying drawings. Before describing the embodiments of the present disclosure in detail, for ease of understanding, common technical terms are first introduced:

[0049] LLM is a type of natural language processing model based on deep learning. Its main features are huge model parameters and complex neural network structure, strong language understanding, context perception and language generation capabilities. It can automatically learn useful feature representations from input data and generate relevant text.

[0050] Figure 1 A flowchart of the interactive method provided in the first embodiment of the present disclosure.

[0051] The interactive method of the embodiment of the present disclosure can be applied to a client, wherein the client refers to a software program running on an electronic device and providing services to a user, etc. Exemplarily, the client can be an application client with video playback capability.

[0052] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, etc. The mobile terminal can be, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0053] like Figure 1 As shown, the interaction method may include the following steps S101 to S103:

[0054] Step S101, playing a target video on a video playing page; wherein the target video is generated based on the plot of a long text, and the audio stream of the target video is obtained by dubbing the long text.

[0055] The long text may be an online article, a literary work (such as a novel, a drama, etc.), a chapter or part of a chapter in a literary work, etc.

[0056] The target video is generated based on the plot of the long text. For example, the target video can be loaded by the client from the server, and the target video is generated, for example, by the server using AIGC (Artificial Intelligence Generated Content) technology, combining the plot of the long text and the object role (such as a person) and other information to generate a video screen, and generating the target video based on the video screen and the cover of the long text.

[0057] The audio stream of the target video is obtained by dubbing the long text. For example, the server can use telepresence TTS (Text to Sound / Speech) dubbing technology to vividly dub or read the long text content to obtain the video stream of the target video.

[0058] In the disclosed embodiment, the client may load the target video from the server and play the target video on the video playback page.

[0059] As an example, taking the client as a short video application client, the client can play the video stream recommended by the server by sliding down. When the video stream recommended by the server includes a target video, the target video can be played through the video playback page.

[0060] Step S102, when the first interaction condition is met, entering the clear screen mode to switch from the video playback page to the text browsing page.

[0061] Among them, the clear screen mode (or text consumption mode) is used to display the long text corresponding to the target video in full screen.

[0062] In the embodiment of the present disclosure, when the client satisfies the first interaction condition, it can enter the clear screen mode to switch from the video playback page to the text browsing page (or called text consumption page).

[0063] Step S103, continue to broadcast the audio stream on the text browsing page, and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0064] Among them, prominent display includes but is not limited to at least one of the following: highlighted display, bold display, enlarged display, eye-catching color marking display, etc.

[0065] In an embodiment of the present disclosure, the client can continue to broadcast the audio stream of the target video on the text browsing page based on the real-time playback progress of the target video, and synchronously scroll and display the long text, wherein the text segment in the long text that matches the current broadcast content of the audio stream (recorded as the first text segment in the present disclosure, that is, the first text segment contains the current broadcast content) is highlighted to facilitate the logged-in user of the client to intuitively browse or locate the current broadcast content from the text browsing page.

[0066] According to the interactive method of the disclosed embodiment, when watching the video content of the target video, the user can be instantly immersed in the long text closely related to the target video through an interactive method. This seamless long text consumption method greatly shortens the user's consumption path for the long text, making the reading of the long text easy and convenient. Among them, the target video itself, as a visual interpretation of the plot of the long text, continues to evolve with the ups and downs of the plot of the long text, which not only improves the creative quality of the video content, but also gives the audience an unprecedented visual feast and optimizes the overall viewing experience. In addition, the audio stream can use the telepresence TTS dubbing technology to vividly dub or read the long text, and the tone fits the emotional context of the plot of the long text. The scene atmosphere is more intense and realistic, allowing the audience to enjoy a more profound and shocking audio-visual feast and improve the user's immersive experience.

[0067] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0068] In order to clearly explain how to determine whether the client satisfies the first interaction condition in any embodiment of the present disclosure, the present disclosure also proposes an interaction method.

[0069] Figure 2 A flowchart of the interactive method provided in the second embodiment of the present disclosure.

[0070] like Figure 2 As shown, the interaction method may include the following steps:

[0071] Step S201, playing a target video on a video playing page; wherein the target video is generated based on the plot of a long text, and the audio stream of the target video is obtained by dubbing the long text.

[0072] For the explanation of step S201, reference may be made to the relevant description in any embodiment of the present disclosure, and no further details will be given here.

[0073] Step S202: when a first touch event triggered on the screen is monitored, the number of first touch points and the coordinates of the first touch points in the first touch event are obtained.

[0074] In an embodiment of the present disclosure, when the client monitors a touch event triggered by a user on the screen (referred to as a first touch event in the present disclosure), the number of touch points in the first touch event (referred to as first touch points in the present disclosure) and the coordinates of the first touch points can be obtained.

[0075] Step S203: determining the first touch gesture based on the number and coordinates of the first touch points.

[0076] In the embodiment of the present disclosure, the client may determine the first touch gesture triggered or executed by the user based on the number and coordinates of the first touch points.

[0077] As a possible implementation method, first, it can be determined whether the first designated gesture for entering the clear screen mode is a single-finger triggered gesture or a multi-finger triggered gesture. In the case that the first designated gesture is a single-finger triggered gesture, it can be determined whether the number of first touch points is 1. If the number of first touch points is 1, the coordinate change trend of the first touch point can be further determined based on the coordinates of the first touch point, and then the first touch gesture performed by the user can be determined based on the coordinate change trend.

[0078] If the number of the first touch points is multiple, it can be directly determined that the client does not meet the first interaction condition. At this time, step S207 can be executed to continue playing the target video on the video playback page.

[0079] As another possible implementation method, when the first designated gesture is a multi-finger triggered gesture, it can be determined whether the number of first touch points matches or is consistent with the number of fingers associated with the first designated gesture (for example, when the first designated gesture is a two-finger triggered gesture (such as a two-finger 'zoom-in' gesture), the number of fingers is 2). If the number of first touch points is consistent with the number of fingers associated with the first designated gesture, the coordinate change trend of the multiple first touch points can be further determined based on the coordinates of the multiple first touch points, and then the first touch gesture performed by the user can be determined based on the coordinate change trend of the multiple first touch points.

[0080] If the number of the first touch points is inconsistent with the number of fingers associated with the first designated gesture, it can be directly determined that the client does not meet the first interaction condition. At this time, step S207 can be executed to continue playing the target video on the video playback page.

[0081] As an example, the first designated gesture for entering the clear screen mode is a two-finger zoom gesture for illustrative explanation. The client can first detect and obtain the number of first touch points in the first touch event and their respective coordinates. If the number of detected first touch points is two, the coordinate change trend of the two first touch points is further analyzed to determine the first touch gesture. For example, the client can track the distance change between the two first touch points, and the movement direction of the two first touch points relative to each other and the center of the screen. Based on the coordinates of the two first touch points and their dynamic distance changes, the client can identify whether the first touch gesture performed by the user is a two-finger pinch gesture, that is, the two first touch points gradually approach each other and may be accompanied by a certain rotation or translation, but the main feature is that the distance between the two first touch points gradually decreases.

[0082] In summary, by determining whether the number of the first touch points is consistent with the number of fingers associated with the first designated gesture, if not, the user may have accidentally touched the screen and executed other gestures. At this time, there is no need to enter the clear screen mode (or text consumption mode), but continue to stay on the video playback page to play the target video, which can maintain a smooth user experience and improve the user's video viewing experience.

[0083] If so, then based on the coordinates of each first touch point, the coordinate change trend of each first touch point is further determined, and based on the coordinate change trend of each first touch point, the first touch gesture performed by the user is identified, which can improve the accuracy of gesture recognition, and then improve the accuracy of page switching to meet the user's actual interaction needs.

[0084] Step S204, determining whether the first touch gesture matches the first designated gesture for entering the clear screen mode, if so, executing steps S205 to S206, if not, executing step S207.

[0085] It should be noted that steps S205 to S206 and step S207 are two parallel implementation methods. In actual application, only one needs to be executed.

[0086] Step S205, determining that the first interaction condition is met, entering a clear screen mode to switch from the video playback page to the text browsing page.

[0087] In an embodiment of the present disclosure, when the first touch gesture performed by the user matches or is consistent with the first designated gesture for entering the clear screen mode, it can be determined that the client satisfies the first interaction condition. At this time, the clear screen mode can be entered to switch from the video playback page to the text browsing page (or text consumption page).

[0088] Step S206, continue to broadcast the audio stream on the text browsing page, and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0089] For the explanation of step S206, reference may be made to the relevant description in any embodiment of the present disclosure, and no further details will be given here.

[0090] Step S207, determining that the first interaction condition is not met, and continuing to play the target video on the video playback page.

[0091] In an embodiment of the present disclosure, when the first touch gesture performed by the user does not match or is inconsistent with the first designated gesture for entering the clear screen mode, it can be determined that the client does not meet the first interaction condition. At this time, there is no need to enter the clear screen mode, that is, the target video can be continued to be played on the video playback page.

[0092] In the interactive method of the disclosed embodiment, the user can directly enter the clear screen mode (or text consumption mode) from the current video playback page through the first designated gesture (such as the two-finger "zoom in" gesture), and the user can browse long texts and manually slide to view texts in the text browsing page corresponding to the clear screen mode (or text consumption mode), which is convenient for users to quickly browse, filter, and locate content. Among them, the consistency of gestures and interactions can ensure that users have zero learning costs when accessing long text content, and can quickly get started with operations, thereby improving the user experience.

[0093] In order to clearly explain how to determine whether the client satisfies the first interaction condition in the above embodiment, the present disclosure also proposes an interaction method.

[0094] Figure 3 This is a flow chart of the interactive method provided in the third embodiment of the present disclosure.

[0095] like Figure 3 As shown, the interaction method may include the following steps:

[0096] Step S301, playing a target video on a video playing page; wherein the target video is generated based on the plot of a long text, and the audio stream of the target video is obtained by dubbing the long text.

[0097] For the explanation of step S301, reference may be made to the relevant description in any embodiment of the present disclosure, and no further details will be given here.

[0098] In any embodiment of the present disclosure, the target video may be generated by the server by following steps A to C:

[0099] Step A: semantically analyze the long text to extract at least one highlight from the long text.

[0100] Among them, highlights include but are not limited to: wonderful highlights, highlight moments, etc.

[0101] As an example, LLM may be used to perform semantic analysis on a long text to extract at least one highlight from the long text.

[0102] Exemplarily, at least one reference example can be obtained, wherein the reference example includes a reference text and highlights in the reference text. In the present disclosure, prompt information can be generated based on the reference example and the long text, wherein the prompt information is used to prompt the LLM with task information to be executed. Thereafter, the LLM can be called to process the prompt information to obtain the highlights in the long text.

[0103] Step B: Generate the video frame corresponding to the highlight based on the plot of the highlight and the context information of the highlight in the long text.

[0104] The number of frames of the video picture corresponding to each bright spot may be at least one frame.

[0105] As an example, AIGC technology may be used to generate a video picture corresponding to each highlight based on the plot of each highlight and the context information of the highlight in the long text.

[0106] Step C: Generate a target video based on the video screen and the cover of the long text.

[0107] As an example, the cover of the long text is displayed by default in the target video. If there is a highlight in the long text, when the start playback time of the highlight is reached, the video screen corresponding to the highlight is displayed, and when the end playback time of the highlight is reached, it switches back to the cover of the long text.

[0108] In summary, by combining the text plot, object roles (such as characters) and other information in long texts, dynamic video images can be mass-produced through AIGC technology, which can be displayed in real time in response to the text, making the long text concrete and allowing users to be immersed in the scene. This can solve the problem of no video images in long text content under immersive containers.

[0109] In any embodiment of the present disclosure, the audio stream of the target video may be generated by the server by following steps A' to C':

[0110] Step A': Recognize the long text to obtain at least one object role to be dubbed and the portrait features of each object role.

[0111] The target characters may be human characters, animal characters, or other characters that require dubbing.

[0112] Among them, the portrait features include but are not limited to the attribute characteristics of the object role (such as age, gender, etc.), personality characteristics (such as introversion, extroversion, optimism, etc.), psychological characteristics, etc.

[0113] As an example, each object role to be dubbed and the portrait features of each object role can be extracted from a long text based on deep learning technology or LLM technology.

[0114] Step B': Based on the portrait features of each target character, a target timbre adapted to each target character is determined from a plurality of timbres in the timbre library.

[0115] As an example, each timbre in the timbre library may have corresponding characteristic information or description information (such as the timbre of a middle-aged calm male). In the present disclosure, for any object character, the portrait features of the object character may be similarly matched with the characteristic information (or description information) of multiple timbres in the timbre library to determine a target timbre that is suitable for the object character from the multiple timbres.

[0116] Step C': Based on the target timbre of each object character and the plot of the long text, dub the long text to obtain an audio stream.

[0117] In summary, it is possible to use immersive TTS technology to read or dub long texts vividly based on the target timbre that matches the various object roles in the long text and the plot of the long text, so that the tone fits the emotional context of the plot and the sense of scene is more realistic and intense, thus solving the problem of no audio in long text content under immersive interaction.

[0118] Step S302: when a trigger operation on a first page element in a video playback page is detected, determining that a first interaction condition is satisfied.

[0119] Among them, the first page element (such as the "Full Text" button) is used to display the long text corresponding to the target video currently being played.

[0120] In the embodiment of the present disclosure, when the client detects the user's triggering operation on the first page element in the video playback page, it can be directly determined that the client meets the first interaction condition.

[0121] Step S303, entering the clear screen mode to switch from the video playback page to the text browsing page.

[0122] Step S304, continue to broadcast the audio stream on the text browsing page, and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0123] The explanation of steps S303 to S304 can be found in the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0124] According to the interactive method of the disclosed embodiment, the user can directly enter the clear screen mode (or text consumption mode) from the current video playback page by triggering or clicking the first page element (such as the "Full Text" button) in the video playback page. The user can browse long texts and manually slide to view text in the text browsing page corresponding to the clear screen mode (or text consumption mode), which is convenient for the user to quickly browse, filter, and locate content, thereby meeting the user's actual knowledge acquisition needs.

[0125] In order to clearly illustrate any embodiment of the present disclosure, the present disclosure also proposes an interaction method.

[0126] Figure 4 A flowchart of the interactive method provided in the fourth embodiment of the present disclosure.

[0127] like Figure 4 As shown, the interaction method may include the following steps:

[0128] Step S401, playing the target video on the video playback page; wherein the video playback page displays a progress bar, the progress bar is used to indicate the playback progress of the target video, and the progress bar is marked with at least one marking point, the marking point is used to indicate the playback time corresponding to the highlight in the long text.

[0129] The target video is generated based on the plot of the long text, and the audio stream of the target video is obtained by dubbing the long text.

[0130] It should be noted that the explanations of the target video and audio streams in the aforementioned embodiments are also applicable to this embodiment and will not be repeated here.

[0131] In an embodiment of the present disclosure, a progress bar is displayed in the video playback page; wherein, the progress bar is used to indicate the playback progress of the target video, and to indicate the broadcast progress of the audio stream, and the progress bar is marked with at least one marking point, each marking point is used to indicate the playback time corresponding to the highlight in the long text, wherein the highlights include but are not limited to: wonderful highlights, highlight moments, etc.

[0132] As an example, LLM can be used to perform semantic analysis on a long text to extract at least one highlight from the long text, and mark a corresponding mark point on the progress bar according to the playing time or broadcasting time of each highlight in the audio stream.

[0133] Step S402: when a trigger operation on a target mark point among at least one mark point is detected, jump to a target playback time indicated by the target mark point.

[0134] In the embodiment of the present disclosure, when the client detects the user's triggering operation on at least one target mark point on the progress bar, it can jump to the target playback time indicated by the target mark point on the progress bar.

[0135] Step S403, playing the target video from the target playback time on the video playback page.

[0136] In the disclosed embodiment, the client can play the video frames in the target video from the target playback time in the video playback page, and synchronously play the audio stream.

[0137] Step S404, when the first interaction condition is met, enter the clear screen mode to switch from the video playback page to the text browsing page.

[0138] Step S405, continue to broadcast the audio stream on the text browsing page, and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0139] The explanation of steps S404 to S405 can be found in the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0140] In any embodiment of the present disclosure, the long text may be obtained by segmenting the content to be processed (such as an article, a literary work (such as a novel, a drama), etc.) to obtain multiple sub-contents (or text segments), and is determined from the multiple sub-contents. Taking the content to be processed as a novel as an example, the novel may be divided according to its chapters to obtain multiple sub-contents, wherein the long text may be any one of the multiple sub-contents.

[0141] Among them, the video playback page can be a collection playback page (or called a collection landing page), wherein the collection playback page is used to play a video collection, the video collection includes multiple episodes of video, and the multiple episodes of video are generated based on the plots of multiple sub-contents, that is, each episode of the multiple episodes of video (including the target video) is generated based on the plot of a sub-content, and its implementation principle is similar to the generation principle of the target video, which will not be repeated here.

[0142] The collection playback page displays at least one of the following information:

[0143] The first item is the currently playing video frame in the target video.

[0144] The second item is the interface title, wherein the interface title is used to indicate the content title of the content to be processed. Taking the content to be processed as a novel as an example, the interface title can be used to indicate the name of the novel.

[0145] The third item is subtitles; subtitles are used to indicate the current broadcast content of the audio stream.

[0146] The fourth item is the first page element, wherein the first page element is used to display the long text corresponding to the currently playing target video. For example, the first page element can be the full text BTN (button).

[0147] The fifth item is the episode number of the target video in the video collection. For example, the target video can be the first episode, the second episode, the third episode, etc. in the video collection.

[0148] The sixth item is the title of a single episode of the target video. Still taking the novel as an example, the title of a single episode of the target video can be the chapter title of the long text in the novel.

[0149] The seventh item is a third page element, wherein the third page element is used to select a video of a desired number of episodes from the video collection. For example, the third page element can select BTN for the number of episodes.

[0150] Item 8, the fourth page element; wherein the fourth page element is used to adjust the playback speed of the target video.

[0151] As an example, the collection playback page (or collection landing page) can be as follows Figure 5 As shown, Figure 5 Area 51 displays the interface title, area 52 displays subtitles, area 53 displays the first page element (full text BTN), area 54 displays the episode number of the target video in the video collection and the single episode title of the target video, area 55 displays the third page element (episode selection BTN), and area 56 displays the fourth page element (double speed BTN).

[0152] Among them, the audio novel played in the video playback page corresponds to or is synchronized with the subtitles.

[0153] in, Figure 5Only the long text corresponding to the target video is used as an example of a chapter in the novel. For the interface title in area 51, the display format can be "Novel-XXX". If one line is not fully displayed, "..." is displayed. For the subtitles in area 52, the text display effect is: a single sentence scrolls up, and the currently read sentence is highlighted. It should be noted that the number of display lines of subtitles supports cloud or server control; the single-line subtitle display rules are: 1. Sentence segmentation rules: whether each sentence ends with a period, question mark, or exclamation mark; 2. Supplementary single-line display rules: the client pre-layouts the text through rich text, confirms the maximum number of words that can be accommodated in each line, and cuts and decomposes the issued sentences. For the episode number in area 54, it is fixedly displayed as [Episode x], where x is a number without 0 in front, and is not displayed as "01" or "001". For the single-episode title in area 54, at most one line of text is displayed. If it exceeds one line, "..." is displayed; if there is no title, it is not displayed here. For the third page element in area 55 (BTN for selecting the number of episodes), the displayed content is: [Novel icon (icon)] [Novel] [-] [Novel name] [ / ] [Updated to xx episodes / Total xx episodes]. If a row is not fully displayed, [Novel name] is omitted and displayed as “…”. For the fourth page element in area 56 (BTN for multiple speed), the default display is 1.25 times.

[0154] Therefore, rich page elements are displayed in the collection landing page to facilitate users to perform corresponding operations on the playing videos based on actual needs, thereby improving the user's video consumption experience.

[0155] In any embodiment of the present disclosure, the long text may be obtained by segmenting the content to be processed (such as an article, a literary work (such as a novel, a drama), etc.) to obtain multiple sub-contents (or text segments), and is determined from the multiple sub-contents. Taking the content to be processed as a novel as an example, the novel may be divided according to its chapters to obtain multiple sub-contents, wherein the long text may be any one of the multiple sub-contents.

[0156] Among them, the video playback page can be a single episode playback page (or called a fusion page), wherein the single episode playback page is used to play a single episode video in multiple episodes in a video collection. The multiple episodes are generated based on the plots of multiple sub-contents, that is, each episode of the multiple episodes (including the target video) is generated based on the plot of one sub-content. Its implementation principle is similar to the generation principle of the target video, which will not be repeated here.

[0157] The single episode playback page displays at least one of the following information:

[0158] Item 1: The currently playing video frame in the target video.

[0159] Item 2: Subtitles; where subtitles are used to indicate the current broadcast content of the audio stream.

[0160] Item 3: The first page element; wherein the first page element is used to display the long text corresponding to the target video currently being played.

[0161] Item 4: The first information display area; wherein the first information display area displays at least one of the content title of the content to be processed (taking the content to be processed as a novel as an example, the content title can be the name of the novel), the number of episodes of the target video in the video collection, the title of a single episode of the target video, the label of the target video and the author information of the content to be processed.

[0162] Item 5: The fifth page element; wherein the fifth page element is used to display the directory of the video collection. For example, the fifth page element may be the directory BTN.

[0163] Item 6: The sixth page element; wherein the sixth page element is used to display the next episode of the target video in the video collection. For example, the sixth page element may be the next episode of BTN.

[0164] Item 7: The seventh page element; wherein the seventh page element is used to adjust the playback speed of the target video. For example, the seventh page element may be a multiple speed BTN.

[0165] As an example, a single episode playback page (or fusion page) can be as follows Figure 6 As shown, Figure 6 The area 61 in the image has subtitles, the area 62 has the first page element (full text BTN), and the area 63 has the first information display area, wherein: Figure 6 Taking the long text corresponding to the target video as an example of the content of a chapter in the novel, the first information display area displays the name of the novel, the episode number of the target video in the video collection, the single episode title of the target video, the label of the target video, the rating and the author name of the novel, the fifth page element (directory BTN) is displayed in area 64, the sixth page element (next episode BTN) is displayed in area 65, and the seventh page element (double speed BTN) is displayed in area 66.

[0166] Among them, for the subtitles in area 61, the text display effect is: a single sentence scrolls upward, and the currently read sentence is highlighted. It should be noted that the number of display lines of subtitles supports cloud or server control; the single-line subtitle display rules are: 1. Sentence segmentation rules: whether each sentence ends with a period, question mark, or exclamation mark; 2. Supplementary single-line display rules: pre-layout the text through rich text on the terminal, confirm the maximum number of words that can be accommodated in each line, and cut and decompose the issued sentences. For the novel name in area 63, at most one line is displayed. If it exceeds, "..." is displayed; for the episode number in area 63: it is fixedly displayed as [Episode x], where x is a number without a leading 0, and it is not displayed as "01" or "001"; for the single episode title in area 63, at most one line of text is displayed. If it exceeds one line, "..." is displayed; for the tags in area 63, it is limited to displaying 3 at most, and garbled characters / tags greater than or equal to 4 characters are filtered; for the author name in area 63, if this line cannot be displayed, "..." is displayed. If the width of this line of information can only display "rating + classification + tag", the author information is not displayed. For the seventh page element in area 66 (double speed BTN), the default display is 1.25 times.

[0167] Therefore, displaying page elements in a single-episode landing page that are not exactly the same as those in a collection landing page can facilitate users to perform different operations, taking into account the knowledge acquisition needs and video playback needs of different users, and meeting the personalized needs of different users.

[0168] In any one of the embodiments of the present disclosure, at least one of the following information is displayed in the text browsing page:

[0169] The first item: long text; the text fragments in the long text that match the current broadcast content of the audio stream are highlighted.

[0170] The second item: the number of episodes of the target video in the video collection.

[0171] The third item: the title of the single episode of the target video.

[0172] The fourth item: the total number of pages of the long text and the number of pages currently displayed.

[0173] Item 5: The second page element; wherein the second page element is used to exit the display of the long text.

[0174] Item 6: The eighth page element; wherein the eighth page element is used to display the directory of the video collection.

[0175] Item 7: The ninth page element; wherein the ninth page element is used to display the sub-content corresponding to the next video of the target video in the video collection.

[0176] Item 8: The tenth page element; wherein, the tenth page element is used to pause the broadcasting of the audio stream and pause the scrolling display of the long text.

[0177] Item 9: the eleventh page element; wherein the eleventh page element is used to adjust the broadcast speed of the audio stream and the scrolling display speed of the long text.

[0178] As an example, a text browsing page (or text consumption page) can be as follows Figure 7 As shown, Figure 7 Area 71 in the diagram displays the episode number and episode title of the target video; area 72 displays the total number of pages of the long text and the number of pages currently displayed, that is, the current page number / all pages of the long text are displayed; area 73 displays the long text, wherein the currently broadcast sentence in the long text can be highlighted; area 74 displays the second page element (Close BTN); area 75 displays the eighth page element (Directory BTN) and the ninth page element (Next Episode BTN), wherein the Directory BTN: displays [Novel icon] [Directory] [Current Episode / All Episodes]; area 76 displays the tenth page element (Pause / Play icon) and the eleventh page element (Multiples icon).

[0179] Therefore, displaying rich page elements in the text browsing page can facilitate users to perform corresponding operations on long texts based on actual needs, thereby improving users' text consumption experience.

[0180] In any one of the embodiments of the present disclosure, when the client detects a user triggering operation on the fifth page element in a video playback page (such as a single episode playback page), a directory panel may be displayed.

[0181] The catalog panel displays at least one of the following information:

[0182] Item 1: The cover of the content to be processed. Taking the content to be processed as a novel as an example, the cover of the novel can be displayed.

[0183] Item 2, the second information display area; wherein the second information display area displays at least one of the content title of the content to be processed (taking the content to be processed as a novel as an example, the content title can be the name of the novel), classification information (such as novel classification), tag information (such as novel tag), rating information (such as novel rating), author information and updated episode number.

[0184] Item 3: Brief information about the content to be processed.

[0185] Item 4, a directory of a video collection; wherein, an episode information display area is displayed at the currently playing target video in the directory; the episode information display area is used to display at least one of the thirteenth page element, the episode number of the currently playing target video in the video collection, the episode title of the target video, and the playback progress of the target video; the thirteenth page element is used to indicate that the target video is being played.

[0186] Item 5: Number of episodes watched in the video collection.

[0187] Item 6, the twelfth page element; wherein, the twelfth page element is used to switch the display order of the directory between sequential display and reverse display.

[0188] As an example, taking the content to be processed as a novel, the directory panel can be as follows Figure 8 As shown, Figure 8 The middle area 81 displays the novel cover; the area 82 displays the second information display area, in which the novel title, novel category, novel tag, novel rating, novel author name, and the number of updated episodes of the novel are displayed; the area 83 displays the novel's brief introduction information; the area 84 displays the directory; the area 85 displays the number of episodes watched in the video collection; the area 86 displays the twelfth page element (switch directory list order BTN), and the area 87 displays the single episode information display area of ​​the currently playing first episode of the video, and the thirteenth page element (now playing icon) is displayed in the single episode information display area, the playback progress of the first episode of the video, that is, the width of the percentage progress bar background = the proportion of the played time * the screen width.

[0189] Among them, for the novel title in area 82, there is no limit on the number of lines. If it exceeds one line, it will be displayed in a broken line; for the novel rating + novel author name in area 82, at most one line is displayed, and if it exceeds, "..." is displayed; for the updated episodes in area 82, if the novel has not been updated, "Updating·Up to xxx episodes" is displayed, and if the novel has been updated, "Completed·Total xxx episodes" is displayed; for the brief introduction information in area 83, at most 3 lines are displayed, and if it exceeds, "..." + expand is displayed; for the number of episodes watched in area 85, x / episodes watched are displayed, and the criterion for judging the number of episodes watched is: as long as there is a playback record, it is recorded as watched; for the twelfth page element in area 86 (BTN for switching the order of the directory list), the directory list defaults to sequential display, and the BTN is displayed as [Switch reverse order]. When the user clicks the BTN, the directory list immediately switches to reverse order display, and the BTN is displayed as [Switch order]; for the single episode information display area in area 87, the thirteenth page element (now playing icon) is only displayed at the currently playing video.

[0190] Therefore, integrating rich information display and interactive functions on the catalog panel can significantly improve the user experience, enhance the attractiveness and accessibility of content, and promote interactivity between users and short video platforms, thereby improving user satisfaction and the competitiveness of short video platforms.

[0191] In any embodiment of the present disclosure, when the client detects that the user triggers the fourth page element or the seventh page element (such as a speed BTN or a speed icon) in the video playback page, the speech rate adjustment panel can be called up, and when the client detects that the user selects the target playback speed from the multiple candidate playback speeds displayed in the speech rate adjustment panel, the target video continues to be played based on the target playback speed, and when the client detects that the user closes the speech rate adjustment panel, the speech rate adjustment panel is collapsed and the target video continues to be played at the target playback speed.

[0192] As an example, take the video playback page as Fig. 9 As shown in the left area of ​​​​the example, the user can click the speed BTN or speed icon in area 91 to call up the Fig. 9 The speech speed adjustment panel shown in area 92 in the middle right area includes the following multiple candidate playback speeds: 0.5 times, 0.75 times, 1.0 times, 1.25 times, 1.5 times, 2 times, 2.5 times, and 3 times. The user can click on a candidate playback speed to change the audio track to that speed. At this time, the speech speed adjustment panel does not disappear. When the user clicks the "Close" button in the speech speed adjustment panel (as shown in area 93), the speech speed adjustment panel is folded and the target video is played at the selected speed.

[0193] In this way, the user can adjust the playback speed of the target video according to the actual video playback needs, which can meet the personalized playback needs of different users and improve the user experience.

[0194] In any embodiment of the present disclosure, when the client detects the user's triggering operation on the eighth page element (such as the directory BTN) in the text browsing page, a directory pop-up layer can be displayed; wherein the target pop-up layer displays a directory of the video collection.

[0195] Among them, a third information display area is displayed at the target video currently being played in the directory; the third information display area is used to display at least one of the fourteenth page element, the number of episodes of the target video in the video collection corresponding to the currently displayed long text, the title of a single episode of the target video, and the playback progress of the target video; the fourteenth page element is used to indicate that the target video is being played.

[0196] Among them, the fourth information display area is displayed in the other episodes of the video in the directory that are not currently being played; the fourth information display area displays at least one of the episode number of the other episodes of the video in the video collection, the single episode title of the other episodes of the video, and the playback progress of the other episodes of the video.

[0197] As an example, a text browsing page such as Fig.10 As shown in the left area of ​​​​the example, when the user clicks Fig.10 In the case of the eighth page element (such as directory BTN) shown in area 101 in the left area, the following may be displayed: Fig.10 The directory pop-up layer shown in area 102 in the middle right area.

[0198] Therefore, integrating rich information display and interactive functions on the catalog pop-up layer can significantly improve the user experience, enhance the attractiveness and accessibility of the content, and promote the interactivity between users and short video platforms, thereby improving user satisfaction and the competitiveness of short video platforms.

[0199] In any embodiment of the present disclosure, when the client detects that the user triggers the eleventh page element (such as a multiple icon) in the text browsing page, the speed control panel can be called up, and when the client detects that the user selects the target speech rate from the multiple candidate speech rates displayed in the speed control panel, the audio stream is broadcast based on the target speech rate, and the long text is scrolled and displayed synchronously, and when the client detects that the user closes the speed control panel, the speed control panel is folded, and the audio stream continues to be played at the target playback speed, and the long text is scrolled and displayed synchronously.

[0200] As an example, a text browsing page such as Fig.11 As shown in the left area of ​​​​the example, when the user clicks Fig.11 In the case of the eleven page elements (such as multiple icons) shown in the area 111 in the left area, the following can be called up: Fig.11 The speed control panel shown in area 112 in the middle right area includes the following multiple candidate speaking rates: 0.5 times, 0.75 times, 1.0 times, 1.25 times, 1.5 times, 2 times, 2.5 times, and 3 times. The user can click on a candidate speaking rate to change the audio track to that speed. At this time, the speed control panel does not disappear. When the user clicks the "Close" button in the speed control panel (as shown in area 113), the speed control panel is folded, the audio stream is played at the selected speed, and the long text is displayed synchronously.

[0201] In this way, the user can adjust the scrolling display speed of long texts and the broadcasting speed of audio streams according to actual text browsing needs, which can meet the personalized browsing needs of different users and improve the user experience.

[0202] The interactive method of the disclosed embodiment marks the playback time corresponding to each highlight in the long text in the progress bar, which can facilitate users to quickly locate the wonderful points and highlight moments in the target video. This efficient video positioning method can improve the user's video viewing experience.

[0203] In order to clearly illustrate any embodiment of the present disclosure, the present disclosure also proposes an interaction method.

[0204] Fig.12 This is a flowchart of the interactive method provided in Embodiment 5 of the present disclosure.

[0205] like Fig.12 As shown, the interaction method may include the following steps:

[0206] Step S1201, playing the target video on the video playback page; wherein the video playback page displays a progress bar, and the progress bar is used to indicate the playback progress of the target video.

[0207] The target video is generated based on the plot of the long text, and the audio stream of the target video is obtained by dubbing the long text.

[0208] For the explanation of step S1201, please refer to the relevant description in any embodiment of the present disclosure, which will not be repeated here.

[0209] Step S1202, when a dragging operation on the progress bar in the video playback page is detected, enter the full-screen mode to continue broadcasting the audio stream, and synchronously scroll the long text and the mask in full screen; wherein the mask displays the enlarged progress bar and the real-time dragging time to which the enlarged progress bar is dragged; the second text segment in the long text that matches the real-time dragging time is highlighted.

[0210] Among them, the full screen mode is used to display long text in full screen without displaying page elements except the long text and the progress bar.

[0211] Among them, prominent display includes but is not limited to at least one of the following: highlighted display, bold display, enlarged display, eye-catching color marking display, etc.

[0212] In the disclosed embodiment, when the client detects the user dragging the progress bar in the video playback page, it can enter the full-screen mode to continue playing the audio stream according to the real-time playback progress of the target video, and synchronously scroll and display the long text and the mask in full screen.

[0213] The mask layer displays the enlarged progress bar and the real-time dragging time to which the enlarged progress bar is dragged. The second text segment matching the real-time dragging time in the long text is highlighted.

[0214] As an example, the video playback page can be as follows Fig.13 As shown in the left area of ​​​​the Fig.13 When the progress bar in the left area of ​​​​the Fig.13 The page shown in the right area of ​​​​the Fig.13 A long text is displayed in full screen in the middle right area, and the second text segment in the long text that matches the real-time dragging moment is highlighted, and, Fig.13 A mask is displayed in the middle right area, which displays the enlarged progress bar, the real-time dragging time to which the enlarged progress bar is dragged (such as 04:12), and the complete playback time of the target video (such as 09:25).

[0215] Step S1203, in response to the stop dragging operation on the enlarged progress bar, determining the target dragging time to which the enlarged progress bar is dragged, and exiting the full screen mode to start playing the target video from the target dragging time on the video playback page.

[0216] In the embodiment of the present disclosure, when the user stops dragging the progress bar, such as when the user releases his finger, the full screen mode can be exited, and the target video can continue to be played based on the target dragging time that the user finally drags to.

[0217] That is, when the client detects that the user stops dragging the enlarged progress bar in the mask, it can determine the target dragging time to which the enlarged progress bar is dragged, and exit the full-screen mode to play the target video on the video playback page from the target dragging time.

[0218] Step S1204, when the first interaction condition is met, enter the clear screen mode to switch from the video playback page to the text browsing page.

[0219] Step S1205, continue to broadcast the audio stream on the text browsing page, and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0220] The explanation of steps S1204 to S1205 can be found in the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0221] The interactive method of the disclosed embodiment can display a long text closely related to the video when consuming the video by dragging the progress bar. The user can locate the video by reading the long text. For example, when the user reads a text segment of interest, the user drags the progress bar to the playback time corresponding to the text segment. When the user stops dragging the progress bar, the video can be played from the target dragging time that the user finally drags to. This method of locating the video based on the linkage of text and progress bar can not only enrich the video positioning method, but also improve the video positioning efficiency and improve the user experience.

[0222] In order to clearly illustrate any embodiment of the present disclosure, the present disclosure also proposes an interaction method.

[0223] Fig.14 This is a flowchart of the interactive method provided in Embodiment 6 of the present disclosure.

[0224] like Fig.14 As shown, the interaction method may include the following steps:

[0225] Step S1401, playing the target video on the video playback page; wherein the video playback page displays a progress bar, and the progress bar is marked with at least one marking point, and the marking point is used to indicate the playback time corresponding to the highlight in the long text.

[0226] The progress bar is used to indicate the playback progress of the target video.

[0227] The target video is generated based on the plot of the long text, and the audio stream of the target video is obtained by dubbing the long text.

[0228] Step S1402, when the first interaction condition is met, enter the clear screen mode to switch from the video playback page to the text browsing page.

[0229] Step S1403, continue to broadcast the audio stream on the text browsing page, and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0230] The explanation of steps S1401 to S1403 can be found in the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0231] Step S1404: When a sliding operation on the long text displayed on the text browsing page is detected, the long text is displayed by sliding.

[0232] Among them, the long text is marked with highlights, which are obtained by semantic analysis of the long text. It should be noted that the explanation of highlights in the above embodiment is also applicable to this embodiment, and will not be repeated here.

[0233] In an embodiment of the present disclosure, when the client detects a user's sliding operation on a long text displayed on a text browsing page, the long text can be displayed by synchronous sliding in response to the sliding operation.

[0234] Step S1405 , in response to detecting a triggering operation on a highlight in the long text, displaying the long text starting from the highlight.

[0235] In the disclosed embodiment, when the client detects the user's triggering operation on a bright spot in a long text, the long text can be displayed starting from the bright spot. Alternatively, when the stop sliding position of the sliding operation matches the position of the bright spot, the client can display the long text starting from the bright spot.

[0236] Step S1406, determining a target broadcasting time of the audio content corresponding to the highlight in the audio stream, and starting to broadcast the audio stream from the target broadcasting time.

[0237] In an embodiment of the present disclosure, the broadcast time (recorded as the target playback time in the present disclosure) of the audio content corresponding to the highlight in the audio stream (that is, the audio content is obtained by dubbing the highlight) can be determined, and the audio stream can be broadcast from the target broadcast time.

[0238] The interactive method of the disclosed embodiment marks various highlights (such as wonderful points and highlights) in a long text. This efficient and practical way of presenting information can significantly improve the reading experience, enhance the efficiency and attractiveness of information transmission, promote understanding and memory, and meet the readers' personalized reading needs.

[0239] The above are various embodiments corresponding to the interaction method for entering the clear screen mode. The present disclosure also proposes an interaction method for exiting the clear screen mode.

[0240] Fig.15 This is a flowchart of the interactive method provided in Embodiment 7 of the present disclosure.

[0241] like Fig.15 As shown, the interaction method may include the following steps:

[0242] Step S1501, play the target video on the video playback page, and enter the clear screen mode when the first interaction condition is met to switch from the video playback page to the text browsing page.

[0243] The target video is generated based on the plot of the long text, and the audio stream of the target video is obtained by dubbing the long text.

[0244] Step S1502, continue to broadcast the audio stream on the text browsing page, and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0245] Step S1503, when the second interaction condition is met, exit the clear screen mode to switch from the text browsing page to the video playing page.

[0246] In the embodiment of the present disclosure, when the client satisfies the second interaction condition, the screen clearing mode is exited to switch from the text browsing page to the video playing page.

[0247] In any embodiment of the present disclosure, when the client monitors the second touch event triggered by the user on the screen, the number of second touch points and the coordinates of the second touch points in the second touch event are obtained, and based on the number and coordinates of the second touch points, the second touch gesture is determined, and it is determined whether the second touch gesture matches the second designated gesture (such as a two-finger "zoom-in" gesture) for exiting the clear screen mode. If so, it is determined that the client satisfies the second interaction condition and exits the clear screen mode to switch from the text browsing page to the video playback page; if not, it is determined that the client does not meet the second interaction condition. At this time, it can stay on the text browsing page to continue scrolling and displaying the long text. Its implementation principle is similar to steps S202 to S205, which will not be repeated here.

[0248] Therefore, the user can directly exit the clear screen mode (or text consumption mode) from the current text browsing page through the second designated gesture (such as the two-finger "pinch" gesture), and the user can watch the video screen in the video playback page corresponding to the non-clear screen mode, which is convenient for the user to quickly watch, filter, and locate the video content. Among them, the consistency of gestures and interactions can ensure that users have zero learning cost when accessing video content, and can quickly get started and improve the user experience.

[0249] In any one of the embodiments of the present disclosure, when the client detects the user's triggering operation on the second page element (such as a "Close" button) in the text browsing page, it can directly determine that the client meets the second interaction condition and exit the clear screen mode to switch from the text browsing page to the video playback page.

[0250] Among them, the second page element is used to exit the display of the long text to continue playing the target video.

[0251] Therefore, the user can directly exit the clear screen mode (or text consumption mode) from the current text browsing page by triggering or clicking the second page element (such as the "Close" button) in the text browsing page. The user can watch the video in the video playback page corresponding to the non-clear screen mode, which is convenient for the user to quickly watch, filter, and locate the video content, thereby meeting the user's actual viewing needs.

[0252] Step S1504, continue to broadcast the audio stream on the video playback page, and synchronously play the video frame in the target video corresponding to the real-time broadcast time.

[0253] In the disclosed embodiment, the client can continue to broadcast the audio stream on the video playback page based on the real-time broadcast progress of the audio stream, and synchronously play the video frame in the target video corresponding to the real-time broadcast time.

[0254] According to the interactive method of the disclosed embodiment, when a user is immersively browsing a long text content, the user can use the interactive method to instantly play a target video closely related to the long text. This seamless video consumption method can greatly shorten the user's video consumption path and make video playback easy and convenient.

[0255] In any embodiment of the present disclosure, the browsing experience of long text content under immersive interaction can be compatible through "AIGC generated screen + TTS dubbing + LLM intelligent highlight recognition + text browsing and progress adjustment fine adaptation", and the two can be integrated with each other. It mainly includes the following parts:

[0256] Part 1: AIGC generates pictures (or AIGC illustrations supplement pictures): Combined with the plot, character information and other information of the text, AIGC technology is used to batch produce dynamic pictures, which are displayed in real time in response to the text, making the text concrete and making the user feel as if they are in the scene, solving the problem of no pictures for long text content in immersive containers.

[0257] The second part, TTS dubbing (or TTS technology supplemented with audio): using the immersive TTS technology, the text content is read aloud vividly. Different object roles in the content to be processed (such as novels, articles, etc.) support different timbres. The tone fits the plot and the sense of scene is stronger, solving the problem of no audio in long text content under immersive interaction.

[0258] Part 3: Greatly reduce the cost of text reading: The text always scrolls with the TTS reading, and the user drags the progress bar to fill the full screen, which can easily locate the video. At the same time, it supports one-click to enter the full-screen text reading mode. While fitting the immersive interactive experience, you can directly consume long texts without jumping out of the scene, greatly reducing the user's interaction cost.

[0259] Part 4: Intelligent identification of highlight content: Through the cutting-edge LLM technology, AI intelligently identifies the highlights and highlights in long text content to quickly attract users. Users can directly perceive the value of the content and quickly decide whether to continue consuming.

[0260] As an example, the interaction process mainly includes the following steps:

[0261] Step 1: When you see a long text in the immersive recommendation stream of a short video platform, play the video corresponding to the long text and consume it immersively.

[0262] After the user enters the immersive recommendation flow, he or she can swipe down to the recommended long text and start playing directly from the wonderful highlights identified by AI. The video screen is a dynamic picture related to the plot of the long text produced by AIGC, which continues to change as the plot progresses. The immersive TTS technology reads the long text vividly. Different object roles in the long text support different timbres, the tone fits the plot, and the sense of scene is stronger. The text always scrolls and displays as the TTS reading.

[0263] Step 2: Users can control the playback progress by long pressing to fast forward or adjusting the progress bar.

[0264] Users can control the playback progress of the video through the immersive and universal long-press fast-forward function and the sliding adjustment progress bar at the bottom. The video picture, text, and sound will change in real time, and the text will fill the screen, making it easier for users to locate specific locations based on the context.

[0265] It should be noted that the consistency of gestures and interactions ensures that users of the immersive recommendation flow have zero learning cost when exposed to long text content and can quickly get started.

[0266] Step 3: Click on the first page element (such as the "Full Text" button) to enter the text consumption mode.

[0267] The user can directly enter the text consumption mode (referred to as the clear screen mode in this disclosure) from the current video playback page through the first designated gesture (such as the two-finger 'zoom in' gesture) or by clicking the first page element (such as the 'full text' button). The user can manually slide to view the text in the text browsing page corresponding to the text consumption mode, which is convenient for the user to quickly browse, filter, and locate the content. Among them, the text consumption mode also supports basic playback control capabilities such as fast forward and speed. The user can also return to the picture mode through the second designated gesture corresponding to the first designated gesture (such as the two-finger 'pinch' gesture) or by clicking the second page element (such as the 'exit' button) to enter the video playback page corresponding to the picture mode.

[0268] In summary, the technical solution provided by the present disclosure has at least the following advantages over the existing technical solutions:

[0269] 1. Short consumption path: Users can complete all mainstream operations of long text content consumption without jumping out of the page, such as precise progress control and quick browsing of text, which greatly shortens the user's consumption path and makes interaction more convenient;

[0270] 2. Native experience: Make full use of the most cutting-edge AIGC technology capabilities to supplement the missing video images and audio in long text content. In addition to being close to immersive interaction in presentation form, it also gives text content more consumer value and enhances the user's immersive consumption experience;

[0271] 3. High consumption efficiency: Through the most cutting-edge LLM technology, it fully understands the content of long texts, and intelligently identifies the highlights and highlights to quickly attract users. Users can directly perceive the value of the content and quickly decide whether to continue consuming, greatly improving consumption efficiency.

[0272] With the above Figures 1 to 15 Corresponding to the interaction method provided in the embodiment, the present disclosure also provides an interaction device. Figures 1 to 15 The interaction method provided in the embodiment corresponds to the above, so the implementation of the interaction method is also applicable to the interaction device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0273] Fig.16 This is a schematic diagram of the structure of the interactive device provided in Embodiment 8 of the present disclosure.

[0274] like Fig.16 As shown, the interactive device 1600 may include: a video playing module 1610 , a switching module 1620 and a broadcast display module 1630 .

[0275] The video playback module 1610 is used to play the target video on the video playback page; wherein the target video is generated based on the plot of the long text, and the audio stream of the target video is obtained by dubbing the long text;

[0276] The switching module 1620 is used to enter the screen clearing mode to switch from the video playback page to the text browsing page when the first interaction condition is met;

[0277] The broadcast display module 1630 is used to continue broadcasting the audio stream on the text browsing page and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

[0278] In a possible implementation of the embodiments of the present disclosure, the switching module 1620 is used to: upon monitoring a first touch event triggered on the screen, obtain the number of first touch points and the coordinates of the first touch points in the first touch event; determine a first touch gesture based on the number and coordinates of the first touch points; when the first touch gesture matches a first designated gesture for entering a clear screen mode, determine that a first interaction condition is satisfied; and enter the clear screen mode to switch from a video playback page to a text browsing page.

[0279] In a possible implementation of the embodiment of the present disclosure, the switching module 1620 is used to: determine whether the number of first touch points is consistent with the number of fingers associated with the first specified gesture; if consistent, determine the coordinate change trend of each first touch point based on the coordinates of each first touch point; determine the first touch gesture based on the coordinate change trend of each first touch point.

[0280] In a possible implementation of the embodiment of the present disclosure, the interaction device 1600 further includes:

[0281] A first determining module, configured to determine that the first interaction condition is not satisfied if there is no inconsistency;

[0282] The video playback module is also used to continue playing the target video on the video playback page.

[0283] In a possible implementation of the embodiment of the present disclosure, the switching module 1620 is used to: determine that the first interaction condition is met when a trigger operation on the first page element in the video playback page is monitored; enter the clear screen mode to switch from the video playback page to the text browsing page; wherein the first page element is used to display the long text corresponding to the target video currently being played.

[0284] In a possible implementation of the embodiment of the present disclosure, the target video is generated by using the following modules:

[0285] An analysis module, used for performing semantic analysis on the long text to extract at least one highlight from the long text;

[0286] A first generation module is used to generate a video screen corresponding to the highlight based on the plot of the highlight and the context information of the highlight in the long text;

[0287] The second generation module is used to generate a target video based on the video screen and the long text cover.

[0288] In a possible implementation of the embodiment of the present disclosure, the audio stream is generated by using the following modules:

[0289] A recognition module is used to recognize the long text to obtain at least one object role to be dubbed and the portrait features of each object role;

[0290] A second determination module is used to determine a target timbre adapted to each object role from a plurality of timbres in a timbre library based on the image features of each object role;

[0291] The dubbing module is used to dub the long text based on the target timbre of each object role and the plot of the long text to obtain an audio stream.

[0292] In a possible implementation of the embodiment of the present disclosure, a progress bar is displayed on the video playback page, and the progress bar is used to indicate the playback progress of the target video. The progress bar is marked with at least one mark point, and the mark point is used to indicate the playback time corresponding to the highlight in the long text; the interactive device 1600 also includes:

[0293] A jump module, configured to jump to a target playback time indicated by a target mark point when a trigger operation on a target mark point among at least one mark point is detected;

[0294] The video playing module 1610 is also used to play the target video from the target playing time on the video playing page.

[0295] In a possible implementation of the embodiment of the present disclosure, the interaction device 1600 further includes:

[0296] An entry module is used to enter the full-screen mode when a drag operation on the progress bar in the video playback page is detected, so as to continue to broadcast the audio stream, and to synchronously scroll and display the long text and the mask in full screen; wherein the mask displays the enlarged progress bar and the real-time dragging time to which the enlarged progress bar is dragged; and the second text segment in the long text that matches the real-time dragging time is highlighted;

[0297] A third determining module is used to determine the target dragging time to which the enlarged progress bar is dragged in response to the stop dragging operation on the enlarged progress bar, and exit the full screen mode;

[0298] The video playing module 1610 is also used to play the target video from the moment the target is dragged on the video playing page.

[0299] In a possible implementation of the embodiment of the present disclosure, a highlight is marked in the long text, and the interactive device 1600 further includes:

[0300] A sliding display module, used for slidingly displaying the long text when a sliding operation on the long text displayed on the text browsing page is detected;

[0301] A highlight display module, for displaying the long text starting from the highlight in response to detecting a triggering operation on a highlight in the long text;

[0302] The audio broadcast module is used to determine the target broadcast time of the audio content corresponding to the highlight in the audio stream, and start broadcasting the audio stream from the target broadcast time.

[0303] In a possible implementation of the embodiment of the present disclosure, the switching module 1620 is further configured to exit the screen clearing mode to switch from the text browsing page to the video playing page when the second interaction condition is met;

[0304] The video playing module 1610 is also used to continue to broadcast the audio stream on the video playing page, and synchronously play the video frame in the target video corresponding to the real-time broadcasting time.

[0305] In a possible implementation of the embodiments of the present disclosure, the switching module 1620 is used to: upon monitoring a second touch event triggered on the screen, obtain the number of second touch points and the coordinates of the second touch points in the second touch event; determine the second touch gesture based on the number and coordinates of the second touch points; upon matching the second touch gesture with a second designated gesture for exiting the clear screen mode, determine that the second interaction condition is satisfied; and exit the clear screen mode to switch from the text browsing page to the video playback page.

[0306] In a possible implementation of the embodiment of the present disclosure, the switching module 1620 is used to: determine that the second interaction condition is met when a triggering operation on the second page element in the text browsing page is monitored; exit the clear screen mode to switch from the text browsing page to the video playback page; wherein the second page element is used to exit the display of the long text to continue playing the target video.

[0307] In a possible implementation of the embodiment of the present disclosure, the long text is obtained by segmenting the content to be processed into multiple sub-contents, and is determined from the multiple sub-contents; the video playback page is a collection playback page, and the collection playback page is used to play a video collection, and the video collection includes multiple episodes of video, and the multiple episodes of video are generated according to the plots of multiple sub-contents; the collection playback page displays at least one of the following information: the video frame currently being played in the target video; the interface title; the interface title is used to indicate the content title of the content to be processed; subtitles; subtitles are used to indicate the current broadcast content of the audio stream; a first page element; wherein the first page element is used to display the long text corresponding to the target video currently being played; the number of episodes of the target video in the video collection; the title of a single episode of the target video; a third page element; wherein the third page element is used to select a video with a required number of episodes from the video collection; a fourth page element; wherein the fourth page element is used to adjust the playback speed of the target video.

[0308] In a possible implementation of the embodiment of the present disclosure, the long text is obtained by segmenting the content to be processed into multiple sub-contents, and is determined from the multiple sub-contents; the video playback page is a single-episode playback page, and the single-episode playback page is used to play a single-episode video in a multiple-episode video in a video collection, and the multiple-episode video is generated according to the plot of multiple sub-contents; the single-episode playback page displays at least one of the following information: the video frame currently being played in the target video; subtitles; subtitles are used to indicate the current broadcast content of the audio stream; a first page element; wherein the first page element is used to display the long text corresponding to the target video currently being played; a first information display area; wherein the first information display area displays at least one of the content title of the content to be processed, the number of episodes of the target video in the video collection, the single-episode title of the target video, the label of the target video and the author information of the content to be processed; a fifth page element; wherein the fifth page element is used to display the directory of the video collection; a sixth page element; wherein the sixth page element is used to display the next episode of the target video in the video collection; a seventh page element; wherein the seventh page element is used to adjust the playback speed of the target video.

[0309] In a possible implementation of the embodiment of the present disclosure, at least one of the following information is displayed in the text browsing page: a long text; wherein a text segment in the long text that matches the current broadcast content of the audio stream is highlighted; the number of episodes of the target video in the video collection; the title of a single episode of the target video; the total number of pages of the long text and the number of pages currently displayed; a second page element; wherein the second page element is used to exit the display of the long text; an eighth page element; wherein the eighth page element is used to display the directory of the video collection; a ninth page element; wherein the ninth page element is used to display the sub-content corresponding to the next episode of the target video in the video collection; a tenth page element; wherein the tenth page element is used to pause the broadcast of the audio stream and pause the scrolling display of the long text; an eleventh page element; wherein the eleventh page element is used to adjust the broadcast speed of the audio stream and the scrolling display speed of the long text.

[0310] In a possible implementation of the embodiment of the present disclosure, the interaction device 1600 further includes:

[0311] A panel display module, used for displaying a directory panel when a trigger operation on the fifth page element is detected;

[0312] Among them, at least one of the following information is displayed in the directory panel: the cover of the content to be processed; the second information display area; wherein the second information display area displays at least one of the content title, classification information, tag information, rating information, author information and updated episode number of the content to be processed; brief information of the content to be processed; the directory of the video collection; wherein a single episode information display area is displayed at the target video currently being played in the directory; the single episode information display area is used to display at least one of the thirteenth page element, the episode number of the currently playing target video in the video collection, the single episode title of the target video, and the playback progress of the target video; the thirteenth page element is used to indicate that the target video is being played; the number of episodes watched in the video collection; the twelfth page element; wherein the twelfth page element is used to switch the display order of the directory between sequential display and reverse display.

[0313] In a possible implementation of the embodiment of the present disclosure, the interaction device 1600 further includes:

[0314] A first calling module is used to call up a speech speed adjustment panel when a trigger operation on the fourth page element or the seventh page element in the video playback page is detected; wherein the speech speed adjustment panel displays a plurality of candidate playback speeds;

[0315] The video playing module 1610 is further configured to play the target video based on the target playing speed when a selection operation of the target playing speed in the speech speed adjustment panel is detected;

[0316] The first retracting module is used to retract the speech rate adjustment panel and continue playing the target video at the target playback speed when a closing operation on the speech rate adjustment panel is detected.

[0317] In a possible implementation of the embodiment of the present disclosure, the interaction device 1600 further includes:

[0318] The second calling module is used to call up the speed control panel when a trigger operation on the eleventh page element in the text browsing page is detected; wherein the speed control panel displays a plurality of candidate speech rates;

[0319] The broadcasting and displaying module 1630 is further used to broadcast the audio stream based on the target speech speed and synchronously scroll and display the long text when a selection operation of the target speech speed in the speed control panel is detected;

[0320] The second folding module is used to fold the speed control panel when a closing operation on the speed control panel is detected, and continue to play the audio stream at the target playback speed and synchronously scroll and display the long text.

[0321] In a possible implementation of the embodiment of the present disclosure, the interaction device 1600 further includes:

[0322] A pop-up display module is used to display a directory pop-up when a trigger operation on the eighth page element in the text browsing page is detected; wherein the target pop-up displays a directory of the video collection;

[0323] Wherein, a third information display area is displayed at the target video currently being played in the directory; the third information display area is used to display at least one of the following: the fourteenth page element, the number of the target video in the video collection corresponding to the currently displayed long text, the title of a single episode of the target video, and the playing progress of the target video; the fourteenth page element is used to indicate that the target video is being played;

[0324] Among them, the fourth information display area is displayed in the other episodes of the video in the directory that are not currently being played; the fourth information display area displays at least one of the episode number of the other episodes of the video in the video collection, the single episode title of the other episodes of the video, and the playback progress of the other episodes of the video.

[0325] The interactive device of the disclosed embodiment allows users to instantly immerse themselves in a long text closely related to the target video through an interactive method when watching the video content of the target video. This seamless long text consumption method greatly shortens the user's consumption path for the long text, making the reading of the long text easy and convenient. Among them, the target video itself, as a visual interpretation of the plot of the long text, continues to evolve with the ups and downs of the plot of the long text, which not only improves the creative quality of the video content, but also gives the audience an unprecedented visual feast and optimizes the overall viewing experience. In addition, the audio stream can use the telepresence TTS dubbing technology to vividly dub or read the long text, and the tone fits the emotional context of the plot of the long text. The scene atmosphere is more intense and realistic, allowing the audience to enjoy a more profound and shocking audio-visual feast and improve the user's immersive experience.

[0326] In order to implement the above embodiments, the present disclosure also provides an electronic device, which may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the interaction method proposed in any of the above embodiments of the present disclosure.

[0327] In order to implement the above embodiments, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the interaction method proposed in any of the above embodiments of the present disclosure.

[0328] In order to implement the above embodiments, the present disclosure further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the interaction method proposed in any of the above embodiments of the present disclosure.

[0329] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0330] Fig.17 A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. Among them, the electronic device may include the server and client in the above-mentioned embodiment. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0331] like Fig.17 As shown, the electronic device 1700 includes a computing unit 1701, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1702 or a computer program loaded from a storage unit 1707 to a RAM (Random Access Memory) 1703. In the RAM 1703, various programs and data required for the operation of the device 1700 can also be stored. The computing unit 1701, the ROM 1702, and the RAM 1703 are connected to each other via a bus 1704. An I / O (Input / Output) interface 1705 is also connected to the bus 1704.

[0332] A number of components in the device 1700 are connected to the I / O interface 1705, including: an input unit 1706, such as a keyboard, a mouse, etc.; an output unit 1707, such as various types of displays, speakers, etc.; a storage unit 1708, such as a disk, an optical disk, etc.; and a communication unit 1709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1709 allows the device 1700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0333] The computing unit 1701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1701 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1701 performs the various methods and processes described above, such as the above-mentioned interaction method. For example, in some embodiments, the above-mentioned interaction method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1700 via the ROM 1702 and / or the communication unit 1709. When the computer program is loaded into the RAM 1703 and executed by the computing unit 1701, one or more steps of the interaction method described above may be performed. Alternatively, in other embodiments, the computing unit 1701 may be configured to execute the above interaction method in any other appropriate manner (for example, by means of firmware).

[0334] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0335] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0336] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0337] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0338] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0339] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0340] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0341] According to the technical solution of the embodiment of the present disclosure, when watching the video content of the target video, the user can be instantly immersed in the long text closely related to the target video through an interactive method. This seamless long text consumption method greatly shortens the user's consumption path for the long text, making the reading of the long text easy and convenient. Among them, the target video itself, as a visual interpretation of the plot of the long text, continues to evolve with the ups and downs of the plot of the long text, which not only improves the creative quality of the video content, but also gives the audience an unprecedented visual feast and optimizes the overall viewing experience. In addition, the audio stream can use the telepresence TTS dubbing technology to vividly dub or read the long text, and the tone fits the emotional context of the plot of the long text. The scene atmosphere is more intense and realistic, allowing the audience to enjoy a more profound and shocking audio-visual feast and improve the user's immersive experience.

[0342] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0343] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An interactive method, comprising: Play the target video on the video playback page; wherein the target video is generated based on the plot of the long text, and the audio stream of the target video is obtained by dubbing the long text; When the first interaction condition is met, entering a clear screen mode to switch from the video playback page to a text browsing page; The audio stream continues to be broadcasted on the text browsing page, and the long text is synchronously scrolled and displayed; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

2. The method according to claim 1, wherein: When the first interaction condition is met, entering the clear screen mode to switch from the video playback page to the text browsing page includes: In the case of monitoring a first touch event triggered on the screen, acquiring the number of first touch points in the first touch event and the coordinates of the first touch points; determining a first touch gesture based on the number and coordinates of the first touch points; In a case where the first touch gesture matches a first designated gesture for entering the clear screen mode, determining that the first interaction condition is satisfied; Enter the clear screen mode to switch from the video playback page to the text browsing page.

3. The method according to claim 2, wherein: The determining the first touch gesture based on the number and coordinates of the first touch points includes: Determining whether the number of the first touch points is consistent with the number of fingers associated with the first designated gesture; If they are consistent, determining a coordinate change trend of each of the first touch points according to the coordinates of each of the first touch points; The first touch gesture is determined according to a coordinate change trend of each of the first touch points.

4. The method according to claim 3, wherein: The method further comprises: If they are inconsistent, it is determined that the first interaction condition is not met; Continue playing the target video on the video playback page.

5. The method according to claim 1, wherein: When the first interaction condition is met, entering the clear screen mode to switch from the video playback page to the text browsing page includes: In a case where a triggering operation on a first page element in the video playback page is monitored, determining that the first interaction condition is satisfied; Entering the clear screen mode to switch from the video playback page to the text browsing page; Among them, the first page element is used to display the long text corresponding to the target video currently being played.

6. The method according to claim 1, wherein: The target video is generated by following the steps below: Performing semantic analysis on the long text to extract at least one highlight from the long text; Based on the plot of the highlight and the context information of the highlight in the long text, generating a video picture corresponding to the highlight; The target video is generated based on the video screen and the cover of the long text.

7. The method according to claim 1, wherein: The audio stream is generated by following the steps below: Recognize the long text to obtain at least one object character to be dubbed and portrait features of each object character; Based on the portrait features of each of the target characters, determining a target timbre adapted to each of the target characters from a plurality of timbres in a timbre library; Based on the target timbre of each of the target characters and the plot of the long text, the long text is dubbed to obtain the audio stream.

8. The method according to claim 1, wherein: The video playback page displays a progress bar, the progress bar is used to indicate the playback progress of the target video, and the progress bar is marked with at least one marking point, the marking point is used to indicate the playback time corresponding to the highlight in the long text; When the first interaction condition is met, before entering the screen clearing mode, the method further includes: When a trigger operation on a target marking point among the at least one marking point is detected, jumping to a target playback time indicated by the target marking point; The target video is played on the video playback page starting from the target playback time.

9. The method according to claim 8, wherein: When the first interaction condition is met, before entering the screen clearing mode, the method further includes: When a dragging operation on the progress bar in the video playback page is detected, the full-screen mode is entered to continue broadcasting the audio stream, and the long text and the display mask are displayed in full-screen synchronous scrolling; wherein the mask displays the enlarged progress bar and the real-time dragging time to which the enlarged progress bar is dragged; and a second text segment in the long text that matches the real-time dragging time is highlighted; In response to a stop dragging operation on the enlarged progress bar, determining a target dragging time to which the enlarged progress bar is dragged, and exiting the full screen mode; The target video is played on the video playing page starting from the moment the target is dragged.

10. The method according to claim 8, wherein: The long text is marked with the highlight, and after the text browsing page continues to broadcast the audio stream and synchronously scrolls and displays the long text, the method further includes: In case a sliding operation on the long text displayed on the text browsing page is detected, sliding and displaying the long text; In response to detecting a triggering operation on the highlight in the long text, displaying the long text starting from the highlight; A target broadcasting time of the audio content corresponding to the highlight in the audio stream is determined, and the audio stream is broadcasted starting from the target broadcasting time.

11. The method according to claim 1, wherein: After the text browsing page continues to broadcast the audio stream and synchronously scrolls and displays the long text, the method further includes: When the second interaction condition is met, exiting the screen clearing mode to switch from the text browsing page to the video playing page; The audio stream continues to be broadcast on the video playback page, and the video frame corresponding to the real-time broadcasting moment in the target video is synchronously played.

12. The method according to claim 11, wherein: When the second interaction condition is met, exiting the screen clearing mode to switch from the text browsing page to the video playing page includes: In the case of monitoring a second touch event triggered on the screen, acquiring the number of second touch points in the second touch event and the coordinates of the second touch points; determining a second touch gesture based on the number and coordinates of the second touch points; In a case where the second touch gesture matches a second designated gesture for exiting the clear screen mode, determining that the second interaction condition is satisfied; Exit the clear screen mode to switch from the text browsing page to the video playing page.

13. The method according to claim 11, wherein: When the second interaction condition is met, exiting the screen clearing mode to switch from the text browsing page to the video playing page includes: In case of monitoring a triggering operation on a second page element in the text browsing page, determining that the second interaction condition is satisfied; Exit the clear screen mode to switch from the text browsing page to the video playing page; Among them, the second page element is used to exit the display of the long text to continue playing the target video.

14. The method according to claim 1, wherein: The long text is a plurality of sub-contents obtained by segmenting the content to be processed, and is determined from the plurality of sub-contents; The video play page is a collection play page, and the collection play page is used to play a video collection, and the video collection includes multiple video episodes, and the multiple video episodes are generated according to the plots of the multiple sub-contents; The collection playback page displays at least one of the following information: The video frame currently being played in the target video; Interface title; the interface title is used to indicate the content title of the content to be processed; Subtitles; the subtitles are used to indicate the current broadcast content of the audio stream; A first page element; wherein the first page element is used to display the long text corresponding to the target video currently being played; The number of episodes of the target video in the video collection; The title of a single episode of the target video; A third page element; wherein the third page element is used to select a required number of videos from the video collection; The fourth page element; wherein the fourth page element is used to adjust the playback speed of the target video.

15. The method according to claim 1, wherein: The long text is a plurality of sub-contents obtained by segmenting the content to be processed, and is determined from the plurality of sub-contents; The video play page is a single-episode play page, which is used to play a single-episode video from a plurality of episodes in a video collection, wherein the plurality of episodes are generated according to the plots of the plurality of sub-contents; The single episode playback page displays at least one of the following information: The video frame currently being played in the target video; Subtitles; the subtitles are used to indicate the current broadcast content of the audio stream; A first page element; wherein the first page element is used to display the long text corresponding to the target video currently being played; A first information display area; wherein the first information display area displays at least one of the content title of the content to be processed, the number of episodes of the target video in the video collection, the title of a single episode of the target video, the tag of the target video, and the author information of the content to be processed; A fifth page element; wherein the fifth page element is used to display the directory of the video collection; The sixth page element; wherein the sixth page element is used to display the next video in the video collection that is located at the target video; The seventh page element; wherein the seventh page element is used to adjust the playback speed of the target video.

16. The method according to claim 14 or 15, wherein: The text browsing page displays at least one of the following information: The long text; wherein the text segment in the long text that matches the current broadcast content of the audio stream is highlighted; The number of episodes of the target video in the video collection; The title of a single episode of the target video; The total number of pages of the long text and the number of pages currently displayed; A second page element; wherein the second page element is used to exit the display of the long text; An eighth page element; wherein the eighth page element is used to display the directory of the video collection; The ninth page element; wherein the ninth page element is used to display the sub-content corresponding to the next video of the target video in the video collection; The tenth page element; wherein the tenth page element is used to pause the broadcast of the audio stream and pause the scrolling display of the long text; The eleventh page element; wherein the eleventh page element is used to adjust the broadcasting speed of the audio stream and the scrolling display speed of the long text.

17. The method according to claim 15, wherein: The method further comprises: In case a triggering operation on the fifth page element is detected, displaying a directory panel; The directory panel displays at least one of the following information: The cover of the content to be processed; A second information display area; wherein the second information display area displays at least one of the content title, classification information, tag information, rating information, author information, and update number of the content to be processed; Brief introduction information of the content to be processed; The directory of the video collection; wherein a single episode information display area is displayed at the target video currently being played in the directory; the single episode information display area is used to display at least one of the thirteenth page element, the number of the currently playing target video in the video collection, the single episode title of the target video, and the playing progress of the target video; the thirteenth page element is used to indicate that the target video is being played; The number of episodes watched in the video collection; The twelfth page element; wherein the twelfth page element is used to switch the display order of the directory between sequential display and reverse display.

18. The method according to claim 14 or 15, wherein: The method further comprises: In case of detecting a trigger operation on the fourth page element or the seventh page element in the video playback page, calling up a speech speed adjustment panel; wherein the speech speed adjustment panel displays a plurality of candidate playback speeds; When a selection operation of a target playback speed in the speech speed adjustment panel is detected, playing the target video based on the target playback speed; When a closing operation of the speech rate adjustment panel is detected, the speech rate adjustment panel is collapsed, and the target video continues to be played at the target playback speed.

19. The method according to claim 16, wherein: The method further comprises: In case of detecting a trigger operation on the eleventh page element in the text browsing page, calling up a speed control panel; wherein the speed control panel displays a plurality of candidate speech rates; When a selection operation of a target speech rate in the speed control panel is detected, the audio stream is broadcast based on the target speech rate, and the long text is synchronously scrolled and displayed; When a closing operation of the speed control panel is detected, the speed control panel is folded, and the audio stream continues to be played at the target playback speed, and the long text is synchronously scrolled and displayed.

20. The method according to claim 16, wherein: The method further comprises: When a trigger operation on the eighth page element in the text browsing page is detected, a directory pop-up layer is displayed; wherein the directory of the video collection is displayed in the target pop-up layer; Wherein, a third information display area is displayed at the target video currently being played in the directory; the third information display area is used to display at least one of the fourteenth page element, the number of the target video corresponding to the currently displayed long text in the video collection, the title of a single episode of the target video, and the playing progress of the target video; the fourteenth page element is used to indicate that the target video is being played; Among them, a fourth information display area is displayed in the other episodes of the video in the directory that are not currently being played; the fourth information display area displays at least one of the episode number of the other episodes of the video in the video collection, the single episode title of the other episode of the video, and the playback progress of the other episode of the video.

21. An interactive device, comprising: A video playback module, used to play a target video on a video playback page; wherein the target video is generated based on the plot of a long text, and the audio stream of the target video is obtained by dubbing the long text; A switching module, configured to enter a clear screen mode to switch from the video playback page to the text browsing page when the first interaction condition is met; The broadcast display module is used to continue broadcasting the audio stream on the text browsing page and synchronously scroll and display the long text; wherein the first text segment in the long text that matches the current broadcast content of the audio stream is highlighted and displayed.

22. The device according to claim 21, wherein The switching module is used for: In the case of monitoring a first touch event triggered on the screen, acquiring the number of first touch points in the first touch event and the coordinates of the first touch points; determining a first touch gesture based on the number and coordinates of the first touch points; In a case where the first touch gesture matches a first designated gesture for entering the clear screen mode, determining that the first interaction condition is satisfied; Enter the clear screen mode to switch from the video playback page to the text browsing page.

23. The device according to claim 22, wherein: The switching module is used for: Determining whether the number of the first touch points is consistent with the number of fingers associated with the first designated gesture; If they are consistent, determining a coordinate change trend of each of the first touch points according to the coordinates of each of the first touch points; The first touch gesture is determined according to a coordinate change trend of each of the first touch points.

24. The device according to claim 23, wherein: The device also includes: A first determining module, configured to determine that the first interaction condition is not satisfied if there is inconsistency; The video playing module is also used to continue playing the target video on the video playing page.

25. The device according to claim 21, wherein The switching module is used for: In a case where a triggering operation on a first page element in the video playback page is monitored, determining that the first interaction condition is satisfied; Entering the clear screen mode to switch from the video playback page to the text browsing page; Among them, the first page element is used to display the long text corresponding to the target video currently being played.

26. The device according to claim 21, wherein The target video is generated using the following modules: An analysis module, configured to perform semantic analysis on the long text to extract at least one highlight from the long text; A first generating module, configured to generate a video screen corresponding to the highlight based on the plot of the highlight and context information of the highlight in the long text; The second generating module is used to generate the target video based on the video screen and the cover of the long text.

27. The device according to claim 21, wherein The audio stream is generated using the following modules: A recognition module, used for recognizing the long text to obtain at least one object role to be dubbed and a portrait feature of each object role; A second determination module is used to determine a target timbre adapted to each of the target roles from a plurality of timbres in a timbre library based on the portrait features of each of the target roles; The dubbing module is used to dub the long text based on the target timbre of each of the object characters and the plot of the long text to obtain the audio stream.

28. The device according to claim 21, wherein The video playback page displays a progress bar, the progress bar is used to indicate the playback progress of the target video, and the progress bar is marked with at least one marking point, the marking point is used to indicate the playback time corresponding to the highlight in the long text; The device also includes: A jump module, configured to jump to a target playback time indicated by the target mark point when a trigger operation on a target mark point among the at least one mark point is detected; The video playing module is also used to play the target video on the video playing page starting from the target playing time.

29. The device according to claim 28, wherein The device also includes: An entry module is used to enter a full-screen mode when a drag operation on the progress bar in the video playback page is detected, so as to continue to broadcast the audio stream, and to synchronously scroll and display the long text and the display mask in full screen; wherein the mask displays the enlarged progress bar and the real-time dragging time to which the enlarged progress bar is dragged; and a second text segment in the long text that matches the real-time dragging time is highlighted; a third determining module, configured to determine, in response to a stop dragging operation on the enlarged progress bar, a target dragging time to which the enlarged progress bar is dragged, and exit the full screen mode; The video playing module is also used to play the target video on the video playing page starting from the moment the target is dragged.

30. The device according to claim 28, wherein The long text is marked with the bright spot, and the device further includes: A sliding display module, configured to display the long text by sliding when a sliding operation on the long text displayed on the text browsing page is detected; A highlight display module, configured to display the long text starting from the highlight in response to detecting a triggering operation on the highlight in the long text; The audio broadcast module is used to determine a target broadcast time of the audio content corresponding to the highlight in the audio stream, and start broadcasting the audio stream from the target broadcast time.

31. The device according to claim 21, wherein The switching module is further configured to exit the screen clearing mode to switch from the text browsing page to the video playing page when a second interaction condition is met; The video playback module is also used to continue to broadcast the audio stream on the video playback page, and synchronously play the video frame in the target video corresponding to the real-time broadcasting moment.

32. The device according to claim 31, wherein The switching module is used for: In the case of monitoring a second touch event triggered on the screen, acquiring the number of second touch points in the second touch event and the coordinates of the second touch points; determining a second touch gesture based on the number and coordinates of the second touch points; In a case where the second touch gesture matches a second designated gesture for exiting the clear screen mode, determining that the second interaction condition is satisfied; Exit the clear screen mode to switch from the text browsing page to the video playing page.

33. The device according to claim 31, wherein The switching module is used for: In case of monitoring a triggering operation on a second page element in the text browsing page, determining that the second interaction condition is satisfied; Exit the clear screen mode to switch from the text browsing page to the video playing page; Among them, the second page element is used to exit the display of the long text to continue playing the target video.

34. The device according to claim 21, wherein: The long text is a plurality of sub-contents obtained by segmenting the content to be processed, and is determined from the plurality of sub-contents; The video play page is a collection play page, and the collection play page is used to play a video collection, and the video collection includes multiple video episodes, and the multiple video episodes are generated according to the plots of the multiple sub-contents; The collection playback page displays at least one of the following information: The video frame currently being played in the target video; Interface title; the interface title is used to indicate the content title of the content to be processed; Subtitles; the subtitles are used to indicate the current broadcast content of the audio stream; A first page element; wherein the first page element is used to display the long text corresponding to the target video currently being played; The number of episodes of the target video in the video collection; The title of a single episode of the target video; A third page element; wherein the third page element is used to select a required number of videos from the video collection; The fourth page element; wherein the fourth page element is used to adjust the playback speed of the target video.

35. The apparatus of claim 21, wherein: The long text is a plurality of sub-contents obtained by segmenting the content to be processed, and is determined from the plurality of sub-contents; The video play page is a single-episode play page, which is used to play a single-episode video from a plurality of episodes in a video collection, wherein the plurality of episodes are generated according to the plots of the plurality of sub-contents; The single episode playback page displays at least one of the following information: The video frame currently being played in the target video; Subtitles; the subtitles are used to indicate the current broadcast content of the audio stream; A first page element; wherein the first page element is used to display the long text corresponding to the target video currently being played; A first information display area; wherein the first information display area displays at least one of the content title of the content to be processed, the number of episodes of the target video in the video collection, the title of a single episode of the target video, the tag of the target video, and the author information of the content to be processed; A fifth page element; wherein the fifth page element is used to display the directory of the video collection; The sixth page element; wherein the sixth page element is used to display the next video in the video collection that is located at the target video; The seventh page element; wherein the seventh page element is used to adjust the playback speed of the target video.

36. The device according to claim 34 or 35, wherein: The text browsing page displays at least one of the following information: The long text; wherein the text segment in the long text that matches the current broadcast content of the audio stream is highlighted; The number of episodes of the target video in the video collection; The title of a single episode of the target video; The total number of pages of the long text and the number of pages currently displayed; A second page element; wherein the second page element is used to exit the display of the long text; An eighth page element; wherein the eighth page element is used to display the directory of the video collection; The ninth page element; wherein the ninth page element is used to display the sub-content corresponding to the next video of the target video in the video collection; The tenth page element; wherein the tenth page element is used to pause the broadcast of the audio stream and pause the scrolling display of the long text; The eleventh page element; wherein the eleventh page element is used to adjust the broadcasting speed of the audio stream and the scrolling display speed of the long text.

37. The device according to claim 35, wherein: The device also includes: A panel display module, used for displaying a directory panel when a trigger operation on the fifth page element is detected; The directory panel displays at least one of the following information: The cover of the content to be processed; A second information display area; wherein the second information display area displays at least one of the content title, classification information, tag information, rating information, author information, and update number of the content to be processed; Brief introduction information of the content to be processed; The directory of the video collection; wherein a single episode information display area is displayed at the target video currently being played in the directory; the single episode information display area is used to display at least one of the thirteenth page element, the number of the currently playing target video in the video collection, the single episode title of the target video, and the playing progress of the target video; the thirteenth page element is used to indicate that the target video is being played; The number of episodes watched in the video collection; The twelfth page element; wherein the twelfth page element is used to switch the display order of the directory between sequential display and reverse display.

38. The device according to claim 34 or 35, wherein: The device also includes: A first calling module is used to call up a speech speed adjustment panel when a trigger operation on the fourth page element or the seventh page element in the video playback page is detected; wherein the speech speed adjustment panel displays a plurality of candidate playback speeds; The video playing module is further configured to play the target video based on the target playing speed when a selection operation of the target playing speed in the speech speed adjustment panel is detected; The first retracting module is used to retract the speech rate adjustment panel and continue playing the target video at the target playback speed when a closing operation on the speech rate adjustment panel is detected.

39. The device according to claim 36, wherein: The device also includes: A second calling module is used to call up a speed control panel when a trigger operation on the eleventh page element in the text browsing page is detected; wherein the speed control panel displays a plurality of candidate speech rates; The broadcasting and displaying module is further used to broadcast the audio stream based on the target speech speed and synchronously scroll and display the long text when a selection operation of the target speech speed in the speed control panel is detected; The second folding module is used to fold the speed control panel when a closing operation on the speed control panel is detected, and continue to play the audio stream at the target playback speed and synchronously scroll and display the long text.

40. The apparatus of claim 36, wherein: The device also includes: A pop-up display module is used to display a directory pop-up when a trigger operation on the eighth page element in the text browsing page is detected; wherein the directory of the video collection is displayed in the target pop-up; Wherein, a third information display area is displayed at the target video currently being played in the directory; the third information display area is used to display at least one of the fourteenth page element, the number of the target video corresponding to the currently displayed long text in the video collection, the title of a single episode of the target video, and the playing progress of the target video; the fourteenth page element is used to indicate that the target video is being played; Among them, a fourth information display area is displayed in the other episodes of the video in the directory that are not currently being played; the fourth information display area displays at least one of the episode number of the other episodes of the video in the video collection, the single episode title of the other episode of the video, and the playback progress of the other episode of the video.

41. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the interaction method according to any one of claims 1 to 20.

42. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the interaction method according to any one of claims 1-20.

43. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the interaction method according to any one of claims 1 to 20.

Images