Multimedia interaction method, information interaction method, device, equipment, and medium

Abstract

A multimedia interaction method, device, equipment and medium, the method includes: a step (101) of receiving a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia display interface, where the multimedia, the multimedia subtitle content and the subtitle interaction aggregation indicator are displayed in the multimedia display interface; and a step (102) of displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation indicator, where the subtitle interaction content is an interaction content for the subtitle content. According to the above technical solution, the subtitle interaction aggregation sign is an aggregation representation of multiple subtitle interaction contents. Therefore, by triggering the subtitle interaction aggregation sign, multiple subtitle interaction contents for the multimedia subtitle content can be quickly displayed, improving the display efficiency of the subtitle interaction content. When the subtitle interaction aggregation sign is not triggered, only one sign is displayed. Compared with the traditional display method, it can be displayed according to the user's choice, which is more flexible and has a higher display effect.

Description

[Technical field]

[0001] This application claims priority to a Chinese patent application filed on November 18, 2020, bearing application number 202011296626.3 and entitled "Multimedia interaction method, information interaction method, device, equipment and medium," the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates to the field of multimedia technology, and in particular to a multimedia interaction method, information interaction method, device, equipment and medium. [Background technology]

[0003] 2. Description of the Related Art With the development of smart devices and video technology, playing audio and video on smart devices has become an increasingly essential part of people's lives. Summary of the Invention [Problem to be solved by the invention]

[0004] During the playback of audio or video, users can simultaneously view the corresponding subtitle text to accurately understand the relevant content of the audio or video. Currently, the interaction functions supported by audio or video subtitles are limited, which cannot meet the interaction needs of various scenarios and results in a poor user interaction experience.

[0005] SUMMARY OF THE DISCLOSURE In order to solve the above technical problems or at least some of them, the present disclosure provides a multimedia interaction method, an information interaction method, an apparatus, a device and a medium. [Means for solving the problem]

[0006] An embodiment of the present disclosure provides a multimedia interaction method, the method comprising: receiving a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia display interface, in which multimedia, subtitle contents of the multimedia, and the subtitle interaction aggregation indicator are displayed in the multimedia display interface; and displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation indicator, the subtitle interaction content being an interaction content for the subtitle content.

[0007] An embodiment of the present disclosure further provides a multimedia interaction method, the method including: receiving an interaction input trigger operation, the interaction input trigger operation triggering an interaction input to a multimedia and / or a subtitle content of the multimedia; acquiring interaction input data triggered based on the interaction input trigger operation; and displaying the interaction input data in association with the multimedia and the subtitle content.

[0008] An embodiment of the present disclosure further provides an information interaction method, the method including: obtaining user interaction content with a selected character in the text content; and displaying an interaction aggregation indicator in association with a text snippet in which the selected character is located, the text content including at least one text snippet, each text snippet including interaction content corresponding to one interaction aggregation indicator.

[0009] An embodiment of the present disclosure further provides a multimedia interaction device, the device comprising: A trigger module for receiving a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia display interface, the multimedia display interface displaying multimedia, subtitle contents of the multimedia, and the subtitle interaction aggregation indicator; and an interaction content display module for displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation mark, the subtitle interaction content being an interaction content for the subtitle content.

[0010] An embodiment of the present disclosure further provides a multimedia interaction device, the device comprising: an interaction input trigger module that receives an interaction input trigger operation, the interaction input trigger operation triggering an interaction input to a multimedia and / or an interaction input to a subtitle content of the multimedia; an interaction data acquisition module for acquiring interaction input data triggered based on the interaction input trigger operation; and an interaction data display module that displays the interaction input data in association with the multimedia and the subtitle content.

[0011] An embodiment of the present disclosure further provides an interaction device, the device comprising: an interaction acquisition module for acquiring user interaction content with respect to a selected character in the text content; a sign display module that displays an interaction aggregation sign in association with a text snippet in which the selected character is located, the text content including at least one text snippet, and each text snippet including interaction content includes a sign display module corresponding to one interaction aggregation sign.

[0012] An embodiment of the present disclosure further provides an electronic device, the electronic device including a processor and a memory storing instructions executable by the processor, the processor reading the executable instructions from the memory and executing the instructions to realize a multimedia interaction method or an information interaction method provided by an embodiment of the present disclosure.

[0013] An embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored, the computer program implementing the multimedia interaction method or information interaction method provided by the embodiment of the present disclosure. Effect of the Invention

[0014] The technical solution provided in the embodiments of the present disclosure has the following advantages compared with the prior art: according to the multimedia interaction solution provided in the embodiments of the present disclosure, a user's trigger operation on a subtitle interaction aggregation mark in a multimedia display interface is accepted, and the multimedia display interface displays (presents / displays) multimedia, multimedia subtitle content, and subtitle interaction aggregation mark, and displays at least one subtitle interaction content corresponding to the subtitle interaction aggregation mark, where the subtitle interaction content is interaction content for the subtitle content. According to the above technical solution, the subtitle interaction aggregation sign is an aggregation expression of multiple subtitle interaction contents. Therefore, by triggering the subtitle interaction aggregation sign, multiple subtitle interaction contents for the multimedia subtitle content can be quickly displayed, improving the display efficiency of the subtitle interaction content. When the subtitle interaction aggregation sign is not triggered, only one sign is displayed. Compared with the traditional display method, it can be displayed according to the user's choice, which is more flexible and has a higher display effect. [Brief description of the drawings]

[0015] The above and other features, advantages, and aspects of each embodiment of the present disclosure will become more apparent with reference to the following specific embodiments in conjunction with the drawings, in which like or similar reference numerals indicate like or similar elements, where the drawings are schematic and parts and elements are not necessarily drawn to scale. [Figure 1] 1 is a flowchart of a multimedia interaction method according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram of a multimedia exhibit interface according to an embodiment of the present disclosure. [Diagram 3] 1 is a schematic diagram of another multimedia exhibit interface according to an embodiment of the present disclosure. [Figure 4]1 is a schematic diagram of another multimedia exhibit interface according to an embodiment of the present disclosure. [Diagram 5] 4 is a flowchart of another multimedia interaction method according to an embodiment of the present disclosure. [Figure 6] 1 is a flowchart of an information interaction method according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a schematic diagram of a configuration of a multimedia interaction device according to an embodiment of the present disclosure. [Figure 8] FIG. 2 is a schematic configuration diagram of another multimedia interaction device according to an embodiment of the present disclosure. [Figure 9] 1 is a schematic configuration diagram of an information interaction device according to an embodiment of the present disclosure. [Figure 10] FIG. 1 is a schematic diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, the embodiments of the present disclosure will be described in more detail with reference to the drawings. The drawings show some embodiments of the present disclosure, but the present disclosure is not limited to the embodiments described herein and may be realized in various forms, and these embodiments are provided for a more thorough and complete understanding of the present disclosure. Hereinafter, the drawings and embodiments of the present disclosure are merely illustrative and are not intended to limit the scope of protection of the present disclosure.

[0017] The steps described in the method embodiments of the present disclosure may be performed in a different order and / or in parallel, and the method embodiments may include additional steps and / or omit the performance of steps shown, and the scope of the present disclosure is not limited in this respect.

[0018] As used herein, the term "including" and variations thereof are used in an open inclusion, including but not limited to. The term "based on" means "based at least in part on." The term "in one embodiment" refers to "at least one embodiment," the term "in another embodiment" refers to "at least one other embodiment," and the term "in some embodiments" refers to "at least some embodiments." Relevant definitions of other terms are provided below.

[0019] It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are merely intended to distinguish different devices, modules or units, and do not limit the order or interdependence of functions performed by these devices, modules or units.

[0020] It should be noted that the modifications "one" and "multiple" referred to in the present disclosure are general rather than limiting, and as will be understood by one of ordinary skill in the art, unless otherwise expressly indicated in the specification, they are to be understood as "one or multiple."

[0021] The names of messages or information interacted between multiple devices in the embodiments of the present disclosure do not limit the scope of these messages or information, but are merely for explanation purposes.

[0022] 1 is a flowchart of a multimedia interaction method according to an embodiment of the present disclosure, the method being performed by a multimedia interaction device, which is implemented in software and / or hardware and generally integrated in an electronic device. As shown in FIG. 1, the method includes the following steps: Step 101: receiving a user's trigger operation on a subtitle interaction aggregation mark in a multimedia display interface, and displaying multimedia, multimedia subtitle content, and the subtitle interaction aggregation mark in the multimedia display interface;

[0023] A multimedia display interface is an interface for displaying multiple types of multimedia information, and the multimedia and subtitle contents are respectively displayed in different areas of the multimedia display interface, i.e., the multimedia display interface is provided with areas such as a multimedia area and a subtitle area, which are respectively used to display multimedia, multimedia subtitle contents, and subtitle interaction aggregation indicators, etc., where the multimedia may include audio and / or video and is not specifically limited, and the subtitle interaction aggregation indicator is a presentation indicator for indicating that the user has interaction content with the subtitles, and the presentation indicator is an aggregation indicator corresponding to one or more interaction contents.

[0024] Specifically, when a user views a multimedia display interface, a trigger operation by the user on an existing subtitle interaction aggregation sign of the multimedia display interface can be accepted. Specifically, the trigger operation can be of various types. For example, when the user performs a click operation or a hover operation on the subtitle interaction aggregation sign, it can be determined that the trigger operation has been accepted.

[0025] In an embodiment of the present disclosure, before receiving a user's trigger operation on a subtitle interaction aggregation indicator in the multimedia interface, the multimedia interaction method may further include displaying a subtitle interaction aggregation indicator in the multimedia display interface in response to obtaining a user's subtitle interaction content for a target subtitle in the subtitle content. When receiving a user's interaction input trigger operation on the target subtitle in the subtitle content in the multimedia display interface, an interaction input interface including an interaction component is displayed to the user, and the subtitle interaction content input by the user can be obtained according to the interaction component, the number of interaction components may be multiple, including a comment component and / or an emoticon component, and the corresponding subtitle interaction content includes a subtitle comment and / or a subtitle emoticon. After obtaining the subtitle interaction content, a subtitle interaction aggregation indicator is set in the multimedia display interface and displayed to the user, and the target subtitle is highlighted in a set manner. The setting manner is any feasible display manner that can be distinguished from other parts of the subtitle content, for example, the setting manner may include at least one of adding background color, displaying in bold, and adding underline, but is not limited thereto.

[0026] Illustratively, FIG. 2 is a schematic diagram of a multimedia exhibition interface according to an embodiment of the present disclosure. As shown in FIG. 2, an audio playback time axis 30 is displayed above the multimedia exhibition interface 10. The multimedia exhibition interface 10 includes a first area 11 and a second area 12. The second area 12 displays subtitle content. “1234” with added background color and underline is the target subtitle currently selected by the user. The diagram also displays comments on the target subtitle entered by user 3 and user 4, which are “This conclusion should be left undecided” and “I agree with this”, respectively.

[0027] The first area 11 displays information related to the current audio, the top area of ​​the first area 11 in the figure displays a picture or a current video related to the current audio, and the participant information is displayed below the picture or video. In FIG. 2, the profile pictures of the participants are displayed, and the bottom area of ​​the first area 11 displays three related links, each of which includes a title and a link source. For example, the first related link is a document with a document title of 1 shared by user 1, and the current user clicks the link addition button on the right to add more related links according to the actual situation. In FIG. 2, the top area of ​​the multimedia display interface 10 further displays the title of the multimedia "Team Review Meeting" and other related contents, and "2019.12.20 10:00 AM" in the figure indicates the start time of the multimedia, "1h30m30s" indicates the duration of the multimedia is 1 hour 30 minutes 20 seconds, and "16" indicates the number of participants.

[0028] The subtitle content is obtained, for example, by recognizing multimedia, specifically, by adopting a speech recognition (Automatic Speech Recognition, ASR) technology for the multimedia to obtain speech information through recognition, and converting the speech information into text content to obtain the subtitle content. In the embodiments of the present disclosure, the specific speech recognition technology is not limited, and for example, a random model method or an artificial neural network method may be adopted. The subtitle content includes at least two subtitle snippets, and the subtitle snippets are obtained by performing semantic segmentation on the subtitle content. Specifically, one or more subtitle snippets are obtained by performing semantic segmentation on the subtitle content, and each subtitle snippet includes a part of the subtitle content, and the specific number of subtitle snippets is not limited.

[0029] In an embodiment of the present disclosure, the step of displaying a subtitle interaction aggregation indicator in a multimedia display interface includes the following steps: displaying a subtitle interaction aggregation indicator in a multimedia display interface in association with a subtitle snippet in which a target subtitle is located, and each subtitle snippet containing subtitle interaction content corresponds to one subtitle interaction aggregation indicator. The subtitle interaction aggregation indicator includes the number of subtitle interaction contents for the associated subtitle snippet, and the subtitle interaction content includes comments and / or emoticons. Preferably, the method may further include the step of respectively displaying corresponding subtitle interaction aggregation indicators for at least two consecutive subtitle snippets when the same target subtitle exists in at least two consecutive subtitle snippets.

[0030] Since the subtitle content includes at least one subtitle snippet, when displaying the subtitle interaction aggregation indicator, the subtitle interaction aggregation indicator is displayed in the multimedia display interface in association with the subtitle snippet where the target subtitle is located. The number of target subtitles may be one or more. If the number of target subtitles is one, the target subtitle may be in one subtitle snippet or in at least two consecutive subtitle snippets, and the corresponding subtitle interaction aggregation indicator is displayed for at least two consecutive subtitle snippets. If the number of target subtitles is two or more, the target subtitle may be in one subtitle snippet or in multiple discontinuous subtitle snippets, and the subtitle snippet where each target subtitle is located will display the corresponding subtitle interaction aggregation indicator. The specific location of the subtitle interaction aggregation indicator is not limited and may be set according to the actual situation. For example, if the subtitle snippets in which the target subtitles are located are the first and third subtitle snippets, two subtitle interaction aggregation indicators are displayed in association with the first and third subtitle snippets, respectively, and no subtitle interaction aggregation indicator is displayed for the other subtitle snippets. Preferably, since the number of subtitle interaction contents is one or more, the subtitle interaction aggregation indicator further displays the specific number of subtitle interaction contents included in the associated subtitle snippet.

[0031] For example, refer to FIG. 2. In the figure, the subtitle content in the second region 12 includes three subtitle snippets, and user 3 and user 4 make comments on the target subtitle "1234", and the subtitle snippet where the target subtitle "1234" is located is the second subtitle snippet, i.e., the subtitle snippet below user 2, and therefore the subtitle interaction aggregation indicator 14 is displayed in association with the right side of the first line of text in the second subtitle snippet, and the "2" in the subtitle interaction aggregation indicator 14 indicates that there are two subtitle interaction contents, i.e., two comments. The display position of the subtitle interaction aggregation indicator 14 in FIG. 2 is only an example, and other positions displayed in association with the subtitle snippets are also applicable, for example, the subtitle interaction aggregation indicator 14 may be displayed to the left side of the first line of text in the second subtitle snippet.

[0032] Step 102: display at least one subtitle interaction content corresponding to the subtitle interaction aggregation mark, where the subtitle interaction content is an interaction content for the subtitle content.

[0033] As described above, the subtitle interaction content is interaction content that is pre-entered for some subtitles in the subtitle content, and specifically includes comments and / or emoticons. The embodiments of the present disclosure do not limit the number and contributors of the subtitle interaction content.

[0034] After receiving a user's trigger operation on a subtitle interaction aggregation indicator, one or more subtitle interaction contents corresponding to the subtitle interaction aggregation indicator can be displayed. Preferably, the step of displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation indicator can include displaying at least one subtitle interaction content of a subtitle snippet associated with the subtitle interaction aggregation indicator in a subtitle interaction window in the multimedia display interface.

[0035] The subtitle interaction window is a window for displaying subtitle interaction content, and switches the subtitle interaction content corresponding to the subtitle interaction aggregation indicator from hidden to expanded display. The subtitle interaction window is displayed in association with the subtitle interaction aggregation indicator, that is, the subtitle interaction window is displayed near the position of the subtitle interaction aggregation indicator. After receiving a user's trigger operation on the subtitle interaction aggregation indicator, the subtitle interaction window is displayed in association with the subtitle interaction aggregation indicator, and the corresponding subtitle interaction content is displayed in the subtitle interaction window.

[0036] For example, refer to FIG. 2. In the drawing, an arrow in the subtitle content indicates a user's trigger operation on the subtitle interaction aggregation indicator 14, and the subtitle snippet associated with the subtitle interaction aggregation indicator 14 is the second subtitle snippet in the drawing. The corresponding comments of User 3 and User 4 are displayed in the subtitle interaction window 13, and the posting times of the two comments are also displayed, that is, the comment posting time by User 3 is September 16th 14:38, and the comment posting time by User 4 is September 10th 11:16.

[0037] Preferably, the multimedia interaction method in the embodiment of the present disclosure further includes the step of displaying at least one multimedia interaction content in a subtitle interaction window, where the multimedia interaction content corresponds to a time period corresponding to the subtitle snippet associated with the subtitle interaction aggregation indicator, and the multimedia interaction content is an interaction content for the multimedia obtained based on a playback time axis of the multimedia.

[0038] The above multimedia interaction content is different from the subtitle interaction content, and is the user's interaction content with respect to the multimedia, and the interaction content is input based on the multimedia playback timeline, and the multimedia interaction content may include comments and / or emoticons, etc., and is not specifically limited. For example, when playing a video, the user can input comments or emoticons for the current video screen frame by triggering the interaction component displayed on the video playback timeline. In the embodiment of the present disclosure, the subtitle interaction aggregation indicator corresponds to the multimedia interaction content included in the time period in the multimedia of the associated subtitle snippet, and the corresponding multimedia interaction content can be displayed in the subtitle interaction window after receiving the user's trigger operation on the subtitle interaction aggregation indicator.

[0039] Preferably, the subtitle interaction window can simultaneously display the subtitle interaction content and the multimedia interaction content, and can be specifically distinguished in various ways, which are not limited in the embodiment of the present disclosure. For example, the subtitle interaction content and the multimedia interaction content can be displayed by arranging two regions, or the subtitle interaction content and the multimedia interaction content can be represented by adding different indicators. Preferably, the subtitle interaction aggregation indicator includes the number of subtitle interaction contents for the associated subtitle snippet as well as the number of multimedia interaction contents for the time period corresponding to the associated subtitle snippet. The subtitle interaction aggregation indicator can represent the sum of the number of corresponding subtitle interaction contents and the number of multimedia interaction contents, or can represent the number of subtitle interaction contents or the number of multimedia interaction contents, respectively.

[0040] Preferably, the multimedia interaction method may further include a step of accepting a user's trigger operation for a selected subtitle in the subtitle content, and displaying the subtitle interaction content of the selected subtitle in a subtitle interaction window in the multimedia display interface, or displaying the subtitle interaction content of a subtitle snippet in which the selected subtitle is located. The selected subtitle is a subtitle having a subtitle interaction content, i.e., one of a plurality of target subtitles including the above-mentioned subtitle interaction content. The selected subtitle is highlighted and displayed to the user, and when the user triggers the selected subtitle including the subtitle interaction content, the subtitle interaction content corresponding to the selected subtitle is displayed in the subtitle interaction window, or all the subtitle interaction contents of the subtitle snippet in which the selected subtitle is located are displayed in the subtitle interaction window, and the subtitle interaction content of the selected subtitle is located in the middle of them, and the subtitle interaction window jumps to the subtitle interaction content corresponding to the selected subtitle and displays it. By triggering the subtitles having the above subtitle interaction contents, the hidden subtitle interaction contents can also be displayed, providing more ways of displaying the subtitle interaction contents.

[0041] For example, referring to Fig. 2, after user 3 and user 4 make comments on the target subtitle "1234", "1234" becomes a selected subtitle, and after the user triggers the selected subtitle, the subtitle interaction contents of user 3 and user 4 can be displayed in the subtitle interaction window 13. Here, the subtitle interaction window 13 may display only the subtitle interaction contents of the currently selected subtitle, or may display the subtitle interaction contents of the entire subtitle snippet. If there are too many subtitle interaction contents displayed in the subtitle interaction window 13, the subtitle interaction contents in the subtitle interaction window 13 may jump to the position of the subtitle interaction contents corresponding to the selected subtitle and be displayed in an emphasized manner.

[0042] According to the multimedia interaction solution provided by the embodiment of the present disclosure, a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia display interface is accepted, and the multimedia display interface displays multimedia, multimedia subtitle content, and a subtitle interaction aggregation indicator, and displays at least one subtitle interaction content corresponding to the subtitle interaction aggregation indicator, and the subtitle interaction content is an interaction content for the subtitle content. According to the above technical solution, the subtitle interaction aggregation indicator is an aggregation expression for multiple subtitle interaction contents, so that the trigger by the subtitle interaction aggregation indicator can quickly display multiple subtitle interaction contents for the multimedia subtitle content, and improve the display efficiency of the subtitle interaction content; when the subtitle interaction aggregation indicator is not triggered, only one indicator is displayed, which can be displayed according to the user's choice, compared with the normal display mode, which is more flexible and has a higher display effect.

[0043] Specifically, in an actual usage scenario, subtitle data is associated with the multimedia data, and the user may interact, for example, provide feedback such as comments, in the subtitle data display interface, or may interact with the multimedia data being played in the playback area of ​​the multimedia data (for example, the video playback area or the audio playback area), and these interactions are actually targeted at the same or related information source, and the acquired interaction input is associated and displayed in the multimedia data and the subtitle data, thereby facilitating the input and distribution of the interaction input information, so that the interaction input input entered by the user in the subtitle data is displayed for viewing when playing a multimedia file associated with the subtitle data, or the interaction input entered by the user in the multimedia data playback interface is viewed in the display interface of the subtitle data associated with the multimedia data, which makes the user's interaction more convenient, and the interaction content can be conveniently entered in different scenarios, and the interaction content will not be lost when viewed in other scenarios, and the information interaction effect is better. For example, in a viewing mode (e.g., a mode primarily for viewing the subtitle content), an interaction input with respect to the subtitle content may be displayed in a playback scenario, or an interaction content input in the playback mode may be displayed in a subtitle data display interface in the viewing mode.

[0044] In some embodiments, the multimedia interaction method may further include: receiving a user's trigger operation on a subtitle interaction prompting indicator on a multimedia playback time axis of the multimedia display interface, and displaying subtitle interaction content corresponding to the subtitle interaction prompting indicator in a subtitle interaction window. Preferably, one subtitle interaction prompting indicator corresponds to at least one subtitle interaction content of a related subtitle, and the related subtitle is a subtitle at a related time point or a related time period associated with the subtitle interaction prompting indicator.

[0045] The subtitle interaction presentation indicator is a mark that is set after a user inputs interaction content for the subtitle content, to indicate that there is subtitle interaction content at the position. The subtitle interaction presentation indicator and the subtitle interaction aggregation indicator may have the same or different expression forms, and may be set specifically according to the actual situation. Since the subtitle interaction presentation indicator is set on the multimedia playback time axis, the subtitle interaction presentation indicator may be associated with one related time point on the playback time axis, or may be associated with one related time slot on the playback time axis. When the subtitle interaction presentation indicator is associated with one related time point, the related subtitle corresponding to the subtitle interaction presentation indicator is the subtitle corresponding to the related time point, and when the subtitle interaction presentation indicator is associated with one related time slot, the related subtitle corresponding to the subtitle interaction presentation indicator is the subtitle corresponding to the related time slot. In this way, one subtitle interaction presentation indicator corresponds to the subtitle interaction content for the related subtitle, and the number of subtitle interaction contents may be at least one.

[0046] Specifically, after receiving a user's trigger operation on a subtitle interaction presentation indicator on a multimedia playback timeline of the multimedia display interface, the subtitle interaction content corresponding to the subtitle interaction presentation indicator can be displayed in the subtitle interaction window, and the subtitle interaction window may be displayed in association with the subtitle interaction aggregation indicator.

[0047] In addition, in the embodiment of the present disclosure, a user can input multimedia interaction content for a multimedia based on the playback timeline of the multimedia, and a multimedia interaction presentation indicator corresponding to the multimedia interaction content is further displayed on the playback timeline to present to the user that there is multimedia interaction content. Here, the multimedia interaction presentation indicator is different from the specific expression form of the above-mentioned subtitle interaction presentation indicator in order to distinguish and present different interaction contents, and when the user triggers the subtitle interaction presentation indicator, the subtitle interaction content can be displayed in the subtitle interaction window, and when the user triggers the multimedia interaction presentation indicator, the multimedia interaction content can be displayed in the interaction card associated with the playback timeline.

[0048] For example, refer to FIG. 2, in the figure, four subtitle interaction prompting indicators, including three comment indicators and one emoticon indicator, are displayed on the audio playback time axis 30 in the multimedia display interface 10, which is only an exemplary display of indicators. The arrow of the first subtitle interaction prompting indicator on the left side can indicate a user's trigger operation on the subtitle interaction prompting indicator, and the specific form of the trigger operation can be various, for example, a user's click on the subtitle interaction prompting indicator. The relevant time period associated with the first subtitle interaction prompting indicator corresponds to the second subtitle snippet in the subtitle content, so that after the trigger operation, all the subtitle interaction content of the second subtitle snippet can be displayed in the subtitle interaction window 13.

[0049] In the above solution, based on the trigger operation of the subtitle interaction presentation indicator on the multimedia playback time axis, the multimedia interaction content for the multimedia corresponding to the subtitle interaction presentation indicator can be unfolded from a hidden state and displayed in the subtitle interaction window. The trigger of the subtitle interaction aggregation indicator and the subtitle interaction presentation indicator can both display the corresponding subtitle interaction content, which further improves the display effect of the subtitle interaction content and makes the display more flexible.

[0050] Preferably, the method may further include a step of playing multimedia based on a time stamp of a related subtitle corresponding to the subtitle interaction presentation mark after receiving a user's trigger operation on the subtitle interaction presentation mark on the multimedia playback time axis of the multimedia display interface. Preferably, the method may further include a step of jumping the subtitle content of the multimedia to a position where a related subtitle corresponding to the subtitle interaction presentation mark is located, and highlighting and displaying the related subtitle and the subtitle interaction content of the related subtitle after receiving a user's trigger operation on the subtitle interaction presentation mark on the multimedia playback time axis of the multimedia display interface.

[0051] In the embodiment of the present disclosure, the subtitle content is obtained by speech recognition of the multimedia, and each text in the subtitle content has corresponding voice information and corresponds to one time stamp in the multimedia. In the embodiment of the present disclosure, after receiving a user's trigger operation on the subtitle interaction presentation indicator on the multimedia playback time axis, a time stamp of the related subtitle corresponding to the subtitle interaction presentation indicator is determined, and the multimedia is played according to the time stamp. Then, the subtitle content individually displayed in the multimedia display interface is jumped to the position where the related subtitle is located, and the related subtitle and the subtitle interaction content of the related subtitle are highlighted. Preferably, in the subsequent playback process of the multimedia, the subtitle phrases after the related subtitle corresponding to the playback progress of the multimedia can be highlighted in sequence. The subtitle interaction content of the related subtitle is highlighted in the subtitle interaction window. In the embodiment of the present disclosure, the method of highlighting is not limited, and for example, at least one of highlighting, bolding, and underlining may be adopted for highlighting.

[0052] In the above solution, based on the trigger operation on the subtitle interaction prompting indicator on the playback time axis of the multimedia, the subtitle interaction prompting indicator interacts with the corresponding parts of the multimedia and subtitle content in a related manner, so that the user can better understand the related content of the subtitle interaction prompting indicator, better meet the user's interaction needs, and improve the user's interaction experience effect.

[0053] In some embodiments, the multimedia interaction method may further include the steps of: receiving a user's interaction display trigger operation for a position corresponding to a target time point on a multimedia playback time axis; and displaying an interaction card, the interaction card including subtitle interaction data of subtitles corresponding to the target time point, the subtitle interaction data including subtitle interaction content and / or statistical data of the subtitle interaction content. Preferably, the interaction card further includes multimedia interaction data of the target time point for the multimedia, and the multimedia interaction data includes multimedia interaction content and / or statistical data of the multimedia interaction content.

[0054] The target time point may be a time point on the playback time axis corresponding to the interaction display trigger operation, or may be a key time point closest to the time point on the playback time axis of the interaction display trigger operation, and the key time point is set on the playback time axis according to a predetermined rule. A user inputs multimedia interaction content for the multimedia based on the playback time axis of the multimedia, so that a multimedia interaction indication is displayed at the position of the target time point on the playback time axis to indicate to the user that there is multimedia interaction content here. The multimedia interaction indication indication has a specific expression form different from that of the above subtitle interaction indication indication in order to distinguish and present different interaction contents.

[0055] The interaction display trigger operation refers to a trigger operation for a user to display interaction content for the current multimedia, and in the embodiment of the present disclosure, the interaction display trigger operation includes a setting operation on the video playback timeline or a setting button, and the setting operation may include a hover operation or a click operation, that is, the interaction display trigger operation may include a hover operation or a click operation on the playback timeline, or a hover operation or a click operation on the setting button, and the setting button is a pre-installed button, and the specific position and manner of the button are not limited and may be set according to actual circumstances.

[0056] The interaction card may be a card that can display related interaction contents of multimedia, and the embodiment of the present disclosure does not limit the specific aspect of the interaction card. The subtitle interaction data is the interaction contents posted by all users for the subtitles corresponding to the target time point, the subtitle interaction data includes the subtitle interaction contents and / or the statistical data of the subtitle interaction contents, and the statistical data of the subtitle interaction contents includes at least one of the number of the subtitle interaction contents, user information, and posting time, etc. The multimedia interaction data is the interaction contents posted by the multimedia users corresponding to the target time point, the multimedia interaction data includes the multimedia interaction contents and / or the statistical data of the multimedia interaction contents, and the statistical data of the multimedia interaction contents includes at least one of the number of the multimedia interaction contents, user information, and posting time, etc.

[0057] In the embodiment of the present disclosure, when a user's interaction display trigger operation for a position on a multimedia playback time axis is accepted, a target time point for the position can be determined and an interaction card can be displayed, and the interaction card includes subtitle interaction data of subtitles corresponding to the target time point and / or multimedia interaction data of the target time point for the multimedia.Preferably, when a user's interaction display trigger operation for a multimedia interaction presentation sign on the playback time axis is accepted, the interaction card only includes multimedia interaction data of the target time point for the multimedia.

[0058] In the above solution, an interaction card can display associated subtitle interaction data and / or multimedia interaction data when a user triggers a timeline, so that the user can more intuitively understand various types of interaction data corresponding to the current point on the timeline, the displayed interaction data is more comprehensive and more targeted, and the display effect of the interaction data is better.

[0059] In some embodiments, the multimedia interaction method further includes: accepting a user's delete operation on a first subtitle snippet in the subtitle content; and after deleting the first subtitle snippet, displaying a subtitle interaction aggregation indicator corresponding to the first subtitle snippet in an area associated with a historical position of the first subtitle snippet.

[0060] The first subtitle snippet is any one of the subtitle snippets. In the embodiment of the present disclosure, the subtitle content includes at least two subtitle snippets, so that each subtitle snippet including the subtitle interaction content corresponds to one subtitle interaction aggregation indicator. When a user's delete operation for the first subtitle snippet is accepted, the first subtitle snippet can be deleted, and then, if the first subtitle snippet has an associated subtitle interaction aggregation indicator, the subtitle interaction aggregation indicator associated with the deleted first subtitle snippet is displayed in an area associated with the history position of the first subtitle snippet, and the position of the associated area is not specifically limited.

[0061] Illustratively, Fig. 3 is a schematic diagram of another multimedia exhibition interface according to an embodiment of the present disclosure, in which, as shown in Fig. 2 and Fig. 3, after deleting the subtitle snippet of User 2, the subtitle interaction aggregation indicator 14 associated with the subtitle snippet is displayed between the subtitle snippet of User 1 and the subtitle snippet of User 3, and is associated with the historical position of the subtitle snippet of User 2 between the subtitle interaction aggregation indicator 15 associated with the subtitle snippet of User 1 and the subtitle interaction aggregation indicator 16 associated with the subtitle snippet of User 3. In Fig. 3, after deleting the subtitle snippet, the display position of the subtitle interaction aggregation indicator is not limited but is only an example.

[0062] In some embodiments, the multimedia interaction method may further include: receiving a user's split operation for a second subtitle snippet in the subtitle content, the split operation being used to split the second subtitle snippet into at least two sub-subtitle snippets; and splitting subtitle interaction content of the second subtitle snippet based on the split sub-subtitle snippets. Preferably, the multimedia interaction method may further include deleting a historical subtitle interaction aggregation indicator of the second subtitle snippet and associating it with the sub-subtitle snippet having the subtitle interaction content to display a new subtitle interaction aggregation indicator.

[0063] The second subtitle snippet is also any one of subtitle snippets. After receiving a user's split operation for the second subtitle snippet, a split operation can be performed on the second subtitle snippet to split the second subtitle snippet into at least two subsubtitle snippets, and a subtitle interaction content of the second subtitle snippet can be split based on the subsubtitle snippets. Then, after splitting the second subtitle snippet, since the second subtitle snippet no longer exists, a corresponding historical subtitle interaction aggregation indicator can be deleted, and a new subtitle interaction aggregation indicator can be associated and displayed for the subsubtitle snippet including the subtitle interaction content, but a subtitle interaction aggregation indicator does not need to be displayed for the subsubtitle snippet not including the subtitle interaction content.

[0064] In the above solution, when a subtitle snippet is deleted, the subtitle interaction content for the subtitle snippet is retained, and a subtitle interaction aggregation indicator can be retained to display the subtitle interaction content thereafter; when a subtitle snippet is split, the subtitle interaction content for the subtitle snippet is also split, and a new subtitle interaction aggregation indicator is displayed for the subtitle snippet according to the actual subtitle interaction content after splitting. The display of the subtitle interaction content can also change according to the user's actual operation, which is more flexible.

[0065] In some embodiments, the multimedia interaction method may further include the steps of: accepting a user's reply operation to the subtitle interaction content; displaying a reply editing interface and acquiring the subtitle interaction reply content input by the user in the reply editing interface. Preferably, the multimedia interaction method may further include the steps of displaying the subtitle interaction reply content in association with the subtitle interaction content.

[0066] The reply operation is an operation of replying to an existing subtitle interaction content, and specific operation forms are diverse, for example, a hover operation on the subtitle interaction content and a click operation on a displayed reply button. The reply editing interface is an interface provided to a user so that the user can edit and post reply content. After the subtitle interaction content is displayed in the subtitle window, a reply operation from the user to the subtitle interaction content can be accepted, and then the reply editing interface can be displayed to the user to obtain the subtitle interaction reply content input by the user in the reply editing interface, and the subtitle interaction reply content may be a comment reply and / or an emoticon reply. Preferably, the subtitle interaction reply content is displayed in association with the subtitle interaction content, i.e., the subtitle interaction reply content may also be displayed in the subtitle interaction window.

[0067] For example, referring to FIG. 2, an arrow in the subtitle display window 13 indicates the selection of the current user 5 for the comment “I agree with this” made by the user 4, and a reply button is displayed to the user on the right side of the target subtitle “1234”. After receiving the trigger of the current user 5 on the reply button, it is determined that the reply operation for the target subtitle “1234” is accepted, and then a reply editing interface is displayed to the user 5 (not shown) and the subtitle interaction reply content entered by the user 5 is obtained. “You should also take a look” in the drawing is the subtitle interaction reply content entered by the user 5, and the subtitle interaction reply content is displayed below the target comment “I agree with this” and presented to the user with different gradations of background color.

[0068] In the above solution, after the subtitle interaction content is displayed, the user can further make a reply interaction on the subtitle interaction content according to actual needs, and the reply interaction content is related to the previous subtitle interaction content, which further meets the user's interaction needs for the subtitle interaction content and improves the user's interaction experience.

[0069] In some embodiments, the multimedia interaction method may further include receiving a user's search trigger operation for the search keyword, and highlighting a target keyword that is a search hit based on the search keyword in the subtitle content and the subtitle interaction content. The subtitle interaction content in which the target keyword exists is displayed in a subtitle interaction window.

[0070] The search keyword is a keyword currently input by the user for performing a search operation. The search trigger operation is a trigger operation for a search, and the specific form is not limited, for example, triggering a pre-installed search button. The target keyword is a keyword that is the same as or similar to the search keyword. After receiving the user's search trigger operation for the search keyword, a search operation is performed on the subtitle content and the subtitle interaction content, and a word having a matching degree greater than a matching degree threshold is determined as a search hit target keyword, and the target keyword is highlighted. In the embodiment of the present disclosure, the method of highlighting the target keyword is not limited, and may be, for example, highlighting, bolding, underlining, etc., and the target keyword in the subtitle content may be the same as or different from the method of highlighting the target keyword in the subtitle interaction content. The subtitle interaction content is displayed in a subtitle interaction window, and the subtitle interaction window may jump to the position of the target keyword.

[0071] Preferably, the multimedia interaction method further includes displaying a search hit indication indicator at a position corresponding to the target keyword timestamp on the multimedia playback timeline. The search hit indication indicator indicates to the user that the subtitle and / or subtitle interaction content corresponding to the current position has been searched for, and the specific form of the search hit indication indicator is not limited, and for example, the search hit indication indicator may be displayed as a circular point. After determining the target keyword, the time point of the target keyword in the multimedia may be determined, and the search hit indication indicator may be displayed at a position corresponding to the timestamp where the target keyword is located on the multimedia playback timeline.

[0072] Illustratively, FIG. 4 is a schematic diagram of another multimedia display interface according to an embodiment of the present disclosure. Referring to FIG. 4, a search area 17 is displayed in the multimedia display interface 10, a user inputs a search keyword in the search area 17 and triggers a search button to perform a search operation for the search keyword, after the search is completed, the hit target keyword in the subtitle content and the subtitle interaction content is highlighted and displayed, for example, the search keyword in the drawing is “innovation”, in the subtitle content and the subtitle interaction window 13, “innovation” in the subtitle interaction content is both displayed with background color added, and the circle point under the audio playback time axis 30 above the multimedia display interface indicates a search hit presentation indicator, for example, the four search hit presentation indicators for “innovation” in the drawing are all displayed, and one subtitle content and the corresponding subtitle interaction content in the drawing both contain “innovation”, and only one search hit presentation indicator is displayed on the playback time axis.

[0073] In the embodiment of the present disclosure, the subtitle content and the subtitle interaction content can support a search function, and the post-search keywords can be associated with and interacted with the subtitle content, the subtitle interaction content, and the multimedia, so that the user can intuitively and quickly understand the corresponding relationship between different contents of the search results, and improve the user's search experience.

[0074] Preferably, the multimedia interaction method may further include a step of displaying a total number of hit target keywords in a search display area, and a step of highlighting a currently selected keyword in the target keywords and a corresponding search hit display indicator to display a sorted result of the currently selected keyword in the target keywords, and a manner of highlighting the currently selected keyword and the search hit display indicator is different from a manner of highlighting the target keyword. Preferably, the multimedia interaction method may further include a step of receiving a user's jump operation for the currently selected keyword, the jump operation including an upward jump and / or a downward jump, and a step of displaying the currently selected keyword after the jump.

[0075] In an embodiment of the present disclosure, after a search operation is performed on a search keyword, the total number of hit target keywords is displayed in the search display area, the currently selected keyword and a search hit display indicator corresponding to the currently selected keyword are highlighted, and the sorting result of the currently selected keyword in the target keywords, i.e., the number of the currently selected keyword among a plurality of target keywords, is displayed. The manner of highlighting the currently selected keyword and the search hit display indicator is different from the manner of highlighting the target keyword in order to distinguish it from other target keywords. Preferably, a jump operation by a user for the currently selected keyword is accepted, the jump operation being used to jump to other target keywords, specifically including an upward jump and / or a downward jump, and after the jump, the currently selected keyword after the jump and the corresponding search hit display indicator are highlighted in a second manner.

[0076] For example, referring to Fig. 4, "1 / 4" in the search area 17 in Fig. 4 indicates that the total number of "innovation" hits is 4 and the currently selected "innovation" is the first, the first "innovation" in the subtitle content is highlighted in a bold manner, and the corresponding search prompt mark is displayed as a square dot. Two jump buttons, an upward jump button and a downward jump button, are displayed behind "1 / 4" in the search area 17, and when a user clicks the downward jump button, a downward jump operation is performed on the currently selected keyword, and the picture jumps to the second "innovation" in the subtitle content, and "1 / 4" in the picture changes to "2 / 4", the second "innovation" is highlighted in a bold manner, the corresponding search prompt mark is displayed as a square dot, and the first "innovation" and the corresponding search prompt mark return to being highlighted in the first manner.

[0077] In the above solution, after a user performs a search on subtitle content and subtitle interaction content, the search results can be further displayed, and the user may further select the search results, so that the corresponding relationships between different contents of different search results can be understood through different highlight displays after selection, and the search experience effect of the user is further improved.

[0078] In some embodiments, the multimedia interaction method further includes receiving a user's translation trigger operation for the multimedia subtitle content and the subtitle interaction content, and translating the multimedia subtitle content and the subtitle interaction content from an initial language to a target language.

[0079] The translation trigger operation is a trigger operation for translating subtitle content and subtitle interaction content between different languages. After receiving a user's translation trigger operation for the subtitle content and the subtitle interaction content, the subtitle content and the subtitle interaction content can be translated from a current initial language to a target language, and the target language may be a translation language specified by the user, may include multiple languages, and is not specifically limited.

[0080] For example, referring to FIG. 4, a translation button 18 is displayed on the multimedia exhibition interface 10. When a user triggers the translation button 18, translation may be performed on all texts in the multimedia exhibition interface 10, or translation may be performed on the subtitle content or subtitle interaction content, specifically, translation from an initial language to a target language, for example, translation of the subtitle content and subtitle interaction content from Chinese to English.

[0081] In the embodiments of the present disclosure, the subtitle content and the subtitle interaction content can support a translation function, which meets the translation needs of users for different languages ​​and is beneficial for users to understand the subtitle content and the subtitle interaction content.

[0082] Also, referring to FIG. 4, the multimedia exhibition interface 10 further includes a keyword display area 19 and a share button 20, and when a user triggers one of the buttons, the user can perform a corresponding operation. The keyword display area 19 exemplarily displays five keywords, namely, "innovation", "size", "frame", "component" and "rename". The keywords are obtained by performing keyword extraction on the subtitle content and the subtitle interaction content. When a user triggers one of the keywords, the keyword in the subtitle content and the subtitle interaction content is highlighted. For example, when "innovation" is triggered, "innovation" in the subtitle content and the subtitle interaction content is highlighted. The manner of highlighting is not specifically limited. When a user triggers the share button 20, the entire multimedia exhibition interface 10 is shared with other users. And the audio playback time axis 30 may be displayed at the bottom or top of the multimedia exhibition interface 10, and is not specifically limited and may be set according to the actual situation.

[0083] It should be noted that the multimedia display interfaces of Figures 2, 3 and 4 in the embodiments of the present disclosure are only examples and are not limited to the multimedia display interface. The multimedia display interface may be set in various situations according to actual situations and user needs.

[0084] According to the multimedia interaction method provided by the embodiment of the present disclosure, the trigger by the subtitle interaction aggregation indicator can quickly display multiple subtitle interaction contents for the multimedia subtitle content, improving the display efficiency of the subtitle interaction content, and when the subtitle interaction aggregation indicator is not triggered, only one indicator is displayed, which can be displayed according to the user's selection, and is more flexible and has a higher display effect than the normal display method. In addition, the trigger by the subtitle interaction aggregation indicator can quickly display multiple multimedia interaction contents for the multimedia, and both types of interaction contents can be displayed by triggering the subtitle interaction aggregation indicator, which further improves the display efficiency and display effect of the interaction content, and based on the trigger operation of the subtitle interaction presentation indicator on the multimedia playback time axis, the multimedia interaction content can be expanded from the hidden state and displayed in the subtitle interaction window, which further improves the display effect of the subtitle interaction content and makes the display more flexible. An interaction card allows users to display associated subtitle interaction data and / or multimedia interaction data when triggering the timeline, so that the displayed interaction data is more comprehensive and more targeted, and the display effect of the interaction data is better; the subtitle snippets in the subtitle content support splitting and deletion functions, and the display of the subtitle interaction content can also change according to the user's actual operation, which is more flexible; after the subtitle interaction content is displayed, users can further reply to the subtitle interaction content according to their actual needs, and the subtitle interaction content supports operations such as search and translation, making the interaction functions more diversified and more convenient for meeting the diverse needs of users.

[0085] 5 is a flowchart of another multimedia interaction method according to an embodiment of the present disclosure, the method being performed by a multimedia interaction device, which is realized by software and / or hardware and generally integrated in electronic equipment. As shown in FIG. 5, based on the above embodiment, the method includes the following steps: Step 201: receiving an interaction input trigger operation, which triggers an interaction input to a multimedia and / or an interaction input to a subtitle content of the multimedia.

[0086] An interaction input trigger operation is a trigger operation in which a user performs an interaction input on the multimedia and / or subtitle content of the multimedia currently being viewed. In the embodiments of the present disclosure, the interaction input trigger operation is not limited to a specific one. For example, the interaction input trigger operation may be a hover operation or a click operation, etc.

[0087] When a user views the multimedia and / or multimedia subtitle content, a user interaction input trigger operation for the multimedia and / or multimedia subtitle interaction can be accepted.

[0088] Step 202: Obtain interaction input data triggered based on an interaction input trigger operation.

[0089] The interaction input data may include multimedia interaction content and / or subtitle interaction content in the above embodiments, where the multimedia interaction content is the interaction content input for the multimedia, and the subtitle interaction content is the interaction content input for the subtitle content.

[0090] Specifically, after receiving an interaction input trigger operation, an interaction input interface including an interaction component is displayed to a user, and multimedia interaction content and / or subtitle interaction content input by the user can be obtained based on the interaction component, and the number of interaction components may be multiple and may include a comment component and / or an emoticon component.

[0091] Step 203: The interaction input data is associated with the multimedia and subtitle contents and displayed.

[0092] In an embodiment of the present disclosure, the multimedia interaction content of the interaction input data may be displayed in association with the multimedia and subtitle content, and the subtitle interaction content of the interaction input data may be displayed in association with the multimedia and subtitle content.

[0093] In some application scenarios, the interaction input data is displayed in association with the subtitle content according to the timestamp of the subtitle content and the multimedia. Specifically, a constituent unit of the subtitle content (e.g., a word, a phrase, and / or a paragraph in the subtitle content) has a timestamp, which represents the time information of the sound corresponding to the constituent unit appearing in the multimedia. In this way, the interaction input data triggered for the multimedia is displayed at the subtitle content corresponding to the trigger time (the progress of the playback of the multimedia indicated by the trigger operation) (i.e., displayed in association with the subtitle content), and the interaction input data triggered for the subtitle content may also be displayed in association with the multimedia snippet corresponding to the timestamp of the triggered subtitle content.

[0094] For the multimedia interaction content in the interaction input data, after the multimedia interaction content is obtained, the multimedia interaction content is displayed in a pop-up manner or in association with a multimedia time axis, and the multimedia interaction content is displayed in association with the subtitle interaction window of the area where the subtitle content is located.For the subtitle interaction content in the interaction input data, after the subtitle interaction content is obtained, the subtitle interaction content is displayed in a pop-up manner or in association with a multimedia time axis, and the subtitle interaction content is displayed in association with the subtitle interaction window of the area where the subtitle content is located.Different types of interaction contents in the interaction input data may be displayed in the same window or interface or in different windows or interfaces when displayed, and are not specifically limited.

[0095] In addition, different types of interaction contents in the interaction input data may be distinguished in different ways, for example, by different indicators, when they are associated and displayed.

[0096] In addition, the manner in which the interaction input data is obtained and then associated with the multimedia and subtitle content and displayed includes multiple methods, and the above are merely examples, and will not be described in detail here. In addition, provided that there is no contradiction, each step and feature in the embodiment of the present disclosure may be combined with other embodiments of the present disclosure (including, but not limited to, the embodiment shown in FIG. 1, the embodiment shown in FIG. 6, and specific implementation means of these embodiments).

[0097] According to the multimedia interaction method provided by the embodiment of the present disclosure, an interaction input trigger operation is received, the interaction input trigger operation triggers an interaction input for multimedia and / or an interaction input for multimedia subtitle content, the interaction input data triggered according to the interaction input trigger operation is obtained, and the interaction input data is associated with the multimedia and subtitle content for display. According to the above technical solution, after the interaction content for multimedia and the interaction content for the multimedia subtitle content are obtained, both can be associated with the multimedia and subtitle content for display, which realizes the cross-association display between the interaction data of the multimedia and the subtitle content, improves the association display effect, further enhances the user's understanding of the mutual correspondence between the three entities of multimedia, subtitles and interaction data, makes the association of the interaction function stronger and more diversified, and improves the user experience effect.

[0098] 6 is a flowchart of an information interaction method according to an embodiment of the present disclosure, the method being executed by an information interaction device, which is realized by software and / or hardware and generally integrated in electronic equipment. As shown in FIG. 6, based on the above embodiment, the method includes the following steps: Step 301: Obtain the user's interaction content with respect to the selected character in the text content.

[0099] The text content may be understood as any content composed of characters, and the specific origin is not limited, and the subtitle content obtained by recognizing multimedia audio in the above embodiment is an example. The selected character is a corresponding character in the above text content triggered by a user. The interaction content is an interaction content input by a user for the selected character, specifically including comments and / or emoticons, and the embodiment of the present disclosure does not limit the number and posters of the interaction content. If the text content is a subtitle content, the interaction content is a subtitle interaction content for the subtitle content.

[0100] When a user browses the text content, an interaction input interface including an interaction component is displayed to the user after receiving an interaction input trigger operation of the user for a selected character in the text content, and interaction content input by the user can be obtained based on the interaction component. Since the interaction content can include multiple types, the number of interaction components can be multiple, and can include, for example, a comment component and / or an emoticon component.

[0101] Step 302: Display an interaction aggregation indicator associated with a text snippet in which the selected character is located, where the text content includes at least one text snippet, and each text snippet including an interaction content corresponds to one interaction aggregation indicator.

[0102] The text snippet is obtained by dividing the above text content, specifically, it can be divided based on semantics, the text content includes one or more text snippets, each text snippet includes a part of the text content, and the number of text snippets is not limited. The interaction aggregation indicator is a presentation indicator for indicating that a user has interaction content with the text content, and the presentation indicator is an aggregation indicator and can correspond to one or more interaction contents.

[0103] After obtaining the user's interaction content for the selected character, display the interaction aggregation indicator in association with the text snippet where the selected character is located. The number of selected characters may be one or more. If the number of selected characters is one, the selected character may be present in one text snippet or at least two consecutive text snippets, and at least two consecutive text snippets will display the corresponding interaction aggregation indicator. If the number of selected characters is two or more, the selected character may be present in one text snippet or at least two different non-consecutive text snippets, and the text snippet where each selected character is located will display the corresponding interaction aggregation indicator. There is no specific limit on the location of the interaction aggregation indicator, and it may be set according to the actual situation.

[0104] Preferably, the number of interaction contents is one or more, so the interaction aggregation indicator further displays the specific number of interaction contents included in the associated text snippet.

[0105] Preferably, after displaying an interaction aggregation indicator, when a user's trigger operation on the interaction aggregation indicator is accepted, at least one interaction content included in a text snippet associated with the interaction aggregation indicator is displayed.

[0106] Furthermore, provided there is no contradiction, each step and feature in the embodiments of the present disclosure may be combined with other embodiments of the present disclosure (including, but not limited to, the embodiment shown in FIG. 1, the embodiment shown in FIG. 5, and specific implementation means of these embodiments).

[0107] According to the information interaction method provided by the embodiment of the present disclosure, the user's interaction content for the selected characters in the text content is obtained, and the interaction aggregation indicator is displayed in association with the text snippet where the selected characters are located, and the text content includes at least one text snippet, and each text snippet containing the interaction content corresponds to one interaction aggregation indicator. According to the above technical solution, after the user's interaction content for the selected characters in the text content is obtained, the interaction aggregation indicator is displayed in association with the text snippet where the selected characters are located, and the interaction aggregation indicator is an aggregation expression for one or more interaction contents, so that the interaction aggregation indicator can be triggered to quickly display multiple associated interaction contents, which improves the efficiency of displaying the interaction content, is more flexible, and has a high display effect.

[0108] 7 is a schematic diagram of a multimedia interaction device provided by an embodiment of the present disclosure, which is realized by software and / or hardware and is generally integrated into an electronic device. As shown in FIG. 7, the device includes: A trigger module 401 for receiving a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia display interface, where the multimedia display interface displays multimedia, subtitle contents of the multimedia, and the subtitle interaction aggregation indicator; an interaction content display module 402 for displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation label, the subtitle interaction content being an interaction content for the subtitle content; Includes.

[0109] Preferably, the apparatus comprises: The display device further includes a sign display module that displays the subtitle interaction aggregation sign in the multimedia display interface in response to obtaining a user's subtitle interaction content for a target subtitle in the subtitle content before receiving a user's trigger operation on the subtitle interaction aggregation sign in the multimedia display interface.

[0110] Preferably, the sign display module specifically comprises: The subtitle interaction aggregation indicator is displayed on the multimedia display interface in association with the subtitle snippet in which the target subtitle is located, and each subtitle snippet containing subtitle interaction content corresponds to one subtitle interaction aggregation indicator.

[0111] Preferably, the sign display module specifically comprises: If the same target subtitle exists in at least two consecutive subtitle snippets, display corresponding subtitle interaction aggregation indicators for the at least two consecutive subtitle snippets, respectively.

[0112] Preferably, the subtitle interaction aggregation indicator includes a number of subtitle interaction contents for an associated subtitle snippet, the subtitle interaction contents including comments and / or emoticons.

[0113] Preferably, the interaction content display module 402 specifically includes: At least one subtitle interaction content of the subtitle snippet associated with the subtitle interaction aggregation indicator is displayed in a subtitle interaction window in the multimedia display interface.

[0114] Preferably, the apparatus comprises: a multimedia interaction content module for displaying at least one multimedia interaction content in the subtitle interaction window; The multimedia interaction content corresponds to a time period corresponding to a subtitle snippet associated with the subtitle interaction aggregation indicator, and the multimedia interaction content is interaction content with the multimedia obtained based on a playback time axis of the multimedia.

[0115] Preferably, the subtitle content includes at least two subtitle snippets, the subtitle snippets being obtained by performing semantic segmentation on the subtitle content.

[0116] Preferably, the apparatus comprises: The multimedia display interface further includes a trigger module for receiving a user's trigger operation on a subtitle interaction prompting indicator on the multimedia playback time axis of the multimedia display interface, and displaying subtitle interaction content corresponding to the subtitle interaction prompting indicator in a subtitle interaction window.

[0117] Preferably, one of the subtitle interaction presentation indicators corresponds to at least one subtitle interaction content of a related subtitle, and the related subtitle is a subtitle at a related time point or time period corresponding to the subtitle interaction presentation indicator.

[0118] Preferably, the subtitle interaction window is displayed in association with the subtitle interaction aggregation indicator.

[0119] Preferably, the apparatus comprises: The multimedia display interface further includes a playback module for playing the multimedia according to a timestamp of a related subtitle corresponding to the subtitle interaction prompting mark after receiving a user's trigger operation on a subtitle interaction prompting mark on the playback time axis of the multimedia of the multimedia display interface.

[0120] Preferably, the apparatus comprises: The multimedia display interface further includes a highlight display module that, after receiving a user's trigger operation on a subtitle interaction presentation indicator on the playback time axis of the multimedia of the multimedia display interface, jumps subtitle content of the multimedia to a position where a related subtitle corresponding to the subtitle interaction presentation indicator is located, and highlights and displays the related subtitle and the subtitle interaction content of the related subtitle.

[0121] Preferably, the apparatus comprises: Further comprising a subtitle interaction data display module for receiving a user's interaction display trigger operation for a corresponding position of a target time point on the playback time axis of the multimedia, and displaying an interaction card; The interaction card includes subtitle interaction data of the subtitle corresponding to the target time point, and the subtitle interaction data includes subtitle interaction content and / or statistical data of the subtitle interaction content.

[0122] Preferably, the interaction card further includes multimedia interaction data for the multimedia at the target time point, the multimedia interaction data including multimedia interaction content and / or statistical data of the multimedia interaction content.

[0123] Preferably, the apparatus comprises: The display device further includes a subtitle deletion module that receives a user's delete operation for a first subtitle snippet in the subtitle content, and displays a subtitle interaction aggregation indicator corresponding to the first subtitle snippet in an area associated with a history position of the first subtitle snippet after deleting the first subtitle snippet.

[0124] Preferably, the apparatus comprises: The display device further includes a subtitle splitting module that accepts a user's splitting operation for a second subtitle snippet in the subtitle content, the splitting operation being to split the second subtitle snippet into at least two sub-subtitle snippets, and splits subtitle interaction content of the second subtitle snippet based on the split sub-subtitle snippets.

[0125] Preferably, the subtitle segmentation module specifically comprises: The historical subtitle interaction aggregation indicator of the second subtitle snippet is deleted, and a new subtitle interaction aggregation indicator is displayed in association with the subtitle interaction content of the subtitle snippet.

[0126] Preferably, the apparatus comprises: The multimedia display interface further includes an interaction display module for receiving a trigger operation by a user for a selected subtitle in the subtitle content, and displaying a subtitle interaction content of the selected subtitle or a subtitle interaction content of a subtitle snippet in which the selected subtitle is located in a subtitle interaction window in the multimedia display interface.

[0127] Preferably, the apparatus comprises: The display device further includes a subtitle interaction reply module for receiving a user's reply operation to the subtitle interaction content, displaying a reply editing interface, and acquiring the subtitle interaction reply content input by the user in the reply editing interface.

[0128] Preferably, the subtitle interaction reply module specifically comprises: The subtitle interaction reply content is displayed in association with the subtitle interaction content.

[0129] Preferably, the apparatus comprises: The display device further includes a search module that receives a user's search trigger operation for a search keyword, and highlights target keywords that are searched based on the search keyword in the subtitle content and the subtitle interaction content.

[0130] Preferably, the subtitle interaction content in which the target keyword exists is displayed in a subtitle interaction window.

[0131] Preferably, the search module specifically: A search hit indication indicator is displayed on the multimedia playback timeline at a position corresponding to the timestamp of the target keyword.

[0132] Preferably, the apparatus comprises: The search result display module further includes a search result display module that displays the total number of hit target keywords in a search display area, highlights a currently selected keyword and a corresponding search hit display indicator in the target keywords, and displays a sorted result of the currently selected keyword in the target keywords, The currently selected keyword and the search hit indicator are highlighted in a different manner than the target keyword.

[0133] Preferably, the search result display module specifically includes: receiving a user jump operation, including an upward jump and / or a downward jump, for the currently selected keyword; The currently selected keywords will be displayed after the jump.

[0134] Preferably, the apparatus comprises: The video processing apparatus further includes a translation module for receiving a translation trigger operation from a user for the multimedia subtitle content and the subtitle interaction content, and translating the multimedia subtitle content and the subtitle interaction content from an initial language to a target language.

[0135] Preferably, the multimedia and the subtitle content are each exhibited in different areas of the multimedia exhibit interface.

[0136] The multimedia interaction device provided by the embodiments of the present disclosure implements the multimedia interaction method provided by any of the embodiments of the present disclosure, and has function modules and beneficial effects corresponding to the implementation method.

[0137] 8 is a schematic diagram of another multimedia interaction device according to an embodiment of the present disclosure, which is implemented by software and / or hardware and generally integrated into an electronic device. As shown in FIG. 8, the device includes: an interaction input trigger module 501 for receiving an interaction input trigger operation, the interaction input trigger operation triggering an interaction input to a multimedia and / or an interaction input to a subtitle content of the multimedia; an interaction data acquisition module 502 for acquiring interaction input data triggered based on the interaction input trigger operation; and an interaction data display module 503 for displaying the interaction input data in association with the multimedia and the subtitle content.

[0138] The multimedia interaction device provided by the embodiments of the present disclosure implements the multimedia interaction method provided by any of the embodiments of the present disclosure, and includes functional modules and beneficial effects corresponding to the implementation method.

[0139] 9 is a schematic diagram of an information interaction device according to an embodiment of the present disclosure, which is realized by software and / or hardware and is generally integrated into an electronic device. As shown in FIG. 9, the device includes: An interaction acquisition module 601 for acquiring user interaction content with respect to a selected character in the text content; and a sign display module 602 for displaying an interaction aggregation sign in association with a text snippet in which the selected character is located, the text content including at least one text snippet, and each text snippet including interaction content includes a sign display module 602 corresponding to one interaction aggregation sign.

[0140] The information interaction device provided by the embodiments of the present disclosure implements the information interaction method provided by any of the embodiments of the present disclosure, and has functional modules and beneficial effects corresponding to the implementation method.

[0141] FIG. 10 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. Hereinafter, with specific reference to FIG. 10, it is a schematic diagram of a configuration suitable for realizing an electronic device 700 in an embodiment of the present disclosure. The electronic device 700 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device in FIG. 10 does not limit the functions and scope of use of the embodiment of the present disclosure, and is only one example.

[0142] 10, the electronic device 700 includes a processing unit (e.g., a central processing unit, a graphic processor, etc.) 701, which executes various appropriate operations and processes based on a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 further stores various programs and data for the operation of the electronic device 700. The processing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0143] Typically, input devices 706 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyro, etc., output devices 707 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc., storage devices 708 including, for example, a magnetic tape, hard disk, etc., and communication devices 709 may be connected to the I / O interface 705. The communication devices 709 allow the electronic device 700 to communicate wirelessly or via wires with other devices for data exchange. Although FIG. 10 illustrates the electronic device 700 having various devices, it is not required that the electronic device 700 implement or include all of the devices shown herein. Instead, more or fewer devices may be implemented or included.

[0144] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts are implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product, the computer program product includes a computer program carried on a non-transitory computer-readable medium, the computer program including a program code for performing the method shown in the flowchart. In this embodiment, the computer program is downloaded and installed from a network via a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, it performs the above-mentioned functions limited to the multimedia interaction method and / or information interaction method of the embodiment of the present disclosure.

[0145] It should be noted that the above computer readable medium of the present disclosure may be a computer readable signal medium, or a computer readable storage medium, or any combination of the above. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection having one or more leads, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a compact disk read only memory (CD-ROM), an optical memory, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer readable signal medium includes a data signal propagated in baseband or as part of a carrier, the data signal carrying the computer readable program code. Such propagated data signals may take many forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which transmits, propagates, or transmits a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code contained in a computer-readable medium may be transmitted over any suitable medium, including, but not limited to, optical cables, RF (radio frequency), or the like, or any suitable combination of the above.

[0146] In some embodiments, clients and servers communicate using any now known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and are connected to each other via any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include local area networks ("LANs"), wide area networks ("WANs"), international networks (e.g., the Internet), end-to-end networks (e.g., ad hoc end-to-end network), and any now known or later developed networks.

[0147] The computer-readable medium may be included in the electronic device, or may exist separately and not be attached to the electronic device.

[0148] The computer-readable medium has one or more programs stored therein, which, when executed by the electronic device, causes the electronic device to execute the following steps: accepting a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia display interface, in which the multimedia, subtitle content of the multimedia, and the subtitle interaction aggregation indicator are displayed on the multimedia display interface; and displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation indicator, in which the subtitle interaction content is an interaction content for the subtitle content.

[0149] Alternatively, the computer-readable medium has one or more programs stored therein, which, when executed by the electronic device, causes the electronic device to execute the following steps: receiving an interaction input trigger operation, the interaction input trigger operation triggering an interaction input to multimedia and / or to subtitle content of the multimedia; obtaining interaction input data triggered based on the interaction input trigger operation; and displaying the interaction input data in association with the multimedia and the subtitle content.

[0150] Alternatively, the computer-readable medium may include one or more programs, which, when executed by the electronic device, causes the electronic device to perform the following steps: obtaining user interaction content with a selected character in a text content; and displaying an interaction aggregation indicator in association with a text snippet in which the selected character is located, wherein the text content includes at least one text snippet, and each text snippet including interaction content corresponds to one interaction aggregation indicator.

[0151] Computer program code for carrying out the operations of the present disclosure is written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be entirely executed on the user computer, partially executed on the user computer, executed as a separate software package, partially executed on the user computer and partially executed on a remote computer, or entirely executed on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., connected via the Internet using an Internet Service Provider).

[0152] The flowcharts and block diagrams in the drawings illustrate possible system architectures, functions, and operations of the systems, methods, and computer program products of various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams represents a module, program segment, or part of code, which includes one or more executable instructions for implementing a certain logical function. In addition, in some alternative implementations, the functions described in the blocks may occur in a different order from that shown in the drawings. For example, two blocks shown in succession in practice may be executed substantially in parallel, or in some cases may be executed in a reverse order, depending on the functions involved. In addition, each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts may be implemented by a specialized system with hardware that executes a certain function or operation, or may be implemented by a combination of specialized hardware and computer instructions.

[0153] The units according to the embodiments of the present disclosure may be implemented in the form of software or hardware. In some cases, the names of the units do not limit the units themselves.

[0154] The functionality described above may be implemented at least in part in one or more hardware logic components. For example, and without limitation, types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0155] In this disclosure, a machine-readable medium is a tangible medium that can contain or store a program used by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include an electrical connection by one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0156] According to one or more embodiments of the present disclosure, the present disclosure provides a multimedia interaction method, comprising: receiving a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia display interface, in which multimedia, subtitle contents of the multimedia, and the subtitle interaction aggregation indicator are displayed in the multimedia display interface; and displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation indicator, the subtitle interaction content being an interaction content for the subtitle content.

[0157] According to one or more embodiments of the present disclosure, in a multimedia interaction method provided by the present disclosure, before receiving a user's trigger operation on a subtitle interaction aggregation indicator in the multimedia exhibition interface, in response to obtaining subtitle interaction content input by a user for a target subtitle in the subtitle content, displaying a subtitle interaction aggregation indicator corresponding to the subtitle interaction content in the multimedia exhibition interface.

[0158] According to one or more embodiments of the present disclosure, in the multimedia interaction method provided by the present disclosure, the step of displaying the subtitle interaction aggregation indicator on the multimedia display interface includes: The method includes a step of displaying the subtitle interaction aggregation indicator on the multimedia display interface in association with a subtitle snippet in which the target subtitle is located, wherein each subtitle snippet including subtitle interaction content corresponds to one subtitle interaction aggregation indicator.

[0159] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: The method further includes, when the same target subtitle exists in at least two consecutive subtitle snippets, respectively displaying corresponding subtitle interaction aggregation indicators for the at least two consecutive subtitle snippets.

[0160] According to one or more embodiments of the present disclosure, in a multimedia interaction method provided by the present disclosure, the subtitle interaction aggregation indicator includes a number of subtitle interaction contents for an associated subtitle snippet, and the subtitle interaction contents include comments and / or emoticons.

[0161] According to one or more embodiments of the present disclosure, in the multimedia interaction method provided by the present disclosure, the step of displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation mark includes: The method includes displaying at least one subtitle interaction content of the subtitle snippet associated with the subtitle interaction aggregation indicator in a subtitle interaction window in the multimedia display interface.

[0162] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: The method further includes the step of displaying at least one multimedia interaction content in a subtitle interaction window, the multimedia interaction content corresponding to a time period corresponding to a subtitle snippet associated with the subtitle interaction aggregation indicator, and the multimedia interaction content being an interaction content for the multimedia obtained based on a playback time axis of the multimedia.

[0163] According to one or more embodiments of the present disclosure, in a multimedia interaction method provided by the present disclosure, the subtitle content includes at least two subtitle snippets, and the subtitle snippets are obtained by performing semantic segmentation on the subtitle content.

[0164] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: receiving a user's trigger operation on a subtitle interaction presentation indicator on the multimedia playback timeline of the multimedia exhibition interface; The method further includes displaying subtitle interaction content corresponding to the subtitle interaction display mark in a subtitle interaction window.

[0165] According to one or more embodiments of the present disclosure, in the multimedia interaction method provided by the present disclosure, one of the subtitle interaction presentation indicators corresponds to at least one subtitle interaction content of a related subtitle, and the related subtitle is a subtitle at a related time point or related time period corresponding to the subtitle interaction presentation indicator.

[0166] According to one or more embodiments of the present disclosure, in a multimedia interaction method provided by the present disclosure, the subtitle interaction window is displayed in association with the subtitle interaction aggregation indicator.

[0167] According to one or more embodiments of the present disclosure, in a multimedia interaction method provided by the present disclosure, after receiving a user's trigger operation on a subtitle interaction presentation mark on the multimedia playback timeline of the multimedia exhibition interface, The method further includes playing the multimedia based on a timestamp of an associated subtitle corresponding to the subtitle interaction presentation indicator.

[0168] According to one or more embodiments of the present disclosure, in a multimedia interaction method provided by the present disclosure, after receiving a user's trigger operation on a subtitle interaction presentation mark on the multimedia playback timeline of the multimedia exhibition interface, The method further includes a step of jumping to a position of a related subtitle corresponding to the subtitle interaction indication mark, and highlighting and displaying the related subtitle and the subtitle interaction content of the related subtitle.

[0169] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: receiving a user's interaction display trigger operation for a corresponding position of the target time point on the multimedia playback time axis; and displaying an interaction card, the interaction card including subtitle interaction data of subtitles corresponding to the target time point, the subtitle interaction data including subtitle interaction content and / or statistical data of the subtitle interaction content.

[0170] According to one or more embodiments of the present disclosure, in the multimedia interaction method provided by the present disclosure, the interaction card further includes multimedia interaction data for the multimedia at the target time point, and the multimedia interaction data includes multimedia interaction content and / or statistical data of the multimedia interaction content.

[0171] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: receiving a delete operation by a user for a first subtitle snippet in the subtitle content; After deleting the first subtitle snippet, displaying a subtitle interaction aggregation indicator corresponding to the first subtitle snippet in an area associated with a historical position of the first subtitle snippet.

[0172] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: receiving a user's split operation on a second subtitle snippet in the subtitle content, the split operation being used to split the second subtitle snippet into at least two sub-subtitle snippets; The method further includes dividing a subtitle interaction content of the second subtitle snippet based on the divided sub-subtitle snippet.

[0173] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: The method further includes deleting a historical subtitle interaction aggregation indicator of the second subtitle snippet and displaying a new subtitle interaction aggregation indicator associated with the subtitle snippet having subtitle interaction content.

[0174] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: The method further includes accepting a user's trigger operation on a selected subtitle in the subtitle content, and displaying a subtitle interaction content of the selected subtitle or a subtitle interaction content of a subtitle snippet in which the selected subtitle is located in a subtitle interaction window in the multimedia display interface.

[0175] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: receiving a response operation from a user to the subtitle interaction content; The method further includes displaying a reply editing interface, and receiving subtitle interaction reply content input by the user in the reply editing interface.

[0176] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: The method further includes displaying the subtitle interaction reply content in association with the subtitle interaction content.

[0177] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: receiving a user's search trigger operation for a search keyword; The method further includes highlighting target keywords that are search hits based on the search keywords in the subtitle content and the subtitle interaction content.

[0178] According to one or more embodiments of the present disclosure, in the multimedia interaction method provided by the present disclosure, the subtitle interaction content in which the target keyword exists is displayed in a subtitle interaction window.

[0179] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: The method further includes displaying a search hit indication indicator at a position on the multimedia playback timeline corresponding to a timestamp of the target keyword.

[0180] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: displaying a total number of hits for the target keywords in a search display area; The method further includes a step of highlighting a currently selected keyword and a corresponding search hit presentation indicator in the target keyword, and displaying a sorted result of the currently selected keyword in the target keyword, wherein a manner of highlighting the currently selected keyword and the search hit presentation indicator is different from a manner of highlighting the target keyword.

[0181] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: receiving a user jump operation for the currently selected keyword, the jump operation including an upward jump and / or a downward jump; The method further includes the step of displaying the currently selected keyword after the jump.

[0182] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction method comprising: receiving a translation trigger operation by a user for the multimedia subtitle content and the subtitle interaction content; The method further includes translating the multimedia subtitle content and the subtitle interaction content from an initial language to a target language.

[0183] According to one or more embodiments of the present disclosure, in a multimedia interaction method provided by the present disclosure, the multimedia and the subtitle content are respectively displayed in different regions in the multimedia display interface.

[0184] According to one or more embodiments of the present disclosure, the present disclosure provides a method for manufacturing a semiconductor device comprising: receiving an interaction input trigger operation, the interaction input trigger operation triggering an interaction input to a multimedia and / or a subtitle content of the multimedia; acquiring interaction input data triggered based on the interaction input trigger operation; displaying the interaction input data in association with the multimedia and the subtitle content; The present invention provides a method for multimedia interaction, including

[0185] According to one or more embodiments of the present disclosure, the present disclosure provides a method for manufacturing a semiconductor device comprising: obtaining user interaction content with a selected character in the text content; displaying an interaction aggregation indicator in association with a text snippet in which the selected character is located, the text content including at least one text snippet, each text snippet including interaction content corresponding to one interaction aggregation indicator; The present invention provides methods of information interaction including:

[0186] According to one or more embodiments of the present disclosure, the present disclosure provides a method for manufacturing a semiconductor device comprising: A trigger module for receiving a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia display interface, the multimedia display interface displaying multimedia, subtitle contents of the multimedia, and the subtitle interaction aggregation indicator; an interaction content display module for displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation mark, the subtitle interaction content being an interaction content for the subtitle content; The company provides multimedia interaction devices including

[0187] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The multimedia display interface further includes a sign display module for, in response to obtaining subtitle interaction content input by a user for a target subtitle in the subtitle content before receiving a user's trigger operation on a subtitle interaction aggregation sign in the multimedia display interface, displaying a subtitle interaction aggregation sign corresponding to the subtitle interaction content in the multimedia display interface.

[0188] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the sign display module specifically comprises: The subtitle interaction aggregation indicator is displayed on the multimedia display interface in association with the subtitle snippet in which the target subtitle is located, and each subtitle snippet containing subtitle interaction content corresponds to one subtitle interaction aggregation indicator.

[0189] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the sign display module specifically comprises: When the same target subtitle exists in at least two consecutive subtitle snippets, corresponding subtitle interaction aggregation indicators are displayed in the at least two consecutive subtitle snippets, respectively.

[0190] According to one or more embodiments of the present disclosure, in a multimedia interaction device provided by the present disclosure, the subtitle interaction aggregation indicator includes a number of subtitle interaction contents for an associated subtitle snippet, and the subtitle interaction contents include comments and / or emoticons.

[0191] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the interaction content display module specifically includes: At least one subtitle interaction content of the subtitle snippet associated with the subtitle interaction aggregation indicator is displayed in a subtitle interaction window in the multimedia display interface.

[0192] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: a multimedia interaction content module for displaying at least one multimedia interaction content in the subtitle interaction window; The multimedia interaction content corresponds to a time period corresponding to a subtitle snippet associated with the subtitle interaction aggregation indicator, and the multimedia interaction content is interaction content with the multimedia obtained based on a playback time axis of the multimedia.

[0193] According to one or more embodiments of the present disclosure, in a multimedia interaction device provided by the present disclosure, the subtitle content includes at least two subtitle snippets, and the subtitle snippets are obtained by performing semantic segmentation on the subtitle content.

[0194] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The multimedia display interface further includes a trigger module for receiving a user's trigger operation on a subtitle interaction prompting indicator on the multimedia playback time axis of the multimedia display interface, and displaying subtitle interaction content corresponding to the subtitle interaction prompting indicator in a subtitle interaction window.

[0195] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, one of the subtitle interaction presentation indicators corresponds to at least one subtitle interaction content of a related subtitle, and the related subtitle is a subtitle at a related time point or related time period corresponding to the subtitle interaction presentation indicator.

[0196] According to one or more embodiments of the present disclosure, in a multimedia interaction device provided by the present disclosure, the subtitle interaction window is displayed in association with the subtitle interaction aggregation indicator.

[0197] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The multimedia display interface further includes a playback module for playing the multimedia according to a timestamp of a related subtitle corresponding to the subtitle interaction prompting mark after receiving a user's trigger operation on a subtitle interaction prompting mark on the playback time axis of the multimedia of the multimedia display interface.

[0198] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The multimedia display interface further includes a highlight display module that, after receiving a user's trigger operation on a subtitle interaction presentation indicator on the playback time axis of the multimedia of the multimedia display interface, jumps subtitle content of the multimedia to a position where a related subtitle corresponding to the subtitle interaction presentation indicator is located, and highlights and displays the related subtitle and the subtitle interaction content of the related subtitle.

[0199] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: Further comprising a subtitle interaction data display module for receiving a user's interaction display trigger operation for a corresponding position of a target time point on the playback time axis of the multimedia, and displaying an interaction card; The interaction card includes subtitle interaction data of the subtitle corresponding to the target time point, and the subtitle interaction data includes subtitle interaction content and / or statistical data of the subtitle interaction content.

[0200] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the interaction card further includes multimedia interaction data for the multimedia at the target time point, and the multimedia interaction data includes multimedia interaction content and / or statistical data of the multimedia interaction content.

[0201] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The display device further includes a subtitle deletion module that receives a user's delete operation for a first subtitle snippet in the subtitle content, and displays a subtitle interaction aggregation indicator corresponding to the first subtitle snippet in an area associated with a history position of the first subtitle snippet after deleting the first subtitle snippet.

[0202] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The display device further includes a subtitle splitting module that accepts a user's splitting operation for a second subtitle snippet in the subtitle content to split the second subtitle snippet into at least two sub-subtitle snippets, and splits subtitle interaction content of the second subtitle snippet based on the split sub-subtitle snippets.

[0203] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the subtitle segmentation module specifically comprises: The historical subtitle interaction aggregation indicator of the second subtitle snippet is deleted, and a new subtitle interaction aggregation indicator is displayed in association with the subtitle interaction content of the subtitle snippet.

[0204] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The multimedia display interface further includes an interaction display module for receiving a user's trigger operation for a selected subtitle in the subtitle content, and displaying a subtitle interaction content of the selected subtitle or a subtitle interaction content of a subtitle snippet in which the selected subtitle is located in a subtitle interaction window in the multimedia display interface.

[0205] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The display device further includes a subtitle interaction reply module for receiving a user's reply operation to the subtitle interaction content, displaying a reply editing interface, and acquiring the subtitle interaction reply content input by the user in the reply editing interface.

[0206] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the subtitle interaction reply module specifically includes: The subtitle interaction reply content is displayed in association with the subtitle interaction content.

[0207] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The display device further includes a search module that receives a user's search trigger operation for a search keyword, and highlights target keywords that are searched based on the search keyword in the subtitle content and the subtitle interaction content.

[0208] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the subtitle interaction content in which the target keyword exists is displayed in a subtitle interaction window.

[0209] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the search module specifically comprises: A search hit indication indicator is displayed on the multimedia playback timeline at a position corresponding to the timestamp of the target keyword.

[0210] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The search result display module further includes a search result display module that displays the total number of hit target keywords in a search display area, highlights a currently selected keyword and a corresponding search hit display indicator in the target keywords, and displays a sorted result of the currently selected keyword in the target keywords, The currently selected keyword and the search hit indicator are highlighted in a different manner than the target keyword.

[0211] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the search result display module specifically includes: receiving a jump operation including an upward jump and / or a downward jump from the user for the currently selected keyword; The currently selected keywords will be displayed after the jump.

[0212] According to one or more embodiments of the present disclosure, there is provided a multimedia interaction device, the device comprising: The video processing apparatus further includes a translation module for receiving a translation trigger operation from a user for the multimedia subtitle content and the subtitle interaction content, and translating the multimedia subtitle content and the subtitle interaction content from an initial language to a target language.

[0213] According to one or more embodiments of the present disclosure, in the multimedia interaction device provided by the present disclosure, the multimedia and the subtitle content are respectively displayed in different regions in the multimedia display interface.

[0214] According to one or more embodiments of the present disclosure, the present disclosure provides a method for producing a liquid crystal display comprising: an interaction input trigger module that receives an interaction input trigger operation, the interaction input trigger operation triggering an interaction input to a multimedia and / or an interaction input to a subtitle content of the multimedia; an interaction data acquisition module for acquiring interaction input data triggered based on the interaction input trigger operation; an interaction data display module that displays the interaction input data in association with the multimedia and the subtitle content; The company provides multimedia interaction devices including

[0215] According to one or more embodiments of the present disclosure, the present disclosure provides a method for producing a liquid crystal display comprising: an interaction acquisition module for acquiring user interaction content with respect to a selected character in the text content; a sign display module for displaying an interaction aggregation sign in association with a text snippet in which the selected character is located, the text content including at least one text snippet, each text snippet including an interaction content corresponding to one interaction aggregation sign; The present invention provides an interaction device including:

[0216] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, the electronic device comprising: A processor; a memory storing instructions executable by said processor; Including, The processor reads the executable instructions from the memory and executes the instructions to perform any one of the multimedia interaction methods and / or information interaction methods provided by the present disclosure.

[0217] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored therein, the computer program performing any one of the multimedia interaction methods and / or information interaction methods provided by the present disclosure.

[0218] The above is only a description of the preferred embodiments of the present disclosure and the technical principles used. As can be understood by those skilled in the art, the disclosure scope of the present disclosure is not limited to the technical solution composed of a specific combination of the above technical features, and also covers other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, a technical solution formed by replacing the above features with technical features having similar functions disclosed in the present disclosure (but not limited to these).

[0219] Also, although operations have been described in a particular sequence, there is no requirement that these operations be performed in the particular sequence or order shown. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, the above discussion includes some specific implementation details, but these should not be construed as limitations on the scope of the present disclosure. Some features described in separate embodiments may be implemented in combination in a single embodiment. Also, various features described in a single embodiment may be implemented in multiple embodiments, either alone or in any suitable subcombination.

[0220] Although the present subject matter has been described in specific language of structural features and / or methodological operations, the subject matter defined in the appended claims is not limited to the specific features or operations described above, and the specific features and operations described above are merely example forms of implementing the claims.

Claims

1. 1. A method of multimedia interaction performed by an electronic device, comprising: receiving a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia exhibition interface, where the multimedia exhibition interface displays multimedia, subtitle contents of the multimedia, and the subtitle interaction aggregation indicator; displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation indicator, the subtitle interaction content being an interaction content for the subtitle content; A method comprising:

2. before receiving a user's trigger operation on a subtitle interaction aggregation indicator in the multimedia exhibition interface; In response to obtaining a subtitle interaction content input by a user for a target subtitle in the subtitle content, displaying a subtitle interaction aggregation indicator corresponding to the subtitle interaction content on the multimedia display interface; The method of claim 1 further comprising:

3. The step of displaying the subtitle interaction aggregation indicator on the multimedia display interface includes: displaying the subtitle interaction aggregation indicator in the multimedia display interface in association with the subtitle snippet in which the target subtitle is located, where each subtitle snippet containing subtitle interaction content corresponds to one subtitle interaction aggregation indicator; The method of claim 2, comprising:

4. if the same target subtitle exists in at least two consecutive subtitle snippets, respectively displaying corresponding subtitle interaction aggregation indicators for the at least two consecutive subtitle snippets; The method of claim 3 further comprising:

5. 4. The method of claim 3, wherein the subtitle interaction aggregation indicator includes a number of subtitle interaction contents for an associated subtitle snippet, the subtitle interaction contents including comments and / or emoticons.

6. The step of displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation mark includes:

4. The method of claim 3, further comprising displaying at least one subtitle interaction content of a subtitle snippet associated with the subtitle interaction aggregation indicator in a subtitle interaction window in the multimedia exhibition interface.

7. 2. The method of claim 1, further comprising: displaying at least one multimedia interaction content in a subtitle interaction window, the multimedia interaction content corresponding to a time period corresponding to a subtitle snippet associated with the subtitle interaction aggregation indicator.

8. The method according to claim 7, wherein the multimedia interaction content is an interaction content for the multimedia obtained based on a playback time axis of the multimedia.

9. The multimedia interaction content includes: determining, during generation of the multimedia, in response to receiving interaction content, time information at which the interaction content was received; 9. The method according to claim 7 or 8, characterized in that the multimedia interaction content is generated based on the interaction content and the time information.

10. 9. The method according to claim 7 or 8, characterized in that the multimedia interaction content is displayed in relation to a playback timeline of the multimedia.

11. The method of claim 1 , wherein the subtitle content includes at least two subtitle snippets, the subtitle snippets being obtained by performing semantic segmentation on the subtitle content.

12. The method according to claim 2, further comprising the step of highlighting and displaying the target subtitles in a set manner in the multimedia display interface.

13. 13. The method of claim 12, wherein the setting scheme includes at least one of adding background color, making the text bold, and adding an underline.

14. receiving a user's trigger operation on a subtitle interaction presentation indicator on the multimedia playback timeline of the multimedia exhibition interface; The method of claim 1 , further comprising: displaying subtitle interaction content corresponding to the subtitle interaction indication mark in a subtitle interaction window.

15. The method according to claim 11, characterized in that one subtitle interaction presentation indicator corresponds to at least one subtitle interaction content of a related subtitle, and the related subtitle is a subtitle at a related time point or a related time period corresponding to the subtitle interaction presentation indicator.

16. 15. The method of claim 6, 7 or 14, wherein the subtitle interaction window is displayed in association with the subtitle interaction aggregation indicator.

17. After receiving a user's trigger operation on a subtitle interaction presentation mark on the multimedia playback time axis of the multimedia exhibition interface, 16. The method of claim 15, further comprising playing the multimedia based on timestamps of associated subtitles corresponding to the subtitle interaction presentation indicia.

18. After receiving a user's trigger operation on a subtitle interaction presentation mark on the multimedia playback time axis of the multimedia exhibition interface, The method of claim 15, further comprising: jumping subtitle content of the multimedia to a position where a related subtitle corresponding to the subtitle interaction indication sign is located, and highlighting and displaying the related subtitle and the subtitle interaction content of the related subtitle.

19. receiving a user's trigger operation on a multimedia interaction presentation indicator on the multimedia playback timeline of the multimedia exhibition interface; displaying the multimedia interaction content corresponding to the multimedia interaction presentation sign on an interaction card associated with a playback time axis; The method of claim 1 further comprising:

20. receiving a user's interaction display trigger operation for a corresponding position of the target time point on the multimedia playback time axis; displaying an interaction card, the interaction card including subtitle interaction data of the subtitle corresponding to the target time point, the subtitle interaction data including subtitle interaction content and / or statistical data of the subtitle interaction content; The method of claim 1 further comprising:

21. 21. The method of claim 20, wherein the interaction card further comprises multimedia interaction data for the multimedia at the target time point, the multimedia interaction data comprising multimedia interaction content and / or statistical data of the multimedia interaction content.

22. receiving a delete operation by a user for a first subtitle snippet in the subtitle content; after deleting the first subtitle snippet, displaying a subtitle interaction aggregation indicator corresponding to the first subtitle snippet in an area associated with a historical position of the first subtitle snippet; The method of claim 1 further comprising:

23. receiving a user's split operation on a second subtitle snippet in the subtitle content, the split operation being used to split the second subtitle snippet into at least two sub-subtitle snippets; Dividing a subtitle interaction content of the second subtitle snippet based on the divided sub-subtitle snippet; The method of claim 1 further comprising:

24. deleting a historical subtitle interaction aggregation indicator of the second subtitle snippet and displaying a new subtitle interaction aggregation indicator associated with the subtitle snippet having subtitle interaction content; 24. The method of claim 23, further comprising:

25. receiving a user's trigger operation for a selected subtitle in the subtitle content, and displaying the subtitle interaction content of the selected subtitle or the subtitle interaction content of a subtitle snippet in which the selected subtitle is located in a subtitle interaction window in the multimedia display interface, the selected subtitle being a subtitle having subtitle interaction content; The method of claim 1 further comprising:

26. receiving a response operation from a user to the subtitle interaction content; The method of claim 1 , further comprising: displaying a reply editing interface; and receiving subtitle interaction reply content input by a user in the reply editing interface.

27. displaying the subtitle interaction reply content in association with the subtitle interaction content; 27. The method of claim 26, further comprising:

28. receiving a user's search trigger operation for a search keyword; highlighting target keywords in the subtitle content and the subtitle interaction content that are searched based on the search keywords; The method of claim 1 further comprising:

29. The method of claim 28, wherein the subtitle interaction content in which the target keyword exists is displayed in a subtitle interaction window.

30. displaying a search hit indication mark at a position corresponding to a timestamp of the target keyword on the multimedia playback time axis; 30. The method of claim 28, further comprising:

31. displaying a total number of hits for the target keywords in a search display area; highlighting a currently selected keyword and a corresponding search hit display indicator in the target keyword and displaying a sorted result of the currently selected keyword in the target keyword, wherein a manner of highlighting the currently selected keyword and the search hit display indicator is different from a manner of highlighting the target keyword; 30. The method of claim 28, further comprising:

32. receiving a user's jump operation for the currently selected keyword, the jump operation including an upward jump and / or a downward jump; After the jump, a step to display the currently selected keyword, 32. The method of claim 31 further comprising:

33. receiving a translation trigger operation by a user for the multimedia subtitle content and the subtitle interaction content; translating the multimedia subtitle content and the subtitle interaction content from an initial language to a target language; The method of claim 1 further comprising:

34. 10. The method of claim 1, wherein the multimedia and the subtitle content are each exhibited in different areas of the multimedia exhibit interface.

35. A multimedia interaction device comprising: A trigger module for receiving a user's trigger operation on a subtitle interaction aggregation indicator in a multimedia exhibition interface, where the multimedia exhibition interface displays multimedia, subtitle contents of the multimedia, and the subtitle interaction aggregation indicator; and an interaction content display module for displaying at least one subtitle interaction content corresponding to the subtitle interaction aggregation indicator, the subtitle interaction content being an interaction content for the subtitle content.

36. An electronic device, A processor; a memory storing instructions executable by said processor; Including, 35. An electronic device, characterized in that the processor performs the multimedia interaction method according to any one of claims 1 to 34 by reading the executable instructions from the memory and executing the instructions.

37. 1. A computer-readable storage medium, comprising: A computer readable storage medium having stored thereon a computer program, the computer program executing the multimedia interaction method according to any one of claims 1 to 34.

Images