Method, apparatus, device, medium and program product for interacting with media item

Abstract

The invention provides a method, a device, equipment, a medium and a program product for interacting with a media item. The method provided by the invention comprises the following steps: responding to received voice data, displaying prompt information on a display page of a media item, the prompt information being used for indicating that the voice data is being received; displaying an operation instruction on the display page, wherein the operation instruction is a predetermined operation instruction and is matched with the voice data; and displaying an execution result of the operation instruction on the display page. In this way, interaction can be executed on the media items based on the voice data of the user, manual operation of the user is not needed, flexible control over the media items can be achieved, and convenience of interaction execution by the user can be improved. The operation instruction displayed on the display page of the media item can help the user to conveniently and quickly know the operation instruction corresponding to the current voice data, and the user can conveniently know the interaction state.

Description

Technical Field

[0001] The examples in this article generally relate to the field of computers, and in particular to methods, apparatuses, electronic devices, computer-readable storage media, and computer program products for interacting with media items. Background Technology

[0002] More and more applications are now designed to provide users with various services. For example, users can publish, browse, and view media items within an application. Media items can also be called media content, media content items, or content items. Media items can include various types of content such as videos, images, image sets, text, and audio. Users can interact with media items on the display page used to show them; for example, users can perform various interactions such as liking, fast-forwarding, and toggling on the media items. Summary of the Invention

[0003] In a first aspect, a method for interacting with a media item is provided. The method includes: in response to receiving voice data, displaying a prompt message on a display page of the media item, the prompt message indicating that voice data is being received; displaying an operation instruction on the display page, the operation instruction being a predetermined operation instruction and matching the voice data; and displaying the execution result of the operation instruction on the display page.

[0004] In a second aspect, an apparatus for interacting with a media item is provided. The apparatus includes: an information display module configured to display a prompt message on a display page of the media item in response to receiving voice data, the prompt message indicating that voice data is being received; an instruction display module configured to display an operation instruction on the display page, the operation instruction being a predetermined operation instruction and matching the voice data; and a result display module configured to display the execution result of the operation instruction on the display page.

[0005] In a third aspect, an electronic device is provided. The electronic device includes: at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect, a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0008] In this way, users can interact with media items based on their voice data without having to manually operate them. This allows for flexible interaction with media items and improves the convenience of user interaction.

[0009] It should be understood that the content described in this section is not intended to limit the key or important features of the examples in this article, nor is it intended to restrict the scope of the solution. Other features will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the various examples herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic diagram of the example environment is shown; Figures 2A to 2I Example pages show some scenarios; Figure 3 The flowcharts show example methods for interacting with media items in several scenarios; Figure 4 Schematic block diagrams of example devices for interacting with media items in several scenarios are shown; and Figure 5 A block diagram of an electronic device capable of implementing multiple illustrative scenarios is shown. Detailed Implementation

[0011] The examples in the text will now be described in more detail with reference to the accompanying drawings. While some examples are shown in the drawings, it should be understood that solutions can be implemented in various forms and should not be construed as limited to the examples presented herein. Rather, these examples are provided to provide a more thorough and complete understanding of the solutions. It should be understood that the drawings and examples in this document are for illustrative purposes only and are not intended to limit the scope of protection of the solutions.

[0012] It should be noted that the headings of any section / subsection provided herein are not restrictive. Various examples are described throughout this document, and examples of any type may be included under any section / subsection. Furthermore, examples described in any section / subsection may be combined in any way with any other examples described in the same section / subsection and / or different sections / subsections.

[0013] In the description of the examples in this document, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an example" or "the example" should be understood as "at least one example". The term "some examples" should be understood as "at least some examples". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. The term "trigger" refers to one or more interactive actions by a user on a terminal device. Further, these interactive actions may be triggered within the same user interface / pop-up window or within different user interfaces / pop-up windows. There is no limitation in this regard. Other explicit and implicit definitions may also be included below.

[0014] It should be noted that, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0015] The examples in this article may involve user data, data acquisition, and / or use. All of these aspects comply with relevant laws, regulations, and rules. In the examples presented here, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, when implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means, in accordance with relevant laws and regulations. The specific methods of notification and / or authorization can vary depending on the actual situation and application scenario; the scope of the solution is not limited in this regard.

[0016] In this manual and the sample solutions, any processing of personal information will be conducted only under legal grounds (such as obtaining the consent of the data subject or being necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0017] As mentioned above, on the display page of the application used to show media items, users can interact with these items. For example, users can perform various interactions such as liking, fast-forwarding, and switching between media items. The goal is to achieve flexible control over media items, such as hands-free playback. Hands-free playback refers to users controlling the application's playback through contactless interaction methods in specific scenarios (such as cooking, exercising, or doing housework) when their hands are occupied or inconvenient to directly touch the screen. These interaction methods mainly include voice commands, camera visual recognition (such as gesture recognition and head posture recognition), and linkage with external devices (such as in-vehicle systems, smart speakers, and Bluetooth remote controls). Its core objective is to "free up hands," maintaining an immersive viewing or listening experience while ensuring ease of operation and safety.

[0018] An improved scheme for interacting with media items is proposed. The scheme includes: in response to receiving voice data, displaying a prompt message on the media item's display page, the prompt message indicating that voice data is being received; displaying an operation command on the display page, the operation command being a predefined operation command matched with the voice data; and displaying the execution result of the operation command on the display page.

[0019] In this way, interaction with media items can be performed based on the user's voice data, eliminating the need for manual operation and enabling hands-free control of media items, thus improving the convenience of user interaction. Displaying operation instructions on the media item's display page also helps users quickly and easily understand the operation instructions corresponding to the current voice data, making it easier for them to understand the interaction status.

[0020] The following describes various examples of this scheme in further detail with reference to the accompanying drawings.

[0021] Example Environment Figure 1 A schematic diagram of example environment 100 is shown. (e.g.) Figure 1 As shown, example environment 100 may include terminal device 110.

[0022] In this example environment 100, terminal device 110 may run an application 120 that supports interaction with media items. Application 120 may be any suitable type of application for interacting with media items, including but not limited to: audio / video playback applications, content sharing applications, social applications, audio / video editing applications, etc. User 140 may interact with application 120 via terminal device 110 and / or its attached devices.

[0023] exist Figure 1In environment 100, if application 120 is active, terminal device 110 can use application 120 to display page 150 for supporting interaction with media items. Page 150 may include various types of pages provided by application 120, such as content display pages, content creation pages, content publishing pages, etc.

[0024] In some cases, terminal device 110 communicates with server 130 to provide services to application 120. Terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some cases, terminal device 110 may also support any type of user-facing interface (such as "wearable" circuitry).

[0025] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide backend services for applications 120 in terminal devices 110 that support interaction with media items.

[0026] A communication connection can be established between server 130 and terminal device 110. This communication connection can be established via wired or wireless means. The communication connection can include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections. In some cases, server 130 and terminal device 110 can exchange signaling information through their communication connection.

[0027] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the scheme.

[0028] The following description of the example will continue with reference to the accompanying drawings.

[0029] Example Interaction Figures 2A to 2I Example pages 200A to 200I illustrate some scenarios. Example pages 200A to 200I can be, for example, provided by... Figure 1 The terminal device 110 shown is provided. It should be understood that the various pages shown in the figures are merely examples, and various page designs may exist in practice. The various graphic elements in the page may have different arrangements and different visual representations, one or more of which may be omitted or replaced, and one or more other elements may also be present. It should be understood that the actions described relative to terminal device 110 may also be performed by an application on terminal device 110 (e.g., application 120), or may be performed by the application in conjunction with its server (e.g., server 130).

[0030] As mentioned above, when application 120 is active, terminal device 110 can use application 120 to display page 150 for supporting interaction with media items. Figure 2A The example page 200A shown illustrates an example of a display page. The display page's control modes include voice mode and non-voice mode. In some cases, when the display page's control mode is voice mode, the terminal device 110 can receive voice data and perform operations on the media items displayed on the display page based on the voice data.

[0031] In some cases, terminal device 110 may display a settings page in response to receiving an interactive operation on the display page. This settings page may include switching controls for switching control modes. For example... Figure 2A and Figure 2B As shown, terminal device 110 can display example page 200B in response to receiving an interactive operation on the display page shown in example page 200A. For example, a control in the upper left corner of example page 200A can be clicked to display a settings page. Example page 200B includes an operation panel 210, which includes multiple operation controls, including at least a switching control for switching control modes. These multiple operation controls may also include operation controls with other functions (such as the first to fifth settings controls shown in the figure).

[0032] Terminal device 110 can switch control modes in response to receiving an interactive operation on the switching control. For example, if the current control mode of the displayed page is voice mode, terminal device 110 can switch the control mode of the displayed page from voice mode to non-voice mode in response to receiving an interactive operation on the switching control. If the current control mode of the displayed page is non-voice mode, terminal device 110 can switch the control mode of the displayed page from non-voice mode to voice mode in response to receiving an interactive operation on the switching control. In this way, the control mode of the displayed page can be switched via the switching control, which improves the convenience of switching control modes.

[0033] In some situations, when the interactive operation instruction switches from non-voice mode to voice mode, the terminal device 110 may display at least one setting control, which is used to set at least one operation command. In other words, in voice mode, the operation command displayed on the screen is set via the setting control in at least one setting control. Figures 2A to 2C As shown, when the control mode of the display page shown in Example Page 200A is in non-voice mode, the terminal device 110 can respond to receiving an interactive operation on the display page shown in Example Page 200A and present Example Page 200B. The operation panel 210 of Example Page 200B includes a switching control for switching control modes. The terminal device 110 can respond to receiving a trigger operation on the switching control in Example Page 200B and present Example Page 200C. Example Page 200C includes at least one operation control (e.g., operation controls 223, 224, and 225 shown in the figure). These operation controls can be set with corresponding operation instructions. For example, operation control 223 can be set to the instruction "swipe up / swipe down one video," operation control 224 can be set to the instruction "pause / play," and so on. In some cases, the user can click on the operation controls to set the corresponding operation instructions.

[0034] Here, the operation control involves several attributes: function, keyword, and display text. Specifically, the function represents the function that the operation instruction performs on the display page, such as swiping up / down a video, etc.; the keyword can represent the keyword used to invoke the operation instruction, such as "Hello, swipe up", etc.; the display text can represent the text that will be displayed on the display page. Considering the limitations of screen size, the display text can usually include less text to avoid interfering with viewing the normally displayed content. It should be understood that although the operation control in the example page 200C involves multiple attributes (e.g., function, keyword, and display text), alternatively and / or additionally, the operation control can involve more or fewer attributes. For example, it can allow the user to set only the function; or it can allow the user to set the function and the corresponding keyword, etc. It should be understood that the keywords shown in the attached figures are merely exemplary, and other keywords can be set. For example, the keywords can only include core information related to the function, such as "swipe up / down". Or, for example, the keywords can further include words used to initiate voice interaction, such as "Hello", "Listen to me", etc.

[0035] Alternatively and / or additionally, a settings control 222 may be provided for this at least one operation instruction. In some cases, the terminal device 110 may display a setting control 222 in response to a triggering operation of an example page 200C. Figure 2D The example page 200D is shown. Example page 200D may include, for example, a settings control 226 for setting the display mode of an operation command. Example page 200D may display the currently set display mode of the operation command; for example, example page 200D may include the text "Current display mode: Display command horizontally at the top". Terminal device 110 may, for example, respond to a trigger operation on the settings control 226 to display a list of display modes, which may include multiple display modes (specifically, display position, display font, display font size, etc. may be specified). Terminal device 110 may then, in response to receiving a selection of a display mode, set the selected display mode as the display mode of the operation command.

[0036] Example page 200D may also include at least one operation instruction and at least one setting control as shown in box 227. Each of the multiple setting controls shown in box 227 can be used to set the corresponding operation instruction. This at least one operation instruction may include a default operation instruction or an operation instruction previously set by the user. Specifically, each setting control shown in box 227 can be used to set the text of the corresponding operation instruction. For each of the at least one operation instruction, example page 200D may also display at least one of the following: the function, keywords, display text, etc., of the operation instruction. Terminal device 110, for example, may respond to a trigger operation of the setting control corresponding to an operation instruction, receive the user's editing of the function, keywords, display text, etc., of the operation instruction, and set the operation instruction based on the editing result.

[0037] The functions of operation commands can include switching, fast forwarding, rewinding, pausing, canceling pause, volume adjustment, liking, forwarding, favorited, commenting, etc. Keywords for operation commands can be used to match voice data. For example, the terminal device 110 can determine the operation command that matches the voice data from at least one operation command based on the matching results between various operation commands and voice data. The displayed text of the operation command is the text displayed on the display page when the corresponding operation command is invoked. In this way, operation commands can achieve multiple functions, enriching the diversity of user interaction in voice mode and improving user interaction efficiency. The functions and keywords of operation commands can be set via a setting control, reducing the difficulty of setting operation commands and improving the convenience of setting operation commands.

[0038] For example, for an operation instruction in the diagram with the function "swipe up / swipe down one video", terminal device 110 can respond to the trigger operation of the setting control for the operation instruction and receive the user's editing operation for the operation instruction. If the user's editing operation instructs the display text of the operation instruction to be adjusted to "swipe up / swipe down one video", then terminal device 110 can set the operation instruction based on the editing operation. The set operation instruction can be displayed in the example page 200D as "Function: Swipe up / swipe down one video Keywords: 'Hello, swipe up / swipe down' Display text: Swipe up / swipe down one video".

[0039] It should be noted that example page 200D may also include more operation controls. For example, example page 200D may also include a control for adding an operation command, a control for deleting an operation command, etc. It should also be noted that, due to the screen size limitation of terminal device 110, when at least one operation command includes multiple operation commands, terminal device 110 may only display a portion of the multiple operation commands in example pages 200C and 200D. Terminal device 110 may, in response to receiving a swipe operation (e.g., at least an up swipe and a down swipe) on example pages 200C and 200D, present the remaining portions of the multiple operation commands. It should also be noted that at least one operation command shown on example pages 200C and 200D may be the same, and once this at least one operation command is successfully set, it can be determined as at least one predetermined operation command. In some cases, the status of each operation command can be displayed. For example, at least one operation command may all be in an active state, or at least one operation command may be partially active and partially inactive. In this case, the user can only invoke the operation command in the active state using voice data.

[0040] In some cases, terminal device 110 may, in response to receiving a trigger operation on the switching control in example page 200B, directly determine that the user wants to switch the control mode of the display page from non-voice mode to voice mode, and switch to example page 200C, which presents at least one operation instruction and a setting control 222 for that operation instruction. In other cases, to avoid accidental touches, terminal device 110 may, in response to receiving a trigger operation on the switching control in example page 200B, present example page 200C. In this case, example page 200C may also include a switching control 221 for switching control modes. Terminal device 110 may, in response to receiving a trigger operation on the switching control 221, determine that the user wants to switch the control mode of the display page from non-voice mode to voice mode.

[0041] In some situations, when switching from non-voice mode to voice mode, it can be determined whether access to the device's audio capture device (e.g., microphone) is permitted. If microphone access is not permitted, a prompt can be provided to guide the user to grant microphone access.

[0042] In some cases, terminal device 110 can provide mode information that indicates the current control mode of the display page. Terminal device 110 can provide mode information, for example, via the display page. In this case, the mode information can include any appropriate information such as text or icons. For example, when the control mode of the display page switches from non-voice mode to voice mode, terminal device 110 can display the text "Voice mode enabled" on the display page. Alternatively and / or additionally, a "microphone" icon 240 can be displayed at a predetermined location on the display page to indicate voice mode. Alternatively and / or additionally, other icons can be displayed, such as an icon representing sound wave bands, etc.

[0043] In some situations, to minimize interference with normally displayed content, the mode information can be displayed within a display box that presents various information related to voice interaction. For example, it can be displayed within a display box that presents various information related to voice interaction. Figure 2E The prompt message 231 is displayed within the display box, such as... Figure 2F The mode information is displayed within the display box of the display text 232, or at other locations. In some cases, the terminal device 110 may also provide mode information based on its own indicator. The indicator may, for example, include the terminal device's notification light. The terminal device 110 can provide mode information by controlling the on / off state of the notification light. For example, the terminal device 110 can keep the notification light constantly on to indicate that the current control mode of the display page is voice mode.

[0044] In some cases, when the control mode of the display page is voice mode, the terminal device 110 may display a prompt message on the media item display page in response to receiving voice data. This prompt message indicates that voice data is being received. The prompt message may include any appropriate information such as text, icons, or symbols. (See reference) Figure 2E Terminal device 110 can display prompt information 231 on example page 200E, which indicates that terminal device 110 is currently receiving voice data. The prompt information can be displayed dynamically, such as the waveform of the voice data, etc.

[0045] After receiving voice data, terminal device 110 can display operation instructions on the display page. The operation instructions can be one of the predetermined operation instructions (such as at least one predetermined operation instruction shown on example page 200D above). Terminal device 110 can determine the degree of matching between the voice data and at least one predetermined operation instruction, and based on the degree of matching corresponding to each of the at least one predetermined operation instruction, determine the predetermined operation instruction that matches the voice data from the at least one predetermined operation instruction.

[0046] In some cases, terminal device 110 can determine the degree of matching between at least one predetermined operation instruction and the voice data based on the degree of matching between the keywords of each predetermined operation instruction and the keywords of the voice data. In some cases, terminal device 110 can determine the degree of matching between at least one predetermined operation instruction and the voice data based on the degree of matching between the semantics of each predetermined operation instruction (e.g., the semantics of a function) and the semantics of the voice data. In this way, the degree of matching can be determined based on at least one of keywords and semantics, improving the convenience and accuracy of determining the degree of matching. In this way, the operation instruction matching the voice data can be determined from at least one operation instruction based on the degree of matching between each operation instruction and the voice data, improving the accuracy of the determined operation instruction.

[0047] In some cases, multilingual functionality can be provided. For example, although the above example shows operation commands specified in Chinese, it can also receive voice data expressed in other languages ​​(e.g., English, Japanese, etc.) and allow users to control the display page in other languages. In some cases, the settings page can further support users specifying their own voice tone, and the operation command will only be invoked when the received voice data matches that voice tone. This helps prevent accidental operation by other users.

[0048] In some cases, terminal device 110 may determine a predetermined operation instruction as matching the voice data in response to a threshold degree of matching between one of the at least one predetermined operation instructions and the voice data. In other cases, terminal device 110 may determine the predetermined operation instruction with the highest matching degree among the at least one predetermined operation instructions as matching the voice data. A predetermined operation instruction matching the voice data is also an operation instruction matching the voice data.

[0049] In some cases, terminal device 110 may display the operation instruction to indicate the corresponding function. In other cases, terminal device 110 may, for example, display the text corresponding to the operation instruction. Figure 2F As shown, terminal device 110 can respond to determining that an operation instruction in at least one predetermined operation instruction, which functions to swipe up / down on a video, matches the voice data, and display the operation instruction on the display page shown in example page 200F. Specifically, terminal device 110 can display the display text 232 corresponding to the operation instruction (e.g., the text "swipe down") on the display page.

[0050] See Figure 2E , 2F and Figure 2IThis describes the interaction process in some scenarios. In voice mode, the user can express the desired action by voice, for example, the user can say "Please help me scroll down one video." Terminal device 110 can receive this voice data, perform speech recognition on the received voice data, and obtain the text "Please help me scroll down one video." During the receiving process, such as... Figure 2E As shown, a prompt message 231 can be displayed on example page 200E, indicating that voice data is being received.

[0051] Terminal device 110 can determine an operation instruction matching the text from at least one pre-set operation instruction. For example, the operation instruction can be determined via keyword matching. Assuming the keyword is "swipe down" and the text includes the keyword "swipe down", the corresponding operation instruction can be determined as "swipe down to the next video". Alternatively, the corresponding operation instruction can be determined via semantic similarity. In this way, a mapping relationship can be established between voice data and operation instructions, thereby supporting voice-based operation of the display page.

[0052] Operation commands can be displayed on the display page, along with various information about them. For example, the function or keywords of the operation command can be displayed directly. However, due to the screen size limitations of the terminal device 110, it may be difficult to display all content, such as... Figure 2E As shown, the user-defined text "Slide down" can be displayed. Terminal device 110 can execute the operation command "Slide down to the next video," at which point the page content will switch from the currently playing video to the next video. Figure 2I As shown, terminal device 110 can display the execution result of the operation command (i.e., the next video) on the display page. It should be understood that, although... Figure 2I Only the final execution result is shown; alternatively and / or additionally, the dynamic process of the slide may be shown.

[0053] The terminal device 110 can then display the execution results of the operation commands on the display page. For example, if the voice data is "Please help me swipe down one video," and the corresponding operation command is to swipe up / swipe down one video, then the terminal device 110 can switch the currently displayed media item on the display page. The media item displayed after the switch is the media item displayed after the one displayed before the switch. As another example, if the voice data is "Please help me mute," and the corresponding operation command is to mute / unmute, then the terminal device 110 can switch to playing the media item in mute mode. In this way, interaction with media items can be performed based on the user's voice data, eliminating the need for manual operation by the user. This enables hands-free control of media items, improving the convenience of user interaction and user experience. Displaying the operation commands on the media item's display page also helps users quickly and easily understand the operation commands corresponding to the current voice data, further enhancing the user experience.

[0054] In some scenarios, voice interaction can be supported in a multi-level page environment. For example, in response to a confirmation instruction of "comment," a comment page can be further displayed on the main page. Prompts can be provided to the user via voice or text, such as "Please enter your comment." The system can receive the comment content entered by the user via voice or text and ask the user whether to submit. Upon receiving a "submit" request, the comment can be submitted.

[0055] In some situations, if the voice data is too noisy or incomplete, the terminal device 110 may be unable to determine an operation instruction matching the voice data from at least one predetermined operation instruction. In some cases, if the terminal device 110 cannot determine an operation instruction matching the voice data from at least one predetermined operation instruction, the terminal device 110 may display a notification message on the display page to indicate that no operation instruction matching the voice data exists. (Reference) Figure 2G Terminal device 110 may display notification information 233 on the display page shown in example page 200G in response to determining that no operation instruction matching the voice data exists among at least one predetermined operation instruction. Notification information 233 may include any appropriate information such as text, icons, and symbols. As an example only, notification information 233 may include the text "No valid instruction recognized". Alternatively and / or additionally, notification information may be provided via voice.

[0056] In some cases, terminal device 110 may also determine that the exit condition for voice mode has been met in response to a received interactive operation instruction to exit voice mode, or in response to no voice data received within a second predetermined period. The second predetermined period can be any suitable duration; for example, it could be 15 minutes, half an hour, etc. Terminal device 110 may display exit information in response to the voice mode exit condition being met, indicating that the control mode on the displayed page has exited voice mode. For example, terminal device 110 may determine that an interactive operation instructing to exit voice mode has been received in response to receiving voice data "Exit voice mode". Terminal device 110 may also present a settings page including a switching control in response to receiving an interactive operation on the displayed page. Terminal device 110 may determine that a received interactive operation instruction to exit voice mode has been received in response to receiving a trigger operation on the switching control. (Reference) Figure 2H Terminal device 110 may display exit information 234 on the display page shown in example page 200H in response to determining that the exit condition of the voice mode has been met. Exit information 234 may include any appropriate information such as text, icons, and symbols. For example only, exit information 234 may include the text "Voice control exited".

[0057] In some situations, to avoid prolonged video obstruction caused by displaying operation instructions, notification messages, or exit messages for extended periods, the terminal device 110 may also cancel the display of operation instructions, notification messages, or exit messages after the display duration of these messages has reached a first predetermined duration. The first predetermined duration can be any appropriate length; for example, it could be 10 seconds, 15 seconds, etc. This is just an example. Figure 2F As shown in the example page 200F, the terminal device 110 can cancel the display of the display text 232 in response to the operation command reaching a first predetermined duration.

[0058] Example process Figure 3 A flowchart of an example method 300 for interacting with media items is shown, based on several scenarios. Method 300 can be implemented at terminal device 110. See below for reference. Figure 1 Let's describe method 300.

[0059] In box 310, in response to receiving voice data, terminal device 110 displays a prompt message on the media item display page, indicating that voice data is being received.

[0060] In frame 320, terminal device 110 displays operation instructions on a display page. The operation instructions are predefined operation instructions and are matched with voice data.

[0061] In frame 320, terminal device 110 displays the execution result of the operation command on the display page.

[0062] This method allows for hands-free control of media items based on the user's voice data, eliminating the need for manual operation and improving the convenience and user experience. Displaying operation commands on the media item's screen also helps users quickly and easily understand the corresponding commands for the current voice data, further enhancing the user experience.

[0063] In some cases, the control modes of the display page include a voice mode and a non-voice mode, the voice data is received in the voice mode, and method 300 further includes: displaying a settings page in response to receiving an interactive operation on the display page, the settings page including a toggle control for switching control modes; and switching control modes in response to receiving an interactive operation on the toggle control.

[0064] In this way, the control mode of the display page can be switched by switching controls, which can improve the convenience of switching control modes.

[0065] In some cases, method 300 further includes: in response to an interactive operation indication switching from a non-voice mode to a voice mode, displaying at least one setting control, the at least one setting control being used to set at least one operation instruction, and the operation instruction being set via the setting control in the at least one setting control.

[0066] In this way, users can personalize at least one operation command, which can improve the user's interactive experience.

[0067] In some cases, method 300 further includes: in response to receiving an interactive operation on the setting control, setting an operation instruction.

[0068] In this way, operation commands can be configured via settings controls, which improves the ease of setting operation commands.

[0069] In some cases, setting controls are used to set at least one of the following: the function of the operation command; or the keyword of the operation command.

[0070] In this way, the functions and keywords of operation commands can be set through the setting control, which can reduce the difficulty of setting operation commands and improve the convenience of setting operation commands.

[0071] In some cases, the functions include at least one of the following: toggle, fast forward, rewind, pause, cancel pause, volume adjustment, like, share, favorite, or comment.

[0072] In this way, operation commands can achieve multiple functions, enrich the diversity of user interaction in voice mode, and improve the user's interactive experience.

[0073] In some cases, displaying operation instructions on the display page includes: displaying operation instructions in response to voice data matching keywords.

[0074] In this way, operation instructions that match keywords with voice data can be displayed, which can improve the accuracy of the displayed operation instructions.

[0075] In some cases, method 300 further includes: canceling the display of the operation instruction in response to the display duration of the operation instruction reaching a first predetermined duration.

[0076] This method avoids prolonged obscuring of media items caused by displaying operation commands for extended periods, thus improving the user experience when browsing media items.

[0077] In some cases, method 300 further includes: displaying a notification message in response to a mismatch between voice data and at least one operation command.

[0078] In this way, notifications can be displayed to inform users that no matching voice command exists, allowing them to provide the command again and improving the user experience.

[0079] In some cases, method 300 further includes: displaying exit information in response to an exit condition being met, the exit information indicating that the voice mode has been exited, the exit condition including at least one of the following: no voice data is received within a second predetermined duration, or an interactive operation indicating exiting the voice mode is received.

[0080] In this way, users can be notified that they have exited voice mode by displaying an exit message, making it convenient for them to understand the change in control mode in a timely manner.

[0081] In some cases, method 300 further includes providing mode information that indicates a control mode.

[0082] In this way, the current control mode can be indicated by providing mode information, which can improve the user experience.

[0083] In some cases, method 300 is performed by the terminal device, and providing mode information includes providing mode information based on an indicator of the terminal device.

[0084] In this way, pattern information can be provided via indicators, making it easier for users to understand the current pattern information.

[0085] In some cases, the settings control is also used to set the text of the operation instructions, as well as to display the operation instructions, including the display text.

[0086] In this way, the operation instructions can be displayed by showing the text of the operation instructions, which makes it easy for users to understand the current operation instructions.

[0087] In some cases, displaying operation instructions on the display page includes: determining the degree of matching between voice data and at least one operation instruction; and displaying the operation instruction on the display page in response to the degree of matching between voice data and operation instructions reaching a threshold.

[0088] In this way, the operation command that matches the voice data can be determined from at least one operation command based on the degree of matching between each operation command and the voice data, which can improve the accuracy of the determined operation command.

[0089] In some cases, the degree of matching is determined based on at least one of the following: the degree of matching between the speech data and the semantics of at least one operation instruction; or the degree of matching between the speech data and the keywords of at least one operation instruction.

[0090] In this way, the degree of matching can be determined based on at least one of keywords and semantics, which can improve the convenience and accuracy of determining the degree of matching.

[0091] Example devices and equipment A corresponding apparatus for implementing the above methods or processes is also provided. Figure 4 A schematic structural block diagram of an example device 400 for interacting with media items is shown, according to several scenarios. Device 400 may be implemented as or included in terminal device 110. The various modules / components in device 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0092] like Figure 4 As shown, the device 400 includes: an information display module 410 configured to display a prompt message on a media item display page in response to receiving voice data, the prompt message indicating that voice data is being received; an instruction display module 420 configured to display an operation instruction on the display page, the operation instruction being a predetermined operation instruction and matching the voice data; and a result display module 430 configured to display the execution result of the operation instruction on the display page.

[0093] In some cases, the control modes of the display page include a voice mode and a non-voice mode, with voice data received in the voice mode. The device 400 also includes: a settings page switching control configured to display a settings page in response to receiving an interactive operation on the display page, the settings page including a switching control for switching control modes; and a mode switching module configured to switch control modes in response to receiving an interactive operation on the switching control.

[0094] In some cases, the device 400 further includes: a setting control display module configured to display at least one setting control in response to an interactive operation instruction switching from a non-voice mode to a voice mode, the at least one setting control being used to set at least one operation instruction, and the operation instruction being set via the setting control in the at least one setting control.

[0095] In some cases, the device 400 further includes an operation instruction setting module, configured to set operation instructions in response to receiving an interactive operation on the setting control.

[0096] In some cases, setting controls are used to set at least one of the following: the function of the operation command; or the keyword of the operation command.

[0097] In some cases, the functions include at least one of the following: toggle, fast forward, rewind, pause, cancel pause, volume adjustment, like, share, favorite, or comment.

[0098] In some cases, the instruction display module 420 is further configured to display operation instructions in response to voice data matching keywords.

[0099] In some cases, the device 400 further includes an instruction cancellation display module, configured to cancel the display of the operation instruction in response to the display duration of the operation instruction reaching a first predetermined duration.

[0100] In some cases, device 400 further includes a notification cancellation module configured to display a notification message in response to a mismatch between voice data and at least one operation command.

[0101] In some cases, the device 400 further includes an exit information display module configured to display exit information in response to an exit condition being met. The exit information indicates that the voice mode has been exited. The exit condition includes at least one of the following: no voice data is received within a second predetermined period of time, or an interactive operation indicating exiting the voice mode is received.

[0102] In some cases, the device 400 further includes a mode information providing module configured to provide mode information indicating a control mode.

[0103] In some cases, device 400 is executed by a terminal device, and the mode information providing module is further configured to provide mode information based on an indicator of the terminal device.

[0104] In some cases, the settings control is also used to set the text of the operation instructions, as well as to display the operation instructions, including the display text.

[0105] In some cases, the instruction display module 420 is further configured to: determine the degree of matching between voice data and at least one operation instruction; and, in response to the degree of matching between voice data and operation instruction reaching a threshold, display the operation instruction on a display page.

[0106] In some cases, the degree of matching is determined based on at least one of the following: the degree of matching between the speech data and the semantics of at least one operation instruction; or the degree of matching between the speech data and the keywords of at least one operation instruction.

[0107] The modules included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some cases, one or more modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 400 can be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0108] Figure 5 A block diagram of an electronic device 500 in which one or more examples may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the examples described herein. Figure 5 The electronic device 500 shown can be used to implement the terminal device 110 discussed above.

[0109] like Figure 5As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processing units or processors 510, memory 520, storage devices 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processor 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0110] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof). Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0111] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various examples.

[0112] The communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 500 can operate in a networked environment using logical connections to one or more other servers, networked personal computers, or another network node.

[0113] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0114] A computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. A computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0115] The flowcharts and / or block diagrams of the methods, apparatus, devices, and computer program products referred to herein describe various aspects. It should be understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0116] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0117] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0118] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products under various scenarios. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0119] Various examples have been described above. The foregoing descriptions are exemplary and not exhaustive, nor are they limited to the proposed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the proposed implementations.

Claims

1. A method for interacting with media items, comprising: In response to receiving voice data, a prompt message is displayed on the media item's display page, indicating that the voice data is being received; The display page shows operation instructions, which are predetermined operation instructions and are matched with the voice data; as well as The execution result of the operation command is displayed on the display page.

2. The method according to claim 1, wherein the control mode of the display page includes a voice mode and a non-voice mode, the voice data is received in the voice mode, and the method further includes: In response to receiving an interactive operation on the display page, a settings page is displayed, the settings page including a switching control for switching the control mode; as well as In response to receiving an interactive operation on the switching control, the control mode is switched.

3. The method according to claim 2, further comprising: In response to the interactive operation instruction to switch from the non-voice mode to the voice mode, at least one setting control is displayed, the at least one setting control being used to set at least one operation instruction, and the operation instruction being set via the setting control in the at least one setting control.

4. The method according to claim 3, further comprising: In response to receiving an interactive operation on the settings control, the operation instruction is set.

5. The method of claim 4, wherein the setting control is used to set at least one of the following: The function of the operation instruction; or The keywords of the operation instructions.

6. The method according to claim 1, further comprising: In response to the display duration of the operation instruction reaching a first predetermined duration, the display of the operation instruction is cancelled.

7. The method according to claim 3, further comprising: In response to the voice data not matching the at least one operation command, a notification message is displayed.

8. The method according to claim 2, further comprising: In response to the exit condition being met, an exit message is displayed, which indicates that the voice mode has been exited. The exit condition includes at least one of the following: no voice data is received within a second predetermined period of time, or the received interactive operation indicates that the voice mode has been exited.

9. The method according to claim 2, further comprising: Provide mode information, which indicates the control mode.

10. An apparatus for interacting with a media item, comprising: The information display module is configured to display a prompt message on the display page of the media item in response to receiving voice data, the prompt message indicating that the voice data is being received; The instruction display module is configured to display operation instructions on the display page, the operation instructions being predetermined operation instructions and matching the voice data; as well as The result display module is configured to display the execution result of the operation instruction on the display page.

11. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 9 when executed by the at least one processor.

12. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 9.

Images