Multimedia resource display method and device, electronic equipment and storage medium

By splitting multimedia resources into segments and displaying step information, the problem of complex operations during long-term viewing is solved, thus improving the user experience.

CN121187485APending Publication Date: 2025-12-23BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410804307.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-20
Publication Date
2025-12-23

AI Technical Summary

Technical Problem

During extended viewing of multimedia resources, users need to frequently interact with the content to view videos or text, resulting in a poor user experience.

Method used

By detecting the content type of multimedia resources and breaking them down into resource fragments, the step information corresponding to each fragment is determined, and this information is displayed on the interface based on user actions, thus simplifying the operation process.

Benefits of technology

It improves the ease of operation and user experience when watching multimedia resources, especially in scenarios such as recipes and online courses, where users can easily switch and view operation steps through actions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121187485A_ABST
    Figure CN121187485A_ABST
Patent Text Reader

Abstract

The invention relates to a multimedia resource display method and device, electronic equipment and a storage medium. The method comprises the following steps: detecting a content type of a currently displayed multimedia resource in response to a detected screen reading operation; when the content type is a preset content type, determining step information corresponding to each resource fragment in the multimedia resource; wherein the step information corresponding to the resource fragments is used for describing operation steps displayed in the resource fragments, and each resource fragment corresponds to one operation step; and displaying the step information in a display interface based on the detected user action. According to the method, the multimedia resources are segmentally displayed according to the operation steps, so that a user can conveniently check different operation steps through user actions, the user operation is simplified, and the use experience of the user is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a multimedia resource display method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the daily use of electronic devices, there are often scenarios that require watching operation steps for extended periods. Examples include viewing recipes on a mobile phone or attending online classes at various learning points. In these scenarios, users need to perform operations step by step. Taking viewing video recipes as an example, users need to constantly pause and rewind the video to watch the cooking process, while also timing themselves according to the time indicated in the recipe. For users, this requires constant operation, is complex, and results in a poor user experience. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a multimedia resource display method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of the present disclosure, a multimedia resource display method is provided, the method comprising:

[0005] In response to the detection of a screen read operation, the content type of the currently displayed multimedia resource is detected;

[0006] When the content type is a preset content type, the step information corresponding to each resource segment in the multimedia resource is determined; wherein, the step information corresponding to the resource segment is used to describe the operation steps displayed in the resource segment, and each resource segment corresponds to one operation step;

[0007] Based on the detected user actions, the step information is displayed on the interface.

[0008] In some embodiments, the step of determining the information corresponding to each resource fragment in the multimedia resource includes:

[0009] Split the multimedia resource into multiple resource fragments;

[0010] Determine the step information corresponding to each resource fragment.

[0011] In some embodiments, splitting multiple resource fragments from the multimedia resource includes:

[0012] The multimedia resource is identified to obtain an identification result, which is used to indicate that there is splittable content in the multimedia resource;

[0013] Based on the identification results, the multimedia resource is divided into the multiple resource fragments.

[0014] In some embodiments, the step of determining the information corresponding to each resource fragment includes:

[0015] When the resource segment is a video, the resource segment and the description information of the resource segment are determined as the step information corresponding to the resource segment;

[0016] When the resource fragment is text and / or an image, the resource fragment is identified as the step information corresponding to the resource fragment.

[0017] In some embodiments, the method further includes:

[0018] When the resource segment includes a timeout duration, the timeout duration is determined as the step information corresponding to the resource segment, and the timeout duration is used to indicate the time required to complete the operation steps shown in the resource segment.

[0019] In some embodiments, the display interface includes multiple display areas, and the step information includes various types of information; displaying the step information in the display interface includes at least one of the following:

[0020] Display videos or images in the first display area;

[0021] Display text or descriptive information in the second display area;

[0022] A timer control is displayed in the third display area. The timer control is used to display the countdown progress, and the maximum duration represented by the countdown progress is the timer duration.

[0023] In some embodiments, the method further includes:

[0024] In response to the detection of a progress adjustment action, the countdown progress is adjusted.

[0025] In some embodiments, before displaying the step information in the display interface based on the detected user action, the method further includes:

[0026] Display the first prompt message, or play the first prompt message; wherein the first prompt message is used to prompt the user whether to enter the display mode;

[0027] In response to the detection of a confirmation operation, the display interface is displayed, and the confirmation operation is used to indicate entry into the display mode.

[0028] In some embodiments, displaying the step information in the display interface based on the detected user action includes:

[0029] The first step information is displayed on the display interface;

[0030] In response to detecting a first user action, the first step information is switched to second step information, where the second step information is the step information preceding the first step information; or...

[0031] In response to detecting a second user action, the first step information is switched to third step information, which is the next step information after the first step information.

[0032] In some embodiments, after displaying the first step information in the display interface, the method further includes:

[0033] When the information from the first step ends, the information from the third step is displayed on the display interface.

[0034] In some embodiments, after switching the first step information to second step information in response to detecting a first user action, the method further includes:

[0035] When the second step information includes a timeout duration, a second prompt message is displayed, or a second prompt message is played; wherein, the second prompt message is used to prompt the user to reset the timeout duration, and the timeout duration is used to indicate the time required to complete the operation steps shown in the resource segment;

[0036] In response to the detection of a setting operation, a new timer duration is determined;

[0037] Based on the new timing duration, the information for the second step is displayed on the display interface.

[0038] In some embodiments, detecting the content type of the currently displayed multimedia resource in response to detecting a screen read operation includes:

[0039] In response to the detection of a screenshot operation, the content type of the multimedia resource is detected.

[0040] In some embodiments, the method further includes:

[0041] When the step information includes a video, and the video playback ends, a first end reminder voice message is played;

[0042] When the step information includes a timeout duration, and the display duration of the step information reaches the timeout duration, a second end reminder voice message is played.

[0043] According to a second aspect of the present disclosure, a multimedia resource display device is provided, the device comprising:

[0044] The type detection module is configured to detect the content type of the currently displayed multimedia resource in response to a screen read operation.

[0045] The information determination module is configured to determine the step information corresponding to each resource segment in the multimedia resource when the content type is a preset content type; wherein, the step information corresponding to the resource segment is used to describe the operation steps displayed in the resource segment, and each resource segment corresponds to one operation step;

[0046] The information display module is configured to display the step information in the display interface based on the detected user actions.

[0047] In some embodiments, the information determining module is configured to:

[0048] Split the multimedia resource into multiple resource fragments;

[0049] Determine the step information corresponding to each resource fragment.

[0050] In some embodiments, the information determining module is configured to:

[0051] The multimedia resource is identified to obtain an identification result, which is used to indicate that there is splittable content in the multimedia resource;

[0052] Based on the identification results, the multimedia resource is divided into the multiple resource fragments.

[0053] In some embodiments, the information determining module is configured to:

[0054] When the resource segment is a video, the resource segment and the description information of the resource segment are determined as the step information corresponding to the resource segment;

[0055] When the resource fragment is text and / or an image, the resource fragment is identified as the step information corresponding to the resource fragment.

[0056] In some embodiments, the information determining module is configured to:

[0057] When the resource segment includes a timeout duration, the timeout duration is determined as the step information corresponding to the resource segment, and the timeout duration is used to indicate the time required to complete the operation steps shown in the resource segment.

[0058] In some embodiments, the display interface includes multiple display areas, and the step information includes various types of information; the information display module is configured to perform at least one of the following:

[0059] Display videos or images in the first display area;

[0060] Display text or descriptive information in the second display area;

[0061] A timer control is displayed in the third display area. The timer control is used to display the countdown progress, and the maximum duration represented by the countdown progress is the timer duration.

[0062] In some embodiments, the information display module is further configured to:

[0063] In response to the detection of a progress adjustment action, the countdown progress is adjusted.

[0064] In some embodiments, the apparatus further includes:

[0065] The first prompt module is configured to display a first prompt message or play a first prompt message; wherein the first prompt message is used to prompt the user whether to enter the display mode;

[0066] The interface display module is configured to display the display interface in response to the detection of a confirmation operation, wherein the confirmation operation is used to indicate entry into the display mode.

[0067] In some embodiments, the information display module is configured as follows:

[0068] The first step information is displayed on the display interface;

[0069] In response to detecting a first user action, the first step information is switched to second step information, where the second step information is the step information preceding the first step information; or...

[0070] In response to detecting a second user action, the first step information is switched to third step information, which is the next step information after the first step information.

[0071] In some embodiments, the information display module is configured as follows:

[0072] When the information from the first step ends, the information from the third step is displayed on the display interface.

[0073] In some embodiments, the apparatus further includes:

[0074] The second prompt module is configured to display a second prompt message or play a second prompt message when the second step information includes a timeout duration; wherein the second prompt message is used to prompt the user to reset the timeout duration, and the timeout duration is used to indicate the time required to complete the operation steps shown in the resource segment;

[0075] The duration determination module is configured to determine a new timeout duration in response to the detection of a setting operation;

[0076] The information display module is configured to display the second step information in the display interface based on the new time interval.

[0077] In some embodiments, the type detection module is configured to:

[0078] In response to the detection of a screenshot operation, the content type of the multimedia resource is detected.

[0079] In some embodiments, the apparatus further includes:

[0080] The third prompt module is configured to play a first end reminder voice when the step information includes a video and the video ends;

[0081] The third prompt module is further configured to play a second end reminder voice when the step information includes a timed duration and the display duration of the step information reaches the timed duration.

[0082] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0083] processor;

[0084] Memory used to store processor-executable instructions;

[0085] The processor is configured to perform the method described in the first aspect of the embodiments of this disclosure.

[0086] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method described in the first aspect of the present disclosure.

[0087] The method described above, as disclosed in this invention, has the following beneficial effects:

[0088] The method provided in this disclosure detects the content type of the currently displayed multimedia resource when a screen reading operation is detected. When the content type is a preset content type, it determines the step information corresponding to each resource segment in the multimedia resource. This step information is used to describe the operation steps displayed in the resource segment. Finally, based on the detected user action, the step information is displayed in the display interface, thereby displaying the multimedia resource in segments according to the operation steps. This allows users to conveniently view different operation steps through user actions, simplifying user operations and improving the user experience.

[0089] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0090] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0091] Figure 1 This is a flowchart illustrating a multimedia resource display method according to an exemplary embodiment;

[0092] Figure 2 This is a flowchart illustrating a multimedia resource display method according to an exemplary embodiment;

[0093] Figure 3 This is a schematic diagram illustrating a display interface according to an exemplary embodiment;

[0094] Figure 4 This is a block diagram illustrating a multimedia resource display device according to an exemplary embodiment;

[0095] Figure 5 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0096] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0097] With the development of electronic devices, AON (always-on camera) functionality and AI (Artificial Intelligence) capabilities have become essential features for some electronic devices. AON functionality, through a low-power always-on mode, can sense user behavior, including but not limited to gestures and eye movements. AI capabilities are being deployed and developed rapidly in electronic devices, and are now capable of complex analysis and feedback. The multimedia resource display method provided in this disclosure, based on the AON functionality and AI capabilities of the electronic device, displays multimedia resources in segments according to operation steps, allowing users to easily view different operation steps through user actions, simplifying user operation and improving the user experience.

[0098] The method provided in this disclosure can be applied to various scenarios, such as when a user is viewing a recipe, a student is taking online classes, a user is watching exercise videos, or a user is searching for information.

[0099] Taking a user viewing a recipe as an example, when a user watches a recipe in video format, the multimedia resource display method provided in this embodiment first breaks down the cooking process shown in the recipe into multiple cooking steps, and then displays each cooking step in sequence. During the display process, when the user needs to view the previous cooking steps or the subsequent cooking steps, the user can operate through actions to switch between different cooking steps, making it convenient for the user to view each cooking step.

[0100] The method provided in this disclosure is executed by an electronic device, which may be a mobile phone, tablet computer, laptop computer, desktop computer, wearable device, vehicle terminal, smart home device, etc.

[0101] Figure 1 This is a flowchart illustrating a multimedia resource display method according to an exemplary embodiment, executed by an electronic device. See also... Figure 1 The method includes the following steps:

[0102] Step S101: In response to the detection of a screen reading operation, the content type of the currently displayed multimedia resource is detected.

[0103] The screen reading operation is used to indicate and detect the content type of the multimedia resource currently displayed on the screen. This screen reading operation can be a screenshot operation, a two-finger swipe down operation, or other preset operations. Multimedia resources can be at least one of video, text, and image. The content type of a multimedia resource refers to the type of its content, which can include various categories based on resource content, such as recipes, instructional videos, learning materials, music, games, and articles. For example, if the currently displayed multimedia resource is a video teaching how to cook, detecting this multimedia resource can determine that its content type is a recipe; similarly, if the currently displayed multimedia resource is a game replay, detecting this multimedia resource can determine that its content type is a game.

[0104] Step S102: When the content type is a preset content type, determine the step information corresponding to each resource segment in the multimedia resource; wherein, the step information corresponding to the resource segment is used to describe the operation steps displayed in the resource segment, and each resource segment corresponds to one operation step.

[0105] In this embodiment of the disclosure, some videos or texts do not have clearly defined operational steps, such as a movie or a poem. However, for videos or texts with clearly defined operational steps, such as a cooking tutorial video or an exercise tutorial video, they can be broken down according to the operational steps. Therefore, given that the content type of the multimedia resource is a preset content type, it is necessary to determine the step information corresponding to each resource segment in the multimedia resource, that is, to break down the multimedia resource according to the operational steps. The preset content type includes types with clearly defined operational steps, such as recipe types and instructional video types.

[0106] The multimedia resource comprises multiple resource segments, each corresponding to an operation step. The step information corresponding to the resource segment describes the operation steps displayed in the resource segment. Optionally, the step information includes at least one of the following: the resource segment, descriptive information about the resource segment, and timing duration included in the resource segment. The operation steps will differ depending on the multimedia resource. For example, when the multimedia resource is a recipe, the operation steps are the steps of cooking; when the multimedia resource is an exercise video, the operation steps are the exercise movements.

[0107] Step S103: Based on the detected user actions, display the step information on the display interface.

[0108] User actions are used to display operation step information on the display interface. For example, user actions can switch the resource information corresponding to one resource fragment to the resource information corresponding to another resource fragment. The display interface is used to display the step information.

[0109] The method provided in this disclosure detects the content type of the currently displayed multimedia resource when a screen reading operation is detected. When the content type is a preset content type, it determines the step information corresponding to each resource segment in the multimedia resource. This step information is used to describe the operation steps displayed in the resource segment. Finally, based on the detected user action, the step information is displayed in the display interface, thereby displaying the multimedia resource in segments according to the operation steps. This allows users to conveniently view different operation steps through user actions, simplifying user operations and improving the user experience.

[0110] Figure 2 This is a flowchart illustrating a multimedia resource display method according to an exemplary embodiment, executed by an electronic device. See also... Figure 2 The method includes the following steps:

[0111] Step S201: In response to the detection of a screen reading operation, the content type of the currently displayed multimedia resource is detected.

[0112] A screen reading operation is used to indicate the content type of the multimedia resource currently displayed on the screen. In some embodiments, the content type of the currently displayed multimedia resource is detected in response to a screenshot operation. Of course, other operations can also be used as screen reading operations, such as a two-finger swipe down operation. This disclosure does not limit the specific operation method of the screen reading operation.

[0113] For content type detection, in some embodiments, a type detection model is invoked to detect the content type of the currently displayed multimedia resource.

[0114] In some embodiments, detecting the content type of the currently displayed multimedia resource includes: acquiring the complete multimedia resource, detecting the complete multimedia resource to obtain the content type; for example, if the multimedia resource is a video, acquiring the complete video and detecting the complete video; or, when the multimedia resource includes a title, acquiring the title, detecting the title to obtain the content type; for example, if the multimedia resource is an article, acquiring the article title and detecting the article title. Of course, other methods can also be used to detect the content type of the multimedia resource, and this disclosure does not limit this approach.

[0115] Step S202: When the content type is a preset content type, split multiple resource segments from the multimedia resources and determine the step information corresponding to each resource segment.

[0116] In this embodiment of the disclosure, when obtaining step information, the multimedia resource is first divided into multiple resource fragments, and then the corresponding step information is obtained for each resource fragment.

[0117] In some embodiments, splitting multiple resource fragments from a multimedia resource includes: identifying the multimedia resource, obtaining an identification result, the identification result being used to indicate that there is splittable content in the multimedia resource; and splitting the multimedia resource into multiple resource fragments based on the identification result.

[0118] Optionally, the recognition model can be invoked to identify multimedia resources and obtain the recognition results.

[0119] Optionally, the recognition result includes the start and end time points of the resource segments. Based on the recognition result, the multimedia resource is split into multiple resource segments, including: splitting the multimedia resource into multiple resource segments based on the start and end time points of each resource segment.

[0120] In some embodiments, when the resource segment is a video, the resource segment and the description information of the resource segment are determined as the step information corresponding to the resource segment, wherein the description information is a text description of the content of the resource segment; when the resource segment is text and / or an image, the resource segment is determined as the step information corresponding to the resource segment.

[0121] In some embodiments, when a resource fragment includes a timeout duration, the timeout duration is determined as the step information corresponding to the resource fragment. The timeout duration indicates the time required to complete the operation steps shown in the resource fragment. For example, in a cooking video, if it indicates that stir-frying takes 5 minutes, the timeout duration can be determined to be 5 minutes.

[0122] Optionally, when identifying multimedia resources, if a timing duration is identified in the multimedia resource, the timing duration is marked, and when determining the step information, the timing duration is used as the step information.

[0123] Step S203: Display the first prompt message, or play the first prompt message; wherein the first prompt message is used to prompt the user whether to enter the display mode.

[0124] In this embodiment of the disclosure, in order to provide users with a better user experience, the user will be prompted whether to enter the display module. The user can choose whether to enter the display mode according to their own needs. If the user confirms to enter the display mode, the subsequent steps will be executed. If the user confirms not to enter the display mode, the multimedia resources will continue to be displayed according to the current display method.

[0125] When the first prompt message can be text, it is displayed; when the first prompt message is voice, it is played. Of course, in some embodiments, the first prompt message can be displayed in text form on the screen while simultaneously playing the voice prompt message.

[0126] Step S204: In response to the detection of a confirmation operation, a display interface is displayed, and the confirmation operation is used to indicate entering the display mode.

[0127] The confirmation operation can be a user's click on the screen. For example, the initial prompt might be text, along with confirmation and cancel options. A confirmation operation is detected when the user clicks the confirmation option. Alternatively, the confirmation operation can be a user action. For instance, a preset action representing the confirmation operation can be pre-set. A confirmation operation is detected when the user action, as indicated by the AON function, matches this preset action. Or, the confirmation operation can be a voice prompt, such as when the user utters the voice command "Enter display mode."

[0128] After the user confirms entry into the display mode, a display interface is shown, which displays the step information.

[0129] Step S205: Display the information from the first step on the display interface.

[0130] The first step information is any one of the multiple step information.

[0131] In some embodiments, the display interface includes multiple display areas, and the step information includes various types of information. Displaying the step information in the display interface includes at least one of the following: displaying a video or image in a first display area; displaying text or descriptive information in a second display area; and displaying a timing control in a third display area, the timing control being used to display the countdown progress, the maximum duration represented by the countdown progress being the timing duration. In one example, see [reference needed]. Figure 3 The diagram shown illustrates the display interface, which includes a video area, a text description area, and a countdown area.

[0132] In some embodiments, the timing control in the third display area can be adjusted to adjust the countdown progress in response to a detected progress adjustment action. The progress adjustment action is used to adjust the countdown progress, for example, by decreasing or increasing the countdown. Optionally, the progress adjustment action can be a hand swing to the right or a hand swing to the left. For example, when a hand swing to the right is detected, the countdown is increased according to the amplitude of the rightward swing; when a hand swing to the left is detected, the countdown is decreased according to the amplitude of the leftward swing. Of course, the progress adjustment action can also take other forms, and this embodiment of the disclosure does not limit this.

[0133] In some embodiments, when a video is displayed in the first display area, the user can also adjust the video's pause, play, fast forward, rewind, etc.

[0134] It should be noted that the implementation method for displaying step information in the above-described display interface is applicable to each piece of step information. Furthermore, when displaying step information, the display area in the display interface can be adaptively adjusted according to the type of information contained within the step information. For example, when the step information includes video, text description, and timeout duration, the display interface displays a first display area, a second display area, and a third display area; when the step information includes video and text description, the display interface displays a first display area and a second display area; and when the step information includes text description and timeout duration, the display interface displays a second display area and a third display area.

[0135] Step S206: In response to detecting the first user action, the first step information is switched to the second step information, which is the previous step information of the first step information.

[0136] The first user action is used to indicate whether to switch the currently displayed step information to the previous step information. For example, the first user action is a finger pointing upwards, an eye looking upwards, or other actions.

[0137] In some embodiments, user actions are detected through the AON function. When a first user action is detected, the first step information is switched to the second step information so that the second step information can be displayed in the display interface.

[0138] In some embodiments, considering that the second step information is the step preceding the first step information, the second step information may have already been displayed on the interface. Therefore, when the second step information includes a timer duration, the original timer duration may no longer be applicable to the current scenario. Thus, when the second step information includes a timer duration, a second prompt message is displayed, or played, where the second prompt message prompts the user to reset the timer duration. In response to detecting a setting operation, a new timer duration is determined; based on the new timer duration, the second step information is displayed on the interface. When displaying the second step information, apart from the timer duration being changed to the new timer duration, other display content remains unchanged.

[0139] Optionally, when the second prompt message is text, the second prompt message is displayed. At the same time as displaying the second prompt message, the timer duration setting options can be displayed. Different timer duration setting options correspond to different timer durations. Users can set a new timer duration by selecting the timer duration setting option.

[0140] Optionally, when the second prompt is a voice message, it is played, and the user can directly reply with a new timer duration via voice after hearing it. For example, if the second prompt area says "Please set a new countdown," the user can reply "Set a 1-minute countdown."

[0141] Step S207: In response to detecting a second user action, the first step information is switched to the third step information, which is the next step information after the first step information.

[0142] The second user action is used to indicate switching the currently displayed step information to the next step information. This second user action is different from the first user action. For example, the second user action is a downward finger gesture, a downward eye gesture, or other gesture. When displaying the third step information, simply display the content from the third step information.

[0143] In some embodiments, user actions are detected through the AON function. When a second user action is detected, the first step information is switched to the third step information so that the third step information can be displayed in the display interface.

[0144] It should be noted that step S207 described above is an example of the user manually switching the first step information to the third step information. In another embodiment, when the display of the first step information ends, the third step information is displayed on the display interface. That is, when the display of a step information ends, the next step information of that step information is displayed directly. The end of the first step information display includes: when the first step information includes video and a timer duration, the display of the first step information ends when the video playback ends and the display duration of the first step information reaches the timer duration; when the first step information includes either video or a timer duration, the display of the first step information ends when either the video playback ends or the display duration of the first step information reaches the timer duration.

[0145] In some embodiments, for any step information, if the step information includes a video and the video playback ends, a first end reminder voice is played; if the step information includes a timeout duration and the display duration of the step information reaches the timeout duration, a second end reminder voice is played. That is, when an operation step is completed, the user can be prompted by voice that the current operation step has been completed, so that the user can follow up promptly.

[0146] The method provided in this disclosure detects the content type of the currently displayed multimedia resource when a screen reading operation is detected. When the content type is a preset content type, it determines the step information corresponding to each resource segment in the multimedia resource. This step information is used to describe the operation steps displayed in the resource segment. Finally, based on the detected user action, the step information is displayed in the display interface, thereby displaying the multimedia resource in segments according to the operation steps. This allows users to easily switch the displayed step information through user actions, simplifying user operations and enabling users to complete operations more conveniently, hygienically, and efficiently, thus improving the user experience.

[0147] Furthermore, a voice assistance function is introduced during user operation, which plays prompts to users through voice, so that users can understand the current step information in a timely manner even when it is inconvenient to look at the screen. In addition, when it is inconvenient for users to operate by hand, they can directly set the timer duration by voice. The combination of voice operation and hand operation further improves the user experience.

[0148] Figure 4 This is a block diagram illustrating a multimedia resource display device according to an exemplary embodiment, performed by an electronic device, see [link to relevant documentation]. Figure 4 The device includes:

[0149] The type detection module 401 is configured to detect the content type of the currently displayed multimedia resource in response to the detection of a screen reading operation;

[0150] The information determination module 402 is configured to determine the step information corresponding to each resource segment in the multimedia resource when the content type is a preset content type; wherein, the step information corresponding to the resource segment is used to describe the operation steps displayed in the resource segment, and each resource segment corresponds to one operation step;

[0151] The information display module 403 is configured to display step information in the display interface based on the detected user actions.

[0152] In some embodiments, the information determination module 402 is configured to:

[0153] Split multimedia resources into multiple fragments;

[0154] Determine the step information corresponding to each resource fragment.

[0155] In some embodiments, the information determination module 402 is configured to:

[0156] Identify multimedia resources and obtain identification results. The identification results are used to indicate that there is splittable content in the multimedia resources.

[0157] Based on the recognition results, the multimedia resources are split into multiple resource fragments.

[0158] In some embodiments, the information determination module 402 is configured to:

[0159] When the resource segment is a video, the resource segment and its description information are identified as the step information corresponding to the resource segment.

[0160] When the resource fragment is text and / or an image, the resource fragment is identified as the step information corresponding to the resource fragment.

[0161] In some embodiments, the information determination module 402 is configured to:

[0162] When a resource fragment includes a timeout duration, the timeout duration is determined as the step information corresponding to the resource fragment. The timeout duration is used to indicate the time required to complete the operation steps shown in the resource fragment.

[0163] In some embodiments, the display interface includes multiple display areas, and the step information includes various types of information; the information display module 403 is configured to perform at least one of the following:

[0164] Display videos or images in the first display area;

[0165] Display text or descriptive information in the second display area;

[0166] The third display area shows a timer control, which is used to display the countdown progress. The maximum duration represented by the countdown progress is the timer duration.

[0167] In some embodiments, the information display module 403 is further configured to:

[0168] In response to the detection of a progress adjustment action, the countdown progress is adjusted.

[0169] In some embodiments, the apparatus further includes:

[0170] The first prompt module is configured to display a first prompt message or play a first prompt message; wherein the first prompt message is used to prompt the user whether to enter the display mode.

[0171] The interface display module is configured to display a display interface in response to the detection of a confirmation operation. The confirmation operation is used to indicate entry into the display mode.

[0172] In some embodiments, the information display module 403 is configured as follows:

[0173] The first step information is displayed on the interface;

[0174] In response to detecting a first user action, the first step information is switched to the second step information, where the second step information is the information preceding the first step information; or...

[0175] In response to the detection of a second user action, the first step information is switched to the third step information, which is the next step information after the first step information.

[0176] In some embodiments, the information display module 403 is configured as follows:

[0177] When the information from the first step is finished being displayed, the information from the third step will be displayed on the screen.

[0178] In some embodiments, the apparatus further includes:

[0179] The second prompt module is configured to display or play a second prompt message when the second step information includes a timeout duration; wherein the second prompt message is used to prompt the user to reset the timeout duration, and the timeout duration is used to indicate the time required to complete the operation steps shown in the resource segment;

[0180] The duration determination module is configured to determine a new timeout duration in response to the detection of a setting operation;

[0181] The information display module 403 is configured to display the second step information in the display interface based on the new time interval.

[0182] In some embodiments, the type detection module 401 is configured to:

[0183] In response to the detection of a screenshot operation, the content type of the multimedia resource is detected.

[0184] In some embodiments, the apparatus further includes:

[0185] The third prompt module is configured to play the first end reminder voice when the step information includes a video and the video playback ends;

[0186] The third prompt module is also configured to play a second end reminder voice when the step information includes a timeout duration and the display duration of the step information reaches the timeout duration.

[0187] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0188] This disclosure also provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the multimedia resource display method described in the above embodiments.

[0189] Figure 5 This is a block diagram of an electronic device 500 according to an exemplary embodiment.

[0190] Reference Figure 5 The electronic device 500 may include one or more of the following components: processing component 502, memory 504, power supply component 506, multimedia component 508, audio component 510, input / output (I / O) interface 512, sensor component 514, and communication component 516.

[0191] Processing component 502 typically controls the overall operation of electronic device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0192] Memory 504 is configured to store various types of data to support the operation of electronic device 500. Examples of such data include instructions for any application or method operating on electronic device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0193] Power supply component 506 provides power to various components of electronic device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 500.

[0194] Multimedia component 508 includes a screen that provides an output interface between the electronic device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and actions on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When the electronic device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0195] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when electronic device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0196] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0197] Sensor assembly 514 includes one or more sensors for providing state assessments of various aspects of electronic device 500. For example, sensor assembly 514 may detect the on / off state of electronic device 500, the relative positioning of components such as the display and keypad of electronic device 500, changes in position of electronic device 500 or a component of electronic device 500, the presence or absence of user contact with electronic device 500, orientation or acceleration / deceleration of electronic device 500, and temperature changes of electronic device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0198] Communication component 516 is configured to facilitate wired or wireless communication between electronic device 500 and other devices. Electronic device 500 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0199] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0200] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by a processor 520 of an electronic device 500 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0201] This disclosure also provides a non-transitory computer-readable storage medium, wherein when the instructions in the storage medium are executed by the processor of an electronic device, the electronic device is able to perform the multimedia resource display method described in the above embodiments.

[0202] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0203] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for displaying multimedia resources, characterized in that, The method includes: In response to the detection of a screen read operation, the content type of the currently displayed multimedia resource is detected; When the content type is a preset content type, the step information corresponding to each resource segment in the multimedia resource is determined; wherein, the step information corresponding to the resource segment is used to describe the operation steps displayed in the resource segment, and each resource segment corresponds to one operation step; Based on the detected user actions, the step information is displayed on the interface.

2. The method according to claim 1, characterized in that, The step information for determining the corresponding resource fragment in the multimedia resource includes: Split the multimedia resource into multiple resource fragments; Determine the step information corresponding to each resource fragment.

3. The method according to claim 2, characterized in that, The step of splitting multiple resource fragments from the multimedia resource includes: Identify the multimedia resource to obtain an identification result, which is used to indicate that there is splittable content in the multimedia resource; Based on the identification results, the multimedia resource is divided into the multiple resource fragments.

4. The method according to claim 2, characterized in that, The steps for determining the information corresponding to each resource fragment include: When the resource segment is a video, the resource segment and the description information of the resource segment are determined as the step information corresponding to the resource segment; When the resource fragment is text and / or an image, the resource fragment is identified as the step information corresponding to the resource fragment.

5. The method according to claim 4, characterized in that, The method further includes: When the resource segment includes a timeout duration, the timeout duration is determined as the step information corresponding to the resource segment, and the timeout duration is used to indicate the time required to complete the operation steps shown in the resource segment.

6. The method according to claim 5, characterized in that, The display interface includes multiple display areas, and the step information includes various types of information; displaying the step information in the display interface includes at least one of the following: Display videos or images in the first display area; Display text or descriptive information in the second display area; A timer control is displayed in the third display area. The timer control is used to display the countdown progress, and the maximum duration represented by the countdown progress is the timer duration.

7. The method according to claim 6, characterized in that, The method further includes: In response to the detection of a progress adjustment action, the countdown progress is adjusted.

8. The method according to claim 1, characterized in that, Before displaying the step information on the display interface based on the detected user action, the method further includes: Display the first prompt message, or play the first prompt message; wherein the first prompt message is used to prompt the user whether to enter the display mode; In response to the detection of a confirmation operation, the display interface is displayed, and the confirmation operation is used to indicate entry into the display mode.

9. The method according to claim 1, characterized in that, The step information displayed on the display interface based on the detected user actions includes: The first step information is displayed on the display interface; In response to detecting a first user action, the first step information is switched to second step information, where the second step information is the step information preceding the first step information; or... In response to detecting a second user action, the first step information is switched to third step information, which is the next step information after the first step information.

10. The method according to claim 9, characterized in that, After displaying the first step information on the display interface, the method further includes: When the information from the first step ends, the information from the third step is displayed on the display interface.

11. The method according to claim 9, characterized in that, After switching the first step information to second step information in response to detecting a first user action, the method further includes: When the second step information includes a timeout duration, a second prompt message is displayed, or a second prompt message is played; wherein, the second prompt message is used to prompt the user to reset the timeout duration, and the timeout duration is used to indicate the time required to complete the operation steps shown in the resource segment; In response to the detection of a setting operation, a new timer duration is determined; Based on the new timing duration, the information for the second step is displayed on the display interface.

12. The method according to claim 1, characterized in that, The step of detecting the content type of the currently displayed multimedia resource in response to a detected screen read operation includes: In response to the detection of a screenshot operation, the content type of the multimedia resource is detected.

13. The method according to claim 1, characterized in that, The method further includes: When the step information includes a video, and the video playback ends, a first end reminder voice message is played; When the step information includes a timeout duration, and the display duration of the step information reaches the timeout duration, a second end reminder voice message is played.

14. A multimedia resource display device, characterized in that, The device includes: The type detection module is configured to detect the content type of the currently displayed multimedia resource in response to a screen read operation. The information determination module is configured to determine the step information corresponding to each resource segment in the multimedia resource when the content type is a preset content type; wherein, the step information corresponding to the resource segment is used to describe the operation steps displayed in the resource segment, and each resource segment corresponds to one operation step; The information display module is configured to display the step information in the display interface based on the detected user actions.

15. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method as described in any one of claims 1-13.

16. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-13.