Multimedia information processing method, device, electronic device and storage medium

Abstract

The present application provides a multimedia information processing method, device, electronic device and storage medium; the method comprises: a terminal presents a multimedia information recording interface for recording multimedia information; in the process of recording multimedia information based on the multimedia information recording interface, detecting a gesture operation performed on the terminal; when a target gesture operation for indicating that multimedia information is to be marked is detected, in response to the target gesture operation, the recorded multimedia information is marked and corresponding marking information is presented; through the present application, the marking of multimedia information can be realized based on gesture operation, the operation is simple, and when necessary, the corresponding marking node can be quickly and accurately located based on the marking information, thereby improving efficiency.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a multimedia information processing method, device, electronic device and storage medium. Background Art

[0002] For some critical occasions, the key content information is generally recorded by recording multimedia information (such as video and audio) throughout the process. However, when reviewing or checking the key content, either all the recorded multimedia information is played back, or the key information points are repeatedly searched and located, which is cumbersome and time-consuming, resulting in a waste of human resources and no guarantee of accurate finding of important content. Summary of the invention

[0003] The embodiments of the present application provide a multimedia information processing method, device, electronic device and storage medium, which can implement the annotation of multimedia information based on gesture operation. The operation is simple and when needed, the corresponding node can be quickly and accurately located based on the annotation information, thereby improving efficiency.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a multimedia information processing method, including:

[0006] The terminal presents a multimedia information recording interface for recording multimedia information;

[0007] In the process of recording multimedia information based on the multimedia information recording interface, detecting a gesture operation performed on the terminal;

[0008] When a target gesture operation for indicating to mark the multimedia information is detected, the recorded multimedia information is marked in response to the target gesture operation, and corresponding marking information is presented.

[0009] In the above solution, the presenting of corresponding annotation information includes:

[0010] In the multimedia information recording interface, an information display area for displaying the marked information is presented.

[0011] In the information display area, the marked information corresponding to the marked multimedia information is displayed.

[0012] In the above solution, the presenting of corresponding annotation information includes:

[0013] Get the number of times the recorded multimedia information is marked;

[0014] Determine the order corresponding to the current annotation of the recorded multimedia information according to the number of acquisitions;

[0015] Generate annotation indication information for indicating the sorting, and present the annotation indication information as annotation information corresponding to the current annotation of the recorded multimedia information.

[0016] In the above scheme, the method further comprises:

[0017] In the multimedia information recording interface, a marking button is presented;

[0018] During the multimedia information recording process, a trigger operation for the marking button is received;

[0019] In response to the triggering operation on the marking button, the recorded multimedia information is marked and corresponding marking information is presented.

[0020] In the above scheme, when a target gesture operation for indicating to mark multimedia information is detected, in response to the target gesture operation, the recorded multimedia information is marked and corresponding marking information is presented, including:

[0021] When a target gesture operation for indicating marking multimedia information is detected, obtaining the operation time when the target gesture operation was last detected;

[0022] When it is determined according to the acquired operation time and current time that the corresponding time difference reaches a time difference threshold, the recorded multimedia information is annotated in response to the target gesture operation, and corresponding annotation information is presented.

[0023] In the above scheme, the method further includes:

[0024] When it is determined based on the acquired operation time and current time that the corresponding time difference does not reach the time difference threshold, a marking prompt message is presented;

[0025] The marking prompt information is used to prompt that the multimedia information cannot be marked at the current time.

[0026] In the above scheme, the method further comprises:

[0027] In the multimedia information recording interface, the remaining number of marking times for the recorded multimedia information is presented;

[0028] When the remaining number of markings presented is zero and the target gesture operation is detected again, a marking deletion prompt message is presented;

[0029] The annotation deletion prompt information is used to prompt the deletion of the historical annotation of the recorded multimedia information in response to the target gesture operation detected again.

[0030] The embodiment of the present application further provides a multimedia information processing device, which is arranged in a terminal and includes:

[0031] A presentation module, used to present a multimedia information recording interface for recording multimedia information;

[0032] A detection module, used to detect the gesture operation performed on the terminal during the process of recording multimedia information based on the multimedia information recording interface;

[0033] The marking module is used to mark the recorded multimedia information and present corresponding marking information in response to the target gesture operation when a target gesture operation for indicating marking the multimedia information is detected.

[0034] In the above solution, the detection module is further used to detect the angular displacement of the terminal in various directions through an angular velocity sensor in the terminal;

[0035] When it is detected that the angular displacement of the terminal in the target direction reaches an angular displacement threshold, it is determined that the target gesture operation performed on the terminal is detected.

[0036] In the above solution, the presenting module is further used to present a marking function switch in the multimedia information recording interface;

[0037] In response to a triggering operation on the marking function switch, turning on a marking function for the recorded multimedia information;

[0038] The detection module is further configured to detect a gesture operation performed on the terminal when a marking function for the recorded multimedia information is enabled.

[0039] In the above solution, the presenting module is further used to present the guidance information corresponding to the target gesture operation in the multimedia information recording interface;

[0040] The guide information is used to guide the execution of the target gesture operation so as to mark the recorded multimedia information.

[0041] In the above solution, the presenting module is further used to present the recording timeline of the corresponding multimedia information in the multimedia information recording interface;

[0042] The marking module is further used to obtain the operation time corresponding to the target gesture operation;

[0043] The recorded multimedia information is annotated according to the operation time, and corresponding annotation information is presented at a position corresponding to the operation time on the recording time axis.

[0044] In the above solution, the annotation module is further used to present an information display area for displaying the annotation information in the multimedia information recording interface.

[0045] In the information display area, the marked information corresponding to the marked multimedia information is displayed.

[0046] In the above scheme, the annotation module is further used to present an information display area for displaying the annotation information in the multimedia information recording interface when the number of the annotation information is multiple and the number of the annotation information reaches the target number;

[0047] Displaying the plurality of annotation information in the information display area, and presenting scroll bars corresponding to the plurality of annotation information on one side of the information display area;

[0048] When a sliding operation on the scroll bar is received, the display positions of the plurality of annotation information in the information display area are synchronously adjusted.

[0049] In the above solution, the marking module is also used to obtain the number of times the recorded multimedia information is marked;

[0050] Determine the order corresponding to the current annotation of the recorded multimedia information according to the number of acquisitions;

[0051] Generate annotation indication information for indicating the sorting, and present the annotation indication information as annotation information corresponding to the current annotation of the recorded multimedia information.

[0052] In the above scheme, when the annotation information is a annotation name corresponding to the annotation, the presentation module is further used to present a name edit box corresponding to the annotation name;

[0053] Based on the name edit box, receiving the edited annotation name corresponding to the annotation;

[0054] The presenting of corresponding annotation information includes:

[0055] In response to a determination instruction for the edited annotation name, a annotation name corresponding to the annotation is presented.

[0056] In the above solution, the presenting module is further used to present a modification function item corresponding to the marked name;

[0057] When a trigger operation for the modification function item is received, presenting the name edit box again;

[0058] In response to a name modification operation triggered based on the name edit box, a modified annotation name is presented.

[0059] In the above solution, the presenting module is further used to present a marking button in the multimedia information recording interface;

[0060] During the multimedia information recording process, a trigger operation for the marking button is received;

[0061] In response to the triggering operation on the marking button, the recorded multimedia information is marked and corresponding marking information is presented.

[0062] In the above solution, the marking module is further used to obtain the operation time when the target gesture operation for indicating marking the multimedia information is detected;

[0063] When it is determined according to the acquired operation time and current time that the corresponding time difference reaches a time difference threshold, the recorded multimedia information is annotated in response to the target gesture operation, and corresponding annotation information is presented.

[0064] In the above scheme, the presentation module is further used to present the annotation prompt information when it is determined that the corresponding time difference does not reach the time difference threshold according to the acquired operation time and current time;

[0065] The marking prompt information is used to prompt that the multimedia information cannot be marked at the current time.

[0066] In the above scheme, the presenting module is further used to present the remaining marking times for the recorded multimedia information in the multimedia information recording interface;

[0067] When the remaining number of markings presented is zero and the target gesture operation is detected again, a marking deletion prompt message is presented;

[0068] The annotation deletion prompt information is used to prompt the deletion of the historical annotation of the recorded multimedia information in response to the target gesture operation detected again.

[0069] In the above solution, the presenting module is further used to present at least two importance level options corresponding to the annotation;

[0070] In response to a selection operation on the importance level option, taking the importance level corresponding to the selected importance level option as the importance level of the marked multimedia information;

[0071] The marking module is further used to present the marking information corresponding to the marking of the recorded multimedia information in a style corresponding to the importance level.

[0072] In the above scheme, the device also includes:

[0073] A playback module, used for presenting a playback interface corresponding to the recording file, presenting a corresponding playback timeline in the playback interface, and presenting at least one of the annotation information on the playback timeline;

[0074] In the process of playing the recorded file, in response to the trigger operation on the annotation information, the playback progress is jumped to the position of the annotation information in the playback timeline, so as to

[0075] Based on the position, the recorded file is played.

[0076] In the above solution, the playback module is further used to present a playback interface corresponding to the recorded file, in which at least one annotation information corresponding to the recorded file is presented, and

[0077] Presenting a play button corresponding to each of the annotation information;

[0078] In response to a triggering operation on the play button, obtaining a marking time of the recording file corresponding to corresponding marking information;

[0079] Jump the playing progress to the marked time to play the recorded file based on the marked time.

[0080] The present application also provides an electronic device, including:

[0081] A memory for storing executable instructions;

[0082] The processor is used to implement the multimedia information processing method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0083] The embodiment of the present application also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implements the multimedia information processing method provided in the embodiment of the present application.

[0084] The embodiments of the present application have the following beneficial effects:

[0085] During the process of recording multimedia information based on the multimedia information recording interface, the terminal detects the gesture operation performed on the terminal. When a target gesture operation for indicating the marking of the multimedia information is detected, the recorded multimedia information is marked in response to the target gesture operation, and the corresponding marking information is presented. In this way, during the recording of the multimedia information, the multimedia information can be marked based on the target gesture operation. The operation is simple, so that the key points in the recorded multimedia information can be recorded. When it is necessary to obtain the key content in the multimedia information, the corresponding node can be quickly and accurately located based on the marking information, without the need for manual search one by one, thereby improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 is a schematic diagram of the architecture of a multimedia information processing system 100 provided in an embodiment of the present application;

[0087] Figure 2 is a structural diagram of an electronic device 500 for implementing a multimedia information processing method provided in an embodiment of the present application;

[0088] Figure 3 It is a flowchart of a multimedia information processing method provided in an embodiment of the present application;

[0089] Figure 4 is a schematic diagram of presenting a multimedia information recording interface provided in an embodiment of the present application;

[0090] Figure 5 is a schematic diagram of a target gesture operation provided by an embodiment of the present application;

[0091] Figure 6 is a schematic diagram of presenting the marking function items provided in the embodiment of the present application;

[0092] Figure 7 is a schematic diagram of presenting guidance information provided by an embodiment of the present application;

[0093] Figure 8 It is a schematic diagram of presenting the annotation information provided in the embodiment of the present application;

[0094] Fig. 9 It is a schematic diagram of presenting the annotation information provided in the embodiment of the present application;

[0095] Fig.10 It is a schematic diagram of presenting the annotation information provided in the embodiment of the present application;

[0096] Fig.11 It is a schematic diagram of presenting the annotation information provided in the embodiment of the present application;

[0097] Fig.12 It is a schematic diagram of editing the annotation information provided in the embodiment of the present application;

[0098] Fig.13 It is a schematic diagram of the modification process of the label name provided in the embodiment of the present application;

[0099] Fig.14 is a schematic diagram of presenting a marking button provided in an embodiment of the present application;

[0100] Fig.15 is a schematic diagram of presenting the annotation prompt information provided in an embodiment of the present application;

[0101] Fig.16 This is a schematic diagram of presenting the annotation deletion prompt information provided in an embodiment of the present application;

[0102] Fig.17 1 is a schematic diagram of a selection process of the importance level of annotation information provided in an embodiment of the present application;

[0103] Fig.18 is a schematic diagram of playing multimedia information provided by an embodiment of the present application;

[0104] Fig.19 is a schematic diagram of playing multimedia information provided by an embodiment of the present application;

[0105] Fig. 20 It is a schematic diagram of the annotation process of multimedia information provided in an embodiment of the present application;

[0106] Fig.21 It is a schematic diagram of the playback process of multimedia information provided in an embodiment of the present application;

[0107] Fig. 22 It is a schematic diagram of the annotation process of multimedia information provided in an embodiment of the present application;

[0108] Fig.23 It is a schematic diagram of the annotation process of multimedia information provided in an embodiment of the present application;

[0109] Fig.24 It is a schematic diagram of the playback process of multimedia information provided in an embodiment of the present application. DETAILED DESCRIPTION

[0110] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0111] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0112] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0113] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0114] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0115] 1) Client: an application running in a terminal to provide various services, such as an instant messaging client or a video player client.

[0116] 2) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0117] 3) Angular velocity sensor: The angular velocity sensor of a mobile phone can accurately measure the rotation and deflection of the mobile phone, so that the actual action of the user can be accurately analyzed and judged. Then, the mobile phone can be operated accordingly according to the action.

[0118] Based on the above explanations of the nouns and terms involved in the embodiments of the present application, the multimedia information processing system provided by the embodiments of the present application is described below. Figure 1 , Figure 1 It is an architectural diagram of a multimedia information processing system 100 provided in an embodiment of the present application. To support an exemplary application, a terminal (terminal 400-1 is shown as an example) is connected to a server 200 via a network 300. The network 300 may be a wide area network or a local area network, or a combination of the two, and data transmission is achieved using wireless or wired links.

[0119] A terminal (such as terminal 400-1) is used to present a multimedia information recording interface for recording multimedia information on a graphical interface 410 (graphical interface 410-1 is shown as an example); in the process of recording multimedia information based on the multimedia information recording interface, the gesture operation performed on the terminal is detected; when a target gesture operation for indicating that the multimedia information is to be marked is detected, the recorded multimedia information is marked in response to the target gesture operation, and corresponding marking information is presented; and the marking information is sent to the server 200 for storage.

[0120] The server 200 is used to receive the annotation information of the corresponding multimedia information sent by the terminal and store it in the database 600, so that when the multimedia information is played again, the corresponding annotation information can be obtained from the database 600.

[0121] In practical applications, server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal (such as terminal 400-1) can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart TV, a smart watch, etc., but is not limited to this. The terminal (such as terminal 400-1) and server 200 can be directly or indirectly connected by wired or wireless communication, and this application is not limited here.

[0122] See also Figure 2 , Figure 2 Schematic diagram of the structure of an electronic device 500 for implementing a multimedia information processing method provided in an embodiment of the present application. Figure 1 The server or terminal shown is an electronic device 500. Figure 1 Taking the terminal shown as an example, an electronic device that implements the multimedia information processing method of the embodiment of the present application is described. The electronic device 500 provided in the embodiment of the present application includes: at least one processor 510, a memory 550, at least one network interface 520 and a user interface 530. The various components in the electronic device 500 are coupled together through a bus system 540. It can be understood that the bus system 540 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 2 Various buses are labeled as bus system 540 .

[0123] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0124] The user interface 530 includes one or more output devices 531 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0125] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices that are physically remote from the processor 510.

[0126] The memory 550 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0127] In some embodiments, the memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0128] Operating system 551, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0129] A network communication module 552, for reaching other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0130] a presentation module 553 for enabling presentation of information via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with the user interface 530 (e.g., a user interface for operating peripherals and displaying content and information);

[0131] The input processing module 554 is used to detect one or more user inputs or interactions from one of the one or more input devices 532 and translate the detected inputs or interactions.

[0132] In some embodiments, the multimedia information processing device provided in the embodiments of the present application can be implemented in software. Figure 2 A multimedia information processing device 555 stored in a memory 550 is shown, which may be software in the form of a program or plug-in, including the following software modules: a presentation module 5551, a detection module 5552, and an annotation module 5553. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0133] In other embodiments, the multimedia information processing device provided in the embodiments of the present application can be implemented by a combination of software and hardware. As an example, the multimedia information processing device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the multimedia information processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0134] Based on the above description of the multimedia information processing system and electronic device provided by the embodiments of the present application, the multimedia information processing method provided by the embodiments of the present application is described below. In some embodiments, the multimedia information processing method provided by the embodiments of the present application can be implemented by a server or a terminal alone, or by a server and a terminal in collaboration. The multimedia information processing method provided by the embodiments of the present application is described below using terminal implementation as an example. Figure 3 , Figure 3 : is a flowchart of a multimedia information processing method provided in an embodiment of the present application. The multimedia information processing method provided in an embodiment of the present application includes:

[0135] Step 101: The terminal presents a multimedia information recording interface for recording multimedia information.

[0136] Step 102: During the process of recording multimedia information based on the multimedia information recording interface, a gesture operation performed on the terminal is detected.

[0137] Here, the terminal may be provided with a client for recording multimedia information (such as audio and video), such as an audio recording client, a video recording client, an audio and video recording client, etc. In actual applications, when a user needs to record multimedia information, an operation instruction for a corresponding client may be triggered, and the terminal responds to the operation instruction, operates the client, and presents a multimedia information recording interface in a view interface, and the multimedia information recording interface is used to record the multimedia information.

[0138] When the terminal is recording multimedia information based on the multimedia information recording interface, the recording time corresponding to the recorded multimedia information can be presented in real time in the multimedia information recording interface. Figure 4 , Figure 4 is a schematic diagram of a multimedia information recording interface provided by an embodiment of the present application. Figure 4 As shown in Figure A, the recording duration corresponding to the audio is 00:05.62 in the audio recording interface; Figure 4 As shown in Figure B, the video recording interface displays the corresponding recording time of the video, 00:06.50, and also displays the video content during the video recording.

[0139] In an embodiment of the present application, in the process of recording multimedia information, the multimedia information can also be annotated based on the gesture operation performed on the terminal. The gesture operation can be a shake (which can be a shake forward and backward or a shake sideways), a left and right turn, etc., which is used to trigger the opening of the annotation function for the recorded multimedia information. When the gesture operation is detected, the recorded multimedia information is annotated. As an example, when the user considers a certain section of the recorded multimedia information to be a key and important content, the section of multimedia information can be annotated to facilitate subsequent quick search and viewing. Therefore, in the process of recording multimedia information based on the above-mentioned multimedia information recording interface, the gesture operation performed on the terminal can also be detected to determine whether the user has performed a gesture operation for triggering the annotation function on the terminal, thereby determining whether the recorded multimedia information needs to be annotated.

[0140] In some embodiments, the terminal may detect gesture operations performed on the terminal in the following manner: detecting the angular displacement of the terminal in various directions through an angular velocity sensor in the terminal; when it is detected that the angular displacement of the terminal in the target direction reaches an angular displacement threshold, it is determined that the target gesture operation performed on the terminal is detected.

[0141] Here, in actual application, when the terminal starts recording multimedia information, the built-in angular velocity sensor of the terminal can also be started at the same time, so as to detect whether the terminal has angular displacement through the angular velocity sensor. Specifically, the angular displacement of the terminal in various directions is detected through the angular velocity sensor, such as the angular displacement caused by shaking back and forth, the angular displacement caused by shaking left and right, etc.

[0142] In actual implementation, an angle displacement threshold that can trigger a target gesture operation can be pre-set. After detecting the angle displacement of the terminal in the target direction (such as turning left), it is determined whether the angle displacement reaches the set angle displacement threshold. When it is detected that the angle displacement of the terminal in the target direction reaches the angle displacement threshold, it is determined that the target gesture operation performed on the terminal is detected.

[0143] As an example, see Figure 5 , Figure 5 is a schematic diagram of a target gesture operation provided by an embodiment of the present application. Figure 5 As shown in Figure A, the target gesture operation is to shake the terminal back and forth; Figure 5 As shown in Figure B, the target gesture operation is to rotate the terminal vertically to the right; Figure 5 As shown in Figure C, the target gesture operation is to rotate the terminal vertically to the left.

[0144] In some embodiments, the terminal may present a marking function switch in the multimedia information recording interface; in response to a trigger operation on the marking function switch, the marking function for the recorded multimedia information is turned on;

[0145] Accordingly, the terminal may detect the gesture operation performed on the terminal in the following manner: when the annotation function for the recorded multimedia information is turned on, the gesture operation performed on the terminal is detected.

[0146] Here, the terminal may present a marking function switch in the multimedia information recording interface, and the marking function switch is used to turn on or off the marking function for the recorded multimedia information. When the marking function switch is in the on state, the marking function for the recorded multimedia information is turned on, and the marking mode is entered at this time; when the marking function switch is in the off state, the marking function for the recorded multimedia information is turned off, and the marking mode is exited at this time. In the embodiment of the present application, a marking function switch is provided to facilitate users to turn on or off the marking function for multimedia information as needed, thereby avoiding the problem of waste of resources caused by turning on the marking function in real time. For example, when multimedia information is recorded and no marking is required, there will always be waste of resources caused by real-time detection of gesture operations, real-time detection of angle displacement, etc.

[0147] The terminal responds to the trigger operation for the marking function item and turns on the marking function for the recorded multimedia information. At this time, when the marking function for the recorded multimedia information is turned on, the terminal detects the gesture operation performed on the terminal, thereby ensuring the reasonable use of hardware resources.

[0148] As an example, see Figure 6 , Figure 6 6 is a schematic diagram of presenting the annotation function items provided in the embodiment of the present application. Here, the terminal presents the annotation function switch 61 in the multimedia information recording interface 60, and in response to the triggering operation on the annotation function switch 61, the annotation function for the recorded multimedia information is turned on. At this time, when the annotation function for the recorded multimedia information is turned on, the terminal detects the gesture operation performed by the user on the terminal.

[0149] In some embodiments, after turning on the annotation function for recorded multimedia information, the terminal may present guidance information corresponding to the target gesture operation in the multimedia information recording interface; wherein the guidance information is used to guide the execution of the target gesture operation to annotate the recorded multimedia information.

[0150] In actual applications, after the terminal turns on the annotation function for recorded multimedia information, in order to guide the user to perform the corresponding target gesture operation to trigger the annotation operation for the multimedia information, the terminal presents the guidance information corresponding to the target gesture operation in the multimedia information recording interface, so as to guide the user to perform the target gesture operation based on the guidance information to annotate the recorded multimedia information. The guidance information can be a graphic (such as a schematic diagram of the target gesture operation) or text, such as "Please shake to annotate the recorded multimedia information"; it can also be a combination of text and graphics, etc.

[0151] As an example, see Figure 7 , Figure 7is a schematic diagram of presenting the guidance information provided by the embodiment of the present application. Figure 7 As shown in Figure A, the guidance information is a graphic, which is reflected by a schematic diagram of the target gesture operation (vertical rotation to the left); Figure 7 As shown in Figure B, the guidance information is the text "Please shake to mark the recorded multimedia information."

[0152] Step 103: When a target gesture operation for indicating to mark the multimedia information is detected, the recorded multimedia information is marked in response to the target gesture operation, and corresponding marking information is presented.

[0153] Here, after the terminal detects the gesture operation performed on the terminal, when a target gesture operation for indicating to mark the multimedia information is detected, the recorded multimedia information is marked in response to the target gesture operation, specifically, at the current moment when the target gesture operation is detected, the multimedia information recorded at the current moment is marked, such as displaying a marking mark on the recording progress bar of the multimedia information, associating and storing the marking information input by the user with the multimedia information at the current moment, etc. At the same time, the corresponding marking information, such as the name, content, label, etc. of the marking, is presented.

[0154] In some embodiments, the terminal may present a recording timeline corresponding to the multimedia information in the multimedia information recording interface; accordingly, the terminal may mark the recorded multimedia information and present the corresponding marking information in the following manner: obtain the operation time corresponding to the target gesture operation; mark the recorded multimedia information according to the operation time, and present the corresponding marking information at the position corresponding to the operation time on the recording timeline.

[0155] Here, when the terminal records multimedia information based on the multimedia information recording interface, it presents the recording timeline corresponding to the multimedia information, that is, the recording progress bar. At this time, the terminal obtains the operation time corresponding to the target gesture operation, and marks the recorded multimedia information corresponding to the operation time. When this marking contains corresponding marking content, the marking content can also be associated with the recorded multimedia information corresponding to the operation time. At the same time, the terminal can present the corresponding marking information at the position corresponding to the operation time on the recording timeline of the multimedia information, so as to indicate through the marking information that the recorded multimedia information corresponding to the operation time has been marked.

[0156] As an example, see Figure 8 , Figure 8Here, taking the multimedia information as audio as an example, when the terminal records audio based on the audio recording interface, it presents the recording timeline 81 corresponding to the audio to be recorded, and presents the corresponding annotation information 82 at the position corresponding to the operation time of the target gesture operation on the recording timeline.

[0157] In some embodiments, the terminal may present the corresponding annotation information in the following manner: in the multimedia information recording interface, an information display area for displaying the annotation information is presented, and in the information display area, the annotation information corresponding to the annotated multimedia information is displayed.

[0158] Here, the terminal may also present an information display area for displaying the marked information in the multimedia information recording interface, so that the marked information corresponding to the marked multimedia information is displayed in the information display area. Fig. 9 , Fig. 9 Here, taking the multimedia information as a video as an example, the terminal presents an information display area 91 in the video recording interface 90, and presents the annotation information 92 in the information display area 91, including "Annotation 1", "Annotation 2" and the like.

[0159] In some embodiments, the terminal may present corresponding annotation information in the following manner: when the number of annotation information is multiple and the number of annotation information reaches a target number, an information display area for displaying the annotation information is presented in the multimedia information recording interface; multiple annotation information are displayed in the information display area, and scroll bars corresponding to the multiple annotation information are presented on one side of the information display area; when a sliding operation on the scroll bar is received, the display positions of the multiple annotation information in the information display area are synchronously adjusted.

[0160] Here, the recorded multimedia information may contain a lot of annotation information, but the display resources of the terminal are limited. At this time, the maximum number of annotation information that can be displayed by the display resources of the terminal corresponding to the annotation information can be set, and the set maximum number is used as the target number.

[0161] When the number of annotated information is multiple and reaches the target number, an information display area for displaying the annotated information is presented in the multimedia information recording interface. At the same time, multiple annotated information is displayed in the information display area, specifically, the target number of annotated information can be displayed, and a scroll bar corresponding to the multiple annotated information is presented on one side of the information display area. In this way, when a sliding operation is received on the scroll bar, the display position of the multiple annotated information in the information display area can be synchronously adjusted, so as to display the annotated multiple annotated information.

[0162] As an example, see Fig.10 , Fig.10 : is a schematic diagram of presenting the annotation information provided by the embodiment of the present application. Here, the terminal presents an information display area for displaying the annotation information in the multimedia information recording interface. At this time, the number of annotation information is multiple, and the number of annotation information reaches the target number, the multiple annotation information is displayed in the information display area, and a scroll bar corresponding to the multiple annotation information is displayed on one side of the information display area, such as Fig.10 As shown in Figure A; when a sliding operation is received on the scroll bar, the display positions of the multiple annotation information in the information display area are adjusted synchronously, such as Fig.10 As shown in Figure B.

[0163] In some embodiments, the terminal may present the corresponding annotation information in the following manner: obtain the number of times the recorded multimedia information is annotated; determine the order corresponding to the current annotation of the recorded multimedia information based on the obtained number of times; generate annotation indication information for indicating the order, and present the annotation indication information as the annotation information corresponding to the current annotation of the recorded multimedia information.

[0164] Here, the terminal can also obtain the number of times the recorded multimedia information is marked, and thus determine the order corresponding to the current marking of the recorded multimedia information based on the number of times obtained. Specifically, the order corresponding to the current marking of the recorded multimedia information can be determined based on the number of times obtained and the time point corresponding to each marking. For example, the number of markings is 3 times, and the time points corresponding to each marking are 00:10:00, 00:20:00, and 00:30:00, respectively. Then the corresponding order can be 00:10:00 as the first place, 00:20:00 as the second place, and 00:30:00 as the third place.

[0165] Continuously, the annotation indication information for indicating the order is generated. For example, if the number of annotations is 3, and the time points corresponding to each annotation are 00:10:00, 00:20:00, and 00:30:00, respectively, the corresponding order may be 00:10:00 as the first place, 00:20:00 as the second place, and 00:30:00 as the third place, and the generated annotation indication information may be annotation 1, annotation 2, and annotation 3. At the same time, when presenting the annotation information, the annotation indication information is presented as the annotation information corresponding to the current annotation of the recorded multimedia information.

[0166] As an example, see Fig.11 , Fig.11This is a schematic diagram of presenting the annotation information provided by the embodiment of the present application. Here, the terminal obtains the number of times the recorded multimedia information is annotated 2 times, and determines the order corresponding to the current annotation of the recorded multimedia information according to the obtained number of times, thereby generating annotation indication information "Annotation 1" and "Annotation 2" for indicating the order, and presents the annotation indication information as the annotation information corresponding to the current annotation of the recorded multimedia information.

[0167] In some embodiments, when the annotation information is a annotation name of a corresponding annotation, the terminal may present a name edit box of the corresponding annotation name; based on the name edit box, the edited annotation name of the corresponding annotation is received;

[0168] Accordingly, the terminal may present the corresponding annotation information in the following manner: in response to a confirmation instruction for the edited annotation name, present the annotation name of the corresponding annotation.

[0169] Here, when multimedia information is annotated, the annotation information may be an annotation name of the corresponding annotation. At this time, the user may edit the annotation name, and the terminal presents a name editing box for the user to edit the annotation name, and the user may enter the annotation name of the corresponding annotation in the name editing box as needed. The terminal receives the edited annotation name of the corresponding annotation based on the name editing box. In actual applications, a default annotation name may also be presented in the name editing box, so that the user can edit the annotation name he needs as needed, or directly use the default annotation name.

[0170] When a determination instruction for the edited annotation name is received, the annotation name of the corresponding annotation is presented in response to the determination instruction.

[0171] As an example, see Fig.12 , Fig.12 Here, the annotation information is the annotation name of the corresponding annotation, and the terminal presents a name editing box for the corresponding annotation name, and based on the name editing box, receives the edited annotation name "Important Content" of the corresponding annotation, and presents the corresponding confirmation function item "Confirm" at the same time, such as Fig.12 As shown in Figure A; receiving a confirmation instruction for the edited annotation name triggered by the confirmation function item "Confirm", the corresponding annotation name "Important content, such as Fig.12 As shown in Figure B”.

[0172] In some embodiments, the terminal may present a modification function item corresponding to the annotation name; when a trigger operation for the modification function item is received, the name editing box is presented again; in response to the name modification operation triggered based on the name editing box, the modified annotation name is presented.

[0173] In actual applications, users can also modify the edited annotation name. The terminal presents a modification function item corresponding to the annotation name. When a trigger operation for the modification function item is received, the terminal presents a name editing box again, in which the user can modify the annotation name; in response to the name modification operation triggered by the name editing box, the modified annotation name is presented.

[0174] As an example, see Fig.13 , Fig.13 : is a schematic diagram of the modification process of the label name provided in the embodiment of the present application. Here, the terminal presents the modification function item "Modify" corresponding to the label name on the right side of the label name "Important Content", such as Fig.13 As shown in Figure A; in response to the trigger operation for "Modify", the name edit box is presented again, such as Fig.13 As shown in Figure B; receiving the name modification operation triggered by the name edit box, modify "Important Content" to "Important Information", such as Fig.13 As shown in Figure C.

[0175] In some embodiments, the terminal may present a marking button in the multimedia information recording interface; during the process of recording the multimedia information, receive a trigger operation for the marking button; in response to the trigger operation for the marking button, mark the recorded multimedia information and present corresponding marking information.

[0176] In actual applications, in addition to the target gesture operation-based triggering annotation function for multimedia information provided in the above embodiments, it can also be triggered by an annotation button. Here, the terminal presents an annotation button in the multimedia information recording interface, and the user can trigger the annotation function for multimedia information based on the annotation button; when the terminal receives a trigger operation for the annotation button during the multimedia information recording process, the terminal annotates the recorded multimedia information in response to the trigger operation for the annotation button, and presents the corresponding annotation information.

[0177] As an example, see Fig.14 , Fig.14 is a schematic diagram of presenting a marking button provided in an embodiment of the present application. Here, the terminal presents a marking button “mark” in the multimedia information recording interface, such as Fig.14 As shown in the first figure from the left; in response to the trigger operation of the annotation button "Annotation", the recorded multimedia information is annotated, and the corresponding annotation information is displayed at the corresponding position of the progress bar, such as Fig.14 Shown in the second picture from the left.

[0178] In some embodiments, the terminal may annotate the recorded multimedia information and present the corresponding annotation information in the following manner: when a target gesture operation for indicating the annotation of the multimedia information is detected, the operation time when the target gesture operation was last detected is obtained; when it is determined based on the obtained operation time and the current time that the corresponding time difference reaches a time difference threshold, the recorded multimedia information is annotated in response to the target gesture operation, and the corresponding annotation information is presented.

[0179] In some embodiments, the terminal may present a marking prompt message when it determines, based on the acquired operation time and current time, that the corresponding time difference does not reach the time difference threshold; wherein the marking prompt message is used to prompt that the current time cannot be marked for the multimedia information.

[0180] Here, in actual application, a time difference threshold between each annotation can be set. When the terminal detects a target gesture operation for indicating the annotation of multimedia information, it first obtains the operation time of the last time the target mobile phone operation was detected, and then determines whether the corresponding time difference reaches the time difference threshold based on the acquired operation time and the current time. When it is determined that the time difference reaches the time difference threshold, the recorded multimedia information is annotated in response to the target gesture operation, and the corresponding annotation information is presented. When it is determined that the time difference does not reach the time difference threshold, an annotation prompt message is presented to prompt the user that the multimedia information cannot be annotated at the current time through the annotation prompt message. In actual implementation, an annotation prompt message carrying the remaining waiting time can be presented. The remaining waiting time is the length of time between the current time and the annotation time when the terminal can perform the annotation operation, so as to facilitate the user to understand how long it will take to continue the annotation operation, and also prevent possible misoperations in a short time.

[0181] As an example, see Fig.15 , Fig.15 2 is a schematic diagram of presenting the annotation prompt information provided by the embodiment of the present application. Here, when the terminal determines that the corresponding time difference does not reach the time difference threshold, the annotation prompt information "Current time annotation is not available, please wait" is presented.

[0182] In some embodiments, the terminal may present the remaining number of markings for the recorded multimedia information in the multimedia information recording interface; when the remaining number of markings presented is zero and the target gesture operation is detected again, a marking deletion prompt message is presented; the marking deletion prompt message is used to prompt the deletion of the historical markings of the recorded multimedia information in response to the target gesture operation detected again.

[0183] Here, a corresponding number of annotations is set for the recorded multimedia information, that is, when annotating the recorded multimedia information, the number of annotations that can be made may be limited, and the number of annotations that can be made is the preset number of annotations. In actual applications, during the process of recording multimedia information, the terminal may present the remaining number of markings for the recorded multimedia information in the multimedia information recording interface. When the remaining number of markings presented is zero, if the target gesture operation is detected again at this time, the terminal may present an annotation deletion prompt message, thereby prompting the deletion of the historical annotations of the recorded multimedia information through the annotation deletion prompt message, in response to the target gesture operation detected again, and then making new annotations to the recorded multimedia information.

[0184] Continuing, when the terminal receives a deletion operation for a historical annotation, the remaining number of markings for the recorded multimedia information is updated according to the number of deleted historical annotations, for example, if the number of deleted historical annotations is 1, the remaining number of markings is increased by 1. When the presented remaining number of markings is not 0, the terminal marks the recorded multimedia information in response to the target gesture operation detected again, and presents the corresponding marking information.

[0185] As an example, see Fig.16 , Fig.16 is a schematic diagram of presenting the mark deletion prompt information provided by an embodiment of the present application. Here, the terminal obtains and presents the remaining mark times for the recorded multimedia information. At this time, the remaining mark times presented are 0. Fig.16 As shown in Figure A; in response to the target gesture operation triggered again, the annotation deletion prompt message "Please delete the historical annotation to re-annotate" is presented, such as Fig.16 As shown in Figure B.

[0186] In some embodiments, the terminal may present at least two importance level options corresponding to the annotation; in response to a selection operation for an importance level option, the importance level corresponding to the selected importance level option is used as the importance level of the annotated multimedia information;

[0187] Accordingly, the terminal may present the corresponding annotation information in the following manner: using a style corresponding to the importance level to present the annotation information corresponding to the annotation of the recorded multimedia information.

[0188] In actual application, the terminal can also provide for setting the importance level of each annotation. Specifically, the terminal can present at least two importance level options corresponding to the annotation, such as "low", "medium" and "high". The user can select the required importance level option as needed to serve as the importance level of the annotated multimedia information. When the terminal receives a selection operation for an importance level option, the importance level corresponding to the selected importance level option is used as the importance level of the annotated multimedia information.

[0189] As an example, see Fig.17 , Fig.17 1 is a schematic diagram of a selection process of the importance level of the annotated information provided in an embodiment of the present application. Here, when the terminal annotates the multimedia information, at least two importance level options "low", "medium" and "high" corresponding to the annotation are displayed, such as Fig.17 As shown in Figure A; in response to the selection operation of the importance level option "Medium", the importance level "Medium" corresponding to the selected importance level option "Medium" is used as the importance level of the annotated multimedia information, that is, the annotated information corresponding to the importance level "Medium" is displayed in a style corresponding to the importance level "Medium", such as using a yellow font, as shown in Figure A; Fig.17 As shown in Figure B.

[0190] In some embodiments, different target gesture operations may correspond to different annotation tags. The annotation tag may be used to indicate the importance level of the corresponding annotation. For example, if the target gesture operation is two shake operations, the corresponding annotation tag indicates that the importance level of the corresponding annotation is "intermediate", that is, when two shake operations are detected, the multimedia information is annotated and the importance level of the annotation is set to "intermediate"; if the target gesture operation is three shake operations, the corresponding annotation tag indicates that the importance level of the corresponding annotation is "high", that is, when three shake operations are detected, the multimedia information is annotated and the importance level of the annotation is set to "high". In addition, the annotation tag may also be used to indicate the prompt of the recording segmentation of the multimedia information, such as the prompt of audio segmentation (corresponding to the target gesture operation of two shake operations), that is, when two shake operations are detected, the recorded audio is annotated and segmented, and the corresponding segmentation mark can be output; it may also be a prompt of video segmentation (corresponding to the target gesture operation of three shake operations), that is, when three shake operations are detected, the recorded video is annotated and segmented, and the corresponding segmentation mark can be output.

[0191] In some embodiments, the target gesture operation can be used to trigger both the start instruction and the close instruction of the annotation function. Specifically, during the recording of multimedia information, when the terminal detects the target gesture operation, the annotation function is turned on, the annotation operation for the recorded multimedia information is started, and the corresponding start annotation information is generated; during the annotation process, when the terminal detects the target gesture operation again, the annotation function is turned off, the annotation operation for the recorded multimedia information is ended, and the corresponding end annotation information is generated. At the same time, between the start annotation information and the end annotation information, the annotation information corresponding to the recorded multimedia information is generated.

[0192] In some embodiments, after generating a recording file corresponding to the multimedia information, the terminal can play the multimedia file in the following manner: presenting a playback interface corresponding to the recording file, presenting a corresponding playback timeline in the playback interface, and presenting at least one annotation information on the playback timeline; in the process of playing the recording file, in response to a trigger operation on the annotation information, jumping the playback progress to the position in the playback timeline where the annotation information is located, so as to play the recording file based on the position.

[0193] Here, after the multimedia information is recorded, a recording file corresponding to the multimedia information is generated. When the terminal plays the multimedia file, that is, when the annotated multimedia information is played, the terminal presents the playback interface corresponding to the recording file, and presents the corresponding playback timeline in the playback interface. In actual applications, at least one annotation information is presented on the playback timeline, that is, the annotation information annotated during the multimedia information recording process. During the process of playing the recorded file, when the terminal receives a trigger operation for the annotation information, it responds to the trigger operation for the annotation information and jumps from the current playback progress to the position in the playback timeline where the annotation information is located, thereby playing the recorded file based on the position.

[0194] As an example, see Fig.18 , Fig.18 Here, the terminal presents a playback interface corresponding to the recorded file, in which a corresponding playback timeline is presented, and three annotation information are presented on the playback timeline: annotation 1, annotation 2, and annotation 3, such as Fig.18 As shown in Figure A; during the playback process, in response to the trigger operation for the annotation information "Annotation 3", the playback progress jumps to the position of the annotation information "Annotation 3" in the playback timeline, so as to play the recorded file based on the position, such as Fig.18 As shown in Figure B.

[0195] In some embodiments, after generating a recording file corresponding to multimedia information, the terminal can play the multimedia file in the following manner: presenting a playback interface corresponding to the recording file, in which at least one annotation information corresponding to the recording file is presented, and a play button corresponding to each annotation information is presented; in response to a trigger operation on the play button, obtaining the annotation time of the recording file corresponding to the corresponding annotation information; jumping the playback progress to the annotation time to play the recording file based on the annotation time.

[0196] Here, after the multimedia information is recorded, a recording file corresponding to the multimedia information is generated. When the terminal plays the multimedia file, that is, when the annotated multimedia information is played, the terminal presents a playback interface corresponding to the recording file, and at the same time presents at least one annotation information corresponding to the recording file and a play button corresponding to each annotation information in the playback interface. The user can quickly play the recording file corresponding to the corresponding annotation information based on the play button. When a trigger operation for the play button is received, in response to the trigger operation for the play button, the annotation information corresponding to the play button and the annotated time corresponding to the recording file are obtained, so as to jump from the current playback progress to the playback progress corresponding to the annotated time, so as to continue playing the recording file based on the annotated time.

[0197] As an example, see Fig.19 , Fig.19 Here, the terminal presents a playback interface corresponding to the recorded file, in which a corresponding playback timeline is presented, and three annotation information are presented on the playback timeline: annotation 1, annotation 2, and annotation 3, and a play button corresponding to each annotation information is presented, such as Fig.19 As shown in Figure A; in response to the triggering operation of the play button corresponding to "mark 3", the marked time "00:50:30" of the recording file corresponding to the corresponding marking information is obtained, so as to jump to the marked time "00:50:30" to play the recording file based on the marked time, such as Fig.19 As shown in Figure B.

[0198] By applying the above-mentioned embodiments of the present application, during the process of recording multimedia information based on the multimedia information recording interface, the terminal detects the gesture operation performed on the terminal. When a target gesture operation for indicating the marking of the multimedia information is detected, the recorded multimedia information is marked in response to the target gesture operation, and the corresponding marking information is presented. In this way, during the recording of the multimedia information, the multimedia information can be marked based on the target gesture operation, and the operation is simple, so that the key points in the recorded multimedia information can be recorded. When it is necessary to obtain the key content in the multimedia information, the corresponding marking nodes can be quickly and accurately located based on the marking information, without the need to manually search one by one, thereby improving efficiency.

[0199] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0200] For some critical occasions, the key content information is generally recorded by recording multimedia information (such as video and audio) throughout the process. However, when reviewing or checking the key content, either all the recorded multimedia information is played back, or the key information points are repeatedly searched and located, which is cumbersome and time-consuming, resulting in a waste of human resources and no guarantee of accurate finding of important content.

[0201] Based on this, the embodiment of the present application provides a multimedia information annotation scheme, which comprehensively considers the pain points of recording important information in the use of video / audio recording, optimizes the function of video / audio recording, and adds an annotation function during the video / audio recording process. The annotation function can be started by triggering the "annotation button" on the video / audio recording page or by shaking the phone and other gesture operations. The name of the annotation node can be customized to facilitate memory, and the recorded video / audio of important information can also be quickly queried. In this way, when playing back the video, you can quickly locate the exact node you want to obtain, without having to manually find the exact time point one by one; by recording the name of the standard point, you can find the key point of the video by name when playing back the video, and there is no need to search blindly; and through the angular velocity sensor of the terminal, you can quickly annotate the key nodes of the video by shaking it, effectively improving work efficiency.

[0202] Taking the multimedia information as video as an example, the multimedia information annotation function process is as follows: Fig. 20 As shown, Fig. 20 This is a schematic diagram of the multimedia information annotation process provided by the embodiment of the present application, including:

[0203] Step 201: Recording a video;

[0204] Step 202: Click to select a marking point;

[0205] Step 203: Synchronize the background to start the mobile phone angular velocity sensor;

[0206] Step 204: whether it is necessary to mark the key nodes of the video, if yes, go to step 205, if no, go to step 206;

[0207] Step 205: Start the annotation function through the angular velocity sensor / APP page;

[0208] Step 206: End recording;

[0209] Step 207: Whether the annotation content needs to be edited; if so, execute step 208; if not, execute step 206;

[0210] Step 208: Edit the annotation content;

[0211] Step 209: Save the edited content and return to step 206;

[0212] Step 210: Turn off the mobile phone angular velocity sensor.

[0213] Taking the multimedia information as a video as an example, the multimedia information playback process is as follows: Fig.21 As shown, Fig.21 This is a schematic diagram of the playback process of multimedia information provided by an embodiment of the present application, including:

[0214] Step 301: Play the recorded video;

[0215] Step 302: whether it is necessary to start playing the video from the marked point, if yes, go to step 304, if no, go to step 303;

[0216] Step 303: the video ends playing;

[0217] Step 304: Select and play the marked video node;

[0218] Step 305: Whether the annotation content needs to be edited, if yes, execute step 306, if no, execute step 303;

[0219] Step 306|: Edit the annotation content;

[0220] Step 307: Save the edited annotation content.

[0221] Next, the multimedia information processing method provided in the embodiment of the present application is first described from the product side.

[0222] First, taking multimedia information as video as an example, quick annotation is performed when recording the video.

[0223] Among them, the annotation function is started through the page annotation button, including: 1. Open the video recording app and start the video recording program; 2. Click the "Annotation" button in the video recording program page to start the annotation function; 3. After the annotation function is started, the annotation mark will be displayed on the video recording progress bar; and the default annotation name will be displayed: for example, "Annotation 1", and the annotation can be customized and deleted during the video recording process. 4. During the video recording process, the annotation function can be used an unlimited number of times. Each time the annotation function is started, a marked record will be added to the page. 5. When there are more than three annotation records, the page will display a dynamic panel to display the annotation records, and you can pull down to view all annotation records.

[0224] In addition to the video annotation function, a new node annotation method is introduced, and the angular velocity sensor can be used to start the annotation function of the recorded video. The annotation function is started by shaking and other target gesture operations, including: 1. When the video recording function is started, the mobile phone background will synchronously start the built-in angular velocity sensor of the mobile phone; 2. When the video recording function is started, the annotation function is started when the mobile phone is lifted forward and backward; 3. When the video recording function is started, the annotation function is started when the mobile phone is moved vertically to the right; 4. When the video recording function is started, the annotation function is started when the mobile phone is moved vertically to the left; Here, when the angular velocity sensor is used, the scene defaults to the mobile phone being in the off-screen state. When the mobile phone is in the on-screen state, the annotation function can also be started through the angular velocity sensor.

[0225] Second, for video files recorded using the annotation function, the playback of the annotated nodes in the recorded video files includes: 1. When you need to play the entire recorded video, click the play icon to play the entire recorded video; 2. When you need to play the video recorded at the annotated node, click the "Play" button corresponding to the annotation name to start playing the recorded video at the annotated point; 3. During the playback of the entire recorded video, if you click the "Play" button corresponding to the annotation name, you can jump to the annotated point and continue to play the recorded video.

[0226] Next, the multimedia information processing method provided by the embodiment of the present application will be described from the technical side. Fig. 22 As shown, taking the multimedia information as video as an example, Fig. 22 This is a schematic diagram of the annotation process of multimedia information provided by an embodiment of the present application, which quickly starts annotation, including:

[0227] Step 401: The terminal starts video recording;

[0228] Step 402: in response to a click operation on a "mark" button in the video recording interface, start the marking function and send a start instruction to the background server;

[0229] Step 403: The backend server determines whether the marking function has been activated 5 seconds ago. If so, execute step 404; if not, execute step 405;

[0230] Step 404: Notify the terminal not to start the marking function;

[0231] Step 405: create a marked node at the time of triggering, generate a marked node name and a corresponding node ID, and send the node ID to the terminal;

[0232] Step 406: The terminal creates a marking icon on the presented recording track at the time point corresponding to when the marking function 1 is started;

[0233] Step 407: The terminal displays the label name, an edit button, and a delete button;

[0234] Step 408: The terminal edits the annotation name and sends the edited name to the backend server;

[0235] Step 409: the backend server modifies the labeled name of the node and saves it;

[0236] Step 410: The terminal deletes the annotation, and the page does not display the annotation name and the annotation icon;

[0237] Step 411: The backend server deletes the node corresponding to the annotation.

[0238] Specifically, when the terminal starts the annotation function, it sends a startup instruction to the background; after receiving the instruction, the background (i.e., the background server) determines whether the instruction to start the annotation function has been executed within 5 seconds. If it has not been started, the annotation function will be started, otherwise it will not be started; after receiving the instruction to start the annotation function, the background will create a node at the time of the trigger. The node name "Annotation 1" and the corresponding node ID are generated by default; after creating the ID, send the ID to the front end, and the front end will create an annotation icon on the track (i.e., timeline) of the recorded video at the time of starting the annotation and display the annotation name and edit and delete buttons; use the edit function on the front end, after modifying the annotation name, the front end sends the modified name and the corresponding ID to the background, and the background receives the ID to modify and save the corresponding annotation name; use the delete function on the front end, the front end sends the deleted ID to the background, and the background deletes it accordingly after receiving the instruction.

[0239] like Fig.23 As shown, taking the multimedia information as video as an example, Fig.23 : is a schematic diagram of the multimedia information annotation process provided by an embodiment of the present application, wherein the shake annotation is realized by an angular velocity sensor, including:

[0240] Step 501, start recording;

[0241] Step 502: Start the angular velocity sensor through the API;

[0242] Step 503, determine whether the terminal has an angular displacement, if so, execute step 505, if not, execute step 504;

[0243] Step 504: do not start the marking function;

[0244] Step 505, determine whether the terminal displacement angle changes by more than 30 degrees, if so, execute step 507, if not, execute step 504;

[0245] Step 506: Start the annotation function.

[0246] Specifically, when the terminal starts the video recording function, the angular velocity sensor is simultaneously started through the API; the angular velocity sensor determines whether the mobile phone has an angular displacement and whether the displacement angle is greater than 30 degrees. If a displacement occurs and is greater than 30 degrees, the marking function is started, otherwise it is not started.

[0247] like Fig.24 As shown, Fig.24 This is a schematic diagram of the playback process of multimedia information provided by an embodiment of the present application, and the nodes marked with the playback video include:

[0248] Step 601: After the terminal finishes recording the video, it clicks the play button on the page, and the front end sends the node ID to the back end; Step 602: After the back end identifies the sent ID, it returns the video data corresponding to the node ID to the terminal; Step 603: The terminal receives the video data and starts playing the recorded video from the time point of the node.

[0249] By applying the above embodiments of the present application, when playing back the video, the exact node that you want to obtain can be quickly located without having to manually search for the exact time point one by one; by recording the names of the marked points, the key points of the video can be found by name when playing back the video, and there is no need to search blindly; this technology uses the angular velocity sensor of the mobile phone to quickly mark the key nodes of the video, effectively improving work efficiency.

[0250] The following is a description of an exemplary structure of a multimedia information processing device 555 provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the multimedia information processing device 555 of the memory 550 may include:

[0251] The presentation module 5551 is used to present a multimedia information recording interface for recording multimedia information;

[0252] A detection module 5552, configured to detect a gesture operation performed on the terminal during recording of multimedia information based on the multimedia information recording interface;

[0253] The marking module 5553 is used to mark the recorded multimedia information and present corresponding marking information in response to the target gesture operation when a target gesture operation for indicating marking the multimedia information is detected.

[0254] In some embodiments, the detection module 5552 is further configured to detect the angular displacement of the terminal in various directions through an angular velocity sensor in the terminal;

[0255] When it is detected that the angular displacement of the terminal in the target direction reaches an angular displacement threshold, it is determined that the target gesture operation performed on the terminal is detected.

[0256] In some embodiments, the presenting module 5551 is further used to present a marking function switch in the multimedia information recording interface;

[0257] In response to a triggering operation on the marking function switch, turning on a marking function for the recorded multimedia information;

[0258] The detection module 5552 is further configured to detect a gesture operation performed on the terminal when a marking function for the recorded multimedia information is enabled.

[0259] In some embodiments, the presenting module 5551 is further used to present guidance information corresponding to the target gesture operation in the multimedia information recording interface;

[0260] The guide information is used to guide the execution of the target gesture operation so as to mark the recorded multimedia information.

[0261] In some embodiments, the presenting module 5551 is further used to present a recording timeline of the corresponding multimedia information in the multimedia information recording interface;

[0262] The marking module 5553 is further used to obtain the operation time corresponding to the target gesture operation;

[0263] The recorded multimedia information is annotated according to the operation time, and corresponding annotation information is presented at a position corresponding to the operation time on the recording time axis.

[0264] In some embodiments, the annotation module 5553 is further used to present an information display area for displaying the annotation information in the multimedia information recording interface.

[0265] In the information display area, the marked information corresponding to the marked multimedia information is displayed.

[0266] In some embodiments, the annotation module 5553 is further configured to present an information display area for displaying the annotation information in the multimedia information recording interface when the number of the annotation information is multiple and the number of the annotation information reaches a target number;

[0267] Displaying the plurality of annotation information in the information display area, and presenting scroll bars corresponding to the plurality of annotation information on one side of the information display area;

[0268] When a sliding operation on the scroll bar is received, the display positions of the plurality of annotation information in the information display area are synchronously adjusted.

[0269] In some embodiments, the marking module 5553 is further used to obtain the number of times the recorded multimedia information is marked;

[0270] Determine the order corresponding to the current annotation of the recorded multimedia information according to the number of acquisitions;

[0271] Generate annotation indication information for indicating the sorting, and present the annotation indication information as annotation information corresponding to the current annotation of the recorded multimedia information.

[0272] In some embodiments, when the annotation information is a annotation name corresponding to the annotation, the presenting module 5551 is further used to present a name editing box corresponding to the annotation name;

[0273] Based on the name edit box, receiving the edited annotation name corresponding to the annotation;

[0274] The presenting of corresponding annotation information includes:

[0275] In response to a determination instruction for the edited annotation name, a annotation name corresponding to the annotation is presented.

[0276] In some embodiments, the presenting module 5551 is further used to present a modification function item corresponding to the marked name;

[0277] When a trigger operation for the modification function item is received, presenting the name edit box again;

[0278] In response to a name modification operation triggered based on the name edit box, a modified annotation name is presented.

[0279] In some embodiments, the presenting module 5551 is further used to present a marking button in the multimedia information recording interface;

[0280] During the multimedia information recording process, a trigger operation for the marking button is received;

[0281] In response to the triggering operation on the marking button, the recorded multimedia information is marked and corresponding marking information is presented.

[0282] In some embodiments, the marking module 5553 is further used to obtain the operation time when the target gesture operation for indicating marking the multimedia information is detected;

[0283] When it is determined according to the acquired operation time and current time that the corresponding time difference reaches a time difference threshold, the recorded multimedia information is annotated in response to the target gesture operation, and corresponding annotation information is presented.

[0284] In some embodiments, the presenting module 5551 is further configured to present a marking prompt message when it is determined that the corresponding time difference does not reach a time difference threshold value according to the acquired operation time and current time;

[0285] The marking prompt information is used to prompt that the multimedia information cannot be marked at the current time.

[0286] In some embodiments, the presenting module 5551 is further used to present the remaining marking times for the recorded multimedia information in the multimedia information recording interface;

[0287] When the remaining number of markings presented is zero and the target gesture operation is detected again, a marking deletion prompt message is presented;

[0288] The annotation deletion prompt information is used to prompt the deletion of the historical annotation of the recorded multimedia information in response to the target gesture operation detected again.

[0289] In some embodiments, the presenting module 5551 is further used to present at least two importance level options corresponding to the annotation;

[0290] In response to a selection operation on the importance level option, taking the importance level corresponding to the selected importance level option as the importance level of the marked multimedia information;

[0291] The marking module 5553 is further used to present the marking information corresponding to the marking of the recorded multimedia information in a style corresponding to the importance level.

[0292] In some embodiments, the apparatus further comprises:

[0293] A playback module, used for presenting a playback interface corresponding to the recording file, presenting a corresponding playback timeline in the playback interface, and presenting at least one of the annotation information on the playback timeline;

[0294] In the process of playing the recorded file, in response to the trigger operation on the annotation information, the playback progress is jumped to the position of the annotation information in the playback timeline, so as to

[0295] Based on the position, the recorded file is played.

[0296] In some embodiments, the playback module is further configured to present a playback interface corresponding to the recording file, wherein at least one annotation information corresponding to the recording file is presented in the playback interface, and

[0297] Presenting a play button corresponding to each of the annotation information;

[0298] In response to a triggering operation on the play button, obtaining a marking time of the recording file corresponding to corresponding marking information;

[0299] Jump the playing progress to the marked time to play the recorded file based on the marked time.

[0300] By applying the above-mentioned embodiments of the present application, during the process of recording multimedia information based on the multimedia information recording interface, the terminal detects the gesture operation performed on the terminal. When a target gesture operation for indicating the marking of the multimedia information is detected, the recorded multimedia information is marked in response to the target gesture operation, and the corresponding marking information is presented. In this way, during the recording of the multimedia information, the multimedia information can be marked based on the target gesture operation, and the operation is simple, so that the key points in the recorded multimedia information can be recorded. When it is necessary to obtain the key content in the multimedia information, the corresponding marking nodes can be quickly and accurately located based on the marking information, without the need to manually search one by one, thereby improving efficiency.

[0301] The present application also provides an electronic device, the electronic device comprising:

[0302] A memory for storing executable instructions;

[0303] The processor is used to implement the multimedia information processing method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0304] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the multimedia information processing method provided in the embodiment of the present application.

[0305] The embodiment of the present application also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implements the multimedia information processing method provided in the embodiment of the present application.

[0306] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0307] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0308] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0309] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0310] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A multimedia information processing method, characterized in that, the method includes: The terminal presents a multimedia information recording interface for recording multimedia information; During the process of recording multimedia information based on the multimedia information recording interface, detect the gesture operations performed on the terminal; When a target gesture operation for indicating annotation of multimedia information is detected, obtain the operation time when the target gesture operation was last detected; When it is determined that the corresponding time difference reaches a time difference threshold according to the obtained operation time and the current time, in response to the target gesture operation, annotate the recorded multimedia information, obtain the number of times of annotating the multimedia information, determine the corresponding sorting for the current annotation of the multimedia information according to the number of annotations, generate annotation indication information for indicating the sorting, and use the annotation indication information as the corresponding annotation information when annotating the multimedia information, and present the corresponding annotation information; when it is determined that the time difference does not reach the time difference threshold, present annotation prompt information carrying the remaining waiting duration; wherein, different target gesture operations correspond to different annotation tags, and the annotation tags are used to indicate the importance level of the corresponding annotation information; the annotation prompt information is used to prompt that the multimedia information cannot be annotated at the current time, and the remaining waiting duration is the time length between the current time and the execution time when the terminal can perform the annotation operation; Present the remaining number of marks for the recorded multimedia information on the multimedia information recording interface; when the remaining number of marks is zero and the target gesture operation is detected again, present annotation deletion prompt information, and the annotation deletion prompt information is used to prompt deleting the historical annotation of the recorded multimedia information in response to the detected target gesture operation again; When the terminal receives a deletion operation for the historical annotation, update the remaining number of marks for the recorded multimedia information according to the number of the deleted historical annotations.

2. The method according to claim 1, characterized in that, the detection of the gesture operations performed on the terminal includes: Detect the angular displacement of the terminal in each direction through an angular velocity sensor in the terminal; When it is detected that the angular displacement of the terminal in the target direction reaches an angular displacement threshold, determine that the target gesture operation performed on the terminal is detected.

3. The method according to claim 1, characterized in that, the method further includes: Present a marking function switch in the multimedia information recording interface; In response to a trigger operation for the marking function switch, turn on the marking function for the recorded multimedia information; the detection of the gesture operations performed on the terminal includes: When the marking function for the recorded multimedia information is turned on, detect the gesture operations performed on the terminal.

4. The method according to claim 3, characterized in that, after turning on the marking function for the recorded multimedia information, the method further includes: In the multimedia information recording interface, presenting guidance information corresponding to the target gesture operation; The guide information is used to guide the execution of the target gesture operation so as to mark the recorded multimedia information.

5. The method according to claim 1, It is characterized in that The method further comprises: In the multimedia information recording interface, a recording time axis corresponding to the multimedia information is presented; The step of marking the recorded multimedia information and presenting corresponding marking information includes: Obtaining the operation time corresponding to the target gesture operation; The recorded multimedia information is annotated according to the operation time, and corresponding annotation information is presented at a position corresponding to the operation time on the recording time axis.

6. The method according to claim 1, It is characterized in that The presenting of corresponding annotation information includes: When the number of the annotation information is multiple and the number of the annotation information reaches the target number, presenting an information display area for displaying the annotation information in the multimedia information recording interface; Displaying the plurality of the annotation information in the information display area, and presenting scroll bars corresponding to the plurality of the annotation information on one side of the information display area; When a sliding operation on the scroll bar is received, the display positions of the plurality of annotation information in the information display area are synchronously adjusted.

7. The method according to claim 1, It is characterized in that When the annotation information is a annotation name corresponding to the annotation, the method further includes: Presenting a name edit box corresponding to the annotation name; Based on the name edit box, receiving the edited annotation name corresponding to the annotation; The presenting of corresponding annotation information includes: In response to a determination instruction for the edited annotation name, a annotation name corresponding to the annotation is presented.

8. The method according to claim 7, It is characterized in that The method further comprises: Presenting a modification function item corresponding to the marked name; When a trigger operation for the modification function item is received, presenting the name edit box again; In response to a name modification operation triggered based on the name edit box, a modified annotation name is presented.

9. The method according to claim 1, It is characterized in that The method further comprises: Presenting at least two importance level options corresponding to the annotation; In response to a selection operation on the importance level option, taking the importance level corresponding to the selected importance level option as the importance level of the marked multimedia information; The presenting of corresponding annotation information includes: The marking information corresponding to the marking of the recorded multimedia information is presented in a style corresponding to the importance level.

10. The method according to claim 1, It is characterized in that After generating a recording file corresponding to the multimedia information, the method further includes: Presenting a playback interface corresponding to the recorded file, presenting a corresponding playback timeline in the playback interface, and presenting at least one of the annotation information on the playback timeline; During the process of playing the recorded file, in response to a trigger operation on the annotation information, the playback progress is jumped to the position in the playback timeline where the annotation information is located, so as to play the recorded file based on the position.

11. The method according to claim 1, It is characterized in that After generating a recording file corresponding to the multimedia information, the method further includes: presenting a playback interface corresponding to the recording file, wherein at least one annotation information corresponding to the recording file is presented in the playback interface, and Presenting a play button corresponding to each of the annotation information; In response to a triggering operation on the play button, obtaining a marking time of the recording file corresponding to corresponding marking information; Jump the playing progress to the marked time to play the recorded file based on the marked time.

12. A multimedia information processing device, It is characterized in that Set at a terminal, the device includes: A presentation module, used to present a multimedia information recording interface for recording multimedia information; A detection module, used to detect the gesture operation performed on the terminal during the process of recording multimedia information based on the multimedia information recording interface; A marking module, used for, when a target gesture operation for indicating marking multimedia information is detected, obtaining the operation time of the last time the target gesture operation was detected; when it is determined based on the obtained operation time and the current time that the corresponding time difference reaches a time difference threshold, in response to the target gesture operation, marking the recorded multimedia information, and obtaining the number of times the multimedia information is marked, determining the order corresponding to the current marking of the multimedia information based on the number of times the marking is performed, generating marking indication information for indicating the order, and using the marking indication information as the corresponding marking information when marking the multimedia information, and presenting the corresponding marking information; when it is determined that the time difference does not reach the time difference threshold, presenting marking prompt information carrying the remaining waiting time; wherein different target gesture operations correspond to different marking tags, and the marking tags are used to indicate the importance level of the corresponding marking information; the marking prompt information is used to prompt that the multimedia information cannot be marked at the current time, and the remaining waiting time is the length of time between the current time and the execution time when the terminal can perform the marking operation; The presentation module is also used to present the remaining number of markings for the recorded multimedia information on the multimedia information recording interface; when the remaining number of markings is zero and the target gesture operation is detected again, present a marking deletion prompt message, and the marking deletion prompt message is used to prompt the deletion of the historical markings for the recorded multimedia information in response to the target gesture operation detected again; when the terminal receives a deletion operation for the historical markings, the remaining number of markings for the recorded multimedia information is updated according to the number of the deleted historical markings.

13. An electronic device, It is characterized in that The electronic device comprises: A memory for storing executable instructions; The processor is used to implement the multimedia information processing method according to any one of claims 1 to 11 when executing the executable instructions stored in the memory.

14. A computer-readable storage medium, It is characterized in that Executable instructions are stored, and when the executable instructions are executed, they are used to implement the multimedia information processing method according to any one of claims 1 to 11.

15. A computer program product comprising computer executable instructions or a computer program, It is characterized in that When the computer executable instructions or computer programs are executed by a processor, the multimedia information processing method according to any one of claims 1 to 11 is implemented.

Images