Information processing method and device, electronic equipment, storage medium and program product

By displaying multimedia information and its associated information, intercepting and assembling processing target frames, and generating information integration results, the multimedia information integration problem in the prior art is solved, and the information intensification and interaction efficiency are improved.

CN118921504APending Publication Date: 2024-11-08TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310505980.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-06
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The prior art is difficult to effectively record and integrate multimedia information with its associated information, resulting in users being unable to perceive more relevant related information when viewing, and the operation is cumbersome and complicated.

Method used

Provide an information processing method and device, by displaying multimedia information and its associated information, displaying target association information in response to acquisition operations, intercepting target frames, and assembling the summary content and associated information generated based on target frames, and generating information integration results .

Benefits of technology

It improves the degree of information intensification and the richness of the integration results, realizes flexible information selection and assembly, simplifies the operation process, and enhances the efficiency of human-computer interaction and the convenience of information sharing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118921504A_ABST
    Figure CN118921504A_ABST
Patent Text Reader

Abstract

The invention provides an information processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps that information to be collected corresponding to multimedia information is displayed, and the information to be collected comprises the multimedia information and associated information of the multimedia information; in response to an acquisition operation for the associated information, displaying that the acquired target associated information is in a selected state, and in response to an interception operation for the multimedia information, acquiring an intercepted target frame in the multimedia information; and in response to an information integration operation for the multimedia information, displaying an information integration result of the multimedia information, the information integration result being obtained by splicing abstract content generated based on the target frame and the target associated information. According to the invention, the information integration efficiency and the integration flexibility can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to human-computer interaction technology, and in particular to an information processing method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of information technology, more and more multimedia information will be publicly released on the Internet. A lot of related information will be generated for the released multimedia information. Sometimes users will record or share valuable multimedia information.

[0003] However, multimedia information is often accompanied by related information, such as comments, author information, etc. The relevant technology can transfer, save, and collect multimedia information, but it is difficult to record more related information related to the multimedia information. Therefore, subsequent users can only view the multimedia information itself when viewing it, but cannot perceive more related information. Summary of the invention

[0004] The embodiments of the present application provide an information processing method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the efficiency and flexibility of information integration.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application embodiment provides a method, including:

[0007] Displaying information to be collected corresponding to the multimedia information, wherein the information to be collected includes the multimedia information and associated information of the multimedia information;

[0008] In response to the collection operation on the associated information, displaying the collected target associated information as being in a selected state, and in response to the interception operation on the multimedia information, acquiring the intercepted target frame in the multimedia information;

[0009] In response to the information integration operation for the multimedia information, an information integration result of the multimedia information is displayed, wherein the information integration result is obtained by assembling the summary content generated based on the target frame and the target associated information.

[0010] The present application provides an information processing device, including:

[0011] A display module, used for displaying the information to be collected corresponding to the multimedia information, wherein the information to be collected includes the multimedia information and the associated information of the multimedia information;

[0012] A collection module, configured to, in response to a collection operation on the associated information, display the collected target associated information as being in a selected state, and, in response to an interception operation on the multimedia information, obtain an intercepted target frame in the multimedia information;

[0013] The integration module is used to display the information integration result of the multimedia information in response to the information integration operation for the multimedia information, wherein the information integration result is obtained by assembling the summary content generated based on the target frame and the target associated information.

[0014] An embodiment of the present application provides an electronic device, including:

[0015] A memory for storing computer executable instructions;

[0016] The processor is used to implement the information processing method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0017] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for causing a processor to execute the information processing method provided in the embodiment of the present application.

[0018] An embodiment of the present application provides a computer program product, including computer executable instructions, which, when executed by a processor, implement the information processing method provided in the embodiment of the present application.

[0019] The embodiments of the present application have the following beneficial effects:

[0020] Display multimedia information corresponding to the multimedia information and associated information of the multimedia information; in response to a collection operation on the associated information, display the collected target associated information in a selected state; in response to an interception operation on the multimedia information, obtain the intercepted target frame in the multimedia information; in response to an information integration operation on the multimedia information, assemble the summary content generated based on the target frame and the target associated information to obtain an information integration result; through the embodiments of the present application, the associated information can be flexibly selected and the multimedia information can be intercepted, thereby achieving flexibility in information assembly; since the summary content is generated based on the target frame, the information concentration can be improved; and since the information integration result is obtained by assembling the summary content and the target associated information, the information richness of the information integration result can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a schematic diagram of the structure of the information processing system provided by the embodiment of the present application;

[0022] Figure 2is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0023] Figure 3A-3H It is a flowchart of the information processing method provided in the embodiment of the present application;

[0024] Figure 4A-4B It is a schematic diagram of information sharing provided in the related art;

[0025] Figure 5A-Figure 5B It is a schematic diagram of the interface of the information processing method provided in the embodiment of the present application;

[0026] Figure 6A-6E It is a schematic diagram of the interface of the information processing method provided in the embodiment of the present application;

[0027] Figure 7A-7B It is a schematic diagram of the interface of the information processing method provided in the embodiment of the present application;

[0028] Figure 8A-8B It is a schematic diagram of the interface of the information processing method provided in the embodiment of the present application;

[0029] Figure 9A-9B It is a schematic diagram of the interface of the information processing method provided in the embodiment of the present application;

[0030] Figure 10A-10B It is a schematic diagram of the interface of the information processing method provided in the embodiment of the present application;

[0031] Fig.11 It is a technical architecture diagram of the information processing method provided in the embodiment of the present application;

[0032] Fig.12 It is a schematic diagram of the overall workflow of the information processing method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0034] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0035] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0037] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0038] 1) Multimedia information: The multimedia information involved in the embodiments of the present application may be audio information, video information, or image information.

[0039] 2) Related information: information that has an interactive relationship with multimedia information. The related information has multiple dimensions, for example, the dimension of the object that uploaded the multimedia information, the interactive dimension of the multimedia information, and the custom dimension of the multimedia information for the object with integration requirements;

[0040] 3) Target frame: The target frame is at least a portion of the data frame intercepted from the multimedia information. For example, for image multimedia information, the target frame can be the entire image or a portion of the image. For audio multimedia information, the target frame can be a portion of the audio frame in the audio. For video multimedia information, the target frame can be a portion of the video frame in the video.

[0041] See also Figure 4A and Figure 4B In the related technology, when users share videos, they usually share them by forwarding video links or taking screenshots. These two forms only share the video itself, and cannot meet the needs of efficient sharing of additional information integration, such as sharing a collection of related information elements of the work, which includes author information, key pictures of the work, selected comments, and impressions after watching the video. The user wants to display an excerpt of the key content of a video and attach his or her own impressions of the video. It is necessary to download the video first, perform secondary editing and assembly, and then disseminate the impressions in the form of dubbing, subtitles, or postscript sharing. The overall process is cumbersome and complicated.

[0042] The embodiments of the present application provide an information processing method, device, electronic device, computer-readable storage medium and computer program product, which can optimize the display effect of various information in a human-computer interaction interface and the human-computer interaction efficiency.

[0043] The following describes an exemplary application of the electronic device provided by the embodiment of the present application. The electronic device provided by the embodiment of the present application can be implemented as various types of user terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), and a vehicle-mounted terminal, and can also be implemented as a server. The following describes an exemplary application when the device is implemented as a terminal or a server.

[0044] refer to Figure 1 , Figure 1 Schematic diagram of the application mode of the information processing method provided in the embodiment of the present application; for example, the server involved includes: server 200, network 300 and terminal 400. Terminal 400 is connected to server 200 via network 300, and network 300 can be a wide area network or a local area network, or a combination of the two.

[0045] For example, the server 200 may be a video sharing server, and the user may be a user using video sharing software.

[0046] The terminal 400 obtains the video from the server and displays the information to be collected corresponding to the video in the terminal 400, wherein the information to be collected includes the video and the related information of the video; the terminal 400 receives the collection operation for the related information triggered by the user, and displays that the collected target related information is in a selected state, that is, the selected target related information; the terminal 400 receives the interception operation for the video, and the terminal 400 obtains the intercepted target frame in the video; the terminal 400 receives the information integration operation for the video triggered by the user, and the terminal 400 assembles the summary content generated based on the target frame and the target related information to obtain the information integration result of the video, and displays the information integration result of the video; the terminal 400 receives the sharing operation for the information integration result triggered by the user, and the terminal 400 shares the information integration result to the sharing object through the server 200.

[0047] In some embodiments, the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0048] The embodiments of the present application can also be implemented through cloud technology. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application. It can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, as well as the promotion of search services, social networks, mobile commerce and open collaboration, each item may have its own hash code identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0049] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may be a terminal or a server. Figure 2 The terminal 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0050] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0051] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0052] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0053] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0054] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0055] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0056] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0057] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0058] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0059] In some embodiments, the information processing device provided in the embodiments of the present application can be implemented in software. Figure 2 The information processing device 455 stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, including the following software modules: display module 4551, acquisition module 4552, integration module 4553, these modules are logical, so according to the functions implemented, they can be arbitrarily combined or further split. The functions of each module will be explained below.

[0060] The information processing method provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the terminal 400 provided in the embodiment of the present application.

[0061] See also Figure 3A , Figure 3A is a flowchart of the information processing method provided in the embodiment of the present application, which will be combined with Figure 3A Steps 101 to 103 are shown for explanation.

[0062] In step 101, the information to be collected corresponding to the multimedia information is displayed.

[0063] As an example, the information to be collected includes multimedia information and associated information of the multimedia information. The multimedia information here can be any one of audio, video or image, and the associated information here can be information of the author who publishes the multimedia information, or comment information on the multimedia information, or a user-defined comment on the multimedia information by the user using the terminal 400.

[0064] In step 102, in response to a collection operation on associated information, the collected target associated information is displayed as being in a selected state, and in response to a capture operation on multimedia information, a captured target frame in the multimedia information is acquired.

[0065] As an example, the collection operation here is triggered by a user using terminal 400. The collection operation here is equivalent to collecting at least part of the related information from multiple related information. The part of the related information here is the target related information. In order to visually reflect whether the related information is collected, the collected target related information will be displayed in a selected state. The interception operation here is used to intercept multimedia information, which can be intercepted by screenshots. For example, for multimedia information in video format, the video frame can be intercepted as the target frame, for multimedia information in audio format, the audio frame can be intercepted as the target frame, and for multimedia information in image format, the image itself is a frame, and the image of a partial area in the image can be intercepted as the target frame.

[0066] In step 103, in response to the information integration operation for the multimedia information, the information integration result of the multimedia information is displayed.

[0067] As an example, the information integration result is obtained by assembling the summary content generated based on the target frame and the target associated information. For the associated information collected and the intercepted target frame in the above step 102, an assembly process can be performed. First, the summary content of the multimedia information is generated based on the target frame, and then the summary content and the associated information are assembled together to obtain the information integration result, so that the relevant content of the multimedia information can be intensively displayed to improve the display efficiency.

[0068] In some embodiments, see Figure 3B In step 101, the information to be collected corresponding to the multimedia information is displayed, which can be obtained by Figure 3B Steps 1011 to 1012 are implemented as shown.

[0069] In step 1011, an information integration portal is displayed.

[0070] As an example, the information integration entrance here can be directly displayed in the interface for playing multimedia information and presented as a new function control, so that users can start using the information integration function with the fastest operation path, or the information integration entrance can be additionally displayed in the sharing-related interface.

[0071] In some embodiments, the information integration entrance is displayed in step 1011, which can be implemented by the following technical solutions: playing multimedia information in the human-computer interaction interface; and displaying the information integration entrance of the multimedia information in response to the sharing preparation operation for the multimedia information. The embodiment of the present application can achieve the effect of information integration sharing when sharing.

[0072] As an example, you can play videos or audios in a mini-program or APP. The videos or audios here can be uploaded and made public by other users. When a video uploaded by a user is played in the human-computer interaction interface, a sharing preparation operation for the video being played can be triggered to display an information integration entrance.

[0073] In some embodiments, the above-mentioned display of the information integration entrance of the multimedia information in response to the preparatory operation for sharing the multimedia information can be implemented by the following technical solutions: when the preparatory operation for sharing is a screenshot operation, in response to the screenshot operation for the multimedia information, a screenshot sending interface is displayed, and the information integration entrance is displayed in the screenshot sending interface; when the preparatory operation for sharing is a trigger operation for the forwarding entrance, in response to the trigger operation for the forwarding entrance, a sharing interface is displayed, and the information integration entrance is displayed in the sharing interface. Through the embodiments of the present application, the effect of information integration sharing can be achieved when sharing.

[0074] As an example, see Figure 5A A video is played in the human-computer interaction interface 501A. In response to a screenshot operation, an image 502A obtained by the screenshot operation and a screenshot sending interface 503A are displayed. The screenshot sending interface 503A includes an information integration entrance 504A. Figure 5A , the sharing preparation operation is a situation where the information integration entrance is displayed under the screenshot operation. In addition, the sharing preparation operation starts as a trigger operation for the forwarding entrance, for example, see Figure 4A , a video is played in the human-computer interaction interface 401A, and in response to a trigger operation on the forwarding entrance, a sharing interface 402A is displayed. The information integration entrance can be displayed in the sharing interface 402A. Figure 4A The interface shown is a schematic diagram of the interface in the related art, so Figure 4A The information integration entrance is not displayed in the sharing interface 402A, but in the embodiment of the present application, the information integration entrance can be displayed in the sharing interface 402A.

[0075] Through the embodiments of the present application, an information integration entrance can be added to the interface for implementing the sharing function, such as a screenshot sending interface or a sharing interface, so that the sharing function can be combined with the information integration function to provide users with an information integration function based on sharing purposes.

[0076] In step 1012, in response to the triggering operation on the information integration portal, the information to be collected corresponding to the multimedia information is displayed.

[0077] In some embodiments, the information to be collected corresponding to the multimedia information in step 1012 can be displayed by the following technical solutions: displaying candidate integration modes, wherein the candidate integration modes include image integration modes and web page integration modes; in response to a selection operation for any candidate integration mode, displaying any candidate integration mode in a selected state; in response to a confirmation operation for the selected state, displaying the information to be collected corresponding to the multimedia information, wherein the interception operation and the information integration operation are implemented based on any candidate integration mode. Through the embodiments of the present application, multiple integration modes can be provided to meet the diverse needs of information integration.

[0078] As an example, see Figure 5B , Figure 5B The home page of the information integration function is shown. In response to the user's selection operation, the final display form of the information integration result (image form or video web page) is selected. See Figure 5BThe human-computer interaction interface 501B shows the homepage of the information integration function. The human-computer interaction interface 501B also displays a selection entry 502B in the form of a picture (image integration mode) and a selection entry 503B in the form of a video web page (web page integration mode). In response to the trigger operation on the above selection entry, a candidate integration mode is selected. The human-computer interaction interface 501B also displays an assembly entry 504B. In response to the trigger operation (i.e., confirmation operation) on the assembly entry 504B, a content check page is displayed, and the content check page displays the information to be collected of the multimedia information.

[0079] In some embodiments, see Figure 3C , before displaying the multimedia information to be collected in step 101, execute Figure 3C Step 104 and step 105 are shown.

[0080] In step 104, at least one candidate multimedia information is displayed.

[0081] In step 105, in response to a selection operation on at least one candidate multimedia information, the selected candidate multimedia information is used as multimedia information.

[0082] The above-mentioned embodiment is to trigger the display of the information integration entrance by sharing the preparatory operation when playing the multimedia information, but the embodiment of the present application is to display at least one candidate multimedia information in response to the triggering operation on the information integration entrance after the information integration entrance is displayed, and then determine the multimedia information from the at least one candidate multimedia information through the selection operation, so as to subsequently display the corresponding multimedia information to be collected. Through the embodiment of the present application, the multimedia information that needs to be integrated and processed can be freely selected after the information integration function is triggered, thereby improving the freedom of using the information integration function.

[0083] In some embodiments, the information to be collected has at least one collection dimension, see Figure 3D In step 101, the information to be collected by displaying the multimedia information can be obtained by Figure 3D Steps 1013 and 1014 are shown to be implemented.

[0084] In step 1013, a selection control corresponding to each acquisition dimension is displayed.

[0085] In step 1014, in response to the triggering operation of the selection control for the target collection dimension, the target collection dimension is displayed in a selected state, and the to-be-collected information corresponding to the target collection dimension of the multimedia information is displayed.

[0086] As an example, at least one acquisition dimension includes: a picture dimension of multimedia information and an association dimension of multimedia information, and the association dimension includes at least one of the following: an attribute dimension of an object uploading multimedia information, an interaction dimension of multimedia information, and an integrated additional dimension of multimedia information.

[0087] As an example, the target collection dimension is the attribute dimension. Fig. 6A , the assembly control in the human-computer interaction interface 601A is highlighted, Fig. 6A The content check page is shown. In the human-computer interaction interface 601A, in response to the trigger operation on the object information selection entry 602A (the object information selection entry 602A here is the selection control of the corresponding attribute dimension, and the attribute dimension here is the target collection dimension), the object information selection entry 602A is in a highlighted state (indicating that the target collection dimension is in a selected state), and options related to the user information to be checked are displayed, including avatar, nickname, authentication, signature, etc. These options related to the user information to be checked are the information to be collected of the corresponding target collection dimension of the multimedia information.

[0088] In some embodiments, when the attribute dimension is in a selected state, the information to be collected is associated information, and the associated information includes attribute information. The information to be collected corresponding to the target collection dimension of the multimedia information displayed in step 1014 can be implemented by the following technical scheme: displaying the attribute information corresponding to each candidate attribute type in a first order; wherein the first order includes at least one of the following: the order of the number of times the candidate attribute type is collected from high to low, the order of the representation ability of the object from high to low, and the order of the most recent collection by the target object, and the target object is the object that triggers the collection operation. The embodiment of the present application can improve the efficiency of the target object in collecting the associated information of the attribute dimension.

[0089] As an example, when displaying the information to be collected in the attribute dimension, the attribute information corresponding to each candidate attribute type can be displayed in the first order. The candidate attribute types can be account nickname, account avatar, personal signature, number of followed users, etc. The first order may be the order of the number of times the candidate attribute types are collected from high to low. For example, if the number of times the account nickname is collected is greater than the number of times the account avatar is collected, then when displaying the attribute information corresponding to each candidate attribute type, the attribute information corresponding to the account nickname (for example, the attribute information may be "Little Cutie *** Loves Pig Pai") is sorted before the attribute information corresponding to the account avatar; the first order may be the order of the representation ability of the object from high to low. For example, the representation ability of the account nickname for the object uploading multimedia information is greater than the representation ability of the account avatar for the object uploading multimedia information. Therefore, when displaying the attribute information corresponding to each candidate attribute type, the attribute information corresponding to the account nickname (for example, the attribute information may be "Little Cutie *** Loves Pig Pai") is sorted before the attribute information corresponding to the account avatar; the first order may be the order of the most recent collection by the target object. For example, when the target object used the information integration function most recently, the attribute information corresponding to the account nickname was first collected, and then the attribute information corresponding to the account avatar was collected. Then the attribute information corresponding to the account nickname (for example, the attribute information may be "Little Cutie *** Loves Pig Pai") is sorted before the attribute information corresponding to the account avatar. Through the embodiments of the present application, attribute information of attribute dimensions that are checked first can be recommended to users, thereby improving the efficiency of human-computer interaction.

[0090] In some embodiments, when the interactive dimension is in a selected state, the information to be collected is related information, and the related information includes comment information. The information to be collected of the corresponding target collection dimension of the multimedia information is displayed in step 1014, which can be achieved by the following technical solution: each comment information for the multimedia information is displayed in a second order; wherein the comment information includes the comment content for the multimedia information and at least one of the following data: the publishing object information of the comment content, the popularity of the comment content, and the publishing time of the comment content; wherein the second order includes at least one of the following: the order of popularity from high to low, the distance of publishing time from near to far, and the order of the data volume of the comment content from large to small.

[0091] As an example, when displaying the information to be collected in the interactive dimension, each comment information can be displayed in the second order. The second order can be the order of popularity from high to low. For example, if the popularity of comment information A is higher than that of comment information B, then comment information A is displayed before comment information B. The second order can be the distance from the release time from near to far. For example, if the release time of comment information A is 1 minute ago and the release time of comment information B is 3 minutes ago, then comment information A is displayed before comment information B. The second order can be the order of the data volume of the comment content from large to small. For example, if the number of characters in comment information A is 100 words and the number of characters in comment information B is 50 words, then comment information A is displayed before comment information B. Through the embodiment of the present application, the comment information of the interactive dimension that is checked first can be recommended to the user, thereby improving the efficiency of human-computer interaction.

[0092] In some embodiments, when the integrated additional dimension is in a selected state, the information to be collected is related information, and the related information includes at least one of the following: a custom statement of the target object for the multimedia information, a search entry for the multimedia information, and verification information of the multimedia information. The target object is the object that triggers the collection operation; when the picture dimension is in a selected state, the information to be collected is multimedia information.

[0093] As an example, see Fig. 6E , the assembly control in the human-computer interaction interface 601E is highlighted, Fig. 6E The content check page is also shown. In the human-computer interaction interface 601E, in response to the trigger operation on the extended type information selection entry 602E, the extended type information selection entry 602E is in a highlighted state, indicating that the integrated additional dimension is in a selected state. A custom information input area 603E and an original video jump QR code 604E are displayed in the human-computer interaction interface 601E. In response to the input operation in the custom information input area 603E, the input comment information (the target object's custom sentence for the multimedia information) is displayed, that is, the screenshot user is provided with a custom sentence display function, which can be used to express summaries, feelings, notes, etc., and a QR code for jumping to the original video is also provided.

[0094] As an example, see Figure 6B , the assembly control in the human-computer interaction interface 601B is highlighted, Figure 6B The content check page is also shown. In the human-computer interaction interface 601B, in response to the triggering operation of the selection control 607B for the picture dimension, the selection control 607B for the picture dimension is in a highlighted state, indicating that the picture dimension is in a selected state, and multimedia information is displayed in the content check page, such as the video cover, audio cover, etc.

[0095] In some embodiments, the information to be collected has at least one collection dimension, see Figure 3EIn step 102, in response to the collection operation for the associated information, the collected target associated information is displayed in a selected state, which can be obtained by Figure 3E Steps 1021 to 1023 are implemented as shown.

[0096] In step 1021, a selection control corresponding to each acquisition dimension is displayed.

[0097] In step 1022, in response to a triggering operation of a selection control for a target acquisition dimension, it is displayed that the target acquisition dimension is in a selected state.

[0098] In step 1023, in response to the collection operation for the associated information of the target collection dimension, the collected target associated information of the target collection dimension is displayed in a selected state.

[0099] As an example, the target collection dimension is the attribute dimension. Fig. 6A , the assembly control in the human-computer interaction interface 601A is highlighted, Fig. 6A A content check page is shown. In the human-computer interaction interface 601A, in response to a trigger operation on the object information selection entry 602A (the object information selection entry 602A here is the selection control for the corresponding attribute dimension, and the attribute dimension here is the target collection dimension), the object information selection entry 602A is in a highlighted state (indicating that the target collection dimension is in a selected state), and options related to user information to be checked are displayed, including avatar, nickname, authentication, signature, etc. These options related to user information to be checked are the information to be collected of the corresponding target collection dimension of multimedia information. In response to a collection operation on a nickname (for example, the nickname is "cute girl gnaws bones"), the target associated information of the nickname is displayed in a selected state.

[0100] In some embodiments, see Figure 3F In step 102, in response to the interception operation for the multimedia information, the intercepted target frame in the multimedia information is obtained, which can be obtained by Figure 3F Steps 1024 to 1025 are implemented as shown.

[0101] In step 1024, the following two interception controls are displayed: a smart interception control and a manual interception control.

[0102] As an example, see Figure 6C , the assembly control in the human-computer interaction interface 601C is highlighted, Figure 6CThe content check page is also shown. In the human-computer interaction interface 601C, in response to the triggering operation of the selection control 602C for the picture dimension, the selection control 602C for the picture dimension is in a highlighted state. The human-computer interaction interface 601C includes a smart capture control 602C and a manual capture control 603C.

[0103] In step 1025, in response to a trigger operation on any interception control, the trigger operation is identified as an interception operation, and an interception process is performed on the multimedia information based on an interception function corresponding to any interception control to obtain an intercepted target frame.

[0104] In some embodiments, the multimedia information is video or audio. When any interception control is a manual interception control, in step 1025, interception processing is performed on the multimedia information based on the interception function corresponding to any interception control to obtain the intercepted target frame. This can be achieved through the following technical solution: during the playback of the multimedia information, the moment when the manual interception control is triggered is obtained as the interception timestamp; the information frame played at the interception timestamp is used as the target frame.

[0105] As an example, see Figure 6B The human-computer interaction interface 601B includes an intelligent interception control 602B and a manual interception control 603B. Manual interception is in response to a click operation on the manual interception control 603B on the right side of the video window 605B. Multiple key time points to be recorded can be marked on the progress bar of the video. For example, during the playback of multimedia information, the terminal receives a trigger operation for the manual interception control, and the information frame being played in the multimedia information at the triggering moment is recorded as the target frame, that is, each time the manual interception control 603B is triggered, a target frame can be obtained.

[0106] In some embodiments, the multimedia information is video or audio. When any interception control is a smart interception control, in step 1025, interception processing is performed on the multimedia information based on the interception function corresponding to any interception control to obtain the intercepted target frame. This can be achieved through the following technical solutions: obtaining at least one interception timestamp that meets the recommended conditions; intercepting the multimedia information based on at least one interception timestamp to obtain the target frame.

[0107] As an example, see Figure 6B The human-computer interaction interface 601B includes an intelligent interception control 602B and a manual interception control 603B. Intelligent interception is in response to the user's triggering operation on the intelligent interception control 602B, and obtains at least one interception timestamp at one time, and then intercepts the information frame corresponding to the interception timestamp from the multimedia information as the target frame, that is, each time the intelligent interception control 602B is triggered, all target frames can be obtained.

[0108] In some embodiments, a frame progress bar is displayed, and the timestamp of the target frame and the timestamp of the current play progress are marked in the frame progress bar. The embodiment of the present application can visualize the marking result, thereby improving the efficiency of human-computer interaction.

[0109] As an example, see Figure 6B In response to the trigger operation on the smart interception control 602B, the timestamp of the target frame (i.e., the interception timestamp) is marked in the frame progress bar. In response to the trigger operation on the manual interception control 603B, the timestamp of the target frame recorded by each trigger operation is marked in the frame progress bar. In addition to marking the timestamp of the target frame in the progress bar, the timestamp corresponding to the current playback progress of the multimedia information is also marked.

[0110] In some embodiments, the above-mentioned acquisition of at least one interception timestamp that meets the recommendation condition can be implemented by the following technical solution: acquiring the interactive data of the multimedia information at each timestamp; sorting the multiple timestamps in descending order based on the popularity represented by the interactive data, and obtaining the multiple timestamps with the highest order as the interception timestamps. Through the embodiments of the present application, the interception timestamps can be automatically acquired, thereby improving the human-computer interaction efficiency of intelligent marking.

[0111] As an example, multimedia information can be a video or audio that has been published, so the interactive data of the video or audio at each timestamp can be obtained, for example, the number of barrages at each timestamp in the video. The more barrages there are, the higher the popularity. Multiple timestamps are sorted in descending order based on the popularity, and the top timestamps are automatically used as intercepted timestamps. When obtaining the interactive data of each timestamp here, the interactive data of all timestamps can be obtained, or multiple timestamps can be obtained by sampling.

[0112] In some embodiments, obtaining at least one interception timestamp that meets the recommendation conditions can be achieved through the following technical solutions: segmenting the multimedia information to obtain multiple candidate segments; performing interest matching processing on each candidate segment and the target object to obtain the target object's interest level in each candidate segment, the target object is the object that triggers the interception operation; sorting the multiple candidate segments in descending order based on the interest level, and obtaining the midpoint timestamps of the top-ranked candidate segments as the interception timestamp. Through the embodiments of the present application, the interception timestamp can be automatically obtained, thereby improving the human-computer interaction efficiency of intelligent marking.

[0113] As an example, the interception timestamp can also be obtained through the content dimension. For example, multimedia information can be divided into multiple candidate segments according to the video storyboard or the audio eight-beat method. The trained neural network model can be used to perform interest matching processing on each candidate segment and the target object, and the target object's interest level in each candidate segment can be predicted, so as to automatically use the midpoint timestamp of the candidate segment with a higher level of interest as the interception timestamp.

[0114] In some embodiments, in step 1025, based on the interception function corresponding to any interception control, the interception processing for the multimedia information is performed to obtain the intercepted target frame, which can be achieved through the following technical solutions: when in the web page integration mode, the fragments corresponding to each interception timestamp in the multimedia information are intercepted, and the information frames in the fragments are used as the target frames; when in the image integration mode, the information frames corresponding to each interception timestamp in the multimedia information are intercepted as the target frames.

[0115] As an example, there are two integration modes when performing information integration. For the web page integration mode, since the information is carried in the web page, the audio clip or video clip can be displayed; and for the image integration mode, since the information is carried in the image, only the puzzle can be displayed, so the multimedia information cannot be audio at this time. Taking video as an example, when in the web page integration mode, the clips corresponding to each interception timestamp in the video are intercepted, and all the information frames included in the clips are used as target frames. When in the image integration mode, the information frames corresponding to each interception timestamp in the video are intercepted as target frames, that is, in the web page integration mode, multiple target frames are generated for each interception timestamp, and in the image integration mode, one target frame is generated for each interception timestamp.

[0116] In some embodiments, before intercepting the segments corresponding to each interception timestamp in the multimedia information, candidate durations of the segments are displayed; in response to a selection operation for any candidate duration, any candidate duration is used as the duration of the segment.

[0117] As an example, see Figure 6C , the human-computer interaction interface 601C also displays a candidate duration 604C. In response to the selection operation on the candidate duration 604C, the video segments corresponding to each interception timestamp can be obtained based on the candidate duration 604C. For example, if the selected candidate duration is 2 seconds, the interception timestamp can be used as the midpoint of 2 seconds to obtain the video segments within 1 second before and 1 second after the interception timestamp.

[0118] In some embodiments, when in image integration mode, candidate puzzle templates matching the number of target frames are displayed; in response to a selection operation on any candidate puzzle template, the target frame is puzzled based on any candidate puzzle template and the puzzle result is displayed.

[0119] As an example, see Figure 6B A puzzle template 604B is also displayed in the human-computer interaction interface 601B. In response to a selection operation on the puzzle template 604B, the marked target frames can be assembled into a puzzle result based on the puzzle template 604B according to the picture layout indicated by the selected puzzle template, and the assembly result can be viewed in the preview area 606B on the left side of the video window 605B.

[0120] In some embodiments, see Figure 3G After the information integration result of the multimedia information is displayed in step 103, you can also execute Figure 3G Step 106 or 107 is shown.

[0121] In step 106, in response to a sharing operation on the information integration result, the information integration result is sent to a sharing object.

[0122] In some embodiments, sending the information integration result to the sharing object in step 106 can be achieved by the following technical solutions: when in image integration mode, sending the information integration result with the image as the carrier to the sharing object; when in web page integration mode, sending the web page link corresponding to the information integration result to the sharing object; when in web page integration mode, copying the web page link to the clipboard for pasting into the information to be sent to the sharing object. Through the embodiment of the present application, the information integration result can play the role of sharing medium, thereby playing the role of efficient information dissemination.

[0123] As an example, see Fig. 8A In the human-computer interaction interface 801A, a forwarding control 802A is displayed. In the picture form, in response to the trigger operation on the forwarding control 802A, a friend list 803A is displayed, and the assembled content picture is sent to the selected friend in the friend list 803A. Figure 8B A forwarding control 802B is displayed in the human-computer interaction interface 801B. In the form of a video web page, in response to a trigger operation on the forwarding control 802B, a friend list 803B is displayed, and the assembled video web page link is sent to the selected friend in the friend list 803B, and the link is copied to the clipboard, so that the link can be shared through other social interfaces.

[0124] In step 107, in response to the save operation on the information integration result, the information integration result is saved.

[0125] In some embodiments, see Figure 3H The information integration result of the multimedia information displayed in step 103 can be obtained by Figure 3H Steps 1031 to 1032 are implemented as shown.

[0126] In step 1031, when the multimedia information is a video and is in an image integration mode, an information integration result is displayed based on the image, wherein the summary content in the information integration result is a puzzle result of the target frame.

[0127] As an example, see Fig. 7A The preview control 703A in the human-computer interaction interface 701A is in a highlighted state. In the form of a picture, the human-computer interaction interface 701A displays object information, for example, the nickname "Universal xx Dream", etc. A long picture 702A is also displayed in the human-computer interaction interface 701A, and the preview supports sliding up and down to view the full picture.

[0128] In step 1032, when in the web page integration mode, the information integration result is displayed based on the web page, wherein the summary content in the information integration result is the video segment splicing result or the audio segment splicing result obtained based on the target frame.

[0129] As an example, see Figure 7B The preview control in the human-computer interaction interface 701B is in a highlighted state. In the form of a video web page, the human-computer interaction interface 701B displays object information, such as the nickname "Universal xx Dream", etc. The human-computer interaction interface 701B also displays a video 702B. During preview, it supports sliding up and down to view the complete content of the page, and supports playing and pausing the video.

[0130] In some embodiments, when the associated information includes verification information of the multimedia information, a verification entry is displayed; in response to the operation of inputting the verification information in the verification entry, the verification information is displayed in the verification entry; in response to the confirmation operation on the verification information, the verification result for the information integration result is displayed, and the verification result is used to prompt that the information integration result is obtained by the target object assembling the multimedia information of the publishing object, wherein the target object is the object that triggers the information integration operation, and the publishing object is the object that uploads the multimedia information. The information integration result may have an exclusive verification code (verification information), which can be entered on a fixed page for verification. The verification code is used to record the creation proof of the information integration result, as well as the authenticity and validity of the video content in the information integration result. It prevents the information integration result from losing credibility after the original video is deleted or edited twice, and can play a role in blocking and preventing the spread of rumors.

[0131] As an example, see Fig. 10A, a verification code input area 1002A is displayed in the human-computer interaction interface 1001A, and in response to the operation of inputting a verification code in the verification code input area 1002A, Fig. 10B The human-computer interaction interface 1001B displays the verification result 1002B and the original video 1003B. By displaying the assembly information and verification code of the content set, the credibility of the content is improved.

[0132] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0133] The video and the associated information of the video are displayed in the video sharing APP of the terminal; the terminal receives the collection operation for the associated information triggered by the user, and displays that the collected target associated information is in a selected state, that is, the selected target associated information; the terminal receives the interception operation for the video, and the terminal obtains the intercepted target frame in the video; the terminal receives the information integration operation for the video triggered by the user, and the terminal assembles the summary content generated based on the target frame and the target associated information to obtain the information integration result of the video, and displays the information integration result of the video; the terminal receives the sharing operation for the information integration result triggered by the user, and the terminal shares the information integration result to the sharing object through the server.

[0134] See also Figure 4A and Figure 4B In the related technology, when users share videos, they usually share them by forwarding video links or taking screenshots. These two forms only share the video itself, and cannot meet the needs of efficient sharing of additional information integration, such as sharing a collection of related information elements of the work, which includes author information, key pictures of the work, selected comments, and impressions after watching the video. The user wants to display an excerpt of the key content of a video and attach his or her own impressions of the video. It is necessary to download the video first, perform secondary editing and assembly, and then disseminate the impressions in the form of dubbing, subtitles, or postscript sharing. The overall process is cumbersome and complicated.

[0135] The embodiment of the present application provides users with an efficient and customized interactive method based on the video sharing scenario to generate sharing media based on the self-selected video content collection. The key points involved are as follows: 1. Self-select the collection of relevant content information of the work to generate sharing media; 2. The scene of the extracted work includes video content, graphic content, live broadcast replay, etc.; 3. Users can extract and integrate multiple media such as author information, video screenshots, user comments, and impressions; 4. The generated sharing media includes two modes: pictures and video web pages. The above functions meet the needs of users to extract and annotate multiple aspects of a video work at one time, eliminating the cumbersome tool switching, lowering the threshold for content integration, and reducing implementation costs.

[0136] Videos can be batch-marked to complete screenshots or clip capture, and automatically assembled after marking to improve production efficiency. Intelligent marking can judge hot content and reduce content sorting costs. In video scenes such as courses and large-scale live broadcasts, by recording the marking point information in the sharing media, there is no need to analyze the full-length video, saving the user experience cost of secondary viewing and server storage costs. In picture mode, it supports the selection of custom layout to meet the user's customized needs. Each sharing medium (information integration result) can generate a unique verification code, which can be entered on a fixed page for verification. The verification code is used to record the creation proof of the collection and the authenticity and validity of the video content generated by the collection. It prevents the sharing media from losing credibility after the original video is deleted or edited twice, and can play a role in blocking and preventing the spread of rumors.

[0137] The interactive details of the above functions are described in detail below:

[0138] 1. Take a screenshot while watching a video to start the assembly mode. The screenshot image will be automatically saved locally. Figure 5A , a video is played in the human-computer interaction interface 501A, and in response to a screenshot operation, an image 502A obtained by the screenshot operation and a screenshot sending interface 503A are displayed, and the sharing page 503A includes an information integration entrance 504A, and in response to a trigger operation on the information integration entrance 504A, the homepage of the collection sharing function is displayed, that is, jumping to Figure 5B .

[0139] 2. Enter the homepage of the collection sharing function, and select the final display format of the shared content collection (image format or video web page) in response to the user's selection operation, see Figure 5B The human-computer interaction interface 501B shows the homepage of the collection sharing function. The human-computer interaction interface 501B also displays a selection entrance 502B in the form of a picture and a selection entrance 503B in the form of a video web page. The human-computer interaction interface 501B also displays an assembly entrance 504B. In response to the trigger operation on the assembly entrance 504B, a content check page is displayed.

[0140] 3. The content selection process is implemented through the content selection page. The assembled content is divided into 4 types: user information, video content, interactive content, and extended information.

[0141] See also Fig. 6A , the assembly control 603A in the human-computer interaction interface 601A is in a highlighted state, Fig. 6AThe content check page is shown. In the human-computer interaction interface 601A, in response to the trigger operation on the object information selection entry 602A, the object information selection entry 602A is highlighted and displays the options to be checked related to the user information, including avatar, nickname, certification, signature, etc. The author's public personal information, such as user avatar, nickname, gender, can be checked. The checked information will finally be displayed in the output content collection.

[0142] See also Figure 6B , the assembly control in the human-computer interaction interface 601B is highlighted, Figure 6B The content check page is also shown. In the human-computer interaction interface 601B, in response to the trigger operation of the selection control 607B for the screen dimension, the selection control 607B for the screen dimension is in a highlighted state. The human-computer interaction interface 601B includes an intelligent capture control 602B and a manual capture control 603B. In response to the dotting operation, the key screen time points are recorded in the video screen progress bar, and the screenshots are batch-produced and automatically composed into a long picture. When in the picture mode, the human-computer interaction interface 601B also displays a puzzle template 604B. In response to the selection operation for the puzzle template 604B, the screenshots can be batch-generated based on the puzzle template 604B to be marked content, and the long picture is finally composed according to the picture layout indicated by the selected puzzle template, and the assembly effect can be viewed in the preview area 606B on the left side of the video window 605B. Intelligent dotting is to identify high-heat clips based on the data feedback of the user's viewing, including the overall high-interaction part, the high-playing part, and the user's interactive content. Manual marking is in response to a click operation on the manual capture control 603B on the right side of the video window 605B, and multiple key time points to be recorded can be marked on the progress bar of the video.

[0143] See also Figure 6C , the assembly control in the human-computer interaction interface 601C is highlighted, Figure 6CThe content check page is also shown. In the human-computer interaction interface 601C, in response to the trigger operation of the selection control 606C for the picture dimension, the selection control 606C for the picture dimension is in a highlighted state. The human-computer interaction interface 601C includes an intelligent interception control 602C and a manual interception control 603C. In response to the trigger operation for the dot entry, the key picture time points are recorded during the video playback. When in video mode, the human-computer interaction interface 601C also displays a candidate duration 604C. In response to the selection operation for the candidate duration 604C, the various point segments can be combined into a new video based on the candidate duration 604C to be displayed on the web page. For example, the intelligent dot will select the content 3s before and after the time point, and the manual dot will be intercepted according to the candidate duration, generating multiple small segments and forming a new video to be displayed on the web page. Intelligent dot is to identify high-heat segments based on the data feedback of the user's viewing, including the overall high-interaction part, the high-play volume part, and the user's interactive content. Manual marking is in response to a click operation on the manual capture control 603C on the right side of the video window 605C, and multiple key time points to be recorded can be marked on the progress bar of the video.

[0144] See also Fig.6D , the assembly control in the human-computer interaction interface 601D is highlighted, Fig.6D A content check page is also shown. In the human-computer interaction interface 601D, in response to a trigger operation on the interactive type information selection entry 602D, the interactive type information selection entry 602D is in a highlighted state. A plurality of interactive information 603D are displayed in the human-computer interaction interface 601D. In response to a selection operation on the interactive information 603D, the interactive information 603D is displayed in a selected state. The content of the comment area is displayed in a tiled and list format, and the user can extract the interactive content that needs to be displayed in the end by checking.

[0145] See also Fig. 6E , the assembly control in the human-computer interaction interface 601E is highlighted, Fig. 6E The content check page is also shown. In the human-computer interaction interface 601E, in response to the trigger operation on the extended type information selection entry 602E, the extended type information selection entry 602E is in a highlighted state. The human-computer interaction interface 601E displays a custom information input area 603E, an original video jump QR code 604E and assembly authentication information. In response to the input operation in the custom information input area 603E, the input comment information is displayed, that is, a custom sentence display function is provided for the screenshot maker, which can be used to express summaries, feelings, notes, etc., and a QR code for jumping to the original video is also provided.

[0146] 4. Switch to the preview page to check the assembly effect.

[0147] See also Fig. 7AThe preview control 703A in the human-computer interaction interface 701A is in a highlighted state. In the form of a picture, the human-computer interaction interface 701A displays object information, for example, the nickname "Universal xx Dream", etc. A long picture 702A is also displayed in the human-computer interaction interface 701A, and the preview supports sliding up and down to view the full picture.

[0148] See also Figure 7B The preview control in the human-computer interaction interface 701B is in a highlighted state. In the form of a video web page, the human-computer interaction interface 701B displays object information, such as the nickname "Universal xx Dream", etc. The human-computer interaction interface 701B also displays a video 702B. During preview, it supports sliding up and down to view the complete content of the page, and supports playing and pausing the video.

[0149] 5. Trigger the share icon to share the assembly content.

[0150] See also Fig. 8A A forwarding control 802A is displayed in the human-computer interaction interface 801A. In the picture form, in response to the trigger operation on the forwarding control 802A, a friend list 803A is displayed, and the assembled content picture is sent to the selected friend in the friend list 803A.

[0151] See also Figure 8B A forwarding control 802B is displayed in the human-computer interaction interface 801B. In the video web page form, in response to the trigger operation on the forwarding control 802B, a friend list 803B is displayed, and the assembled video web page link is sent to the selected friend in the friend list 803B, and the link is copied to the clipboard.

[0152] 6. Display the exported renderings.

[0153] See also Fig. 9A The human-computer interaction interface 901A displays object information 902A, interactive content 903A, screenshot stitching results 904A, extended information 905A, verification code 906A and a QR code 907A for jumping to the original video.

[0154] See also Fig. 9B The human-computer interaction interface 901B displays object information 902B, interactive content 903B, video stitching results 904B, extended information 905B, a verification code 906B, and a QR code 907B for jumping to the original video.

[0155] 7. Verify the web page through the verification code

[0156] See also Fig. 10A , a verification code input area 1002A is displayed in the human-computer interaction interface 1001A, and in response to the operation of inputting a verification code in the verification code input area 1002A, Fig. 10BThe human-computer interaction interface 1001B displays the verification result 1002B and the original video 1003B. By displaying the assembly information and verification code of the content set, the credibility of the content is improved.

[0157] The technical side of the embodiment of the present application is divided into two working modules, the front-end and the back-end. The front-end is responsible for providing an interface to respond to user operations, formatting screenshot content, generating and displaying preview images; the back-end is responsible for storing videos, user-related information, generating QR codes, and other processing, and assembling video-related information according to user needs and returning it to the front-end.

[0158] The following introduces the technical architecture provided by the embodiments of the present application.

[0159] See also Fig.11 The front end is divided into three parts according to their responsibilities: UI interaction layer, media rendering layer and data layer. The UI interaction layer is responsible for drawing user interaction pages and responding to user gestures; the media rendering layer is responsible for video processing, including video playback / frame extraction / excerpting, image layout and exporting images and videos. The data layer is responsible for local storage of front-end data and pulling back-end data. The screenshot function is operated through the UI interaction layer, and information is pulled through the data layer, and finally a preview image is obtained from the media rendering layer. See Fig.11 The backend is divided into two parts according to its responsibilities: the logic layer and the data layer. The logic layer is responsible for interacting with the frontend, retrieving data resources, generating video QR codes and URLs, assembling video information, and judging the authenticity of verification codes and popular video clips. The data layer is responsible for storing video files, interactive information databases, verification code libraries, and user information databases. After receiving the frontend pull request, the backend retrieves video-related data in the data layer, assembles it through the logic layer, and generates and returns a new content collection URL to the frontend.

[0160] Combine the following Fig.12 The overall workflow provided by the embodiment of the present application is introduced.

[0161] The front end detects whether the user clicks on the assembly entrance when taking a screenshot. If it is judged to be clicked, the workflow is started. The process is as follows:

[0162] First, according to the different display forms selected by the user, the front end is connected to different capability modules and UI styles;

[0163] Then extract the contents of the following four assembled dimensions and return the extracted contents:

[0164] 1. User information dimension: the front-end requests the video author information from the back-end. After receiving the request, the back-end filters the public fields and returns it to the front-end. The front-end displays the author's public information in a list view for the user to check and record;

[0165] 2. In the video content dimension, the front-end requests the back-end to download the current video. After receiving the request, the back-end retrieves the video file from the video file warehouse and returns it to the front-end in the form of a video stream. If it is determined that the user has used smart dotting, the back-end will extract the peak time periods of the number of viewers and interactive information based on the data statistics of the video playback and provide feedback to the front-end. On this basis, the front-end adopts a download-while-playing method. For the downloaded part, it supports users to dot and extract frames, or generate pictures or capture video clips based on the time points of the hottest clips; according to the rules selected by the user, the extracted frame pictures are stitched together using a table view;

[0166] 3. In the interactive content dimension, the front-end pulls interactive content such as likes / reposts / comments of the current video from the back-end. After receiving the request, the back-end searches the interactive information database and returns it to the front-end. After receiving the information returned by the back-end, the front-end presents it to the user in a list view for the user to select and record;

[0167] 4. Expand the content dimension. The front end pulls the original video jump QR code and the exclusive verification code of the entire puzzle from the back end, and provides a custom text input component for users to input their thoughts when taking screenshots.

[0168] Regenerate and render assembled content;

[0169] For example, the assembled content can be previewed or exported. The front end generates images in real time in the form of pictures according to the splicing style selected by the user for the user to preview. For example, in the form of a video web page, the selected content is returned to the background for rendering to generate a new video and send it down.

[0170] The advantage of the embodiments of the present application is that it extends the possibility of video content precipitation and distribution. Ordinary screenshots are transformed into more comprehensive and refined rich information. By extracting key information such as video content and commentary content, the user experience and efficiency of secondary viewing and dissemination are improved, and server storage costs are saved. The information verification code can play a role in information evidence collection and enhance the credibility of the excerpted content.

[0171] It is understandable that in the embodiments of the present application, related data such as user information is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0172] The following is a description of an exemplary structure of the information processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2As shown, the software modules stored in the information processing device 455 of the memory 450 may include: a display module, used to display the information to be collected corresponding to the multimedia information, wherein the information to be collected includes the multimedia information and the associated information of the multimedia information; a collection module, used to respond to the collection operation on the associated information, display the collected target associated information in a selected state, and respond to the interception operation on the multimedia information to obtain the intercepted target frame in the multimedia information; an integration module, used to respond to the information integration operation on the multimedia information, display the information integration result of the multimedia information, wherein the information integration result is obtained by assembling the summary content generated based on the target frame and the target associated information.

[0173] In some embodiments, the display module is further used to: display an information integration entrance; and display the to-be-collected information corresponding to the multimedia information in response to a trigger operation on the information integration entrance.

[0174] In some embodiments, the display module is further used to: play the multimedia information in the human-computer interaction interface; and display the information integration entrance of the multimedia information in response to the sharing preparation operation for the multimedia information.

[0175] In some embodiments, the display module is also used for: when the sharing preparation operation is a screenshot operation, in response to the screenshot operation for multimedia information, displaying a screenshot sending interface, and displaying an information integration entrance in the screenshot sending interface; when the sharing preparation operation is a trigger operation for a forwarding entrance, in response to the trigger operation for the forwarding entrance, displaying a sharing interface, and displaying an information integration entrance in the sharing interface.

[0176] In some embodiments, the display module is further used to: display at least one candidate multimedia information before displaying the information to be collected of the multimedia information; in response to a selection operation on at least one candidate multimedia information, use the selected candidate multimedia information as the multimedia information.

[0177] In some embodiments, the collection module is also used to: display a selection control corresponding to each collection dimension; in response to a trigger operation on the selection control for the target collection dimension, display the target collection dimension as being in a selected state, and display the information to be collected of the corresponding target collection dimension of the multimedia information; wherein, at least one collection dimension includes: a picture dimension of the multimedia information and an associated dimension of the multimedia information, and the associated dimension includes at least one of the following: an attribute dimension of the object uploading the multimedia information, an interactive dimension of the multimedia information, and an integrated additional dimension of the multimedia information.

[0178] In some embodiments, when the attribute dimension is in a selected state, the information to be collected is associated information, and the associated information includes attribute information. The collection module is also used to: display the attribute information corresponding to each candidate attribute type in a first order; wherein the first order includes at least one of the following: the order of the number of times the candidate attribute type is collected from high to low, the order of the ability to characterize the object from high to low, and the order of the most recent collection by the target object, and the target object is the object that triggers the collection operation.

[0179] In some embodiments, when the interactive dimension is in a selected state, the information to be collected is related information, and the related information includes comment information, and the collection module is also used to: display each comment information for the multimedia information in a second order; wherein the comment information includes the comment content for the multimedia information and at least one of the following data: the publishing object information of the comment content, the popularity of the comment content, and the publishing time of the comment content; wherein the second order includes at least one of the following: the order of popularity from high to low, the distance of publishing time from near to far, and the order of the data volume of the comment content from large to small.

[0180] In some embodiments, when the integrated additional dimension is in a selected state, the information to be collected is related information, and the related information includes at least one of the following: a custom statement of the target object for the multimedia information, a search entry for the multimedia information, and verification information of the multimedia information. The target object is the object that triggers the collection operation; when the picture dimension is in a selected state, the information to be collected is multimedia information.

[0181] In some embodiments, the information to be collected has at least one collection dimension, and the collection module is further used to: display a selection control corresponding to each collection dimension; in response to a trigger operation of the selection control for the target collection dimension, display the target collection dimension as being in a selected state; in response to a collection operation for associated information of the target collection dimension, display the collected target associated information of the target collection dimension as being in a selected state.

[0182] In some embodiments, the multimedia information is a video, and the acquisition module is further used to: display the following two interception controls: an intelligent interception control and a manual interception control; in response to a trigger operation on any interception control, identify the trigger operation as an interception operation, and perform interception processing on the multimedia information based on the interception function corresponding to any interception control to obtain an intercepted target frame.

[0183] In some embodiments, when any interception control is a manual interception control, the acquisition module is further used to: during the process of playing multimedia information, obtain the moment when the manual interception control is triggered as the interception timestamp; and use the information frame played at the interception timestamp as the target frame.

[0184] In some embodiments, when any interception control is a smart interception control, the acquisition module is further used to: obtain at least one interception timestamp that meets the recommendation conditions; and intercept and process the multimedia information based on the at least one interception timestamp to obtain a target frame.

[0185] In some embodiments, the acquisition module is further used to: display a frame progress bar, and mark the timestamp of the target frame and the timestamp of the current playback progress in the frame progress bar.

[0186] In some embodiments, the acquisition module is further used to: obtain the interactive data of the multimedia information at each timestamp; sort the multiple timestamps in descending order based on the heat represented by the interactive data, and obtain the top timestamps as the intercepted timestamps.

[0187] In some embodiments, the acquisition module is also used to: segment the multimedia information to obtain multiple candidate segments; perform interest matching between each candidate segment and the target object to obtain the target object's interest level in each candidate segment; sort the multiple candidate segments in descending order based on the interest level, and obtain the midpoint timestamps of the top candidate segments as the capture timestamps.

[0188] In some embodiments, the acquisition module is also used to: when in a web page integration mode, intercept the fragments in the multimedia information corresponding to each interception timestamp, and use the information frame in the fragment as the target frame; when in an image integration mode, intercept the information frame in the multimedia information corresponding to each interception timestamp, and use it as the target frame.

[0189] In some embodiments, before intercepting the segments corresponding to each interception timestamp in the multimedia information, the acquisition module is also used to: display candidate durations of the segments; and in response to a selection operation for any candidate duration, use any candidate duration as the duration of the segment.

[0190] In some embodiments, when in image integration mode, the acquisition module is also used to: display candidate puzzle templates that match the number of target frames; in response to a selection operation for any candidate puzzle template, perform puzzle processing on the target frame based on any candidate puzzle template, and display the puzzle result.

[0191] In some embodiments, after displaying the information integration result of the multimedia information, the integration module is further used to: in response to a sharing operation on the information integration result, send the information integration result to the sharing object; or in response to a saving operation on the information integration result, save the information integration result.

[0192] In some embodiments, the integration module is also used to: when in image integration mode, send the information integration result with the image as the carrier to the sharing object; when in web page integration mode, send the web page link corresponding to the information integration result to the sharing object; when in web page integration mode, copy the web page link to the clipboard for pasting into the information to be sent to the sharing object.

[0193] In some embodiments, the integration module is also used to: when the multimedia information is a video and is in an image integration mode, display the information integration result based on the image, wherein the summary content in the information integration result is a puzzle result of the target frame; when in a web page integration mode, display the information integration result based on the web page, wherein the summary content in the information integration result is a video segment splicing result or an audio segment splicing result obtained based on the target frame.

[0194] In some embodiments, when the associated information includes verification information of the multimedia information, the integration module is further used to: display a verification entry; in response to the operation of inputting verification information in the verification entry, display the verification information in the verification entry; in response to the confirmation operation on the verification information, display the verification result for the information integration result, and the verification result is used to prompt that the information integration result is obtained by assembling the multimedia information of the publishing object by the target object, wherein the target object is the object that triggers the information integration operation, and the publishing object is the object that uploads the multimedia information.

[0195] The embodiment of the present application provides a computer program product, which includes computer executable instructions, and the computer executable instructions are stored in a computer readable storage medium. The processor of the computer device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device performs the above-mentioned method of the embodiment of the present application.

[0196] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the information processing method provided by the present application embodiment, for example, Figure 3A-3H The information processing method shown.

[0197] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0198] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0199] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0200] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0201] In summary, through the embodiments of the present application, multimedia information corresponding to the multimedia information and the associated information of the multimedia information are displayed, in response to the collection operation on the associated information, the collected target associated information is displayed in a selected state, and in response to the interception operation on the multimedia information, the intercepted target frame in the multimedia information is obtained, and in response to the information integration operation on the multimedia information, the summary content generated based on the target frame and the target associated information are assembled to obtain the information integration result. Through the embodiments of the present application, the associated information can be flexibly selected and the multimedia information can be intercepted, so that the flexibility of information assembly can be achieved. Since the summary content is generated based on the target frame, the information concentration can be improved, and since the information integration result is obtained by assembling the summary content and the target associated information, the information richness of the information integration result can be improved.

[0202] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An information processing method, characterized in that: The method comprises: Displaying information to be collected corresponding to the multimedia information, wherein the information to be collected includes the multimedia information and associated information of the multimedia information; In response to the collection operation on the associated information, displaying the collected target associated information as being in a selected state, and in response to the interception operation on the multimedia information, acquiring the intercepted target frame in the multimedia information; In response to the information integration operation for the multimedia information, an information integration result of the multimedia information is displayed, wherein the information integration result is obtained by assembling the summary content generated based on the target frame and the target associated information.

2. The method according to claim 1, characterized in that The information to be collected has at least one collection dimension, and the information to be collected for displaying multimedia information includes: Displaying a selection control corresponding to each of the acquisition dimensions; In response to a triggering operation of a selection control for a target acquisition dimension, displaying that the target acquisition dimension is in a selected state, and displaying information to be collected corresponding to the target acquisition dimension of the multimedia information; Among them, the at least one collection dimension includes: the picture dimension of the multimedia information and the association dimension of the multimedia information, and the association dimension includes at least one of the following: the attribute dimension of the object uploading the multimedia information, the interaction dimension of the multimedia information, and the integrated additional dimension of the multimedia information.

3. The method according to claim 2, characterized in that When the attribute dimension is in a selected state, the information to be collected is the associated information, and the associated information includes attribute information, and the information to be collected corresponding to the target collection dimension for displaying the multimedia information includes: Display the attribute information corresponding to each candidate attribute type in the first order; The first order includes at least one of the following: the order of the number of times the candidate attribute types are collected from high to low, and the order of the most recent collection by the target object, where the target object is the object that triggers the collection operation.

4. The method according to claim 2, characterized in that: When the interactive dimension is in a selected state, the information to be collected is the associated information, and the associated information includes comment information, and the information to be collected corresponding to the target collection dimension for displaying the multimedia information includes: displaying each comment information for the multimedia information in a second order; The comment information includes the comment content for the multimedia information and at least one of the following data: information about the publishing object of the comment content, the popularity of the comment content, and the publishing time of the comment content; Among them, the second order includes at least one of the following: the order of the popularity from high to low, the distance of the release time from recent to distant, and the order of the data volume of the comment content from large to small.

5. The method according to claim 2, characterized in that: When the integrated additional dimension is in a selected state, the information to be collected is the associated information, and the associated information includes at least one of the following: a custom statement of the target object for the multimedia information, a search entry for the multimedia information, and verification information of the multimedia information, and the target object is an object that triggers the collection operation; When the picture dimension is in a selected state, the information to be collected is the multimedia information.

6. The method according to claim 1, characterized in that The information to be collected has at least one collection dimension, and in response to the collection operation on the associated information, displaying that the collected target associated information is in a selected state includes: Displaying a selection control corresponding to each of the acquisition dimensions; In response to a triggering operation of a selection control for a target acquisition dimension, displaying that the target acquisition dimension is in a selected state; In response to the collection operation for the associated information of the target collection dimension, the collected target associated information of the target collection dimension is displayed to be in the selected state.

7. The method according to claim 1, characterized in that The step of obtaining the intercepted target frame in the multimedia information in response to the interception operation on the multimedia information comprises: Display the following two interception controls: smart interception control and manual interception control; In response to a trigger operation on any interception control, the trigger operation is identified as the interception operation, and an interception process is performed on the multimedia information based on an interception function corresponding to any interception control to obtain an intercepted target frame.

8. The method according to claim 7, characterized in that The multimedia information is video or audio, and when any of the interception controls is the manual interception control, performing interception processing on the multimedia information based on an interception function corresponding to any of the interception controls to obtain an intercepted target frame includes: During the playing of the multimedia information, obtaining the time when the manual interception control is triggered as an interception timestamp; The information frame played at the interception timestamp is used as the target frame.

9. The method according to claim 7, characterized in that: The multimedia information is video or audio, and when any of the interception controls is the smart interception control, performing interception processing on the multimedia information based on an interception function corresponding to any of the interception controls to obtain an intercepted target frame includes: Obtain at least one interception timestamp that meets the recommendation condition; The multimedia information is intercepted and processed based on the at least one interception timestamp to obtain the target frame.

10. The method according to claim 8 or 9, characterized in that: The method further comprises: A frame progress bar is displayed, and the timestamp of the target frame and the timestamp of the current playback progress are marked in the frame progress bar.

11. The method according to claim 9, characterized in that The obtaining of at least one interception timestamp that meets the recommendation condition includes: Obtaining interactive data of the multimedia information at each timestamp; The multiple timestamps are sorted in descending order based on the heat represented by the interactive data, and the multiple timestamps with the highest order are obtained as the intercepted timestamps.

12. The method according to claim 7, characterized in that The performing interception processing on the multimedia information based on the interception function corresponding to any one of the interception controls to obtain the intercepted target frame includes: When in the webpage integration mode, intercepting the segments in the multimedia information corresponding to each of the interception timestamps, and using the information frames in the segments as the target frames; When in the image integration mode and the multimedia information is a video, an information frame corresponding to each of the interception timestamps in the multimedia information is intercepted as the target frame.

13. The method according to claim 12, characterized in that Before intercepting the segments in the multimedia information corresponding to each of the interception timestamps, the method further includes: Display candidate durations of the segment; In response to a selection operation on any candidate duration, the any candidate duration is used as the duration of the segment.

14. The method according to claim 12, characterized in that When in the image integration mode, the method further comprises: displaying candidate puzzle templates that match the number of target frames; In response to a selection operation on any candidate puzzle template, puzzle processing is performed on the target frame based on the any candidate puzzle template, and a puzzle result is displayed.

15. The method according to claim 1, characterized in that After displaying the information integration result of the multimedia information, the method further includes: In response to a sharing operation on the information integration result, sending the information integration result to a sharing object; or In response to a save operation on the information integration result, a save process is performed on the information integration result.

16. The method according to claim 15, characterized in that The step of sending the information integration result to a sharing object includes: When in the image integration mode, the information integration result with the image as the carrier is sent to the sharing object; When in web page integration mode, a web page link corresponding to the information integration result is sent to the sharing object; The method comprises: When in the web page integration mode, the web page link is copied to a clipboard for pasting into the information to be sent to the sharing object.

17. The method according to claim 1, characterized in that The information integration result of displaying the multimedia information includes: When the multimedia information is a video and is in an image integration mode, the information integration result is displayed based on the image, wherein the summary content in the information integration result is a jigsaw puzzle result of the target frame; When in the web page integration mode, the information integration result is displayed based on the web page, wherein the summary content in the information integration result is a video segment splicing result or an audio segment splicing result obtained based on the target frame.

18. The method according to claim 1, characterized in that When the association information includes verification information of the multimedia information, the method further includes: Display the verification entrance; In response to an operation of inputting the verification information in the verification entry, displaying the verification information in the verification entry; In response to a confirmation operation on the verification information, a verification result on the information integration result is displayed, and the verification result is used to prompt that the information integration result is obtained by assembling the multimedia information of the publishing object by the target object, wherein the target object is the object that triggers the information integration operation, and the publishing object is the object that uploads the multimedia information.

19. The method according to claim 1, characterized in that The display of the information to be collected corresponding to the multimedia information includes: Display information integration entrance; In response to a trigger operation on the information integration portal, information to be collected corresponding to the multimedia information is displayed.

20. The method according to claim 19, characterized in that The displaying of the information to be collected corresponding to the multimedia information includes: displaying candidate integration modes, wherein the candidate integration modes include an image integration mode and a web page integration mode; In response to a selection operation on any candidate integration mode, displaying that the any candidate integration mode is in a selected state; In response to a confirmation operation for the selected state, information to be collected corresponding to the multimedia information is displayed, wherein the interception operation and the information integration operation are implemented based on any one of the candidate integration modes.

21. The method according to claim 19, characterized in that The display information integration entrance includes: Playing the multimedia information in the human-computer interaction interface; In response to the sharing preparation operation for the multimedia information, an information integration portal of the multimedia information is displayed.

22. An information processing device, characterized in that: The device comprises: A display module, used for displaying the information to be collected corresponding to the multimedia information, wherein the information to be collected includes the multimedia information and the associated information of the multimedia information; A collection module, configured to, in response to a collection operation on the associated information, display the collected target associated information as being in a selected state, and, in response to an interception operation on the multimedia information, obtain an intercepted target frame in the multimedia information; The integration module is used to display the information integration result of the multimedia information in response to the information integration operation for the multimedia information, wherein the information integration result is obtained by assembling the summary content generated based on the target frame and the target associated information.

23. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions; A processor, configured to implement the information processing method according to any one of claims 1 to 21 when executing computer executable instructions stored in the memory.

24. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the information processing method according to any one of claims 1 to 21 is implemented.

25. A computer program product comprising computer executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the information processing method according to any one of claims 1 to 21 is implemented.