Multimedia content-based interaction method, device, equipment, medium and product

Abstract

This application discloses an interactive method, apparatus, device, medium, and product based on multimedia content, relating to the field of computer technology. The method includes: displaying a playback interface for first multimedia content; receiving interactive operations on first interactive information within an interactive area; and adding a first work corresponding to the first multimedia content to a work collection list based on the interactive operations. On the one hand, users can trigger the system to identify the corresponding first work through interactive operations, avoiding the cumbersome process of manually searching or querying to determine the first work. On the other hand, after identifying the first work, the system can automatically add it to the collection list, saving users the step of manually adding the first work to the collection list, making it easier for users to obtain and save works of interest when browsing multimedia content, thus improving human-computer interaction efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an interactive method, apparatus, device, medium and product based on multimedia content. Background Technology

[0002] Online video platforms, also known as video websites or streaming media platforms, refer to platforms that provide video playback functionality to users on the internet. With the rapid development of online video platforms, the video content on these platforms has become increasingly diverse, and this content often includes background music.

[0003] In related technologies, if a user wants to obtain the background music in a video played on an online video platform, they need to first find out the name of the background music through the comments section or information provided by the creator, then open the music playback platform to search for the music name, and finally add the background music to their playlist from the search results.

[0004] However, the process of adding background music to a playlist is too cumbersome and has low human-computer interaction efficiency. Summary of the Invention

[0005] This application provides an interactive method, apparatus, device, medium, and product based on multimedia content. The technical solution is as follows:

[0006] On the one hand, an interactive method based on multimedia content is provided, the method comprising:

[0007] The playback interface displays the first multimedia content, and the playback interface includes an interactive area, which is used to display interactive information published in response to the first multimedia content.

[0008] Receive an interactive operation in the interactive area for a first interactive information, the first interactive information including a first interactive identifier, the first interactive identifier being used for content identification of the first multimedia content;

[0009] Based on the interactive operation, the first work corresponding to the first multimedia content is added to the work collection list. The first work is the work obtained after content recognition of the first multimedia content.

[0010] On the other hand, an interactive device based on multimedia content is provided, the device comprising:

[0011] The display module is used to display the playback interface of the first multimedia content, the playback interface including an interactive area, the interactive area being used to display interactive information published in response to the first multimedia content;

[0012] The receiving module is configured to receive interactive operations in the interactive area in response to the first interactive information, wherein the first interactive information includes a first interactive identifier, and the first interactive identifier is used to identify the content of the first multimedia content.

[0013] The processing module is used to add the first work corresponding to the first multimedia content to the work collection list based on the interactive operation. The first work is the work obtained after content recognition of the first multimedia content.

[0014] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement any of the above-described interactive methods based on multimedia content.

[0015] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement any of the above-described interactive methods based on multimedia content.

[0016] On the other hand, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the above-described interactive methods based on multimedia content.

[0017] The beneficial effects of the technical solutions provided in this application include at least the following:

[0018] In the multimedia content playback interface, users only need to trigger an interactive operation on the first interactive information (including the first interactive identifier), and the system can automatically complete the identification process of the multimedia content and add the identified first work to the work collection list. On the one hand, users can trigger the system to identify the corresponding first work through interactive operations, avoiding the tedious process of manually searching or asking questions to determine the first work; on the other hand, after the system identifies the first work, it can automatically add the first work to the collection list, saving users the step of manually adding the first work to the collection list, making it easier for users to obtain and save works of interest when browsing multimedia content, thus improving the efficiency of human-computer interaction. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of a computer system provided in an exemplary embodiment of this application;

[0021] Figure 2 This is a schematic diagram of an interface for an interactive method based on multimedia content provided in an exemplary embodiment of this application;

[0022] Figure 3 This is a flowchart of an exemplary embodiment of the interactive method based on multimedia content provided in this application;

[0023] Figure 4 This is a flowchart of an interactive method based on multimedia content provided in another exemplary embodiment of this application;

[0024] Figure 5 This is a flowchart of an interactive method based on multimedia content provided in yet another exemplary embodiment of this application;

[0025] Figure 6 This is a structural block diagram of an interactive device based on multimedia content provided in an exemplary embodiment of this application;

[0026] Figure 7 This is a structural block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In this application, the terms "first" and "second" are used to distinguish between identical or similar items that have essentially the same function. It should be understood that there is no logical or temporal dependency between "first" and "second", nor is there any limitation on the quantity or execution order.

[0029] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user data. These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without user confirmation), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with relevant laws, regulations, and standards.

[0030] Online video platforms, also known as video websites or streaming media platforms, refer to platforms that provide video playback functionality to users on the internet. With the rapid development of online video platforms, the video content on these platforms has become increasingly rich, and this content often includes background music. Currently, if a user wants to access the background music in a video played on an online video platform, they need to first obtain the music's name through comments or information provided by the creator, then search for that music on a music player platform, and finally add it to their playlist from the search results. However, this process of adding background music to a playlist is overly cumbersome and inefficient in terms of human-computer interaction.

[0031] Based on this, this application provides an interactive method based on multimedia content. In the multimedia content playback interface, the user only needs to trigger an interactive operation on the first interactive information (including the first interactive identifier), and the system can automatically complete the identification process of the multimedia content and add the identified first work to the work collection list.

[0032] The computer system used to implement the interactive method based on multimedia content of this application will now be described.

[0033] Figure 1 A schematic diagram of a computer system provided in an exemplary embodiment of this application is shown below. Figure 1 As shown, the computer system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a communication network 130. In some optional embodiments, the communication network 130 can be a wired network or a wireless network. This embodiment does not limit this.

[0034] In some embodiments, terminal 110 is a smartphone, tablet computer, laptop computer, desktop computer, smart home appliance, smart vehicle terminal, etc., but is not limited to these. Server 120 provides backend services for the client installed in the terminal. Server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services. Server 120 can also be implemented as a node in a blockchain system.

[0035] In some embodiments, a client of a first application is installed on the terminal 110. The first application can be a traditional application, a cloud application, a mini-program or application module in a host application, or a web platform; this embodiment does not limit this. Optionally, the first application can be an application that provides multimedia content playback functionality, such as a video application, a short video application, an audio playback application, an instant messaging application, etc.; this is not limited here.

[0036] Schematic illustration: The client of the first application on terminal 110 displays a playback interface for the first multimedia content. The client of the first application is logged into a first account. This playback interface includes an interactive area, which provides account interaction functionality. Users can post interactive information about the first multimedia content in the interactive area, such as comments or bullet screen messages. When a user posts first interactive information containing a first interactive identifier (e.g., @musicplayplatformA official account) in the interactive area, the client of the first application forwards the first interactive information to server 120 via communication network 130. After receiving the first interactive information, server 120, upon recognizing the first interactive identifier, triggers a content recognition process for the first multimedia content, such as recognizing the background music of the first multimedia content.

[0037] After the content recognition process is triggered, if the server 120 recognizes the first work corresponding to the first multimedia content, it will obtain the work information of the first work and send the work information of the first work to the client of the first application of the terminal 110. After receiving the work information, the client of the first application will display the reply information for the first interaction information in the interactive area of ​​the playback interface. The reply information includes the work information of the first work. Alternatively, the client of the first application will display the work information of the first work in the conversation interface. The conversation interface refers to the interface for the conversation between the first account and the target account. The target account is associated with the first interaction identifier. For example, if the first interaction identifier is @music playback platform A official account, then the target account is the music playback platform A official account.

[0038] Simultaneously, after recognizing the first work, server 120 will trigger the process of adding the first work to the work collection list. Taking the work collection list as an example provided by a second application, optionally, terminal 110 has a client of the second application installed, and a second account is logged in on the client. This second application can be a traditional application, a cloud application, a mini-program or application module within a host application, or a web platform; this embodiment does not limit this. Optionally, the second application can be an application that provides work collection functionality, such as an audio playback application, instant messaging application, novel reading application, video playback application, image browsing application, etc., without limitation here.

[0039] To illustrate, after server 120 recognizes the first work, if there is a relationship between the second account logged in in the client of the second application and the first account logged in in the client of the first application (e.g., user authorization to bind the first account and the second account), then server 120 will directly send a work addition request to the client of the second application of terminal 110. After receiving the work addition request, the client of the second application will add the first work to the work collection list. For example, taking the first work as song 1, the client of the second application creates a new playlist a for the second account and then adds song 1 to playlist a.

[0040] If the second account logged in on the second application's client is not associated with the first account logged in on the first application's client, server 120 will send the work extraction information to the first application's client on terminal 110. After receiving the work extraction information, the first application's client will display a response to the first interaction in the interactive area of ​​the playback interface, which includes the work extraction information; alternatively, the first application's client will display the work extraction information in the conversation interface. The user can launch the second application's client on terminal 110. When the user pastes the work extraction information into the application interface of the second application's client, or searches for the work extraction information in the search bar of the second application's client, the second application's client will add the first work to the work collection list.

[0041] The following explanation uses the example of the first application being implemented as a video playback platform and the second application being implemented as a music playback platform A. Please refer to [link / reference]. Figure 2This diagram illustrates an interface for an interactive method based on multimedia content. A playback interface is displayed on the client of a video playback platform running on terminal 110. The video playback area 211 of the playback interface plays video content. The playback interface also includes an interactive area 212 where users can post comments on the video content or send bullet comments. For example, if a user posts a comment "@music playback platform A official account" in the interactive area 212, the video playback platform client will display the official account of music playback platform A's reply to the comment 213 in the interactive area 212, namely, "The background music of the video content is as follows: Song name: aaaaa, Artist: bbb." This reply is obtained after identifying the background music of the video content. Furthermore, after identifying the song "aaaaa," the music playback platform A client running on terminal 110 will add the song "aaaaa" to "My Playlist 1" 221.

[0042] Next, the flow of the interactive method based on multimedia content provided in the embodiments of this application will be described.

[0043] Based on the above introduction, Figure 3 This is a flowchart illustrating an interactive method based on multimedia content provided in an embodiment of this application, whereby the method is applied to, for example... Figure 1 Taking the terminal 110 shown as an example, the method is described in steps 310 to 330 below.

[0044] Step 310: Display the playback interface of the first multimedia content.

[0045] Optionally, the first multimedia content includes at least one of video content, audio content, image content, text content, etc., without limitation.

[0046] In some embodiments, the playback interface includes a content playback area, in which the playback interface of the first multimedia content is displayed.

[0047] The playback interface includes an interactive area for displaying interactive information related to the first multimedia content. This interactive information, illustratively, includes at least one of comments, bullet comments, etc.

[0048] In some embodiments, the playback interface of the first multimedia content is an interface provided by the first application. Optionally, the playback interface of the first multimedia content is displayed on the client of the first application, and the client of the first application is logged into a first account.

[0049] Optionally, the first application can be an application that provides multimedia content playback functionality, such as a video application, a short video application, an audio playback application, an instant messaging application, etc., without limitation here.

[0050] Step 320: Receive interactive operations in the interactive area for the first interactive information.

[0051] The first interactive information refers to interactive information published in response to the first multimedia content. Optionally, the first interactive information may be in text form.

[0052] The first interactive information includes the first interactive identifier.

[0053] The first interactive identifier is used to identify the content of the first multimedia content. In other words, the first interactive identifier is used to trigger the content identification process of the first multimedia content.

[0054] In illustrative terms, the first interaction identifier is a pre-defined identifier. When the client of the first application recognizes the first interaction identifier, it will send the first interaction identifier to the server. After receiving the first interaction identifier, the server will trigger the content recognition process of the first multimedia content.

[0055] In some embodiments, the first interaction identifier may be implemented as a notification identifier for the target account, such as "@target account name". Optionally, for the target account: the target account refers to an account registered on the target platform in the first application. If the work collection list described below is a collection list provided by the first application, then the target platform is the first application, for example: the target account is the official account of the first application; if the work collection list described below is a collection list provided by a second application, then the target platform is the second application, for example: the target account is the official account of the second application registered in the first application.

[0056] Optionally, content recognition of the first multimedia content refers to the identification of artistic works within the first multimedia content. Artistic works include at least one of the following: literary works (such as novels and poems), audio works (such as radio dramas and songs), film and television works (such as TV series, movies, and variety shows), and paintings. The following explanation uses video content as an example to illustrate this:

[0057] (1) Identify literary works in the video content.

[0058] This is illustrative of the case where video content involves literary works. For example, the video content might be an introduction to a specific literary work, a TV series or movie based on a particular literary work, or simply mention a literary work. To identify literary works within video content, the text content corresponding to each video frame and each audio frame can be analyzed. This analysis helps determine the literary works mentioned in the video. For instance, a keyword database of literary works can be obtained, containing keywords for multiple literary works. The obtained text content is then matched against this database; if the text content contains a keyword for a particular literary work, that literary work is considered the identification result.

[0059] (2) Identify audio works in video content.

[0060] This is illustrative, referring to video content that includes audio works. Examples include: background music in the video being an audio work, the video content being an introduction to an audio work, or the video mentioning a literary work. To identify the audio work involved in the background music of a video, the background music is first extracted from the video content. Then, the corresponding audio feature representation is extracted. This audio feature representation is matched against an audio work database, which contains audio feature representations for multiple audio works. If the audio feature representation of the background music matches the audio feature representation of an audio work in the database, that audio work is used as the identification result.

[0061] (3) Identify the film and television works in the video content.

[0062] This is illustrative; the video content may involve film or television works, such as narration of a film or television work, or clips from a film or television work. To identify the film or television work in the video content, content feature representations can be extracted. These content feature representations are then analyzed to determine the film or television work involved in the video content. For example, the content feature representations can be matched with a film or television work database, which includes feature representations for multiple film or television works (e.g., character feature representations, scene feature representations, plot feature representations, etc.). If the content feature representation of the video content matches the feature representation of a film or television work in the database, then that film or television work is used as the identification result.

[0063] (4) Identify paintings in the video content.

[0064] This is illustrative, illustrating the use of images in video content, such as a video introducing a particular painting. To identify paintings within a video, image feature representations of the corresponding video frames can be extracted. These image feature representations are then analyzed to determine the paintings featured in the video. For example, the image feature representations of the video content can be matched against a database of paintings, which contains image feature representations for multiple paintings. If the image feature representation of the video content matches the image feature representation of a painting in the database, that painting is then identified as the painting in the database.

[0065] It should be noted that the method for identifying video content described above is only illustrative and is not intended to limit the scope of this application.

[0066] In some embodiments, content recognition of the first multimedia content refers to: performing content recognition on the first multimedia content and other information associated with the first multimedia content. Optionally, the other information associated with the first multimedia content includes information such as the title and description of the first multimedia content.

[0067] In some embodiments, the first interactive information further includes a first prompt, and the first interactive identifier is used to perform content recognition on the first multimedia content under the guidance of the first prompt. For example, if the first multimedia content is recognized using a content recognition model, the first prompt and the first multimedia content can be input into the content recognition model. The content recognition model will then perform content recognition on the first multimedia content under the guidance of the first prompt. For instance, if the first prompt is "Recognize background music," the content recognition model will recognize the background music in the first multimedia content.

[0068] In some embodiments, the first interactive information further includes content recognition prompt information, which is used to instruct content recognition of the first multimedia segment in the first multimedia content; optionally, based on the interactive operation, the first work corresponding to the first multimedia segment in the first multimedia content is added to the work collection list, which is the work obtained after content recognition of the first multimedia segment.

[0069] To illustrate, taking video content as the first multimedia content, the content prompt information can be implemented as "[start time]-[end time]". Then, an example of the first interactive information could be "background music from 00:01:10 to 00:02:30 @ target account name".

[0070] When the server receives the first interactive information containing identification content prompts, it will cut the first multimedia content into segments according to the identification content prompts to obtain the first multimedia segment; then it will perform content recognition on the first multimedia segment to obtain the work corresponding to the first multimedia segment (e.g., the first work).

[0071] In the above embodiments, by recognizing content prompts, users can more accurately obtain works of interest when browsing multimedia content. In addition, content recognition is performed only on multimedia segments rather than the entire multimedia content, which reduces the consumption of computer resources to some extent.

[0072] Optionally, the above-mentioned interactive operation of receiving the first interactive information in the interactive area includes at least one of the following operations:

[0073] (1) Sending the first interactive message.

[0074] Optionally, receiving a sending operation of the first interactive information within the interactive area is considered an interactive operation. For example, the first account sending a bullet screen or comment containing the first interactive identifier on the playback interface of the first multimedia content is considered the first interactive information.

[0075] (2) Agree to the first interactive message.

[0076] Optionally, the interactive area displays first interactive information, which is interactive information posted in the interactive area by other accounts (other than the first account). Receiving approval actions for the first interactive information in the interactive area is considered an interactive action; illustratively, approval actions include a "like" action and a "+1" action. The interactive area displays comments sent by other accounts containing the first interactive identifier. The first account can like or "+1" these comments, where "+1" indicates approval of the comment. In some embodiments, when the number of "+1" actions received for a comment reaches a preset number, the recognition process for the first multimedia content is initiated, and the recognized work is added to the work collection list. Optionally, "+1" refers to copying and sending a comment. Illustratively, a "+1" option is displayed under comments containing the first interactive identifier. When the first account clicks the "+1" option, the comment containing the first interactive identifier is automatically sent to the comment area.

[0077] (3) Forwarding the first interactive information.

[0078] Optionally, the interactive area displays first interactive information; receiving forwarding operations of the first interactive information in the interactive area is regarded as an interactive operation. Schematic, the interactive area displays comments containing the first interactive identifier, and the first account can forward the first interactive identifier to the conversation interface with the target account.

[0079] The examples of interactive operations described above are merely illustrative and are not intended to limit the scope of this application.

[0080] Step 330: Based on the interactive operation, add the first work corresponding to the first multimedia content to the work collection list.

[0081] The first work is the work obtained after content recognition of the first multimedia content.

[0082] In a demonstrative manner, after an interactive operation is triggered, the server performs content recognition on the first multimedia content. Once the first work is identified, the process of adding the first work to the work collection list is triggered.

[0083] Optionally, the first work includes at least one of the following: literary works, audio works, film and television works, paintings, etc.

[0084] Based on interactive operations, adding the first work corresponding to the first multimedia content to the work collection list includes at least one of the following situations:

[0085] (1) The work storage list is the storage list provided by the first application.

[0086] Optionally, based on interactive operations, the first work corresponding to the first multimedia content is added to the work collection list in the client of the first application.

[0087] To illustrate, after the server identifies the first work, it sends a work addition request to the client of the first application. After receiving the work addition request, the client of the first application adds the first work to the client of the first application.

[0088] (2) The artwork storage list is a storage list provided by the second application.

[0089] Optionally, based on interactive operations, the first work corresponding to the first multimedia content is added to the work collection list in the client of the second application.

[0090] Optionally, the current terminal has a client for a second application installed, and a second account is logged in on the client of the second application. The second application can be an application that provides content storage functions, such as an audio playback application, an instant messaging application, a novel reading application, a video playback application, an image browsing application, etc., without limitation. The second application and the first application are different applications.

[0091] In the above embodiments, based on interactive operations, the first work corresponding to the first multimedia content can be directly added to the work collection list in the client of the second application, providing users with a convenient way to organize and collect multimedia works they are interested in, without having to search for or find the work again in the second application, saving users time and energy and improving the efficiency of work management.

[0092] In some embodiments, when there is a relationship between the first account and the second account, the first work corresponding to the first multimedia content is automatically added to the work collection list in the client of the second application.

[0093] The association between the first and second accounts means that the user authorizes the first account to be bound to the second account. For illustrative purposes, in the client of the first application, the user can initiate a request to bind the first and second accounts, thereby binding them together.

[0094] When the first account and the second account are linked, after the server recognizes the first work, it will send a work addition request to the client of the second application. After receiving the work addition request, the client of the second application will add the second work to the client of the first application.

[0095] In the above solution, when the first account and the second account are associated, the first work corresponding to the first multimedia content can be automatically added to the work collection list in the client of the second application. This saves users from manual operation steps, improves operational efficiency, and allows users to organize and collect works of interest in different application clients more quickly and conveniently.

[0096] In other embodiments, when there is no association between the first account and the second account, the work extraction information corresponding to the first work is displayed on the client of the first application; when the work extraction information is used on the client of the second application, the first work corresponding to the first multimedia content is automatically added to the work collection list in the client of the second application.

[0097] The phrase "the first account and the second account are not related" means that the first account and the second account are not linked.

[0098] If the first and second accounts are not bound, after the server recognizes the first work, it will send the work extraction information to the client of the first application. The work extraction information includes a unique extraction code corresponding to the first work, such as "ABC12345". Each work has a different extraction code, which is used to add the corresponding work to the work collection list in the client of the second application.

[0099] In the above solution, when the first account and the second account are not associated, the work is not automatically added directly. Instead, the user needs to use the extracted information in the second application client to complete the addition. This ensures the security of the user account to a certain extent and avoids text such as accidentally adding to the wrong account. At the same time, it gives the user enough autonomy to decide whether to add the work to the second application.

[0100] Optionally, when the client of the second application uses the work extraction information, the method for automatically adding the first work corresponding to the first multimedia content to the work collection list in the client of the second application includes at least one of the following methods:

[0101] (1) The client of the first application receives a copy operation for extracting information of the work; in response to receiving the operation of launching the client of the second application, the first work corresponding to the first multimedia content is automatically added to the work collection list in the client of the second application.

[0102] To illustrate, the extracted information of the work is copied to the clipboard of the terminal. At this time, the client of the second application is launched. The client of the second application will read the extracted information of the work in the clipboard. When the extraction code of the first work in the extracted information is recognized, the client of the second application will automatically add the first work to the work collection list in the client of the second application.

[0103] (2) The client of the second application receives an input operation for extracting information about the work; in response to receiving the input operation, the first work corresponding to the first multimedia content is automatically added to the work collection list in the client of the second application.

[0104] To illustrate, launch the client of the second application and display its application interface. Enter the work extraction information in the extraction code input area or search area of ​​the application interface, and the client of the second application will automatically add the first work to the work collection list in the client of the second application.

[0105] Optionally, different types of first works correspond to different types of work collection lists.

[0106] For illustrative purposes only, if the first work is a novel, the work collection list can be implemented as a novel bookshelf; if the first work is a song, the work collection list can be implemented as a song playlist; if the first work is a film or television work, the work collection list can be implemented as a video collection, etc. There are no restrictions here.

[0107] Optionally, the number of works obtained after content recognition of the first multimedia content can be one or more, and there is no limitation here. If the number of works obtained after content recognition of the first multimedia content is multiple, the multiple recognized works are added to the work collection list. For example, multiple works are added to the same work collection list; or, multiple works are added to the work collection list according to work type.

[0108] Taking the works obtained after content recognition of the first multimedia content as an example, including the first work and the second work: based on interactive operation, the first work corresponding to the first multimedia content is added to the second work collection list, and the second work corresponding to the first multimedia content is added to the third work collection list; wherein, the first work matches the collection work type corresponding to the second work collection list, and the second work matches the collection work type corresponding to the third work collection list.

[0109] For example: If the first work is a literary work a and the second work is an audio work b, you can add literary work a to list 1 for literary works and add audio work b to list 2 for audio works. Or, if the first work is a pop song c and the second work is a rap song d, you can add pop song c to playlist 1 for pop songs and add rap song d to playlist 2 for rap songs.

[0110] In the above embodiments, the addition operation is performed based on the matching of the artwork with the type of artwork to be stored, which enables accurate classification and storage of different types of artwork.

[0111] In some embodiments, when there are multiple candidate works collection lists, the candidate works collection list that matches the work can be determined from the multiple candidate works collection lists based on the work characteristics and collection list characteristics.

[0112] Optionally, based on interactive operations, the method of adding the first work corresponding to the first multimedia content to the second work collection list and adding the second work corresponding to the first multimedia content to the third work collection list further includes the following steps:

[0113] Step 1: Based on interactive operations, obtain a list of multiple candidate works.

[0114] Optionally, the multiple candidate works collection list is a list provided by the client of the second application. Illustratively, the multiple candidate works collection list refers to the works collection list currently created by the second account.

[0115] After the server identifies the first and second works, it sends the work information of the first and second works to the terminal. The client of the first application or the client of the second application in the terminal receives the work information of the first and second works, and obtains the collection information and list attribute information corresponding to the multiple candidate work collection lists in the second account.

[0116] Step 2: Extract the list feature representations corresponding to the multiple candidate works collection lists.

[0117] The list feature representation is used to characterize the types of works included in the candidate works collection list; wherein, the list feature representation is a feature representation determined based on the collection information of the candidate works collection list and the list attribute information.

[0118] The information on the works included in the candidate works list is illustrative and includes, but is not limited to, specific information such as the category of the work, the style of the work (e.g., classical music, rap music, etc.), the source of the work, and the duration of the work.

[0119] List attribute information refers to the basic information corresponding to the candidate works collection list. It is illustrative and includes, but is not limited to, the name of the list, the creation time of the list, and specific theme tags of the list (such as "summer", "nostalgia", etc.).

[0120] For each obtained candidate work collection list, feature extraction is performed on the corresponding collection work information and list attribute information to obtain a list feature representation that characterizes the collection work type of the candidate work collection list.

[0121] Step 3: Extract the first work feature representation of the first work corresponding to the first multimedia content, and extract the second work feature representation of the second work corresponding to the first multimedia content.

[0122] To illustrate, feature extraction is performed on the first work's information to obtain the first work's feature representation, and feature extraction is performed on the second work's information to obtain the second work's feature representation.

[0123] Optionally, the work information includes the work title, creator, creation time, work type, work description, and work duration.

[0124] Step 4: If the feature representation of the first work matches the list feature representation of the first candidate work collection list in the multiple candidate work collection lists, then the first candidate work collection list is determined as the second work collection list.

[0125] For the first work's feature representation, calculate the similarity (e.g., cosine similarity, Euclidean distance, etc.) between the first work's feature representation and the list feature representations corresponding to multiple candidate work collection lists. If the similarity between the list feature representation corresponding to the first candidate work collection list and the first work's feature representation is greater than or equal to a preset matching value, the first candidate work collection list is determined as the second work collection list. Specifically, if the similarity between the list feature representation corresponding to a single candidate work collection list and the first work's feature representation is greater than or equal to the preset matching value, that single candidate work collection list is determined as the second work collection list. If the similarity between the list feature representations corresponding to multiple candidate work collection lists and the first work's feature representation is greater than or equal to the preset matching value, the candidate work collection list with the highest similarity is determined as the second work collection list.

[0126] Step 5: If the feature representation of the first work matches the list feature representation of the second candidate work collection list in the multiple candidate work collection lists, then the second candidate work collection list is determined as the third work collection list.

[0127] For the second work's feature representation, calculate the similarity (e.g., cosine similarity, Euclidean distance, etc.) between the second work's feature representation and the list feature representations corresponding to multiple candidate work collection lists. If the similarity between the list feature representation corresponding to the second candidate work collection list and the second work's feature representation is greater than or equal to a preset matching value, the second candidate work collection list is determined as the third work collection list. Specifically, if the similarity between the list feature representation corresponding to a single candidate work collection list and the second work's feature representation is greater than or equal to the preset matching value, that single candidate work collection list is determined as the third work collection list. If the similarity between the list feature representations corresponding to multiple candidate work collection lists and the second work's feature representation is greater than or equal to the preset matching value, the candidate work collection list with the highest similarity is determined as the third work collection list.

[0128] Step 6: Add the first artwork to the second artwork list, and the second artwork to the third artwork list.

[0129] To illustrate, if the client of the first application determines the second and third work collection lists, the client of the first application sends a work collection request to the server. The server then sends the work collection request to the client of the second application. The client of the second application calls the interface program to add the relevant information of the first work (such as the work's storage path, metadata, etc.) to the database table or storage structure corresponding to the second work collection list, thus realizing the collection of the first work in that list; and adds the relevant information of the second work to the database table or storage structure corresponding to the third work collection list, thus completing the collection operation of the second work.

[0130] If the client of the second application determines the second and third work collection lists, the client of the second application will directly call the interface program to add the first work to the second work collection list and the second work to the third work collection list.

[0131] It should be noted that the process of determining the second and third work collection lists described above can also be implemented using a server, and this application embodiment does not limit this.

[0132] In the above scheme, by extracting the feature representations of the works and the feature representations of the candidate works collection list, and matching them to determine the appropriate collection list, more accurate classification and collection of works can be achieved.

[0133] In some embodiments, the first work is added to a list of works already stored in the client of the first application or the client of the second application; or, a new list of works is created in the client of the first application or the client of the second application, and the first work is added to the list of works.

[0134] In some embodiments, the first interactive information further includes storage instruction information, which instructs that the first work obtained by recognizing the first multimedia content be added to the first work storage list.

[0135] Optionally, based on interactive operations, the first work corresponding to the first multimedia content is added to the first work storage list, which is a list determined according to storage instruction information.

[0136] For illustrative purposes, the storage instructions include the name and identifier of the first work's storage list. There are no restrictions here. For example, if the storage instructions are "Add to work storage list a", then the first work will be added to work storage list a.

[0137] In the above embodiments, by including storage instructions in the first interactive information, the first artwork is explicitly specified to be added to a specific first artwork storage list, making the artwork storage process more precise. Users can accurately place artworks in designated storage lists according to their needs, satisfying their personalized requirements.

[0138] In summary, the interactive method based on multimedia content provided in this application allows users to automatically complete the identification process of multimedia content and add the identified first work to the work collection list simply by triggering an interactive operation on the multimedia content playback interface (including the first interactive identifier). On the one hand, users can trigger the system to identify the corresponding first work through interactive operations, avoiding the cumbersome process of manually searching or querying to determine the first work. On the other hand, after identifying the first work, the system can automatically add it to the collection list, saving users the step of manually adding the first work to the collection list. This makes it easier for users to obtain and save works of interest when browsing multimedia content, improving the efficiency of human-computer interaction.

[0139] In some embodiments, after a user triggers an interactive operation, in addition to adding the artwork information of the first artwork to the artwork storage list, the artwork information of the first artwork can also be displayed on the terminal. For illustrative purposes, please refer to [the relevant documentation / reference]. Figure 4 The above Figure 3 The illustrated embodiment can also be implemented as follows: steps 410 to 430.

[0140] Step 410: Display the playback interface of the first multimedia content.

[0141] Optionally, the first multimedia content includes at least one of video content, audio content, image content, text content, etc., without limitation.

[0142] In some embodiments, the playback interface includes a content playback area and an interactive area, wherein the content playback area is used to play first multimedia content, and the interactive area is used to display interactive information published in response to the first multimedia content.

[0143] The interactive area includes at least one of the following: a comment area, a bullet screen area, etc.

[0144] Optionally, the comment area displays interactive information posted in response to the first multimedia content in a static format. For example, the comment area is displayed in a specific location on the playback interface (such as below the content playback area or in the sidebar). After a user posts a comment, the comment is displayed statically in the comment area. Regarding the interactive information displayed in the bullet screen area, interactive information posted by users at different times will be displayed centrally in the comment area, allowing users to systematically browse interactive information posted by different users at different times.

[0145] Optionally, the bullet screen area dynamically displays interactive information posted in response to the first multimedia content. Illustratively, the bullet screen area can be an area covering the content playback area. After a user sends a bullet screen comment, the comment is dynamically displayed in the bullet screen area. The interactive information displayed in the bullet screen area floats across the playing first multimedia content. As the playback time of the first multimedia content progresses, the interactive information scrolls from one side of the screen (e.g., right) to the other (e.g., left). Interactive information posted at different times will float across the screen sequentially according to their posting time, synchronized with the playback of the first multimedia content.

[0146] Step 420: Receive interactive operations for the first interactive information within the interactive area.

[0147] The first interactive information refers to interactive information published in response to the first multimedia content. Optionally, the first interactive information may be in text form.

[0148] The first interactive information includes a first interactive identifier, which is used to identify the content of the first multimedia content. For illustrative purposes, the first interactive identifier is a pre-defined identifier. When the client of the first application identifies the first interactive identifier, it sends the identifier to the server. Upon receiving the first interactive identifier, the server triggers the content identification process for the first multimedia content.

[0149] Regarding the server's content recognition process for the first multimedia content, illustratively speaking, if the server has not triggered the content recognition process for the first multimedia content before receiving the first interaction identifier, the server executes the content recognition process for the first multimedia content and obtains the recognition result (the first work as described below); if the server has already triggered the content recognition process for the first multimedia content before receiving the first interaction identifier, the server directly obtains the currently saved recognition result for the first multimedia content and then feeds the recognition result back to the terminal (that is, to the client of the first application or the client of the second application).

[0150] Optionally, receiving a sending operation of the first interactive information within the interactive area is considered an interactive operation; or, receiving an approval operation of the first interactive information within the interactive area is considered an interactive operation; or, receiving a forwarding operation of the first interactive information within the interactive area is considered an interactive operation.

[0151] Step 430: Based on interactive operation, display the work information of the first work corresponding to the first multimedia content, and add the first work corresponding to the first multimedia content to the work collection list.

[0152] The first work is a work obtained after content recognition of the first multimedia content. Optionally, the first work includes at least one of the following: literary works, audio works, film and television works, and paintings.

[0153] Optionally, if the above playback interface is provided by the first application, then based on interactive operations, the work information of the first work corresponding to the first multimedia content is displayed on the client of the first application.

[0154] In a demonstrative manner, after an interactive operation is triggered, the server performs content recognition on the first multimedia content. After recognizing the first work, the server sends the work information of the first work to the client of the first application.

[0155] Optionally, the information for the first work may include the work's title, creator, creation date, type, description, and duration. Different types of works may have different information, which is not limited here. For example, if the first work is a song, its information may include the song title, creator, and duration.

[0156] Optionally, the method for displaying the work information of the first work corresponding to the first multimedia content includes at least one of the following methods:

[0157] (1) Display the response information for the first interactive message in the interactive area.

[0158] The reply message includes information about the first work corresponding to the first multimedia content.

[0159] For illustrative purposes, if a user sends the first interactive message in the comment area of ​​the playback interface, the client of the first application can display the work information of the first work to the user in the form of a reply. For example, the client can reply to the first interactive message of the first account in the name of the target account, and the reply content of the target account includes the work information of the first work.

[0160] (2) Display the work information of the first work corresponding to the first multimedia content in the chat interface.

[0161] Optionally, the session interface is the interface provided by the client of the first application.

[0162] The conversation interface is used to display conversation messages between the first account and the target account, which is associated with the first interaction identifier.

[0163] For illustrative purposes, if a user sends the first interactive message in the comment area or bullet screen area of ​​the playback interface, the client of the first application can display the work information of the first work to the user via private message. For example, the first account can send a private message in the name of the target account, and the work information of the first work can be displayed in the conversation interface between the first account and the target account.

[0164] The target account mentioned above is associated with the first identification information. In some embodiments, the first interaction identifier can be implemented as a reminder identifier for the target account, such as "@target account name". Optionally, regarding the target account: the target account refers to an account registered on the target platform in the first application. If the work collection list described below is a collection list provided by the first application, then the target platform is the first application, for example: the target account is the official account of the first application; if the work collection list described below is a collection list provided by a second application, then the target platform is the second application, for example: the target account is the official account of the second application registered in the first application.

[0165] As an example, after the server recognizes the first work, it will also trigger the process of adding the first work to the work collection list.

[0166] Optionally, if the above-mentioned list of works is a list provided by the second application, then based on the interactive operation, the first work corresponding to the first multimedia content is added to the list of works in the client of the second application.

[0167] The current terminal has a client of a second application installed, and a second account is logged in on the client of the second application. In some embodiments, if there is an association between the first account and the second account, the first work corresponding to the first multimedia content is automatically added to the work collection list in the client of the second application. If there is no association between the first account and the second account, the work extraction information corresponding to the first work is displayed in the client of the first application; when the work extraction information is used in the client of the second application, the first work corresponding to the first multimedia content is automatically added to the work collection list in the client of the second application.

[0168] It should be noted that during the process of automatically adding the first work corresponding to the first multimedia content to the work collection list in the client of the second application, the current terminal may jump to the client of the second application. After successful addition, the list interface corresponding to the work collection list will be displayed in the client of the second application, which includes the first work; or, during the process of automatically adding the first work corresponding to the first multimedia content to the work collection list in the client of the second application, the terminal may not jump to the client of the second application, and the playback interface of the first multimedia content will still be displayed. After successful addition, a success notification will be displayed in the client of the first application. This success notification is used to notify that the first work has been successfully added to the work collection list in the client of the second application. The form of the success notification may include at least one of the following: pop-up notification, floating icon notification, push message, etc., which is not limited here.

[0169] In some embodiments, when the first multimedia content finishes playing, the client of the second application is automatically launched; the first work is displayed on the client of the second application. If the first multimedia content has not finished playing, a success notification is displayed on the client of the first application, indicating that the first work has been successfully added to the work list in the client of the second application.

[0170] For illustrative purposes, when the playback progress of the first multimedia content reaches the total duration of the multimedia content, it is determined to be in a playback end state; when the playback progress of the first multimedia content does not reach the total duration of the multimedia content, it is determined to be in a playback incomplete state.

[0171] Among them, "displaying the first work" refers to displaying the content of the first work, such as playing a song, opening the homepage of a novel, or playing a video, etc. There is no limitation here.

[0172] To illustrate, let's take a song as an example. If the song is successfully added to the playlist after the first multimedia content playback has finished, the second application automatically launches and plays the song from its playlist. If the song is successfully added to the playlist before the first multimedia content playback has finished, a notification is displayed on the first application's client, indicating that the song has been successfully added to the playlist in the second application's client.

[0173] In the above solution, the second application automatically launches and displays the first work after the first multimedia content finishes playing, providing users with a seamless content experience. If the first work is successfully added to the second application's work list before the first multimedia content finishes playing, a success notification is promptly displayed on the first application's client. This allows users to immediately know the completion status of the addition operation, avoiding the need for additional steps to find out, thus improving human-computer interaction efficiency.

[0174] In some embodiments, the above-mentioned successful addition notification also includes an application redirection entry, which is used to automatically launch the client of the second application and display the first work on the client of the second application. Illustratively, the successful addition notification displays an application redirection control; when the user clicks the application redirection control, the client of the second application is automatically launched and the first work is displayed on the client of the second application.

[0175] In summary, the interactive method based on multimedia content provided in this application allows users to trigger the system to identify the corresponding first work through interactive operations, avoiding the cumbersome process of manually searching or querying to determine the first work. After identifying the first work, the system not only automatically displays the work information to the user, allowing the user to more conveniently obtain work information related to the multimedia content, but also automatically adds the first work to the storage list, saving the user the step of manually adding the first work to the storage list and improving the efficiency of human-computer interaction.

[0176] In some embodiments, the first work includes a first song, which is a song obtained by identifying the background music of the first multimedia content; the work collection list includes a first playlist. (For illustrative purposes only, please refer to the provided text.) Figure 5 The above Figure 3 or Figure 4 The illustrated embodiment can also be implemented as follows: steps 501 to 504.

[0177] Step 501: Display the playback interface of the first multimedia content on the client of the first application.

[0178] The playback interface includes an interactive area used to display interactive information related to the first multimedia content. This first multimedia content includes background music.

[0179] Optionally, the first application can be implemented as a video playback platform, and the first multimedia content can be implemented as first video content. Schematic, the playback interface of the first video content is displayed on the client of the video playback platform, and the first video content includes background music.

[0180] The first application's client is logged into the first account.

[0181] Step 502: In response to receiving a sending operation for the first interactive information in the interactive area, the first interactive information is displayed in the interactive area.

[0182] The first interactive information includes a first interactive identifier, which is used to identify the background music of the first multimedia content.

[0183] To illustrate, when the first account triggers the sending operation of the first interactive information in the client of the first application, the client of the first application first sends the first interactive information to the server. The server, on the one hand, publishes the first interactive information to the interactive area in the client of the first application, so that the interactive area displays the first interactive information. On the other hand, the server triggers the recognition process of the first interactive information. If the first interactive identifier is recognized in the first interactive information, the server will identify the background music of the first multimedia content. The following is a description of the process by which the server identifies the background music of the first multimedia content:

[0184] (1) Obtain the background music of the first multimedia content.

[0185] To illustrate, the server extracts the audio portion from the first multimedia content. Taking the first multimedia content as the first video content as an example, the server will use an audio extraction algorithm to separate the audio stream in the first video content to obtain the audio portion.

[0186] The audio component may include various sound elements, such as background music, dialogue, and environmental sound effects. In this case, audio separation technology is needed to separate the background music from the audio component. Audio separation technology includes spectral subtraction based on signal processing, independent component analysis, etc., which are not limited here.

[0187] (2) Extract the audio feature representation corresponding to the background music.

[0188] As an illustration, the server extracts features from the acquired background music to obtain the corresponding audio feature representation. The audio feature representation includes, but is not limited to, the audio spectral feature representation (such as Mel frequency cepstral coefficients), rhythm feature representation (such as beat, rhythm intensity, etc.), timbre feature representation (such as harmonic structure, formants, etc.), etc., which are not limited here.

[0189] (3) Match the audio feature representation of the background music with the song database to obtain the recognition result.

[0190] For illustrative purposes, the song database is a database provided by the client of the second application. The song database includes audio feature representations corresponding to multiple songs. The audio feature representation corresponding to the background music is matched one-to-one with the multiple audio feature representations in the song database. For example, for each audio feature representation in the song database, the similarity between that audio feature representation and the audio feature representation corresponding to the background music is calculated. Similarity metrics include Euclidean distance, cosine similarity, etc., which are not limited here.

[0191] After determining the similarity between multiple audio feature representations in the song database and the audio feature representations corresponding to the background music, the songs corresponding to the audio feature representations with a similarity greater than or equal to the preset similarity are taken as the recognition results. If there are multiple audio feature representations with a similarity greater than or equal to the preset similarity, the song corresponding to the audio feature representation with the highest similarity is taken as the recognition result.

[0192] Step 503: Based on the first interactive information, display the song information of the first song corresponding to the first multimedia content on the client of the first application.

[0193] Optionally, if the server identifies the background music of the first multimedia content as the first song, it will send the song information of the first song to the client of the first application. After receiving the song information of the first song, the client of the first application will display the target account's reply to the first interactive information in the interactive area, and the reply information includes the song information of the first song; or, the reply information of the first interactive information will be displayed in the conversation interface provided by the client of the first application.

[0194] The aforementioned first interactive identifier can be implemented as a notification identifier for the target account, such as "@target account name". Optionally, the target account refers to an account registered by the second application in the first application, for example, the target account is the official account registered by the second application in the first application.

[0195] Step 504: Add the first song to the first playlist provided by the client of the second application.

[0196] The second application has a second account logged in on its client, and the second application can be implemented as a music playback platform.

[0197] Optionally, if the server identifies the background music of the first multimedia content as the first song, it will also add the first song to the first playlist provided by the client of the second application.

[0198] In some embodiments, if the first account and the second account are associated, the first song is automatically added to the first playlist in the client of the second application. If the first account and the second account are not associated, the work extraction information corresponding to the first song is displayed in the client of the first application; when the song extraction information is used in the client of the second application, the first song is automatically added to the first playlist in the client of the second application.

[0199] In some embodiments, the first interactive information further includes storage instruction information, which instructs that the first song obtained by recognizing the first multimedia content be added to the first playlist.

[0200] Indicatively, the inclusion instruction information includes the name and identifier of the first playlist, etc., which are not limited here. For example, if the inclusion instruction information is: add background music to the "I like" playlist, then when the background music of the first multimedia content is identified as the first song, the first song will be added to the "I like" playlist in the client of the second application, such as adding the relevant icon (such as album cover) and name of the first song to the "I like" playlist.

[0201] In some embodiments, the first song corresponding to the first multimedia content is added to a first playlist in the client of the second application; wherein, the second account in the client of the second application has multiple candidate playlists; if multiple candidate playlists match the first song, the first playlist is the playlist determined from the multiple candidate playlists; if multiple candidate playlists do not match the first song, the first playlist is a playlist created based on the song information of the first song and the first multimedia content.

[0202] To illustrate, after determining that the background music of the first multimedia content includes the first song, the server will obtain multiple candidate playlists associated with the second account in the client of the second application through an interface with the second application or by querying a relevant database. For example, playlists already created in the second account.

[0203] The server will extract the song feature representation corresponding to the first song from the song information of the first song. The song information of the first song includes, but is not limited to: song name, artist information, song style (such as rock, pop, classical, etc.), song duration, song rhythm characteristics (such as average number of beats, rhythm change curve, etc.), song emotional tone (cheerful, sad, lyrical, etc.), and song release year, etc.

[0204] For each candidate playlist, its feature representation is extracted. These features include the playlist name, playlist creator information, playlist theme tags (such as "classic oldies," "popular songs," etc.), the average style characteristics of the songs already included in the playlist (such as overall musical style tendency, average duration range, etc.), and the playlist's creation time and update frequency. Through comprehensive analysis and processing of this information, a playlist feature representation for each candidate playlist is constructed.

[0205] Using a suitable matching algorithm (such as a similarity calculation algorithm based on a vector space model), the song feature representation of the first song is matched one by one with the feature representations of the corresponding multiple candidate playlists to obtain the matching degree between the first song and each of the multiple candidate playlists.

[0206] If a single candidate playlist matches the first song with a match score greater than a preset matching threshold, then that single candidate playlist will be used as the first playlist.

[0207] If multiple candidate playlists match the first song with a match degree greater than a preset matching threshold, the first playlist is determined from these candidate playlists according to a priority rule. The priority rule can be determined based on factors such as playlist usage frequency (e.g., the playlist with the most recent plays takes priority), the match degree between the playlist and the first song (the playlist with the highest match degree takes priority), and the number of times the playlist is favorited (the playlist with the most favorites takes priority). For example, the playlist with the most recent plays and the most favorites among the candidate playlists with a match degree greater than the preset matching threshold can be selected as the first playlist.

[0208] If no candidate playlist matches the first song with a match score greater than the preset threshold, a completely new playlist can be created as the first playlist based on the song information and the first multimedia content. For example, key information from the first song is extracted, such as the song title, artist name, and main genre (e.g., rock, pop, classical, jazz), as well as key information from the first multimedia content, such as its title and type. A playlist name is generated based on at least one of these key information pieces. This playlist name is then used to create the first playlist for the second account, and the name of the first playlist is also the playlist name.

[0209] After determining the first playlist, the server will send a song collection request to the client of the second application. Upon receiving the song collection request, the client of the second application will add the relevant information of the first song (such as the storage path of the song's audio file, the song's metadata information, etc.) to the database record or storage structure corresponding to the first playlist, thereby completing the addition operation of the first song.

[0210] In the above embodiments, by extracting the song feature representation of the first song and the playlist feature representation of each candidate playlist in detail, and using a matching algorithm for calculation, the matching degree between the first song and the candidate playlist can be accurately determined. When a candidate playlist with a high matching degree exists, the first song is added to a suitable existing playlist, making the song arrangement more reasonable; when no candidate playlist matches the first song, a brand new playlist is automatically created based on the song information and the first multimedia content, eliminating the need for the user to create playlists manually and improving the efficiency of human-computer interaction.

[0211] In summary, the interactive method based on multimedia content provided in this application allows users to trigger the system to identify the corresponding first song through interactive operations, avoiding the cumbersome process of manually searching or querying to determine the first song. After identifying the first song, the system not only automatically displays the song information to the user, allowing them to more conveniently obtain information about works related to the multimedia content, but also automatically adds the first song to the playlist, saving the user the step of manually adding the first song to the playlist and improving the efficiency of human-computer interaction.

[0212] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0213] Please refer to Figure 6 This diagram illustrates a structural block diagram of an interactive device based on multimedia content according to an embodiment of this application. The device has the functions described above, which can be implemented in hardware or by hardware executing corresponding software. Figure 6 As shown, the device includes:

[0214] Display module 610 is used to display a playback interface for first multimedia content, the playback interface including an interactive area, the interactive area being used to display interactive information published for the first multimedia content;

[0215] The receiving module 620 is used to receive interactive operations in the interactive area for the first interactive information, wherein the first interactive information includes a first interactive identifier, and the first interactive identifier is used to identify the content of the first multimedia content.

[0216] The processing module 630 is used to add the first work corresponding to the first multimedia content to the work collection list based on the interactive operation. The first work is the work obtained after content recognition of the first multimedia content.

[0217] In some embodiments, the playback interface is an interface provided by a first application, and the work collection list is a collection list provided by a second application; the display module 610 is used to display the playback interface of the first multimedia content on the client of the first application, and a first account is logged in on the client of the first application; the processing module 630 is used to add the first work corresponding to the first multimedia content to the work collection list on the client of the second application based on the interactive operation, and a second account is logged in on the client of the second application.

[0218] In some embodiments, the processing module 630 is configured to automatically add the first work corresponding to the first multimedia content to the work collection list in the client of the second application when there is an association between the first account and the second account.

[0219] In some embodiments, the processing module 630 is configured to display the work extraction information corresponding to the first work in the client of the first application when there is no association between the first account and the second account; and when the work extraction information is used in the client of the second application, automatically add the first work corresponding to the first multimedia content to the work collection list in the client of the second application.

[0220] In some embodiments, the first work includes a first song, which is a song obtained by identifying the background music of the first multimedia content; the work collection list includes a first playlist; the processing module 630 is used to add the first song corresponding to the first multimedia content to the first playlist in the client of the second application; wherein, the second account in the client of the second application has multiple candidate playlists; when the multiple candidate playlists match the first song, the first playlist is a playlist determined from the multiple candidate playlists; when the multiple candidate playlists do not match the first song, the first playlist is a playlist created based on the song information of the first song and the first multimedia content.

[0221] In some embodiments, the processing module 630 is configured to automatically launch the client of the second application when the first multimedia content finishes playing; display the first work on the client of the second application; and display a successful addition notification on the client of the first application when the first multimedia content has not finished playing, wherein the successful addition notification is used to notify that the first work has been successfully added to the work collection list in the client of the second application.

[0222] In some embodiments, the display module 610 is used to display the work information of the first work corresponding to the first multimedia content based on the interactive operation.

[0223] In some embodiments, the first interactive information further includes storage instruction information, which instructs that the first work obtained by identifying the first multimedia content be added to the first work storage list; the processing module 630 is used to add the first work corresponding to the first multimedia content to the first work storage list based on the interactive operation, wherein the first work storage list is a list determined according to the storage instruction information.

[0224] In some embodiments, the works obtained after content recognition of the first multimedia content include a first work and a second work; the processing module 630 is used to add the first work corresponding to the first multimedia content to the second work collection list and add the second work corresponding to the first multimedia content to the third work collection list based on the interactive operation; wherein, the first work matches the collection work type corresponding to the second work collection list, and the second work matches the collection work type corresponding to the third work collection list.

[0225] In some embodiments, the processing module 630 is configured to: obtain multiple candidate work collection lists based on the interactive operation; extract list feature representations corresponding to the multiple candidate work collection lists respectively, wherein the list feature representations are used to characterize the types of works collected in the candidate work collection lists; wherein the list feature representations are feature representations determined based on the collected work information and list attribute information of the candidate work collection lists, and the collected work information refers to the information of the works collected in the candidate work collection lists; extract a first work feature representation of a first work corresponding to the first multimedia content, and extract a second work feature representation of a second work corresponding to the first multimedia content; if the first work feature representation matches the list feature representation corresponding to the first candidate work collection list in the multiple candidate work collection lists, determine the first candidate work collection list as the second work collection list; if the first work feature representation matches the list feature representation corresponding to the second candidate work collection list in the multiple candidate work collection lists, determine the second candidate work collection list as the third work collection list; add the first work to the second work collection list, and add the second work to the third work collection list.

[0226] In some embodiments, the first interactive information further includes content recognition prompt information, which is used to instruct content recognition of the first multimedia segment in the first multimedia content; the processing module 630 is used to add the first work corresponding to the first multimedia segment in the first multimedia content to the work collection list based on the interactive operation, wherein the work collection list is the works obtained after content recognition of the first multimedia segment.

[0227] In summary, the interactive device based on multimedia content provided in this application allows users to automatically complete the identification process of multimedia content and add the identified first work to the work collection list simply by triggering an interactive operation on the multimedia content playback interface (including the first interactive identifier). On the one hand, users can trigger the system to identify the corresponding first work through interactive operations, avoiding the cumbersome process of manually searching or querying to determine the first work. On the other hand, after identifying the first work, the system can automatically add it to the collection list, saving users the step of manually adding the first work to the collection list. This makes it easier for users to obtain and save works of interest when browsing multimedia content, improving the efficiency of human-computer interaction.

[0228] Please refer to Figure 7 This diagram illustrates a structural block diagram of a computer device 700 provided in one embodiment of this application. The computer device 700 may be... Figure 1 The terminal shown can also be Figure 1 The server shown. Specifically:

[0229] Typically, computer device 700 includes a processor 710 and a memory 720.

[0230] Processor 710 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 710 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 710 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 710 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 710 may also include an AI processor for handling computational operations related to machine learning.

[0231] The memory 720 may include one or more computer-readable storage media, which may be non-transitory. The memory 720 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 720 are used to store a computer program configured to be executed by one or more processors to implement the above-described interactive method based on multimedia content.

[0232] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the computer device 700, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0233] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein a computer program is stored in the storage medium, and the computer program, when executed by a processor, implements the above-described interactive method based on multimedia content. Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0234] In an exemplary embodiment, a computer program product is also provided, the computer program product including a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform the above-described interactive method based on multimedia content.

[0235] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0236] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An interactive method based on multimedia content, characterized in that, The method includes: The playback interface displays the first multimedia content, and the playback interface includes an interactive area for displaying interactive information published in response to the first multimedia content. Receive an interactive operation in the interactive area for a first interactive information, the first interactive information including a first interactive identifier, the first interactive identifier being used to identify the first multimedia content; the first interactive identifier is a reminder identifier for mentioning a target account; Based on the interactive operation, the target account's response information to the first interactive information is displayed in the interactive area. The response information includes the work information of the first work corresponding to the first multimedia content. The first work is a work obtained after content recognition of the first multimedia content. Add the first work to the work collection list.

2. The method according to claim 1, characterized in that, The playback interface is provided by the first application, and the work storage list is provided by the second application. The playback interface for displaying the first multimedia content includes: The client of the first application displays the playback interface of the first multimedia content, and the client of the first application is logged in with the first account; Adding the first work to the work collection list includes: The first work is added to the work collection list in the client of the second application, and the client of the second application is logged in with a second account.

3. The method according to claim 2, characterized in that, The process of adding the first work to the work collection list in the client of the second application includes: If the first account and the second account are associated, the first work will be automatically added to the work collection list in the client of the second application.

4. The method according to claim 2, characterized in that, The process of adding the first work to the work collection list in the client of the second application includes: If there is no association between the first account and the second account, the work extraction information corresponding to the first work is displayed on the client of the first application; When the information is extracted from the work in the client of the second application, the first work is automatically added to the work collection list in the client of the second application.

5. The method according to claim 2, characterized in that, The first work includes a first song, which is a song obtained by identifying the background music of the first multimedia content; the work collection list includes a first playlist. The process of adding the first work to the work collection list in the client of the second application includes: Add the first song to the first playlist in the client of the second application; In the client of the second application, the second account corresponds to multiple candidate playlists; when the multiple candidate playlists match the first song, the first playlist is a playlist determined from the multiple candidate playlists; when the multiple candidate playlists do not match the first song, the first playlist is a playlist created based on the song information of the first song and the first multimedia content.

6. The method according to claim 2, characterized in that, After adding the first work to the work collection list in the client of the second application, the method further includes: When the first multimedia content finishes playing, the client of the second application is automatically launched; the first work is displayed on the client of the second application. If the first multimedia content has not finished playing, a notification of successful addition is displayed on the client of the first application. The notification of successful addition is used to notify that the first work has been successfully added to the work collection list in the client of the second application.

7. The method according to any one of claims 1 to 6, characterized in that, The first interactive information also includes storage instruction information, which instructs that the first work obtained by recognizing the first multimedia content be added to the first work storage list; Adding the first work to the work collection list includes: Add the first work to the first work collection list.

8. The method according to any one of claims 1 to 6, characterized in that, The works obtained after content recognition of the first multimedia content include the first work and the second work; Adding the first work to the work collection list includes: Add the first work to the second work list, and add the second work to the third work list; Specifically, the first work matches the storage type of the second work in the storage list, and the second work matches the storage type of the third work in the storage list.

9. The method according to claim 8, characterized in that, Before adding the first work to the second work list and the second work to the third work list, the process includes: Extract list feature representations corresponding to multiple candidate work collection lists respectively. The list feature representations are used to characterize the types of works collected in the candidate work collection lists. The list feature representations are determined based on the work information and list attribute information of the candidate work collection lists. The work information refers to the information of the works collected in the candidate work collection lists. Extract the first work feature representation of the first work, and extract the second work feature representation of the second work; If the first work feature representation matches the list feature representation of the first candidate work collection list in the plurality of candidate work collection lists, the first candidate work collection list is determined as the second work collection list; If the second work feature representation matches the list feature representation of the second candidate work collection list in the plurality of candidate work collection lists, the second candidate work collection list is determined as the third work collection list.

10. The method according to any one of claims 1 to 6, characterized in that, The first interactive information also includes content recognition prompt information, which is used to instruct content recognition of the first multimedia segment in the first multimedia content; Adding the first work to the work collection list includes: Add the first work corresponding to the first multimedia segment to the work collection list. The first work is the work obtained after content recognition of the first multimedia segment.

11. An interactive device based on multimedia content, characterized in that, The device includes: A display module is used to display a playback interface for the first multimedia content. The playback interface includes an interactive area, which is used to display interactive information published in response to the first multimedia content. The receiving module is configured to receive interactive operations in the interactive area in response to first interactive information, the first interactive information including a first interactive identifier, the first interactive identifier being used to identify the first multimedia content; the first interactive identifier is a reminder identifier for mentioning a target account; The processing module is configured to, based on the interactive operation, display the target account's response information to the first interactive information in the interactive area, the response information including the work information of the first work corresponding to the first multimedia content, the first work being the work obtained after content recognition of the first multimedia content; and add the first work to the work collection list.

12. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the interactive method based on multimedia content as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is loaded and executed by a processor to implement the interactive method based on multimedia content as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the interactive method based on multimedia content as described in any one of claims 1 to 10.

Images