Content interaction method, device, computer equipment and storage medium

Abstract

The present application provides a content interaction method, apparatus, computer device, and storage medium that can be applied to the field of artificial intelligence to solve the problem of low content interaction efficiency. The method includes: receiving initial multimedia content sent by a first object to a second object, extracting initial feature information of the initial multimedia content; generating at least one reply multimedia content that matches the initial feature information based on the association relationship between each multimedia content and each feature information contained in a reference mapping relationship; determining target reply multimedia content from the at least one reply multimedia content; and sending the target reply multimedia content to the first object. In this way, reply content that matches the initial multimedia content can be automatically determined, thereby improving the efficiency of content interaction.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a content interaction method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the continuous development of technology, more and more devices can provide content interaction functions for virtual objects. For example, a first virtual object can edit first content on a first device and send the first content to a second virtual object. After receiving the first content from the first virtual object, the second virtual object presents the first content on the second device. The second virtual object can edit second content on the second device and reply to the first virtual object based on the first content. After receiving the second content, the first virtual object presents the second content on the first device, thus achieving the purpose of content interaction.

[0003] However, during the content interaction process, when the virtual object replies, the device needs to wait for the virtual object to view the received content, wait for the virtual object to trigger a reply operation for the content on the interface, and also wait for the virtual object to edit the reply content, making the virtual object's reply process less efficient, and thus the content interaction efficiency is lower. Summary of the Invention

[0004] Embodiments of the present application provide a content interaction method, apparatus, computer device, and storage medium for solving the problem of low efficiency of content interaction.

[0005] In a first aspect, a content interaction method is provided, comprising:

[0006] receiving initial multimedia content sent by the first object to the second object, and extracting initial feature information of the initial multimedia content;

[0007] generating at least one reply multimedia content matching the initial feature information based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship;

[0008] determining target reply multimedia content from the at least one reply multimedia content;

[0009] The target reply multimedia content is sent to the first object.

[0010] In a second aspect, a content interaction method is provided, comprising:

[0011] Presenting, in the first display interface, initial multimedia content sent by the first object to the second object;

[0012] Presenting, in a second display interface, at least one piece of reply multimedia content obtained for the initial multimedia content, wherein the at least one piece of reply multimedia content is generated based on feature information of the initial multimedia content;

[0013] Target reply multimedia content is presented in the first display interface, wherein the target reply multimedia content is obtained based on at least one reply multimedia content.

[0014] In a third aspect, a content interaction device is provided, comprising:

[0015] The transceiver module is configured to receive the initial multimedia content sent by the first object to the second object, and extract the initial feature information of the initial multimedia content;

[0016] A processing module is configured to generate at least one reply multimedia content that matches the initial feature information based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship;

[0017] The processing module is further configured to: determine target reply multimedia content from the at least one reply multimedia content;

[0018] The transceiver module is further configured to send the target reply multimedia content to the first object.

[0019] Optionally, the initial feature information includes the initial image dimension feature vector, and the initial image dimension feature vector is obtained by performing image feature extraction processing on the initial multimedia content. The processing module is specifically configured to:

[0020] Based on the feature information associated with each multimedia content included in the reference mapping relationship, respectively determining a first similarity between the initial image dimension feature vector and the image dimension feature vector included in each feature information, wherein, if the multimedia content is not an image, the image dimension feature vector is obtained by performing image feature extraction processing on a related image associated with the multimedia content in the related mapping relationship, and the related mapping relationship represents the association relationship between each multimedia content and each related image;

[0021] At least one reply multimedia content is generated based on the multimedia content associated with the image dimension feature vector whose first similarity is greater than a first similarity threshold.

[0022] Optionally, the processing module is specifically configured to:

[0023] For the obtained image dimension feature vector having a first similarity greater than a first similarity threshold, determining whether the corresponding feature information includes an association tag, wherein the association tag indicates that a priority of an association relationship between the corresponding feature information and the multimedia content meets a preset priority condition;

[0024] At least one reply multimedia content is generated based on the multimedia content containing the associated tag in the feature information.

[0025] Optionally, the processing module is further configured to:

[0026] After determining the target reply multimedia content from the at least one reply multimedia content, if the initial multimedia content is an image, an association relationship is established in the relevant mapping relationship based on the initial multimedia content and the target reply multimedia content.

[0027] Counting the number of objects that trigger the establishment of an association relationship between the initial multimedia content and the target reply multimedia content;

[0028] If it is determined that the number of objects reaches a preset threshold, feature information associated with the multimedia content is replied to the target in the reference mapping relationship, and an associated tag is added.

[0029] Optionally, the initial feature information includes the initial text dimension feature vector, and the initial text dimension feature vector is obtained by performing text recognition processing on the initial multimedia content. The processing module is specifically configured to:

[0030] Based on the feature information corresponding to each multimedia content included in the reference mapping relationship, respectively determining a second similarity between the initial text dimension feature vector and the text dimension feature vector included in each feature information;

[0031] At least one reply multimedia content is generated based on the multimedia content corresponding to the feature information whose second similarity is greater than a second similarity threshold.

[0032] Optionally, the initial feature information includes the initial attribute dimension feature vector, and the initial attribute dimension feature vector is obtained by performing object recognition processing on the object included in the initial multimedia content. The processing module is specifically configured to:

[0033] Determining, based on the feature information corresponding to each multimedia content included in the reference mapping relationship, a third similarity between the initial attribute dimension feature vector and the attribute dimension feature vector included in each feature information;

[0034] At least one reply multimedia content is generated based on the multimedia content corresponding to the feature information whose third similarity is greater than a third similarity threshold.

[0035] Optionally, the processing module is specifically configured to:

[0036] generating, based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, at least one reference multimedia content whose feature information matches the initial feature information;

[0037] For each piece of reference multimedia content, do the following:

[0038] Determining a magnitude relationship between a first number of frames included in the initial multimedia content and a second number of frames included in the reference multimedia content;

[0039] If the number of the first frames is not equal to the number of the second frames, a frame supplementation strategy is adopted to supplement the initial multimedia content and the reference multimedia content, and the initial multimedia content and the reference multimedia content after the supplementation are merged to obtain corresponding response multimedia content;

[0040] If the number of the first frames is equal to the number of the second frames, the initial multimedia content and the reference multimedia content are merged to obtain corresponding reply multimedia content.

[0041] Optionally, the processing module is specifically configured to:

[0042] If the first number of frames is greater than the second number of frames, performing frame interpolation on the reference multimedia content according to a preset frame interpolation strategy, and fusing the initial multimedia content with the reference multimedia content after the frame interpolation to obtain reply multimedia content;

[0043] If the first frame number is less than the second frame number, the initial multimedia content is interpolated according to the interpolation strategy, and the interpolated initial multimedia content is merged with the reference multimedia to obtain the reply multimedia content.

[0044] Optionally, the processing module is specifically configured to:

[0045] Obtaining each key frame included in the reference multimedia content based on a frame format of each frame included in the reference multimedia content;

[0046] determining a fourth similarity between every two key frames based on the frame features corresponding to each of the key frames;

[0047] If there are two key frames whose fourth similarity is greater than a preset fourth similarity threshold, the frames between the two key frames are looped according to the first number of frames to perform frame supplementation processing on the reference multimedia content.

[0048] Optionally, the processing module is specifically configured to:

[0049] If the at least one reply multimedia content includes a reply multimedia content, determining the reply multimedia content as a target reply multimedia content;

[0050] If the at least one reply multimedia content includes multiple reply multimedia contents, target reply multimedia content is determined from the multiple reply multimedia contents based on the received selection instruction.

[0051] Optionally, the processing module is specifically configured to:

[0052] If the reference mapping relationship includes multiple mapping sub-relationships, then based on the multimedia type of the initial multimedia content included in the initial feature information and a preset screening strategy, determining at least one mapping sub-relationship from the multiple mapping sub-relationships, wherein the multimedia content included in different mapping sub-relationships corresponds to different multimedia types;

[0053] Based on the association relationship between each multimedia content and each feature information contained in each of the at least one mapping sub-relationships, at least one reply multimedia content matching the initial feature information is generated.

[0054] Optionally, the reference mapping relationship is obtained by the processing module using the following method:

[0055] Obtaining various multimedia contents from network resources;

[0056] For each multimedia content, perform the following operations respectively:

[0057] intercepting and processing the multimedia content based on interactive data associated with the multimedia content in the network resource, wherein the interactive data is obtained based on interactive operations triggered by various objects on the multimedia content;

[0058] Determining, based on an audio file corresponding to the intercepted and processed multimedia content, whether the audio file contains specified audio content;

[0059] If the audio file contains the designated audio content, an association relationship is established in the reference mapping relationship based on the intercepted and processed multimedia content and feature information of the intercepted and processed multimedia content.

[0060] In a fourth aspect, a content interaction device is provided, comprising:

[0061] A first display module: configured to present the initial multimedia content sent by the first object to the second object in the first display interface;

[0062] A second display module: configured to present, in a second display interface, at least one reply multimedia content obtained for the initial multimedia content, wherein the at least one reply multimedia content is generated based on feature information of the initial multimedia content;

[0063] The first display module is further configured to present target reply multimedia content in the first display interface, wherein the target reply multimedia content is obtained based on at least one reply multimedia content.

[0064] Optionally, the second display module is specifically used to:

[0065] If the at least one reply multimedia content includes multiple reply multimedia contents, each time a reply multimedia content is obtained, the reply multimedia content is presented in the second display interface.

[0066] Optionally, the first display module is specifically used to:

[0067] If the at least one reply multimedia content includes a reply multimedia content, taking the reply multimedia content as the target reply multimedia content;

[0068] If the at least one reply multimedia content includes multiple reply multimedia contents, determining a target reply multimedia content in response to a selection operation triggered by the second object on the at least one reply multimedia content;

[0069] The target reply multimedia content is presented in the first display interface.

[0070] Optionally, the at least one reply multimedia content is generated by the first display module using the following method:

[0071] Based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, at least one reply multimedia content whose feature information matches the initial feature information is generated.

[0072] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method as described in the first aspect or the second aspect.

[0073] According to a sixth aspect, a computer device is provided, comprising:

[0074] a memory for storing program instructions;

[0075] The processor is configured to call the program instructions stored in the memory and execute the method according to the first aspect or the second aspect according to the obtained program instructions.

[0076] In a seventh aspect, a computer-readable storage medium is provided, wherein the storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method as described in the first aspect or the second aspect.

[0077] In an embodiment of the present application, after receiving the initial multimedia content sent by the first object to the second object, the initial feature information of the initial multimedia content can be extracted. Thus, based on the initial feature information, at least one reply multimedia content whose feature information matches the initial feature information can be determined in the reference mapping relationship. There is no need to wait for a response to the second object's viewing operation on the initial multimedia content, nor is there a need to wait for a response to the second object's reply operation on the initial multimedia content, nor is there a need to wait for a response to the second object's editing operation, nor is there a need to wait for a response to the second object's browsing operation triggered by a large number of multimedia contents. At least one reply multimedia content can be determined, thereby improving the efficiency of content interaction.

[0078] Furthermore, a target reply multimedia content can be determined from the at least one reply multimedia content and sent to the first object. After receiving the initial multimedia content, the initial multimedia content can be replied to without waiting for any response operation, thereby improving the efficiency of content interaction.

[0079] Furthermore, at least one reply multimedia content is related to the initial multimedia content and can be used to reply to the initial multimedia content, rather than randomly generated content or fixed content. Therefore, the target multimedia content determined from the at least one reply multimedia content is more accurate, thereby improving the accuracy of content interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A schematic diagram of the content interaction method provided in this embodiment of the application Figure 1 ;

[0081] Figure 2 An application scenario of the content interaction method provided in the embodiment of the present application;

[0082] Figure 3 A flow diagram of the content interaction method provided in this embodiment of the application Figure 1 ;

[0083] Figure 4a A flow diagram of the content interaction method provided in this embodiment of the application Figure 2 ;

[0084] Figure 4b A schematic diagram of the content interaction method provided in this embodiment of the application Figure 2 ;

[0085] Figure 4c A schematic diagram of the content interaction method provided in this embodiment of the application Figure 3 ;

[0086] Figure 4d A fourth schematic diagram of a principle of a content interaction method provided in an embodiment of the present application;

[0087] Figure 4e A schematic diagram of the content interaction method provided in this embodiment of the application Figure 5 ;

[0088] Figure 4f Schematic diagram 6 of a principle of the content interaction method provided in an embodiment of the present application;

[0089] Figure 4g A schematic diagram of the content interaction method provided in this embodiment of the application Figure 7 ;

[0090] Figure 5 A flow diagram of the content interaction method provided in this embodiment of the application Figure 3 ;

[0091] Figure 6a Schematic diagram 4 of a flow chart of a content interaction method provided in an embodiment of the present application;

[0092] Figure 6b A schematic diagram of the content interaction method provided in this embodiment of the application Figure 8 ;

[0093] Figure 7 A structural diagram of the content interaction method provided in the embodiment of the present application Figure 1 ;

[0094] Figure 8 A structural diagram of a content interaction device provided in an embodiment of the present application Figure 2 ;

[0095] Figure 9 A structural diagram of a content interaction device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0096] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0097] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0098] (1) Multimedia content:

[0099] Multimedia content may include images and sounds. Image content may include pictures, animated images, short videos, or long videos. Sound content may include voice, ringtones, or noise.

[0100] The embodiments of the present application relate to artificial intelligence (AI) technology, which is designed based on computer vision (CV), speech technology, natural language processing (NLP), and machine learning (ML) in AI technology.

[0101] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0102] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0103] Computer vision is the study of how machines can "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. Further image processing is performed to transform the computer's image into an image more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, and smart transportation. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0104] Key technologies in speech technology include automatic speech recognition, speech synthesis, and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech becoming one of the most promising methods of human-computer interaction.

[0105] Natural language processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to linguistics. NLP technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0106] Machine learning is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0107] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, intelligent content interaction, unmanned driving, autonomous driving, drones, robots, smart medical care, intelligent customer service, Internet of Vehicles, autonomous driving, smart transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0108] The following is a brief introduction to the application fields of the content interaction method provided in the embodiments of the present application.

[0109] With the continuous development of technology, more and more devices can provide content interaction functions for virtual objects. For example, a first object can edit first content on a first device and send the first content to a second object. After receiving the first content from the first object, the second object can present the first content on the second device. The second object can edit second content on the second device and reply to the first object based on the first content. After receiving the second content, the first object will present the second content on the first device, thus achieving the purpose of content interaction.

[0110] However, during the content interaction process, when the virtual object replies, the device needs to wait for the virtual object to view the received content, wait for the virtual object to trigger a reply operation for the content on the display interface, and wait for the virtual object to edit the reply content, etc.

[0111] For example, the first object sends an emoticon to the second object through the first device, and the second object receives the emoticon sent by the first object on the second device. If the second object wants to reply with a corresponding emoticon to the emoticon sent by the first object, the second object can trigger a reply operation on the second device. The second device can display various emoticons collected by the second object at historical moments. The second object can select a corresponding emoticon from each emoticon and trigger a selection operation for the corresponding emoticon to send the corresponding emoticon to the first object as a reply. If the second object does not find a corresponding emoticon among the emoticons, the second object can trigger an edit operation on the second device and reply with corresponding text instead of the picture.

[0112] For example, a first party sends an emoticon to a second party via a first device. The second party receives the emoticon on a second device. If the second party wants to reply with an audio message in response to the emoticon sent by the first party, the second party can trigger a reply operation on the second device to record the second party's voice. After the second party completes the audio recording, it responds to the completion operation triggered on the second device and sends the audio message to the first party as a reply.

[0113] It can be seen that the reply process of virtual objects is inefficient, which makes the content interaction less efficient.

[0114] In order to solve the problem of low efficiency of content interaction, this application proposes a content interaction method. Figure 1After receiving initial multimedia content sent by a first object to a second object, the method extracts initial feature information of the initial multimedia content. Based on the association between each multimedia content and each feature information contained in a reference mapping relationship, at least one reply multimedia content matching the initial feature information is generated. After obtaining the at least one reply multimedia content, target reply multimedia content is determined from the at least one reply multimedia content, and the target reply multimedia content is sent to the first object.

[0115] The first object and the second object may be the same object or different objects, and are not specifically limited. The first object and the second object represent two parties interacting with content, such as virtual accounts logged in on a client, and are not specifically limited.

[0116] In an embodiment of the present application, after receiving the initial multimedia content sent by the first object to the second object, the initial feature information of the initial multimedia content can be extracted. Thus, based on the initial feature information, at least one reply multimedia content whose feature information matches the initial feature information can be determined in the reference mapping relationship. There is no need to wait for a response to the second object's viewing operation on the initial multimedia content, nor is there a need to wait for a response to the second object's reply operation on the initial multimedia content, nor is there a need to wait for a response to the second object's editing operation, nor is there a need to wait for a response to the second object's browsing operation triggered by a large number of multimedia contents. At least one reply multimedia content can be determined, thereby improving the efficiency of content interaction.

[0117] Furthermore, a target reply multimedia content can be determined from the at least one reply multimedia content and sent to the first object. After receiving the initial multimedia content, the initial multimedia content can be replied to without waiting for any response operation, thereby improving the efficiency of content interaction.

[0118] Furthermore, at least one reply multimedia content is related to the initial multimedia content and can be used to reply to the initial multimedia content, rather than randomly generated content or fixed content. Therefore, the target multimedia content determined from the at least one reply multimedia content is more accurate, thereby improving the accuracy of content interaction.

[0119] The following describes the application scenarios of the content interaction method provided by this application.

[0120] Please refer to Figure 2, which is an application scenario of the content interaction method provided in an embodiment of the present application. The application scenario includes a client 101 and a server 102. The client 101 includes a first client 1011 and a second client 1012. The first client 1011 and the server 102 can communicate with each other, and the second client 1012 and the server 102 can communicate with each other. The communication method can be to use wired communication technology, such as communicating by connecting a network cable or a serial cable; or to use wireless communication technology, such as communicating by Bluetooth or wireless fidelity (WIFI) and other technologies, without specific limitation.

[0121] The first client 1011 generally refers to a device that can provide initial multimedia content, such as a terminal device, a third-party application accessible to the terminal device, or a webpage accessible to the terminal device. The second client 1012 generally refers to a device that can provide targeted response multimedia content in response to the initial multimedia content, such as a terminal device, a third-party application accessible to the terminal device, or a webpage accessible to the terminal device. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, and in-vehicle terminals. The server 102 generally refers to a device that can send data to the first client 1011 and the second client 1012, and can forward or process data sent by the first client 1011 and the second client 1012, such as a terminal device or a server. Servers include but are not limited to cloud servers, local servers, or associated third-party servers. The first client 1011, the second client 1012, and the server 103 can all use cloud computing to reduce the use of local computing resources; similarly, cloud storage can be used to reduce the use of local storage resources.

[0122] As an embodiment, the first client 1011 and the second client 1012 can be the same device, the first client 1011 and the server 102 can be the same device, and the first client 1011, the second client 1012 and the server 102 can be the same device, without specific limitation. In the embodiment of the present application, the first client 1011, the second client 1012 and the server 102 are different devices as an example for description. In the embodiment of the present application, the first client 1011 is the client corresponding to the first object, and the second client 1012 is the client corresponding to the second object as an example for description. In fact, the first object and the second object can also be the same object and correspond to the same client.

[0123] The following is based on Figure 2 From the perspective of the server, the content interaction method provided in the embodiment of the present application is specifically introduced. In the embodiment of the present application, the second client 1012 corresponding to the second object is used as the target client as an example for introduction.

[0124] Please refer to Figure 3 , which is a flow chart of the content interaction method provided in an embodiment of the present application.

[0125] S301: Receive initial multimedia content sent by a first object to a second object, and extract initial feature information of the initial multimedia content.

[0126] The server may receive initial multimedia content sent by a first object to a second object. The initial multimedia content may be an image, audio, or animated image sent by the first object to the second object. The server may receive initial multimedia content sent by a client corresponding to the first object and forward the initial multimedia content to the target client, without further limitation.

[0127] After obtaining the initial multimedia content, the server can extract the initial feature information of the initial multimedia content. The server can extract the initial feature information of the initial multimedia content after obtaining the initial multimedia content. The target client can also extract the initial feature information of the initial multimedia content when sending the initial multimedia content to the server. When sending the initial multimedia content to the server, the corresponding initial feature information can be sent at the same time. The server can obtain the initial feature information at the same time as obtaining the initial multimedia content, etc. There are no specific restrictions.

[0128] As an embodiment, after obtaining the initial multimedia content, the server may determine the multimedia type of the initial multimedia content based on the format of the initial multimedia content. Multimedia types include, for example, static images, dynamic images, and audio. When the multimedia type of the initial multimedia content is a dynamic image type, the server may further determine whether the initial multimedia content is a long video type or a short video type based on the number of frames contained in the initial multimedia content.

[0129] For multimedia types of static images, the server can directly obtain the initial feature information of the initial multimedia content. For multimedia types of dynamic images, the server can first obtain each frame contained in the initial multimedia content to obtain each image frame. The server can obtain the initial feature information based on each image frame, or extract the key frames in each image frame, and obtain the initial feature information based on each key frame. For multimedia types of audio, the server can first obtain each frame contained in the initial multimedia content to obtain each audio frame. The server can obtain the initial feature information based on each audio frame, or extract the key frames in each audio frame, and obtain the initial feature information based on each key frame, etc., without specific restrictions.

[0130] The initial feature information may include at least one of an initial image dimension feature vector, an initial text dimension feature vector, and an initial attribute dimension feature vector. The initial feature information may also include other feature vectors, without limitation. The method for obtaining the initial feature information varies depending on the feature vectors included in the initial feature information. The following examples illustrate the methods for obtaining the initial image dimension feature vector, the initial text dimension feature vector, and the initial attribute dimension feature vector, respectively.

[0131] Get the initial image dimension feature vector:

[0132] If the initial feature information includes an initial image dimension feature vector, indicating that the initial multimedia content may be a picture or animated image, the server may perform image feature extraction on the initial multimedia content to obtain an initial image dimension feature vector for the initial multimedia content. The image feature extraction process may include grayscale processing, edge detection, and a trained image feature extraction model, among other processing methods, without limitation.

[0133] Get the initial text dimension feature vector:

[0134] If the initial feature information includes an initial text dimension feature vector, it means that the initial multimedia content includes text, for example, a picture with text, an animated image with subtitles, or audio with human voice, etc., then the server can perform text recognition processing on the initial multimedia content to identify the various texts contained in the initial multimedia content. The text can be text, words, or sentences, etc., and obtain the text vector corresponding to each text. After obtaining each text and its corresponding text vector, the server can directly use the text vector corresponding to each text as the initial text dimension feature vector corresponding to the initial multimedia content. The server can also extract key texts from each text and use the text vector corresponding to each key text as the initial text dimension feature vector corresponding to the initial multimedia content. Key text can be text that appears more frequently in each text, or it can be text that is used more frequently based on network resources, or it can be text with a specified part of speech or a specified grammatical structure, or it can be text containing specified content, etc., without specific restrictions.

[0135] Get the initial attribute dimension feature vector:

[0136] If the initial feature information includes an initial attribute dimension feature vector, it means that the initial multimedia content contains a target object, for example, a picture containing a human face or a moving picture containing an object, etc., then the server can perform target object recognition processing on the initial multimedia content, identify the various targets contained in the initial multimedia content, and obtain the attribute features corresponding to each target object. After obtaining each target object and its corresponding attribute features, the server can directly use the attribute features corresponding to each target object as the initial attribute dimension feature vector of the initial multimedia content. The server can also extract key targets from each target object and use the attribute features corresponding to each key target object as the initial attribute dimension feature vector corresponding to the initial multimedia content, etc. The key target object can be a pre-specified target object, a target object that appears more frequently, or a target object that occupies a larger proportion of pixels, etc., and there is no specific limitation.

[0137] As an embodiment, the content included in the initial feature information is not limited to the above three types. When the initial multimedia content is voice, the initial feature information may also include time domain features or frequency domain features, etc., which are not specifically limited.

[0138] S302 : Generate at least one reply multimedia content that matches the initial feature information based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship.

[0139] After obtaining the initial multimedia content and the initial feature information of the initial multimedia content, the server can generate at least one reply multimedia content that matches the initial feature information based on the association relationship between each multimedia content and each feature information contained in the reference mapping relationship. Therefore, there is no need to wait for a response to the viewing operation triggered by the second object on the first display interface after presenting the initial multimedia content in the first display interface, nor is there a need to wait for a response to the reply operation triggered by the second object on the first display interface after viewing the presented initial multimedia content, nor is there a need to wait for a response to the second object's screening operation for a large amount of multimedia content in the third display interface. In this way, reply multimedia content that can be used to reply to the initial multimedia content can be obtained, which simplifies the process of screening reply multimedia content that can be used to reply to the initial multimedia content, improves the efficiency of screening reply multimedia content, and reduces the performance consumption of the target client in the process of waiting for a response to the second object's operation.

[0140] The reference mapping relationship represents the association between multimedia content and feature information. The server can determine the response multimedia content used to respond to the initial multimedia content from the multimedia content included in the reference mapping relationship. Please refer to Table 1 for a possible reference mapping relationship, taking the multimedia content as a song as an example.

[0141] Table 1

[0142]

[0143] Each song number identifies a song, and each song identifies a multimedia content. The text vectors corresponding to the song title, original singer, and key lyrics of each song can be used as text dimension feature vectors in the feature information. The attribute features corresponding to the objects or characters contained in the lyrics of each song can be used as attribute dimension feature vectors in the feature information. The image feature vectors of the images associated with each song in the preset image set can be used as image dimension feature vectors in the feature information. The reference mapping relationship can only store the image numbers of the images associated with the songs, and the image feature vectors of the images can be read from the pre-stored image set based on the image numbers, thereby reducing the amount of data that needs to be stored in the reference mapping relationship.

[0144] As an embodiment, there are various ways to obtain a reference mapping relationship. The server may receive a reference mapping relationship sent by another device, or the server may generate a reference mapping relationship by obtaining data from a network resource in advance, or the server may establish a reference mapping relationship based on historical response records of each object, etc., without limitation. The following example describes the process of a server obtaining data from a network resource to generate a reference mapping relationship.

[0145] The server obtains various multimedia contents from a network resource. The network resource can be any web page, any client that has established a network connection with the server, or any database associated with any device that has established a network connection with the server, etc., without limitation. The multimedia contents obtained by the server from the network resource can be images, audio, or video.

[0146] Multimedia content obtained from network resources may not be directly stored in the reference mapping relationship due to reasons such as size, duration, or content validity. Therefore, after obtaining the multimedia content from the network resource, each multimedia content can be preprocessed so that the preprocessed multimedia content can be stored in the reference mapping relationship. The preprocessing process for each multimedia content is similar. The following describes the preprocessing process for a single multimedia content as an example.

[0147] Since multimedia content is obtained from network resources, when obtaining multimedia content, the interactive data of each object in the network resources on the multimedia content can also be obtained. The interactive data may include data obtained from the interactive operations triggered by each object on the multimedia content. For example, the interactive data may include the barrage issued by each object in each clip of the video; for another example, the interactive data may also include the number of likes for each barrage in each clip; for another example, the interactive data may also include the number of downloads of pictures by each object; for another example, the interactive data may also include the number of plays of audio by each object, etc.

[0148] After obtaining the interactive data associated with the multimedia content, the multimedia content can be intercepted based on the interactive data associated with the multimedia content in the network resource. The interception process is used to obtain the portion of the multimedia content with a large amount of associated interactive data. For example, if a video is long, the segment with a large number of barrage comments associated with the video can be intercepted; for another example, if an audio is long, the segment with the most number of plays can be intercepted; for another example, if an image is large, the local area corresponding to the zoom operation performed on the image by multiple objects can be intercepted, etc.

[0149] If there is a corresponding audio file for the intercepted and processed multimedia content, it can be determined based on the audio file corresponding to the intercepted and processed multimedia content whether to retain the intercepted and processed multimedia content for the subsequent steps of establishing an association relationship. Based on the audio file corresponding to the intercepted and processed multimedia content, it is determined whether the audio file contains specified audio content. The specified audio content can be human voice, etc., and can be specifically set according to the application scenario. For example, for film and television videos, the core video segment corresponds to the audio segment containing human voice. Then, after obtaining the intercepted and processed multimedia content, it can be determined whether the audio file corresponding to the intercepted and processed multimedia content contains human voice.

[0150] If the audio file contains the specified audio content, the feature information of the intercepted and processed multimedia content can be obtained by referring to the method for obtaining initial feature information described above. Based on the intercepted and processed multimedia content and the feature information of the intercepted and processed multimedia content, an association relationship can be established in the reference mapping relationship. If the audio file does not contain the specified audio content, no association relationship is established for the intercepted and processed multimedia content. After performing the above preprocessing on each multimedia content obtained from the network resource, a reference mapping relationship containing multiple association relationships can be obtained.

[0151] As an embodiment, if there is a preset specified duration or specified image size for the multimedia content in the reference mapping relationship, then when the multimedia content is intercepted, it can be intercepted according to the specified duration or specified image size; or it can be intercepted again according to the specified duration or specified image size based on the part of the audio file corresponding to the intercepted multimedia content containing the specified audio content, etc., without specific restrictions.

[0152] The following describes a process in which the server generates at least one piece of reply multimedia content that matches the initial feature information.

[0153] There are various methods for the server to generate at least one piece of reply multimedia content that matches the initial feature information. For example, different feature vectors included in the initial feature information may result in different methods for generating the at least one piece of reply multimedia content. Another example is different priorities of the feature vectors included in the initial feature information may result in different methods for generating the at least one piece of reply multimedia content. Another example is different multimedia types of the initial multimedia content may result in different methods for generating the at least one piece of reply multimedia content. In the embodiments of the present application, several methods for generating at least one piece of reply multimedia content are used as examples.

[0154] Method 1:

[0155] If the initial feature information includes an initial image dimension feature vector, then based on the feature information associated with each multimedia content contained in the reference mapping relationship, the first similarity between the initial image dimension feature vector and the image dimension feature vector contained in each feature information is determined respectively. If the multimedia content is an image, the image dimension feature vector is obtained by performing image feature extraction processing on the multimedia content. If the multimedia content is not an image, the image dimension feature vector is obtained by performing image feature extraction processing on the related images associated with the multimedia content in a preset related mapping relationship, and the related mapping relationship represents the association relationship between the multimedia content and the related images. The related mapping relationship can be pre-set, or the related mapping relationship can be based on a historical moment, where there are one or more objects that can obtain the multimedia content as a reply to the related image, etc.

[0156] The first similarity can be used to represent the degree of similarity between the initial multimedia content and the related image content associated with the multimedia content in the related mapping relationship. The server determines whether each obtained first similarity is greater than a first similarity threshold, and generates at least one piece of reply multimedia content that matches the initial feature information based on the multimedia content corresponding to the image dimension feature vector whose first similarity is greater than the first similarity threshold.

[0157] As an embodiment, after determining the multimedia content corresponding to the image dimension feature vector having a first similarity greater than a first similarity threshold, the server may further determine whether the corresponding feature information includes an associated tag for the obtained image dimension feature vector having a first similarity greater than the first similarity threshold. The feature information may also include information indicating the priority of the association between the feature information and the multimedia content. The priority may be divided into multiple levels according to actual usage scenarios, for example, including level one, level two, and level three.

[0158] The association tag indicates that the priority of the association relationship between the corresponding feature information and the multimedia content meets a preset priority condition. There can be multiple preset priority conditions. For example, the preset priority condition can be the highest priority condition. In this case, if the priority is marked as level 1 in the feature information of the association relationship, it can be determined to have the association tag. For another example, the preset priority condition can be a specified priority condition. In this case, if the priority is marked as specified priority in the feature information of the association relationship, it can be determined to have the association tag.

[0159] If multimedia content containing an associated tag exists in the feature information associated with the multimedia content corresponding to the image dimension feature vector whose first similarity is greater than the first similarity threshold, then at least one piece of reply multimedia content matching the initial feature information may be generated based on the multimedia content containing the associated tag in the feature information. If multimedia content containing an associated tag does not exist in the feature information associated with the multimedia content corresponding to the image dimension feature vector whose first similarity is greater than the first similarity threshold, then at least one piece of reply multimedia content matching the initial feature information may be generated based on the multimedia content corresponding to the image dimension feature vector whose first similarity is greater than the first similarity threshold.

[0160] Method 2:

[0161] If the initial feature information includes an initial text dimension feature vector, then based on the feature information corresponding to each multimedia content included in the reference mapping relationship, a second similarity between the initial text dimension feature vector and the text dimension feature vector included in each feature information is determined. The text dimension feature vector can be obtained by performing text recognition processing on the multimedia content or content related to the multimedia content, can be obtained in advance when establishing the mapping relationship, or can be obtained in real time based on the multimedia content when using each text included in the multimedia content, without specific limitation.

[0162] The second similarity can be used to characterize the degree of similarity between the text contained in the initial multimedia content and the text associated with the multimedia content. The text associated with the multimedia content can be the text contained in the multimedia content, or it can be text related to the multimedia content. For example, the text associated with the audio can be the text contained in the audio obtained by performing text recognition processing on the audio, or it can be text of related content such as the title of the audio, the publisher of the audio, or the creation background of the audio, without specific limitation.

[0163] The server determines whether each obtained second similarity is greater than a second similarity threshold, and generates at least one piece of reply multimedia content that matches the initial feature information based on the multimedia content corresponding to the feature information whose second similarity is greater than the second similarity threshold. If no multimedia content corresponds to the feature information whose second similarity is greater than the second similarity threshold, indicating that no matching multimedia content has been found from the perspective of text matching, at least one piece of reply multimedia content that matches the initial feature information may be generated based on other methods.

[0164] Method 3:

[0165] If the initial feature information includes an initial attribute dimension feature vector, then based on the feature information corresponding to each multimedia content included in the reference mapping relationship, a third similarity is determined between the initial attribute dimension feature vector and the attribute dimension feature vector included in each feature information. The attribute dimension feature vector is obtained by performing target object recognition processing on the multimedia content. It can be obtained in advance when establishing the reference mapping relationship, or it can be obtained in real time based on the multimedia content when using the target object associated with the multimedia content, without specific limitation.

[0166] The third similarity can be used to characterize the degree of similarity between the target object contained in the initial multimedia content and the target object associated with the multimedia content. The target object associated with the multimedia content can be the target object contained in the multimedia content, or it can be a target object related to the multimedia content. For example, the target object associated with the audio can be the target object contained in the audio obtained by performing target object recognition processing on the audio, or it can be the target object corresponding to the content described by the audio or the target object of related content such as the publisher of the audio. There is no specific limitation.

[0167] The server determines whether each obtained third similarity is greater than a third similarity threshold, and generates at least one piece of reply multimedia content that matches the initial feature information based on the multimedia content corresponding to the feature information whose third similarity is greater than the third similarity threshold. If no multimedia content corresponds to the feature information whose third similarity is greater than the third similarity threshold, indicating that no matching multimedia content has been found from the perspective of target object matching, at least one piece of reply multimedia content that matches the initial feature information may be generated based on other methods.

[0168] As an embodiment, various methods for generating at least one piece of reply multimedia content matching the initial feature information may be used in combination. For example, each method may have a pre-set, specified order. If the first-order method fails to determine a set of reply multimedia content, the second-order method may be used, and so on. For another example, multiple pieces of reply multimedia content may be determined based on each method, the number of occurrences of the multimedia content may be counted, and the final at least one piece of reply multimedia content may be generated based on the number of occurrences, etc., without further limitation.

[0169] As an embodiment, in order to avoid the reference mapping relationship containing too many association relationships and reducing the efficiency of generating at least one reply multimedia content, the reference mapping relationship may include multiple mapping sub-relationships. If the reference mapping relationship includes multiple mapping sub-relationships, then based on the multimedia type of the initial multimedia content contained in the initial feature information and the preset screening strategy, at least one mapping sub-relationship can be determined from the multiple mapping sub-relationships. The server can generate at least one reply multimedia content that matches the initial feature information based on the association relationship between each multimedia content and each feature information contained in at least one mapping sub-relationship. The preset screening strategy is used to screen out at least one mapping sub-relationship corresponding to the multimedia type from the multiple mapping sub-relationships. The specific screening strategy can be set according to the actual usage scenario and is not specifically limited here.

[0170] Each mapping sub-relationship can be divided according to the multimedia type of the initial multimedia content. For example, the reference mapping relationship contains three mapping sub-relationships, namely the first mapping sub-relationship, the second mapping sub-relationship and the third mapping sub-relationship. The first mapping sub-relationship corresponds to the multimedia type of static images. The feature information corresponding to the multimedia content contained in the first mapping sub-relationship may include the feature vector of the image dimension, the feature vector of the text dimension, the feature vector of the attribute dimension and the associated number of the related picture, etc. The second mapping sub-relationship corresponds to the multimedia type of dynamic images. The feature information corresponding to the multimedia content contained in the second mapping sub-relationship may include the feature vector of the image dimension, the feature vector of the text dimension, the feature vector of the attribute dimension and the associated number of the related dynamic image, etc. The third mapping sub-relationship corresponds to the multimedia type of audio. The feature information corresponding to the multimedia content contained in the second mapping sub-relationship may include the feature vector of the text dimension, the feature vector of the attribute dimension, time domain features, frequency domain features and the associated number of the related audio, etc.

[0171] Each mapping sub-relationship can also be divided according to the content type of the included multimedia content. For example, content types include dance, foreign language, food broadcast, and scenery. Therefore, after obtaining the initial multimedia content, the server can determine the content type of the initial multimedia content based on the initial multimedia content and determine the corresponding response multimedia content set in the mapping sub-relationship corresponding to the content type of the initial multimedia content.

[0172] As an embodiment, in the process of generating at least one piece of reply multimedia content matching the initial feature information, after determining at least one piece of multimedia content, the server may directly generate at least one piece of reply multimedia content based on the at least one piece of multimedia content. For example, when the initial multimedia content is a picture, an audio file is determined based on the initial feature information of the picture. The server may directly use the audio file as the reply multimedia content for the picture.

[0173] After determining at least one multimedia content, the server may further use each multimedia content as a reference multimedia content, fuse the reference multimedia content with the initial multimedia content, and obtain reply multimedia content based on the fuse-processed multimedia content.

[0174] The following is an example of a process of fusing the reference multimedia content with the initial multimedia content and obtaining the reply multimedia content based on the fused multimedia content.

[0175] After determining at least one reference multimedia content, the server determines corresponding response multimedia content for each reference multimedia content. The process of determining the response multimedia content corresponding to each reference multimedia content is similar. The following describes the process of determining the response multimedia content corresponding to a reference multimedia content.

[0176] The server determines the relationship between the first number of frames corresponding to the initial multimedia content and the second number of frames corresponding to the reference multimedia content. If the first number of frames is not equal to the second number of frames, a preset frame interpolation strategy is used to interpolate the initial multimedia content and a reference multimedia content, and the interpolated initial multimedia content and the reference multimedia content are then merged to obtain the corresponding response multimedia content. If the first number of frames is equal to the second number of frames, the initial multimedia content is directly merged with the reference multimedia content to obtain the corresponding response multimedia content.

[0177] If the first number of frames is greater than the second number of frames, the reference multimedia content is interpolated according to the interpolation strategy. After obtaining the interpolated reference multimedia content, the initial multimedia content is merged with the interpolated reference multimedia content to obtain a response multimedia content corresponding to the reference multimedia content.

[0178] If the first number of frames is less than the second number of frames, then the initial multimedia content is interpolated according to the interpolation strategy. After obtaining the interpolated initial multimedia content, the interpolated initial multimedia content is fused with a reference multimedia content to obtain a response multimedia content corresponding to the reference multimedia content.

[0179] The frame interpolation strategy is used to interpolate the original multimedia content or the reference multimedia content into a form that can be fused. There are many frame interpolation strategies, two of which are described below as examples.

[0180] Taking frame addition processing of reference multimedia content as an example, the server can directly repeatedly arrange the frames included in the reference multimedia content to achieve the purpose of repeatedly playing the frames included in the reference multimedia content, and add the frames of the reference multimedia content to the same number of frames as the original multimedia content. The server can also determine whether the second number of frames is greater than a specified number. If the second number of frames is greater than the specified number, the server determines the key frames included in the reference multimedia content based on the frame formats of the frames included in the reference multimedia content.

[0181] Based on the frame features corresponding to each key frame, a fourth similarity is determined between each two key frames. If there are two key frames whose fourth similarity is greater than a preset fourth similarity threshold, a reference multimedia content is supplemented based on each frame between the two key frames according to the first frame number. The supplementation of the reference multimedia content based on each frame between the two key frames can be that the server determines whether the number of frames between the two key frames reaches a preset frame threshold. If the number of frames reaches the preset frame threshold, the frames between the two key frames are cyclically arranged to achieve the purpose of repeatedly playing each frame between the two key frames, and the reference multimedia content is supplemented to the same number of frames as the initial multimedia content.

[0182] As an embodiment, the at least one reply multimedia content generated may include only one reply multimedia content, or may include multiple reply multimedia contents. After obtaining all the reply multimedia contents contained in the at least one reply multimedia content, the server may send all the reply multimedia contents to the target server, so that the target server presents all the reply multimedia contents on the second display interface. The server may also send the generated reply multimedia content to the target server each time a reply multimedia content is generated, so that the target server presents the received reply multimedia content on the second display interface. The server may send each reply multimedia content included in the at least one reply multimedia content to the target client, or may send a specified number of reply multimedia contents included in the at least one reply multimedia content to the target client, or may stop sending other reply multimedia contents in the at least one reply multimedia content to the target client after responding to the operation triggered by the second object on the reply multimedia content, etc., without specific limitation.

[0183] S303: Determine target reply multimedia content from at least one reply multimedia content.

[0184] After obtaining at least one reply multimedia content, the server may determine target reply multimedia content from the at least one reply multimedia content. If the at least one reply multimedia content includes one reply multimedia content, then the one reply multimedia content may be determined as the target reply multimedia content. If the at least one reply multimedia content includes multiple reply multimedia content, then the target reply multimedia content may be determined from the multiple reply multimedia content based on the received selection instruction. The selection instruction may be obtained in response to a selection operation triggered on the target client.

[0185] As an embodiment, there are multiple methods for determining the target reply multimedia content. For example, if the target reply multimedia content is obtained in response to a selection operation triggered by a second object or another object on a set of reply multimedia content, the selection operation can be a selection of one reply multimedia content from each presented reply multimedia content, or a selection of multiple reply multimedia content from each presented reply multimedia content, without limitation.

[0186] For another example, the server may determine the target reply multimedia content with the highest number of uses based on the number of times each of the at least one reply multimedia content is used as a reply. For another example, the server may determine the target reply multimedia content with the lowest traffic consumption based on the traffic consumed by sending each reply multimedia content, and the like, without limitation.

[0187] As an embodiment, after determining the target reply multimedia content, the server may update the relevant mapping relationship based on this content interaction process. The server may establish an association relationship in the relevant mapping relationship based on the initial multimedia content and the target reply multimedia content.

[0188] As an embodiment, after determining the target reply multimedia content, the server may determine whether to add an association tag. The server may record the second object that triggers the establishment of the association relationship in the relevant mapping relationship. The server may count the number of objects that trigger the establishment of the association relationship. The server determines whether the number of objects reaches a preset quantity threshold. If the number of objects reaches the quantity threshold, then in the reference mapping relationship, for the multimedia content corresponding to the initial multimedia content, an association tag is added in the corresponding reference feature information. If the number of objects does not reach the quantity threshold, then the statistics continue.

[0189] As an embodiment, after determining the target reply multimedia content, the server may update the reference mapping relationship based on the current content interaction process. The server may establish an association relationship in the reference mapping relationship based on the initial feature information and the initial multimedia content.

[0190] S304: Send target reply multimedia content to the first object.

[0191] After determining the target reply multimedia content, the server may send the target reply multimedia content to the client corresponding to the first object as a reply to the initial multimedia content. The server may also send the target reply multimedia content to the target client, so that the client corresponding to the first object and the target client can each present the target reply multimedia content on the first display interface.

[0192] Based on the same inventive concept, an embodiment of the present application provides a content interaction method. This method presents, on a first display interface, initial multimedia content sent by a first object to a second object, and then, on a second display interface, presents at least one piece of reply multimedia content obtained for the initial multimedia content, wherein the at least one piece of reply multimedia content is generated based on feature information of the initial multimedia content. The method also presents, on the first display interface, target reply multimedia content, wherein the target reply multimedia content is obtained based on the at least one piece of reply multimedia content.

[0193] Next, we continue based on Figure 2 , from the perspective of the target client corresponding to the second object, the content interaction method provided in the embodiment of the present application is specifically introduced.

[0194] Please refer to Figure 4a , which is a flow chart of the content interaction method provided in an embodiment of the present application.

[0195] S401: Presenting initial multimedia content sent by a first object to a second object in a first display interface.

[0196] After the target client obtains the initial multimedia content, it can present the initial multimedia content on the first display interface. Figure 4b , which is a schematic diagram of an interface presenting initial multimedia content in a first display interface, in which a first subject sends an image to a second subject. The first display interface can be used to display the initial multimedia content sent by the first subject to the second subject, to display the initial multimedia content sent by the first subject to the first subject, to display the target reply multimedia content in reply to the initial multimedia content to the second subject, and to display the target reply multimedia content in reply to the initial multimedia content to the first subject, etc.

[0197] The target client can obtain the initial multimedia content by receiving the initial multimedia content sent by the client corresponding to the first object. The target client can also obtain the initial multimedia content by receiving the initial multimedia content sent by the client corresponding to the first object forwarded by the server, etc., without specific restrictions.

[0198] As an embodiment, after the initial multimedia content is presented in the first display interface, the target client may wait for a reply operation triggered by the second object for the initial multimedia content, where the reply operation is used to indicate a reply to the initial multimedia content. Figure 4c If the response to the reply operation triggered by the second object for the initial multimedia content indicates that the second object wants to reply to the initial multimedia content, then the target client can execute S402. If there is no response to the reply operation triggered by the second object for the initial multimedia content, it indicates that the second object does not want to reply to the initial multimedia content, then the target client may not perform subsequent steps.

[0199] S402: Present at least one reply multimedia content obtained for the initial multimedia content in the second display interface.

[0200] After obtaining at least one piece of reply multimedia content, the target client may present the at least one piece of reply multimedia content obtained for the initial multimedia content on the second display interface. The target client may obtain the at least one piece of reply multimedia content by receiving the at least one piece of reply multimedia content sent by the server. After obtaining the initial multimedia content and initial feature information of the initial multimedia content, the target client may also generate at least one piece of reply multimedia content that matches the initial feature information based on the association between each piece of multimedia content and each piece of feature information included in the reference mapping relationship. For details, please refer to the above discussion and will not be repeated here.

[0201] As an embodiment, the target client can directly use the multimedia content determined based on the reference mapping relationship as the reply multimedia content. The target client can also use the multimedia content determined based on the reference mapping relationship as the reference multimedia content, and then fuse the reference multimedia content with the initial multimedia content to obtain the reply multimedia content, etc. For details, please refer to the previous introduction and will not be repeated here.

[0202] For example, when the initial multimedia content is a picture, an audio is determined based on the initial feature information of the picture. The target client can directly use the audio as the reply multimedia content of the picture. Please refer to Figure 4d , is a schematic diagram of an interface for presenting at least one reply multimedia content on the second display interface when directly using the audio as the reply multimedia content of the picture.

[0203] For example, when the initial multimedia content is a picture, an audio is determined based on the initial feature information of the picture, that is, the reference multimedia content. The target client can fuse the audio with the picture to obtain a dynamic image as the response multimedia content. Please refer to Figure 4e , which is a schematic diagram of an interface for obtaining reply multimedia content based on the fused multimedia content.

[0204] As an embodiment, at least one reply multimedia content may include only one reply multimedia content, or may include multiple reply multimedia contents. The target client may present all reply multimedia contents on the second display interface after obtaining all reply multimedia contents included in at least one reply multimedia content. The target client may also present the reply multimedia content on the second display interface each time a reply multimedia content is obtained. The target client may present each reply multimedia content included in at least one reply multimedia content, or may present a specified number of reply multimedia contents included in at least one reply multimedia content, or may no longer continue to present other reply multimedia contents in at least one reply multimedia content after responding to the operation triggered by the second object for the reply multimedia content, etc., without specific limitation. The second display interface is used to display the various reply multimedia contents that the second object can select, etc. The first display interface and the second display interface may be the same display interface, or may be different display interfaces, without specific limitation.

[0205] As an embodiment, the second display interface can be a display interface that reuses the display area of ​​the first display interface and is suspended above the first display interface. In this case, the position of the second display interface can be adjusted arbitrarily. The second display interface can also be a display area within the first display interface, located at a specified position within the first display interface. The second display interface can also be a display interface displayed side by side with the first display interface after the screen is split, etc., without specific limitation.

[0206] S403: Present the target reply multimedia content in the first display interface.

[0207] After presenting at least one reply multimedia content, the target client can present the target reply multimedia content in the first display interface after obtaining the target reply multimedia content. The target client can show the second object through the first display interface that the target reply multimedia content has been sent to the first object as a reply to the initial multimedia content. Please refer to Figure 4f , is a schematic diagram of an interface for presenting target reply multimedia content in a first display interface.

[0208] The target client can determine the target reply multimedia content based on at least one reply multimedia content, and the details can be referred to above and will not be repeated here. The target client can also receive the target reply multimedia content sent by the server, and there is no specific limitation.

[0209] As an embodiment, if the at least one reply multimedia content includes one reply multimedia content, the target client may use the reply multimedia content as the target reply multimedia content. If the at least one reply multimedia content includes multiple reply multimedia contents, the target client may determine the target reply multimedia content in response to the selection operation triggered by the second object for the at least one reply multimedia content. Figure 4g , which is a schematic diagram of an interface for selecting a target reply multimedia content. The selection operation can be a selection of one reply multimedia content from among the presented reply multimedia contents, or a selection of multiple reply multimedia contents from among the presented reply multimedia contents, without specific limitation.

[0210] The following is an example of the content interaction method provided in the embodiment of the present application, in which the first object and the second object chat through their respective corresponding clients, the initial multimedia content is an initial picture, and the multimedia content included in the reference mapping relationship is voice. Please refer to Figure 5 .

[0211] S501: A first object sends an initial image to a second object. The second object receives the initial image sent by the first object and presents the initial image on a first display interface.

[0212] S502 , in response to a reply operation triggered by the second object on the initial image, determining initial feature information of the initial image, including a multimedia type, an initial image dimension feature vector, an initial text dimension feature vector, and an initial attribute dimension feature vector.

[0213] S503 : Determine a corresponding mapping sub-relationship among the mapping sub-relationships included in the reference mapping relationship according to the multimedia type of the initial picture.

[0214] S504: Based on the reference speech in the mapping sub-relationship and the related pictures associated with each of the reference speech in the related mapping relationship, determine a first similarity between the initial image dimension feature vector and the reference image dimension feature vector corresponding to each related picture.

[0215] S505: Based on the reference speech corresponding to the reference image dimension feature vector having a first similarity greater than a first similarity threshold, determine whether the reference speech in the mapping sub-relationship has an associated label, obtain a first reference multimedia content subset based on the reference speech with the associated label, and obtain a second reference multimedia content subset based on the reference speech without the associated label.

[0216] S506: Determine a second similarity between the initial text dimension feature vector and each reference text dimension feature vector based on the reference text dimension feature vector corresponding to each reference speech in the mapping sub-relationship.

[0217] S507 : Obtain a third reference multimedia content subset based on the reference speech corresponding to the reference text dimension feature vector having a second similarity greater than a second similarity threshold.

[0218] S508 : Determine a third similarity between the initial attribute dimension feature vector and each reference attribute dimension feature vector based on the reference attribute dimension feature vector corresponding to each reference speech in the mapping sub-relationship.

[0219] S509 : Obtain a fourth reference multimedia content subset based on the reference speech corresponding to the reference attribute dimension feature vector having a third similarity greater than a third similarity threshold.

[0220] S510 , in accordance with the order in which the reference multimedia content subsets are obtained, reference voices in the first reference multimedia content subset are sequentially used as reply voices, and the reply voices are sequentially presented on the second display interface.

[0221] If the number of reference voices included in the first reference multimedia content subset has not reached the specified number, then the reply voices continue to be presented sequentially on the second display interface based on the reference voices in the second reference multimedia content subset. Presentation of the reply voices on the second display interface ceases until a second subject selects a reply voice, or all reference voices in the reference multimedia content subsets have been presented, or the number of presented reply voices reaches the specified number.

[0222] S511 , in response to a selection operation triggered by the second object on the reply multimedia content set, obtaining a target reply voice.

[0223] S512: Send a target reply voice to the first object, and present the target reply voice in the first display interface as a reply to the initial picture.

[0224] The following is an example of the content interaction method provided in the embodiment of the present application, taking the scenario of synthesizing a dynamic emoticon package of the first object, the initial multimedia content is the initial dynamic image, and the reference multimedia content is the reference voice as an example. Please refer to Figure 6a .

[0225] S601, the first object sends an initial animated image to itself on the first display interface, and the first object receives the initial animated image sent by itself on the first display interface. Please refer to Figure 6b Chinese (1).

[0226] S602: Determine, based on the format of the initial animated image, that the multimedia type of the initial animated image is a dynamic image, extract each frame included in the initial animated image, and determine a key frame in each frame.

[0227] S603: Determine a corresponding mapping sub-relationship among the mapping sub-relationships included in the reference mapping relationship according to the multimedia type of the initial dynamic image.

[0228] S604: Based on the reference attribute dimension feature vectors corresponding to the reference speech in the mapping sub-relationship, for each key frame, determine a third similarity between the initial attribute dimension feature vector corresponding to the key frame and each reference attribute dimension feature vector.

[0229] S605: Obtain reference speech corresponding to each reference attribute dimension feature vector having a third similarity greater than a third similarity threshold.

[0230] S606 : Compare the first frame quantity of each frame included in the initial animated image with the second frame quantity of each frame included in each reference speech.

[0231] S607: If the first frame number is smaller than the second frame number, the initial animation is corrected and the frames of the initial animation are played in a loop so that the number of frames of the initial animation after the frame correction processing is the same as the second frame number.

[0232] S608, the initial animated image after the frame supplementation process is merged with the reference voice to obtain a reply multimedia content set, and each reply multimedia content in the reply multimedia content set is presented on the second display interface. Please refer to Figure 6b In (2), the reply multimedia content is a moving picture with voice.

[0233] S609: In response to the selection operation triggered by the first object on the reply multimedia content set, the target reply multimedia content is obtained. Please refer to Figure 6b Chinese (3).

[0234] S610: Display the target reply multimedia content on the first display interface. Figure 6b Chinese (4).

[0235] Based on the same inventive concept, the embodiment of the present application provides a content interaction device, which is equivalent to the target client discussed above and can implement the functions corresponding to the aforementioned content interaction method. Figure 7 , the device includes a transceiver module 701 and a processing module 702, wherein:

[0236] Transceiver module 701: configured to receive initial multimedia content sent by a first object to a second object, and extract initial feature information of the initial multimedia content;

[0237] Processing module 702: configured to generate at least one reply multimedia content matching the initial feature information based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship;

[0238] The processing module 702 is further configured to: determine a target reply multimedia content from the at least one reply multimedia content;

[0239] The transceiver module 701 is further configured to send target reply multimedia content to the first object.

[0240] In a possible embodiment, the initial feature information includes an initial image dimension feature vector, which is obtained by performing image feature extraction processing on the initial multimedia content. The processing module 702 is specifically configured to:

[0241] Based on feature information associated with each multimedia content included in the reference mapping relationship, determining first similarities between the initial image dimension feature vector and the image dimension feature vector included in each feature information, wherein, if the multimedia content is not an image, the image dimension feature vector is obtained by performing image feature extraction processing on related images associated with the multimedia content in the related mapping relationship, and the related mapping relationship represents the association relationship between each multimedia content and each related image;

[0242] At least one reply multimedia content is generated based on the multimedia content associated with the image dimension feature vector whose first similarity is greater than a first similarity threshold.

[0243] In a possible embodiment, the processing module 702 is specifically configured to:

[0244] For the obtained image dimension feature vector having a first similarity greater than a first similarity threshold, determining whether the corresponding feature information includes an association tag, wherein the association tag indicates that a priority of an association relationship between the corresponding feature information and the multimedia content meets a preset priority condition;

[0245] At least one reply multimedia content is generated based on the multimedia content containing the associated tag in the feature information.

[0246] In a possible embodiment, the processing module 702 is further configured to:

[0247] After determining the target reply multimedia content from the at least one reply multimedia content, if the initial multimedia content is an image, an association relationship is established in the relevant mapping relationship based on the initial multimedia content and the target reply multimedia content.

[0248] Counting the number of objects that trigger the establishment of an association relationship between the initial multimedia content and the target reply multimedia content;

[0249] If it is determined that the number of objects reaches a preset threshold, feature information associated with the multimedia content is replied to the target in the reference mapping relationship, and an associated tag is added.

[0250] In a possible embodiment, the initial feature information includes an initial text dimension feature vector, which is obtained by performing text recognition processing on the initial multimedia content. The processing module 702 is specifically configured to:

[0251] Determining, based on the feature information corresponding to each multimedia content included in the reference mapping relationship, a second similarity between the initial text dimension feature vector and the text dimension feature vector included in each feature information;

[0252] At least one reply multimedia content is generated based on the multimedia content corresponding to the feature information whose second similarity is greater than the second similarity threshold.

[0253] In a possible embodiment, the initial feature information includes an initial attribute dimension feature vector, and the initial attribute dimension feature vector is obtained by performing object recognition processing on an object included in the initial multimedia content. The processing module 702 is specifically configured to:

[0254] Determining, based on the feature information corresponding to each multimedia content included in the reference mapping relationship, a third similarity between the initial attribute dimension feature vector and the attribute dimension feature vector included in each feature information;

[0255] At least one reply multimedia content is generated based on the multimedia content corresponding to the feature information whose third similarity is greater than a third similarity threshold.

[0256] In a possible embodiment, the processing module 702 is specifically configured to:

[0257] generating, based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, at least one reference multimedia content whose feature information matches the initial feature information;

[0258] For each piece of reference multimedia content, do the following:

[0259] Determining a magnitude relationship between a first number of frames included in the initial multimedia content and a second number of frames included in the reference multimedia content;

[0260] If the number of the first frames is not equal to the number of the second frames, a frame supplementation strategy is adopted to supplement the initial multimedia content and the reference multimedia content, and the initial multimedia content and the reference multimedia content after the supplementation are merged to obtain corresponding response multimedia content;

[0261] If the number of the first frames is equal to the number of the second frames, the initial multimedia content and the reference multimedia content are merged to obtain corresponding reply multimedia content.

[0262] Optionally, the processing module 702 is specifically configured to:

[0263] If the first frame number is greater than the second frame number, the reference multimedia content is interpolated according to a preset interpolation strategy, and the initial multimedia content is merged with the reference multimedia content after the interpolation to obtain the reply multimedia content;

[0264] If the first frame number is smaller than the second frame number, the initial multimedia content is interpolated according to the interpolation strategy, and the interpolated initial multimedia content is merged with the reference multimedia content to obtain the response multimedia content.

[0265] In a possible embodiment, the processing module 702 is specifically configured to:

[0266] Obtaining each key frame included in the reference multimedia content based on a frame format of each frame included in the reference multimedia content;

[0267] Determining a fourth similarity between every two key frames based on frame features corresponding to each key frame;

[0268] If there are two key frames whose fourth similarity is greater than a preset fourth similarity threshold, each frame between the two key frames is looped according to the first number of frames to perform frame supplementation processing on the reference multimedia content.

[0269] In a possible embodiment, the processing module 702 is specifically configured to:

[0270] If at least one reply multimedia content includes a reply multimedia content, determining the reply multimedia content as a target reply multimedia content;

[0271] If the at least one reply multimedia content includes multiple reply multimedia contents, target reply multimedia content is determined from the multiple reply multimedia contents based on the received selection instruction.

[0272] In a possible embodiment, the processing module 702 is specifically configured to:

[0273] If the reference mapping relationship includes multiple mapping sub-relationships, then based on the multimedia type of the initial multimedia content included in the initial feature information and a preset screening strategy, determining at least one mapping sub-relationship from the multiple mapping sub-relationships, wherein the multimedia content included in different mapping sub-relationships corresponds to different multimedia types;

[0274] Based on the association relationship between each multimedia content and each feature information contained in at least one mapping sub-relationship, at least one reply multimedia content matching the initial feature information is generated.

[0275] In a possible embodiment, the reference mapping relationship is obtained by the processing module 702 using the following method:

[0276] Obtaining various multimedia contents from network resources;

[0277] For each multimedia content, perform the following operations:

[0278] Intercepting and processing the multimedia content based on interactive data associated with the multimedia content in the network resource, wherein the interactive data is obtained based on interactive operations triggered by various objects on the multimedia content;

[0279] Determining, based on the audio file corresponding to the intercepted and processed multimedia content, whether the audio file contains the specified audio content;

[0280] If the audio file contains the specified audio content, an association relationship is established in the reference mapping relationship based on the intercepted and processed multimedia content and the feature information of the intercepted and processed multimedia content.

[0281] Based on the same inventive concept, the embodiment of the present application provides a content interaction device, which is equivalent to the server discussed above and can implement the functions corresponding to the aforementioned content interaction method. Figure 8 , the device includes a first display module 801 and a second display module 802, wherein:

[0282] The first display module 801 is configured to present the initial multimedia content sent by the first object to the second object in the first display interface;

[0283] The second display module 802 is configured to present at least one reply multimedia content obtained for the initial multimedia content in a second display interface, wherein the at least one reply multimedia content is generated based on feature information of the initial multimedia content;

[0284] The first display module 801 is further configured to present target reply multimedia content in the first display interface, wherein the target reply multimedia content is obtained based on at least one reply multimedia content.

[0285] In a possible embodiment, the second display module 802 is specifically configured to:

[0286] If the at least one reply multimedia content includes multiple reply multimedia contents, each time a reply multimedia content is obtained, one reply multimedia content is presented on the second display interface.

[0287] In a possible embodiment, the first display module 801 is specifically configured to:

[0288] If the at least one reply multimedia content includes a reply multimedia content, taking the reply multimedia content as the target reply multimedia content;

[0289] If the at least one reply multimedia content includes multiple reply multimedia contents, determining a target reply multimedia content in response to a selection operation triggered by the second object on the at least one reply multimedia content;

[0290] The target reply multimedia content is presented in the first display interface.

[0291] In a possible embodiment, the at least one reply multimedia content is generated by the first display module 801 using the following method:

[0292] Based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, at least one reply multimedia content whose feature information matches the initial feature information is generated.

[0293] Based on the same inventive concept, an embodiment of the present application provides a computer device, and the computer device 900 is introduced below.

[0294] Please refer to Figure 9 The above-mentioned content interaction device can run on a computer device 900, and the current version and historical version of the data storage program and the application software corresponding to the data storage program can be installed on the computer device 900. The computer device 900 includes a display unit 940, a processor 980 and a memory 920, wherein the display unit 940 includes a display panel 941 for displaying a user interactive operation interface, etc.

[0295] In a possible embodiment, the display panel 941 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0296] The processor 980 is configured to read a computer program and then execute the method defined by the computer program. For example, the processor 980 reads a data storage program or file, thereby running the data storage program on the computer device 900 and displaying a corresponding interface on the display unit 940. The processor 980 may include one or more general-purpose processors and may also include one or more DSPs (Digital Signal Processors) to perform related operations to implement the technical solutions provided in the embodiments of the present application.

[0297] The memory 920 generally includes internal memory and external memory. The internal memory can be a random access memory (RAM), a read-only memory (ROM), and a cache (CACHE), etc. The external memory can be a hard disk, an optical disk, a USB disk, a floppy disk or a tape drive, etc. The memory 920 is used to store computer programs and other data. The computer program includes an application corresponding to each client, etc. Other data may include data generated after the operating system or application is run, and the data includes system data (such as configuration parameters of the operating system) and user data. In the embodiment of the present application, program instructions are stored in the memory 920, and the processor 980 executes the program instructions stored in 920 to implement any of the content interaction methods discussed in the previous figure.

[0298] The display unit 940 is used to receive input digital information, character information, or contact touch operations / contactless gestures, and to generate signal input related to user settings and function control of the computer device 900. Specifically, in the embodiment of the present application, the display unit 940 may include a display panel 941. The display panel 941, such as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or on the display panel 941) and drive corresponding connected devices according to a pre-set program.

[0299] In one possible embodiment, the display panel 941 may include a touch detection device and a touch controller. The touch detection device detects the player's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 980. The touch controller can also receive and execute commands from the processor 980.

[0300] The display panel 941 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 940, the computer device 900 can also include an input unit 930. The input unit 930 can include a graphics input device 931 and other input devices 932. The other input devices can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, and a joystick.

[0301] In addition to the above, the computer device 900 may also include a power supply 990 for powering other modules, an audio circuit 960, a near-field communication module 970, and an RF circuit 910. The computer device 900 may also include one or more sensors 950, such as an accelerometer, a light sensor, a pressure sensor, etc. The audio circuit 960 specifically includes a speaker 961 and a microphone 962. For example, the computer device 900 can use the microphone 962 to collect the user's voice and perform corresponding operations.

[0302] As an embodiment, the number of the processors 980 may be one or more, and the processor 980 and the memory 920 may be coupled or relatively independently configured.

[0303] As an example, Figure 9 The processor 980 in the embodiment can be used to implement the following Figure 7 The functions of the transceiver module 701 and the processing module 702 in FIG. 7 can also be used to implement the following example: Figure 8 The functions of the first display module 801 and the second display module 802 are shown.

[0304] As an example, Figure 9 The processor 980 can be used to implement the corresponding functions of the server 102 discussed above.

[0305] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0306] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0307] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A content interaction method, characterized in that: include: receiving initial multimedia content sent by the first object to the second object, and extracting initial feature information of the initial multimedia content; generating, based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, at least one reference multimedia content whose feature information matches the initial feature information; For each piece of reference multimedia content, do the following: Determining a magnitude relationship between a first number of frames included in the initial multimedia content and a second number of frames included in the reference multimedia content; If the number of the first frames is not equal to the number of the second frames, a frame supplementation strategy is adopted to supplement the initial multimedia content and the reference multimedia content, and the initial multimedia content and the reference multimedia content after the supplementation are merged to obtain corresponding response multimedia content; If the number of the first frames is equal to the number of the second frames, fusing the initial multimedia content with the reference multimedia content to obtain corresponding reply multimedia content; determining target reply multimedia content from the obtained at least one reply multimedia content; The target reply multimedia content is sent to the first object.

2. The method according to claim 1, characterized in that The initial feature information includes an initial image dimension feature vector, and the initial image dimension feature vector is obtained by performing image feature extraction processing on the initial multimedia content; Then, based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, generating at least one reply multimedia content whose feature information matches the initial feature information, including: Based on the feature information associated with each multimedia content included in the reference mapping relationship, respectively determining a first similarity between the initial image dimension feature vector and the image dimension feature vector included in each feature information, wherein, if the multimedia content is not an image, the image dimension feature vector is obtained by performing image feature extraction processing on a related image associated with the multimedia content in the related mapping relationship, and the related mapping relationship represents the association relationship between each multimedia content and each related image; At least one reply multimedia content is generated based on the multimedia content associated with the image dimension feature vector whose first similarity is greater than a first similarity threshold.

3. The method according to claim 2, characterized in that Generating at least one reply multimedia content based on the multimedia content associated with the image dimension feature vector having the obtained first similarity greater than a first similarity threshold includes: For the obtained image dimension feature vector having a first similarity greater than a first similarity threshold, determining whether the corresponding feature information includes an association tag, wherein the association tag indicates that a priority of an association relationship between the corresponding feature information and the multimedia content meets a preset priority condition; At least one reply multimedia content is generated based on the multimedia content containing the associated tag in the feature information.

4. The method according to claim 3, characterized in that After determining target reply multimedia content from the at least one reply multimedia content, the method further includes: If the initial multimedia content is an image, establishing an association relationship in the relevant mapping relationship based on the initial multimedia content and the target reply multimedia content; Counting the number of objects that trigger the establishment of an association relationship between the initial multimedia content and the target reply multimedia content; If it is determined that the number of objects reaches a preset threshold, feature information associated with the multimedia content is replied to the target in the reference mapping relationship, and an associated tag is added.

5. The method according to claim 1, wherein The initial feature information includes an initial text dimension feature vector, and the initial text dimension feature vector is obtained by performing text recognition processing on the initial multimedia content; Then, based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, generating at least one reply multimedia content whose feature information matches the initial feature information, including: Based on the feature information corresponding to each multimedia content included in the reference mapping relationship, respectively determining a second similarity between the initial text dimension feature vector and the text dimension feature vector included in each feature information; At least one reply multimedia content is generated based on the multimedia content corresponding to the feature information whose second similarity is greater than a second similarity threshold.

6. The method according to claim 1, characterized in that The initial feature information includes an initial attribute dimension feature vector, and the initial attribute dimension feature vector is obtained by performing object recognition processing on the object included in the initial multimedia content; Then, based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, generating at least one reply multimedia content whose feature information matches the initial feature information, including: Determining, based on the feature information corresponding to each multimedia content included in the reference mapping relationship, a third similarity between the initial attribute dimension feature vector and the attribute dimension feature vector included in each feature information; At least one reply multimedia content is generated based on the multimedia content corresponding to the feature information whose third similarity is greater than a third similarity threshold.

7. The method according to any one of claims 1 to 6, characterized in that If the number of the first frames is not equal to the number of the second frames, a frame supplementation strategy is adopted to supplement the initial multimedia content and the reference multimedia content, and the initial multimedia content and the reference multimedia content after the supplementation are merged to obtain corresponding reply multimedia content, including: If the first number of frames is greater than the second number of frames, performing frame interpolation processing on the reference multimedia content according to the frame interpolation strategy, and fusing the initial multimedia content with the reference multimedia content after the frame interpolation processing to obtain reply multimedia content; If the first frame number is less than the second frame number, the initial multimedia content is interpolated according to the frame interpolation strategy, and the initial multimedia content after the frame interpolation is merged with the reference multimedia to obtain the reply multimedia content.

8. The method according to any one of claims 1 to 6, characterized in that If the first frame number is greater than the second frame number, performing frame supplementation processing on the reference multimedia content according to a preset frame supplementation strategy, including: Obtaining each key frame included in the reference multimedia content based on a frame format of each frame included in the reference multimedia content; determining a fourth similarity between every two key frames based on the frame features corresponding to each of the key frames; If there are two key frames whose fourth similarity is greater than a preset fourth similarity threshold, the frames between the two key frames are looped according to the first number of frames to perform frame supplementation processing on the reference multimedia content.

9. The method according to any one of claims 1 to 6, characterized in that: Determining target reply multimedia content from the obtained at least one reply multimedia content includes: If the at least one reply multimedia content includes a reply multimedia content, determining the reply multimedia content as a target reply multimedia content; If the at least one reply multimedia content includes multiple reply multimedia contents, target reply multimedia content is determined from the multiple reply multimedia contents based on the received selection instruction.

10. The method according to any one of claims 1 to 6, characterized in that: The reference mapping relationship is obtained by the following method: Obtaining various multimedia contents from network resources; For each multimedia content, perform the following operations respectively: intercepting and processing the multimedia content based on interactive data associated with the multimedia content in the network resource, wherein the interactive data is obtained based on interactive operations triggered by various objects on the multimedia content; Determining, based on an audio file corresponding to the intercepted and processed multimedia content, whether the audio file contains specified audio content; If the audio file contains the designated audio content, an association relationship is established in the reference mapping relationship based on the intercepted and processed multimedia content and feature information of the intercepted and processed multimedia content.

11. A content interaction method, characterized in that: include: Presenting, in the first display interface, initial multimedia content sent by the first object to the second object; Presenting at least one reply multimedia content obtained for the initial multimedia content in the second display interface, wherein the at least one reply multimedia content is generated by the following method: generating, based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, at least one reference multimedia content whose feature information matches the initial feature information of the initial multimedia content; For each piece of reference multimedia content, the following operations are performed: determining a relationship between a first number of frames included in the initial multimedia content and a second number of frames included in the reference multimedia content; if the first number of frames is not equal to the second number of frames, employing a frame-adding strategy to add frames to the initial multimedia content and the reference multimedia content, and fusing the added initial multimedia content and the reference multimedia content to obtain corresponding reply multimedia content; if the first number of frames is equal to the second number of frames, fusing the initial multimedia content with the reference multimedia content to obtain corresponding reply multimedia content; Target reply multimedia content is presented in the first display interface, wherein the target reply multimedia content is obtained based on the at least one reply multimedia content.

12. The method according to claim 11, characterized in that Presenting at least one reply multimedia content obtained for the initial multimedia content in the second display interface includes: If the at least one reply multimedia content includes multiple reply multimedia contents, each time a reply multimedia content is obtained, the reply multimedia content is presented in the second display interface.

13. The method according to claim 11, characterized in that Presenting target reply multimedia content on the first display interface includes: If the at least one reply multimedia content includes a reply multimedia content, taking the reply multimedia content as the target reply multimedia content; If the at least one reply multimedia content includes multiple reply multimedia contents, determining a target reply multimedia content in response to a selection operation triggered by the second object on the at least one reply multimedia content; The target reply multimedia content is presented in the first display interface.

14. A content interaction device, characterized in that: include: The transceiver module is configured to receive the initial multimedia content sent by the first object to the second object, and extract the initial feature information of the initial multimedia content; A processing module is configured to generate at least one reference multimedia content whose characteristic information matches the initial characteristic information based on the association relationship between each multimedia content and each characteristic information included in the reference mapping relationship; The processing module is further configured to: for each piece of reference multimedia content, perform the following operations: Determining a magnitude relationship between a first number of frames included in the initial multimedia content and a second number of frames included in the reference multimedia content; If the number of the first frames is not equal to the number of the second frames, a frame supplementation strategy is adopted to supplement the initial multimedia content and the reference multimedia content, and the initial multimedia content and the reference multimedia content after the supplementation are merged to obtain corresponding response multimedia content; If the number of the first frames is equal to the number of the second frames, fusing the initial multimedia content with the reference multimedia content to obtain corresponding reply multimedia content; The processing module is further configured to: determine target reply multimedia content from the obtained at least one reply multimedia content; The transceiver module is further configured to send the target reply multimedia content to the first object.

15. A content interaction device, characterized in that: include: A first display module: configured to present the initial multimedia content sent by the first object to the second object in the first display interface; A second display module is configured to present, in a second display interface, at least one reply multimedia content obtained in response to the initial multimedia content, wherein the at least one reply multimedia content is generated by the first display module using the following method: generating, based on the association relationship between each multimedia content and each feature information included in the reference mapping relationship, at least one reference multimedia content whose feature information matches the initial feature information of the initial multimedia content; For each piece of reference multimedia content, the following operations are performed: determining a relationship between a first number of frames included in the initial multimedia content and a second number of frames included in the reference multimedia content; if the first number of frames is not equal to the second number of frames, employing a frame-adding strategy to add frames to the initial multimedia content and the reference multimedia content, and fusing the added initial multimedia content and the reference multimedia content to obtain corresponding reply multimedia content; if the first number of frames is equal to the second number of frames, fusing the initial multimedia content with the reference multimedia content to obtain corresponding reply multimedia content; The first display module is further configured to present target reply multimedia content in the first display interface, wherein the target reply multimedia content is obtained based on at least one reply multimedia content.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer device, characterized in that: include: a memory for storing program instructions; The processor is configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 13 according to the obtained program instructions.

18. A computer-readable storage medium, characterized in that The storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 13.

Images