Content recommendation method, apparatus, storage medium, and terminal

Abstract

The application discloses a content recommendation method and device, a storage medium and a terminal. Audio and video data of a current conference is periodically acquired, a multi-modal conference information set of the current conference is recognized based on the audio and video data, the multi-modal conference information set includes at least two conference information, and each conference information represents a feature of the current conference in different aspects; the multi-modal conference information set is input into a multi-modal fusion model, theme information output by the multi-modal fusion model after fusing each conference information in the multi-modal conference information set is determined; target recommendation content meeting a preset correlation condition with the theme information is determined, and the target recommendation content is displayed to the current conference. The multi-modal information in the conference is fused to comprehensively and accurately reflect the theme feature of the conference, and the associated content is derived according to the theme feature of the conference, and the content is recommended to the current conference, so that more effective information related to the current conference can be provided for the conference participants.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a content recommendation method, apparatus, storage medium, and terminal. Background Technology

[0002] Today, online meetings, widely adopted and deeply applied across various industries globally, have become an indispensable part of modern work scenarios. Compared to traditional meeting models, they are not limited by geographical barriers or spatial constraints; participants can easily join various online meetings through internet-connected smart devices, greatly improving meeting efficiency and flexibility. However, simply focusing on a single topic can easily become tedious for participants, and in such a meeting setting, it's often difficult for them to engage in thorough, divergent thinking about the meeting's theme, thus affecting the richness of effective information shared during the meeting. Summary of the Invention

[0003] This application provides a content recommendation method, apparatus, storage medium, and terminal, which can solve the technical problem of limited and monotonous meeting content in related technologies.

[0004] In a first aspect, embodiments of this application provide a content recommendation method, the method comprising:

[0005] The audio and video data of the current meeting are periodically acquired, and a multimodal meeting information set of the current meeting is identified based on the audio and video data. The multimodal meeting information set includes at least two types of meeting information, and each type of meeting information represents the characteristics of the current meeting in different aspects.

[0006] The multimodal meeting information set is input into the multimodal fusion model, and the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set is determined.

[0007] Identify target recommended content that meets preset relevance conditions to the topic information, and display the target recommended content in the current meeting.

[0008] Secondly, embodiments of this application provide a content recommendation device, the device comprising:

[0009] A multimodal data acquisition module is used to periodically acquire audio and video data of the current meeting, and identify a multimodal meeting information set of the current meeting based on the audio and video data. The multimodal meeting information set includes at least two types of meeting information, and each type of meeting information represents the characteristics of the current meeting in different aspects.

[0010] The multimodal information fusion module is used to input the multimodal meeting information set into the multimodal fusion model, and determine the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set;

[0011] The content recommendation module is used to determine target recommended content that meets preset relevance conditions to the topic information, and to display the target recommended content in the current meeting.

[0012] Thirdly, embodiments of this application provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method described above.

[0013] Fourthly, embodiments of this application provide a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being adapted to be loaded by the processor and to execute the steps of the above-described method.

[0014] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following:

[0015] This application provides a content recommendation method. It periodically acquires audio and video data of the current meeting, identifies a multimodal meeting information set based on the audio and video data, and recognizes that the multimodal meeting information set includes at least two types of meeting information, each representing different aspects of the current meeting's characteristics. The multimodal meeting information set is input into a multimodal fusion model, which determines the topic information output by fusing the various meeting information in the multimodal meeting information set. Target recommended content that meets preset relevance conditions with the topic information is identified and displayed in the current meeting. Since meetings contain various types of data that can reflect the characteristics of the topics discussed in the current meeting, fusing multimodal information in the meeting can comprehensively and accurately reflect the topic characteristics of the meeting. Then, relevant and related content is derived from the meeting's topic characteristics and recommended to the current meeting, providing participants with more effective information related to the current meeting, thus enriching meeting content and improving meeting efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1An exemplary system architecture diagram of a content recommendation method provided in this application embodiment;

[0018] Figure 2 A flowchart illustrating a content recommendation method provided in an embodiment of this application;

[0019] Figure 3 A flowchart illustrating a content recommendation method provided in an embodiment of this application;

[0020] Figure 4 This is a structural block diagram of a content recommendation device provided in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0022] To make the features and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. Furthermore, in the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, in the description of the embodiments of this application, "multiple" refers to two or more.

[0024] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0025] Today, online meetings, widely adopted and deeply applied across various industries globally, have become an indispensable part of the modern work environment. Compared to traditional meeting models, they are not limited by geographical barriers or spatial constraints. Participants can easily join various online meetings through internet-connected smart devices, greatly improving meeting efficiency and flexibility.

[0026] In online meeting environments, research and discussion often revolve around a clear and well-defined theme. However, relying solely on this central theme for the entire meeting can lead to another extreme—a monotonous and dull meeting. Focusing solely on a single point can stifle participants' thinking, hindering diverse perspectives on the central theme. Furthermore, the formality and time constraints of meetings often prevent participants from fully exploring the topic in depth and engaging in divergent thinking during the meeting. Consequently, many valuable ideas and insights often emerge only after the meeting has ended. However, lacking the instant messaging tools and centralized discussion platforms available in online meetings, these creative ideas are difficult to translate into team consensus or action plans in a timely manner. This not only hinders the further development of innovative thinking but may also prevent the effective utilization of inspiration and creativity, thus failing to contribute to the long-term development of the team or project.

[0027] Therefore, this application provides a content recommendation method to solve the aforementioned technical problem of monotonous and limited meeting content.

[0028] Please see Figure 1 , Figure 1 An exemplary system architecture diagram of a content recommendation method provided in an embodiment of this application.

[0029] like Figure 1 As shown, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 serves as the medium for providing a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth communication links, Wireless-Fidelity (Wi-Fi) communication links, or microwave communication links, etc.

[0030] Terminal 101 can interact with server 103 via network 102 to receive messages from or send messages to server 103. Alternatively, terminal 101 can interact with server 103 via network 102 to receive messages or data sent to server 103 by other users. Terminal 101 can be hardware or software. When terminal 101 is hardware, it can be various electronic devices, including but not limited to smartwatches, smartphones, tablets, laptops, and desktop computers. When terminal 101 is software, it can be installed in the aforementioned electronic devices and can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module; no specific limitation is made here.

[0031] In this embodiment, terminal 101 first periodically acquires the audio and video data of the current meeting, identifies the multimodal meeting information set of the current meeting based on the audio and video data, the multimodal meeting information set includes at least two types of meeting information, each of which represents the characteristics of the current meeting in different aspects; then, terminal 101 inputs the multimodal meeting information set into a multimodal fusion model, determines the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set; based on this, terminal 101 determines the target recommended content that meets the preset correlation conditions with the topic information, and displays the target recommended content in the current meeting.

[0032] Server 103 can be a business server providing various services. It should be noted that server 103 can be either hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module; no specific limitations are made here.

[0033] Alternatively, the system architecture may not include server 103. In other words, server 103 may be an optional device in the embodiments of this specification. That is, the method provided in the embodiments of this specification can be applied to a system structure that only includes terminal 101. The embodiments of this application do not limit this.

[0034] It should be understood that Figure 1 The number of terminals, networks, and servers shown is only illustrative; the number can be any number of terminals, networks, and servers depending on the implementation requirements.

[0035] Please see Figure 2 , Figure 2This is a flowchart illustrating a content recommendation method provided in an embodiment of this application. The execution entity in this embodiment can be a terminal performing content recommendation, a processor within the terminal performing the content recommendation method, or a content recommendation service within the terminal performing the content recommendation method. For ease of description, the following example uses a processor within the terminal as the execution entity to illustrate the specific execution process of the content recommendation method.

[0036] like Figure 2 As shown, content recommendation methods can include at least:

[0037] S202. Periodically acquire audio and video data of the current meeting, and identify the multimodal meeting information set of the current meeting based on the audio and video data. The multimodal meeting information set includes at least two types of meeting information, and each type of meeting information represents the characteristics of the current meeting in different aspects.

[0038] Optionally, to enrich the information content of the meeting, topics and themes can be identified and analyzed, and then intelligently recommended related content to participants. This intelligent recommendation of rich, comprehensive, and effective information can enhance the interactivity and engagement of the meeting. Participants can quickly understand relevant meeting information, conduct discussions and collaborations more efficiently, thereby improving meeting efficiency and experience.

[0039] Optionally, when recommending content for a meeting, it's crucial to ensure the recommended content is highly relevant to the current meeting topic. Therefore, multimodal information reflecting the meeting's atmosphere and thematic characteristics can be monitored and captured in real time. This includes meeting title information, speech content, participant facial expressions, body language, and shared screen information. This information is then comprehensively processed and analyzed to accurately extract the meeting's thematic characteristics, and based on this, related and relevant content and potential topic development directions can be derived. For example, when the meeting focuses on new product development, the intelligent system can automatically collect and display market data, competitor analysis, and user feedback related to the product. Simultaneously, it can determine the participants' level of interest in certain topics based on their facial expressions and body language, thereby adjusting the meeting's discussion direction and pace accordingly.

[0040] Specifically, to achieve comprehensive analysis of meeting content, audio and video data of the ongoing meeting can be acquired periodically and automatically at set intervals. This acquisition process is typically set with fixed time intervals, such as every one minute or five minutes, to ensure that continuous changes in the meeting are captured without disrupting the normal proceedings due to overly frequent data collection. After successfully acquiring this audio and video data, it is analyzed and identified in depth to extract a multimodal meeting information set about the current meeting from the original audio and video materials.

[0041] Furthermore, this multimodal meeting information set contains at least two types of meeting information. These information pieces are independent yet complementary, collectively forming a comprehensive description of the current meeting from multiple perspectives. For example, one type of meeting information might be speech recognition results based on audio data, accurately converting verbal discussions into text for easy retrieval and organization. The other type might be image recognition results based on video data, capturing nonverbal information such as participants' facial expressions and gestures, providing crucial clues for analyzing meeting atmosphere and participant attitudes. These different types of meeting information characterize the current meeting in various aspects, collectively providing strong support for intelligent meeting management, analysis, and optimization. Comprehensive, in-depth, and intelligent analysis of meeting content through this multimodal meeting information set provides richer, more accurate, and more useful information for meeting participants and managers.

[0042] S204. Input the multimodal meeting information set into the multimodal fusion model, and determine the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set.

[0043] Optionally, to clarify the theme information of the current meeting, it is also necessary to fuse the information from each meeting in the multimodal meeting information set to obtain the most relevant theme information. This fusion process can be achieved using a multimodal fusion model. This model is trained using deep learning and possesses powerful data processing and parsing capabilities. It can perform deep understanding and feature vector extraction for each modality, and by fusing these feature vectors, the most comprehensive feature vector corresponding to the current meeting can be obtained. Therefore, the model can output the theme information of the current meeting based on the fused feature representation.

[0044] Specifically, the multimodal fusion model is trained based on at least one sample meeting information set, which consists of sample data with standard topic information labels. The model's processing of the sample meeting information set is consistent with the processing of meeting information sets in real-world scenarios. During the training of the multimodal fusion model, predicted topic information labels are output for each sample meeting information set, and the loss value of a preset loss function is calculated based on the predicted topic information labels and standard topic information labels of each sample meeting information set. The model parameters are then adjusted based on the loss value until the multimodal fusion model meets the preset convergence condition.

[0045] Furthermore, the multimodal meeting information set is input into a pre-designed and trained multimodal fusion model. Upon receiving this rich information, the multimodal fusion model utilizes deep learning algorithms for cross-modal information alignment, correlation analysis, and feature fusion. It not only understands information within a single modality but also captures the complementarity and correlation between modalities, thereby generating a more comprehensive and accurate meeting topic information. This process involves complex semantic understanding, sentiment analysis, contextual reasoning, and pattern recognition techniques to ensure that the output topic information accurately reflects the core content and discussion focus of the meeting. Ultimately, the multimodal fusion model outputs a topic information set, which may include key elements such as the main topics of the meeting, key points of discussion, consensus reached or decisions made, and next steps in the action plan. In this way, the fusion and output of multimodal meeting information not only improves the efficiency and effectiveness of the meeting but also promotes the effective dissemination and utilization of information.

[0046] S206. Determine the target recommended content that meets the preset relevance conditions of the topic information, and display the target recommended content in the current meeting.

[0047] Optionally, when recommending content related to the topic, the system can filter through a massive information database or content pool based on preset relevance criteria. The final selected recommended content can be of various types, such as at least one of the following: related topic content, related statistical data, prospect analysis, and topic direction prediction information. These relevance criteria may include multiple dimensions such as the content's topic matching degree, timeliness, relevance, authority, and audience preferences. For example, the system might look for recently updated research reports, industry news, expert opinions, or success stories closely related to the conference topic. Simultaneously, the system will also consider the authority and credibility of the content to ensure that the recommended content has reference value. During the filtering process, advanced natural language processing technology and machine learning algorithms can be used to perform deep semantic analysis and sentiment analysis on the content to further improve the accuracy and relevance of the recommendations. Furthermore, personalized content recommendations can be made based on the background, interests, and historical behavioral data of the attendees to meet the needs of different attendees.

[0048] Optionally, after identifying target recommended content that meets preset relevance conditions to the topic information, this content can be further displayed in the current meeting in an appropriate manner. This may include scrolling relevant information on the meeting screen in real time, pushing links or summary information to participants through the meeting system, or providing a detailed list of recommended content after the meeting. In one possible embodiment, when displaying content, a dedicated content recommendation display box can be displayed on all terminals connected to the current meeting. Each generated target recommended content is scrolled and displayed in this dedicated content recommendation display box, and the target recommended content is periodically updated within the content recommendation display box. During the display process, the clarity and readability of the target recommended content must be ensured so that all participants can quickly understand and absorb it. Simultaneously, interactive functions such as comments, likes, or sharing can be further provided to promote communication and interaction between participants and between participants and the recommended content. In this way, the interactivity and participation of the meeting can be enhanced, thereby promoting the depth and expansion of meeting discussions.

[0049] This application provides a content recommendation method that periodically acquires audio and video data of the current meeting, identifies a multimodal meeting information set based on the audio and video data, the multimodal meeting information set including at least two types of meeting information, each representing different aspects of the current meeting's characteristics; inputs the multimodal meeting information set into a multimodal fusion model, determines the topic information output by the multimodal fusion model after fusing the various meeting information in the multimodal meeting information set; identifies target recommended content that meets preset correlation conditions with the topic information, and displays the target recommended content in the current meeting. Since meetings contain various types of data that can reflect the characteristics of the topics discussed in the current meeting, fusing the multimodal information in the meeting can comprehensively and accurately reflect the topic characteristics of the meeting. Then, relevant and related content is derived from the meeting's topic characteristics and recommended to the current meeting, providing participants with more effective information related to the current meeting, thus enriching meeting content and improving meeting efficiency.

[0050] Please see Figure 3 , Figure 3 This is a flowchart illustrating a content recommendation method provided in an embodiment of this application.

[0051] like Figure 3 As shown, content recommendation methods can include at least:

[0052] S302. In response to the user's activation action on the content recommendation control, determine the timestamp of the activation action.

[0053] Optionally, during the meeting, users can choose whether to enable the content recommendation feature according to their needs. A content recommendation control is displayed on the meeting interface. When a user needs intelligent content recommendations for the current meeting content, they can activate this feature by triggering the content recommendation control. In response to the user's activation of the content recommendation control, the timestamp of the activation is determined, and audio and video data for the corresponding time period is recorded for content recommendation.

[0054] S304. Obtain the audio and video data of the current meeting within a preset period containing timestamps, and obtain the audio and video data of the current meeting within that preset period in each period.

[0055] Optionally, when the content recommendation control is activated, it indicates that the user wants to obtain target recommended content related to the current meeting. At this point, audio and video data of the current meeting within a preset period including timestamps can be retrieved. This audio and video data within this period can illustrate the discussion content, topic information, etc., of the meeting in this context. Furthermore, audio and video data of the current meeting within each subsequent preset period can be retrieved, thereby achieving periodic audio and video retrieval operations to support subsequent periodic content recommendations.

[0056] S306. Identify the multimodal meeting information set of the current meeting based on audio and video data. The multimodal meeting information set includes at least two types of meeting information, and each type of meeting information represents the characteristics of the current meeting in different aspects.

[0057] For details regarding step S306, please refer to the description in step S202; it will not be repeated here.

[0058] S308. Input the multimodal meeting information set into the multimodal fusion model, and determine the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set.

[0059] For details regarding step S308, please refer to the description in step S204; it will not be repeated here.

[0060] S310. Generate vector features corresponding to the topic information, and perform vector retrieval in the target knowledge base based on the vector features to obtain retrieval results.

[0061] Alternatively, for enterprise members, if recommended content is generated directly, it is difficult to grasp the relevance between the recommended content and the needs of the participants. Information in a specific knowledge base is more reliable and more likely to meet the actual needs of the meeting. Therefore, when recommending content based on topic information, a specific knowledge base can be used for retrieval.

[0062] Specifically, before conducting a search, the topic information is first preprocessed. This includes extracting keywords, phrases, or sentences from the topic and may involve Natural Language Processing (NLP) techniques such as word segmentation, part-of-speech tagging, and named entity recognition to ensure that we accurately capture the core content of the topic. Next, the processed topic information is converted into vector representations in a high-dimensional space using advanced machine learning or deep learning models (such as Word2Vec, BERT, and GPT). These vector features capture the semantic relationships between words, making similar topic information closer together in the vector space, thus providing a foundation for subsequent vector retrieval.

[0063] Furthermore, having obtained the vector features of the topic information, we compare them with the content in the target knowledge base. The target knowledge base may be a large collection of documents, a database, or a knowledge graph, containing a large amount of materials, articles, and data related to the conference topic. Using vector similarity calculation algorithms (such as cosine similarity, Euclidean distance, etc.), we can quickly find the knowledge base content that is closest to the topic information vector, forming preliminary search results. This process can efficiently filter out information closely related to the conference topic from massive amounts of data, providing a rich candidate set for subsequent filtering.

[0064] S312. Filter out target recommended content from the search results that meets the preset relevance conditions of the topic information, and display the target recommended content in the current meeting.

[0065] Optionally, the initial search results may contain a large amount of content, but not all of it is suitable for direct presentation to conference participants. Therefore, further filtering based on pre-defined relevance criteria is necessary. These criteria may include multiple dimensions such as content relevance, timeliness, authority, and readability. Furthermore, the diversity and complementarity of the content can be considered to ensure that the recommended content comprehensively covers different aspects of the conference topic. After identifying the target recommended content, it is then integrated and presented to conference participants. To ensure effective information delivery, the presentation should be concise and clear, allowing participants to quickly grasp and understand the key points of the recommended content. At the same time, providing sufficient interactivity can further enhance the value and engagement of the recommended content. In summary, this process, through intelligent and automated methods, efficiently connects the conference topic with a wide range of knowledge base resources, providing participants with rich and accurate information support, thus helping to improve the efficiency and effectiveness of the conference.

[0066] In this embodiment, a content recommendation method is provided. When a user needs intelligent content recommendations for the current meeting content, the content recommendation function can be activated by triggering a content recommendation control. Specifically, when making recommendations, vector features corresponding to the topic information can be generated, and vector retrieval can be performed in the target knowledge base based on these vector features to obtain retrieval results. Target recommended content that meets preset correlation conditions with the topic information is then selected from the retrieval results and displayed in the current meeting. Compared to direct generation, direct retrieval in a fixed knowledge base is more conducive to the understanding and use of the target recommended content by the participants.

[0067] Please see Figure 4 , Figure 4 This is a structural block diagram of a content recommendation device provided in an embodiment of this application.

[0068] like Figure 4 As shown, the content recommendation device 400 includes:

[0069] The multimodal data acquisition module 410 is used to periodically acquire the audio and video data of the current meeting, and identify the multimodal meeting information set of the current meeting based on the audio and video data. The multimodal meeting information set includes at least two types of meeting information, and each type of meeting information represents the characteristics of the current meeting in different aspects.

[0070] The multimodal information fusion module 420 is used to input the multimodal meeting information set into the multimodal fusion model and determine the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set.

[0071] The content recommendation module 430 is used to determine target recommended content that meets preset relevance conditions to the topic information and display the target recommended content in the current meeting.

[0072] Optionally, the multimodal meeting information set includes at least two types of meeting information from the following: meeting title information, speech content information, participant facial expressions information, body language information, and shared screen information in the current meeting.

[0073] Optionally, the content recommendation module 430 is also used to generate vector features corresponding to the topic information, perform vector retrieval in the target knowledge base based on the vector features to obtain retrieval results, and filter out target recommended content that meets the preset relevance conditions of the topic information from the retrieval results.

[0074] Optionally, the multimodal data acquisition module 410 is also used to respond to the user's launch operation on the content recommendation control, determine the timestamp of the launch operation; acquire the audio and video data of the current meeting within a preset period containing the timestamp, and acquire the audio and video data of the current meeting within that period in each preset period.

[0075] Optionally, the content recommendation device 400 further includes: a recommended content display module, used to display the content recommendation display box corresponding to the current meeting in all terminals connected to the current meeting; and a content recommendation module 430, used to display the target recommended content in the content recommendation display box and periodically update the target recommended content in the content recommendation display box.

[0076] Optionally, the multimodal fusion model is trained based on at least one sample meeting information set, where each sample meeting information set is sample data with standard topic information labels. During the training process of the multimodal fusion model, predicted topic information labels are output for each sample meeting information set, and the loss value of the preset loss function is calculated based on the predicted topic information labels and standard topic information labels of each sample meeting information set. The model parameters are then adjusted based on the loss value until the multimodal fusion model meets the preset convergence condition.

[0077] Optionally, the target recommended content includes at least one of the following: relevant topic content, related statistical data, prospect analysis, and topic direction prediction information.

[0078] This application provides a content recommendation device, comprising: a multimodal data acquisition module for periodically acquiring audio and video data of the current meeting, identifying a multimodal meeting information set based on the audio and video data, wherein the multimodal meeting information set includes at least two types of meeting information, each representing different aspects of the current meeting; a multimodal information fusion module for inputting the multimodal meeting information set into a multimodal fusion model, determining the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set; and a content recommendation module for determining target recommended content that meets preset correlation conditions with the topic information and displaying the target recommended content in the current meeting. Since meetings contain various types of data that can reflect the characteristics of the topics discussed in the current meeting, fusing the multimodal information in the meeting can comprehensively and accurately reflect the topic characteristics of the meeting. Then, relevant and related content can be derived from the topic characteristics of the meeting and recommended to the current meeting, providing participants with more effective information related to the current meeting, thereby enriching meeting content and improving meeting efficiency.

[0079] This application also provides a computer storage medium that can store multiple instructions adapted for loading by a processor and executing the steps of any of the methods described in the above embodiments.

[0080] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Figure 5As shown, terminal 500 may include: at least one terminal processor 501, at least one network interface 504, user interface 503, memory 505, and at least one communication bus 502.

[0081] The communication bus 502 is used to enable communication between these components.

[0082] The user interface 503 may include a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0083] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0084] The terminal processor 501 may include one or more processing cores. The terminal processor 501 connects to various parts within the terminal 500 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by calling data stored in the memory 505. Optionally, the terminal processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The terminal processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the terminal processor 501 and may be implemented as a separate chip.

[0085] The memory 505 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned terminal processor 501. Figure 5 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a content recommendation program.

[0086] exist Figure 5 In the terminal 500 shown, the user interface 503 is mainly used to provide an input interface for the user and to obtain the user's input data; while the terminal processor 501 can be used to call the content recommendation program stored in the memory 505 and specifically perform the following operations:

[0087] The system periodically acquires audio and video data of the current meeting, and identifies a multimodal meeting information set based on the audio and video data. The multimodal meeting information set includes at least two types of meeting information, and each type of meeting information represents the characteristics of the current meeting in different aspects.

[0088] Input the multimodal meeting information set into the multimodal fusion model, and determine the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set;

[0089] Identify target recommended content that meets preset relevance criteria to the topic information and display the target recommended content in the current meeting.

[0090] In some embodiments, the multimodal meeting information set includes at least two types of meeting information, such as meeting title information, speech content information, participant facial expression information, body movement information, and shared screen information in the current meeting.

[0091] In some embodiments, when the terminal processor 501 determines target recommended content that meets preset relevance conditions to topic information, it specifically performs the following steps: generating vector features corresponding to topic information, performing vector retrieval in the target knowledge base based on the vector features to obtain retrieval results, and filtering out target recommended content that meets preset relevance conditions to topic information from the retrieval results.

[0092] In some embodiments, when the terminal processor 501 performs the periodic acquisition of audio and video data of the current meeting, it specifically performs the following steps: in response to the user's activation operation on the content recommendation control, it determines the timestamp of the activation operation; it acquires the audio and video data of the current meeting within a preset period containing the timestamp, and acquires the audio and video data of the current meeting within that preset period in each preset period.

[0093] In some embodiments, the terminal processor 501 further performs the following steps: displaying the content recommendation display box corresponding to the current meeting in all terminals connected to the current meeting; when the terminal processor 501 performs the task of displaying the target recommended content to the current meeting, it specifically performs the following steps: displaying the target recommended content in the content recommendation display box, and periodically updating the target recommended content in the content recommendation display box.

[0094] In some embodiments, the multimodal fusion model is trained based on at least one sample meeting information set, where each sample meeting information set is sample data with standard topic information labels. During the training process of the multimodal fusion model, predicted topic information labels are output for each sample meeting information set, and the loss value of a preset loss function is calculated based on the predicted topic information labels and standard topic information labels of each sample meeting information set. The model parameters are then adjusted based on the loss value until the multimodal fusion model meets the preset convergence condition.

[0095] In some embodiments, the target recommended content includes at least one of the following: relevant topic content, related statistical data, prospect analysis, and topic direction prediction information.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0097] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0098] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0099] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0100] Furthermore, it should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the traffic data involved in this application were obtained with full authorization.

[0101] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0102] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0103] The above is a description of a content recommendation method, apparatus, storage medium, and terminal provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A content recommendation method, characterized in that, The method includes: The audio and video data of the current meeting are periodically acquired, and a multimodal meeting information set of the current meeting is identified based on the audio and video data. The multimodal meeting information set includes at least two types of meeting information, and each type of meeting information represents the characteristics of the current meeting in different aspects. The multimodal meeting information set is input into the multimodal fusion model, and the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set is determined. Identify target recommended content that meets preset relevance conditions to the topic information, and display the target recommended content in the current meeting.

2. The method according to claim 1, characterized in that, The multimodal meeting information set includes at least two types of meeting information, such as meeting title information, speech content information, participant facial expression information, body movement information, and shared screen information in the current meeting.

3. The method according to claim 1, characterized in that, The determination of target recommended content that meets preset relevance conditions with the topic information includes: Generate vector features corresponding to the topic information, and perform vector retrieval in the target knowledge base based on the vector features to obtain retrieval results; Target recommended content that meets preset relevance conditions to the topic information is selected from the search results.

4. The method according to claim 1, characterized in that, The periodic acquisition of audio and video data for the current meeting includes: In response to a user's activation action on the content recommendation control, determine the timestamp of the activation action; Acquire audio and video data of the current meeting within a preset period containing the timestamp, and acquire audio and video data of the current meeting within each preset period.

5. The method according to claim 1, characterized in that, The method further includes: Display a content recommendation box corresponding to the current meeting in all terminals connected to the current meeting; The step of displaying the target recommended content in the current meeting includes: The target recommended content is displayed in the content recommendation display box, and the target recommended content is periodically updated in the content recommendation display box.

6. The method according to claim 1, characterized in that, The multimodal fusion model is trained based on at least one sample meeting information set, wherein the sample meeting information set consists of sample data with standard topic information labels; During the training process of the multimodal fusion model, predicted topic information labels are output for each sample meeting information set, and the loss value of the preset loss function is calculated based on the predicted topic information labels and standard topic information labels of each sample meeting information set. The model parameters are then adjusted based on the loss value until the multimodal fusion model meets the preset convergence condition.

7. The method according to claim 1, characterized in that, The target recommended content includes at least one of the following: relevant topic content, related statistical data, prospect analysis, and topic direction prediction information.

8. A content recommendation device, characterized in that, The device includes: A multimodal data acquisition module is used to periodically acquire audio and video data of the current meeting, and identify a multimodal meeting information set of the current meeting based on the audio and video data. The multimodal meeting information set includes at least two types of meeting information, and each type of meeting information represents the characteristics of the current meeting in different aspects. The multimodal information fusion module is used to input the multimodal meeting information set into the multimodal fusion model, and determine the topic information output by the multimodal fusion model after fusing the meeting information in the multimodal meeting information set; The content recommendation module is used to determine target recommended content that meets preset relevance conditions to the topic information, and to display the target recommended content in the current meeting.

9. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the steps of the method as described in any one of claims 1 to 7.

10. A terminal, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the method as described in any one of claims 1 to 7.

Images