IVVR-based fusion media content sharing method and device, medium and product

Abstract

The invention discloses an IVVR-based fusion media content sharing method and device, a medium and a product, and the method comprises the steps: carrying out the feature extraction and semantic understanding of heterogeneous data in fusion media content, and constructing a knowledge graph; performing behavior analysis on the historical behavior data of the user, and constructing a user portrait; generating a personalized recommendation sequence in combination with the user portrait according to the knowledge graph; according to real-time environment information of a user, dynamically adjusting a recommendation strategy and an interaction mode of the fusion media content in the personalized recommendation sequence; and outputting the corresponding fusion media content according to the adjusted content recommendation strategy and interaction mode. According to the scheme, more accurate and efficient content matching recommendation can be realized in combination with the environment context of the user.

Description

Technical Field

[0001] The present invention relates to the field of audio and video communication technology, and in particular to an IVVR-based integrated media content sharing method, device, medium and product. Background Art

[0002] IVVR (Interactive Voice and Video Response) is a new value-added wireless voice and video response service. It uses the characteristics of video and voice to highlight the "interactive" function. The IVVR service is an extension of the original voice IVR service and can provide 5G video terminal users with a value-added service that integrates audio and video. Based on IVVR's integrated media content sharing technology, multiple media elements such as text, pictures, audio, and video can be combined together, combined with the interactivity and immediacy of IVVR, to achieve a more vivid and intuitive content sharing experience.

[0003] However, most existing solutions recommend content through user interaction or preference settings, and the quality and efficiency of content recommendation are low. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a method, device, medium and product for sharing integrated media content based on IVVR, which combines the user's environmental context to achieve more accurate and efficient content matching and recommendation.

[0005] An embodiment of the present invention provides a method for sharing media content based on IVVR, including:

[0006] Extract features and understand semantics of heterogeneous data in converged media content to build a knowledge graph;

[0007] Conduct behavioral analysis on users’ historical behavior data and build user portraits;

[0008] Generate a personalized recommendation sequence based on the knowledge graph and the user portrait;

[0009] Dynamically adjust the recommendation strategy and interaction mode of the converged media content in the personalized recommendation sequence according to the user's real-time environmental information;

[0010] Output corresponding integrated media content based on the adjusted content recommendation strategy and interaction method.

[0011] Preferably, the method further comprises:

[0012] Obtain the user's voice commands and gestures to determine the user's interaction intention;

[0013] Understand the user's interaction intention and obtain the user's interaction behavior;

[0014] Convert the user's interactive behavior into control instructions for the integrated media content, and control the output integrated media content.

[0015] Preferably, feature extraction and semantic understanding are performed on heterogeneous data in converged media content to construct a knowledge graph, including:

[0016] Obtain heterogeneous data of different media types in converged media content;

[0017] Standardizing the heterogeneous data to obtain structured data;

[0018] Conduct semantic understanding and association analysis on structured data of different media types to determine the implicit semantic relationship between structured data of different media types;

[0019] Mapping semantically related associations into the semantic space to form a semantic relationship graph;

[0020] By using the semantic relationship graph, the user's interaction needs are understood and analyzed, and the converged media content related to the user's intention and the semantic relationship between the converged media content are obtained;

[0021] The knowledge graph is constructed according to the semantic relationship between the integrated media contents.

[0022] Preferably, the method further comprises:

[0023] Convert the audio data in the audio and video heterogeneous data into text format, obtain the speech transcription text of the video, and obtain the speech recognition result;

[0024] Performing text analysis on the speech transcription text to obtain a text analysis result;

[0025] Perform scene switching detection and shot boundary recognition on heterogeneous audio and video data to obtain scene transition information of the video;

[0026] Extract key frames from heterogeneous audio and video data;

[0027] Positioning and identifying the face in the key frame, and obtaining the appearance position and identity information of the person in the video as a person recognition result;

[0028] The speech recognition result, the text analysis result, the scene conversion information, the key frame and the character recognition result are integrated and associated to determine the multimodal description of data of different media types, and to perform multimodal understanding and annotation on heterogeneous audio and video data.

[0029] Preferably, the user's historical behavior data is analyzed to construct a user profile, including:

[0030] Obtain user's historical behavior data;

[0031] Preprocessing the historical behavior data to obtain a structured user behavior feature vector;

[0032] Obtaining the user's interest and preference distribution for different contents according to the user behavior feature vector;

[0033] Cluster the user's interaction behavior sequences to determine the user's interaction habits and behavior patterns;

[0034] Determine the user's social relationship network and calculate the user's social influence and communication ability based on the user's social relationship network;

[0035] Build user portraits based on different users' interests, preference distribution, interaction habits, behavior patterns, social influence and communication capabilities.

[0036] Preferably, generating a personalized recommendation sequence based on the knowledge graph and in combination with the user portrait includes:

[0037] Performing text analysis and semantic extraction on the integrated media content to obtain the subject semantic elements of the content;

[0038] Taking the subject semantic elements as nodes in the knowledge graph to construct the semantic association network;

[0039] Obtaining the relevance and recommendation strength of the converged media content according to the connection relationship and semantic path of the nodes in the semantic association network;

[0040] Obtain the content nodes that the user is currently interacting with, as well as the user's personalized needs and interaction preferences, calculate the user's interest and relevance in different content nodes, and obtain a preliminary content recommendation list;

[0041] Combining the relevance and recommendation strength of the preliminary content recommendation list with the integrated media content, obtaining a content recommendation list preferred by the user;

[0042] The personalized recommendation sequence that meets the user's preference is generated according to the content recommendation list of the user's preference.

[0043] Preferably, dynamically adjusting the recommendation strategy and interaction mode of the converged media content in the personalized recommendation sequence according to the user's real-time environment information includes:

[0044] Obtain real-time geographic location information of user devices;

[0045] Get the current timestamp of the user's device and convert the timestamp into a date and time in a standard format;

[0046] Acquire real-time weather data of the user's current location according to the geographic location coordinates to determine structured weather environment information;

[0047] The geographical location information, the date and time, and the weather environment information are integrated as the real-time environment information to determine the environment context of the user;

[0048] Obtain users' real-time interactive behavior data and determine their behavioral preferences;

[0049] The interaction mode is dynamically adjusted according to the environmental context and the behavioral preference.

[0050] Preferably, outputting corresponding integrated media content according to the adjusted content recommendation strategy and interaction mode includes:

[0051] According to the user's interaction intention and real-time environment information, the converged media content is matched from the preset knowledge base as the virtual content material to be rendered;

[0052] Determine the spatial mapping relationship and occlusion relationship according to the user's posture data;

[0053] The virtual content material is embedded into the real scene according to the mapping relationship and the occlusion relationship, and the integrated media content is output.

[0054] Preferably, the user's interactive behavior is converted into a control instruction for the integrated media content, and the output integrated media content is controlled, including:

[0055] Matching the keywords and semantic information in the interactive behavior with predefined operation instructions to obtain control instructions for the integrated media content corresponding to the interactive intention;

[0056] When the operation instruction matches the state and attributes of the current converged media content, the output converged media content is adjusted according to the operation instruction.

[0057] The embodiment of the present invention further provides a media content sharing device based on IVVR, the device comprising:

[0058] The graph construction module is used to extract features and understand semantics of heterogeneous data in converged media content and construct a knowledge graph;

[0059] The portrait analysis module is used to analyze the user's historical behavior data and build a user portrait;

[0060] A preference generation module, used to generate a personalized recommendation sequence based on the knowledge graph and the user portrait;

[0061] An adjustment module, used to dynamically adjust the recommendation strategy and interaction mode of the converged media content in the personalized recommendation sequence according to the user's real-time environmental information;

[0062] The output module is used to output corresponding integrated media content according to the adjusted content recommendation strategy and interaction method.

[0063] An embodiment of the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the IVVR-based integrated media content sharing method described in any one of the above embodiments is implemented.

[0064] An embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the IVVR-based integrated media content sharing method described in any one of the above embodiments.

[0065] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

[0066] Compared with the prior art, the present invention provides a method, device, medium and product for sharing media content based on IVVR, which constructs a knowledge graph by extracting features and understanding semantics of heterogeneous data in media content; conducts behavioral analysis on the acquired historical behavior data of users to construct user portraits; establishes a semantic association network between various media content based on the knowledge graph, and generates a personalized recommendation sequence of user preferences in combination with the user portraits; dynamically adjusts the recommendation strategy and interaction method of media content in the personalized recommendation sequence based on the acquired real-time environmental information of the user; and outputs the corresponding media content based on the adjusted content recommendation strategy and interaction method. The present application scheme can combine the user's environmental context to achieve more accurate and efficient content matching and recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a flowchart of a method for sharing media content based on IVVR provided in an embodiment of the present invention;

[0068] Figure 2 It is a structural schematic diagram of an IVVR-based media content sharing device provided in an embodiment of the present invention;

[0069] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The converged media content sharing technology based on IVVR can combine the various content forms of converged media with the interactivity and immediacy of IVVR to achieve a more vivid and intuitive content sharing experience. Through the IVVR system, users can not only watch and receive converged media content, but also interact with other users in real time to share their views and experiences. This technology can be applied to multiple fields such as news media, education and training, advertising, entertainment media, etc., to provide users with richer and more diverse information acquisition and interactive experience. For example, in the field of news media, the converged media content sharing technology based on IVVR can combine multiple media elements such as text, pictures, audio, video, etc. to create a more vivid and intuitive form of news reporting. Users can not only obtain news information through traditional channels such as newspapers and television, but also watch live news broadcasts in real time through the IVVR system, participate in news discussions, and share their views and opinions with other users.

[0072] When sharing traditional IVVR integrated media content, users access the platform through communication terminals. The platform pushes corresponding integrated media content to users based on their requests and preferences. The platform supports real-time interactive sharing and dissemination among users. The platform collects and analyzes user behavior data, understands user preferences and needs, and then optimizes the production and push strategies of integrated media content based on the data analysis results.

[0073] However, the quality and efficiency of content recommendation in the above-mentioned recommendation method adopted in the prior art are low, and the user experience and satisfaction are low.

[0074] In view of the technical problems existing in the above-mentioned prior art, this application proposes a method for sharing media content based on IVVR, see Figure 1 , is a flow chart of a method for sharing media content based on IVVR provided in an embodiment of the present invention, the method comprising the following steps:

[0075] Step S1, extracting features and understanding semantics of heterogeneous data in converged media content to construct a knowledge graph;

[0076] Step S2, analyzing the user's historical behavior data to construct a user profile;

[0077] Step S3, generating a personalized recommendation sequence based on the knowledge graph and the user portrait;

[0078] Step S4, dynamically adjusting the recommendation strategy and interaction mode of the integrated media content in the personalized recommendation sequence according to the user's real-time environment information;

[0079] Step S5: output corresponding integrated media content according to the adjusted content recommendation strategy and interaction method.

[0080] During the specific implementation of this embodiment, feature extraction and semantic understanding are performed on the convergent media content, implicit associations between different media types are discovered, a knowledge graph of the convergent media content is constructed, and semantic relationships between the convergent media content are discovered.

[0081] Obtain users' historical behavior data, analyze their interest preferences, interaction habits, and social influence, and build user portraits. Based on the user portraits, determine the users' personalized needs and interaction preferences.

[0082] Based on the knowledge graph of convergent media content, a semantic association network between each convergent media content is established and analyzed, and the implicit connections and recommendation paths of convergent media content are explored. According to the user's current interactive content, personalized needs and interactive preferences, a personalized content recommendation sequence is generated in real time.

[0083] By obtaining the user's real-time environmental information, including geographic location, time, and weather information, the user's environmental context is determined, and the virtual content recommendation strategy and interaction method are dynamically adjusted according to the user's environmental context, personalized content recommendation sequence, and user interaction behavior.

[0084] The virtual content is output according to the adjusted content recommendation strategy and interaction method, that is, based on the virtual content recommendation strategy and interaction method, the virtual content is integrated into the user's real environment to achieve an interactive effect combining the virtual and the real.

[0085] It should be noted that this case takes into account the user's real-time environmental information to dynamically adjust the content recommendation strategy, and adopts augmented reality and virtual reality technologies to provide users with an immersive interactive experience.

[0086] This case constructs a knowledge graph by extracting features and understanding semantics of heterogeneous data in converged media content, revealing implicit associations and semantic relationships between different media types. In addition, this technical solution constructs a multi-dimensional, dynamic user portrait by analyzing the user's historical behavior data to achieve personalized content recommendation. At the same time, multimodal understanding and annotation of video data enhances its searchability and comprehensibility. In terms of content recommendation, a semantic association network of converged media content is established to generate personalized recommendation sequences in real time. This technical solution also considers the user's real-time environmental information to dynamically adjust the content recommendation strategy, and combines the user's environmental context to achieve more accurate and efficient content matching and promotion.

[0087] In another embodiment provided by the present invention, after outputting the convergent media content, the present solution further provides an interactive solution to control the data of the convergent media content, specifically:

[0088] Determine the user's interaction intention based on the user's voice commands and gestures, and understand the user's interaction intention to obtain the user's interaction behavior;

[0089] Convert user interaction behaviors into control instructions for integrated media content.

[0090] By using voice recognition and gesture recognition technology to judge the user's interaction intention, convert the user's interaction behavior into operation and control instructions for integrated media content, and make personalized recommendations based on the user's instructions, we can provide users with more accurate and efficient personalized content, content accessibility and interactive effects, and promote the development of integrated media in the field of personalized recommendation and interactive experience.

[0091] In another embodiment provided by the present invention, when the step S1 is specifically performed, it specifically includes:

[0092] Obtain heterogeneous data of different media types in converged media content. Heterogeneous data of different media types include heterogeneous data of text, image, audio and video.

[0093] Heterogeneous data is cleaned, formatted and standardized to obtain structured data representation.

[0094] Conduct semantic understanding and association analysis on data of different media types to determine the implicit semantic relationships between different media types.

[0095] If there is a semantic association between different media data, the association relationship with the semantic association is mapped into the semantic space to form a semantic relationship graph.

[0096] By using the semantic relationship graph, we can understand and analyze the user's interaction needs and obtain the integrated media content and semantic relationships related to the user's intentions.

[0097] The knowledge graph is constructed according to the semantic relationship between the integrated media contents.

[0098] By analyzing users' historical behavior data, we build multi-dimensional and dynamic user portraits to achieve personalized content recommendations.

[0099] In another embodiment provided by the present invention, the method further includes:

[0100] Convert the audio data in the video into text format, obtain the speech transcription text of the video, and get the speech recognition result as input for understanding the video content.

[0101] When converting speech to text, the acoustic feature sequence of the user's audio signal is aligned with the transcribed text of the reference data, and the CTC (Connectionist Temporal Classification) method is used to align the output of the acoustic model to the transcribed text, thereby learning the parameters of the acoustic model. Among them, the CTC method is an algorithm in the fields of speech recognition and text recognition, which is used to solve the problem of different lengths of input and output sequences and inability to align. During the training process, the model parameters are continuously updated through the back propagation algorithm so that the model can better fit the data. After the model training is completed, the performance of the model is evaluated on the validation set or test set. Common evaluation indicators include word error rate (WER) and so on. When the model performance is poor, the performance can be improved by adjusting the model parameters and increasing the amount of training data. After the acoustic model is trained, it is deployed in actual application scenarios, such as voice assistants and speech recognition systems.

[0102] The speech transcription text is processed by word segmentation, part-of-speech tagging, and named entity recognition to extract keywords, character names, and location semantic information in the video to obtain text analysis results.

[0103] According to the results of speech recognition and text analysis, scene switching detection and shot boundary recognition are performed on the video content to obtain the shot boundary and scene conversion information of the video.

[0104] Extract key frames from video data to obtain the main pictures and scenes of the video content.

[0105] The video contains information about people, and the faces in the video frames are located and recognized to obtain the location and identity information of the people in the video;

[0106] The speech recognition results, text analysis results, scene transition information, key frames and character recognition results are integrated and associated to construct a multimodal description of the video content. The multimodal description includes text description, image description and speech description.

[0107] Multimodal understanding and annotation of heterogeneous audio and video data.

[0108] Multimodal understanding and annotation of video data to enhance its retrievability and understandability.

[0109] In another embodiment provided by the present invention, the step S2 specifically includes:

[0110] The user's browsing history, interaction history and social relationship data are obtained as the historical behavior data to construct an original user behavior data set.

[0111] The collected user behavior data is preprocessed to obtain a structured user behavior feature vector.

[0112] According to the user behavior feature vector, a collaborative filtering recommendation algorithm is used to train the user interest preference model to obtain the user's interest and preference distribution for different contents.

[0113] Cluster the user's interaction behavior sequences, determine the user's interaction habits and behavior patterns, and mine and analyze the user's social relationship network.

[0114] Calculate the user's social influence and communication ability to obtain the user's social characteristics.

[0115] Build a user portrait model and determine the user's personalized needs and interaction preferences based on the user portrait.

[0116] Build user portraits based on different users' interests, preference distribution, interaction habits, behavior patterns, social influence and communication capabilities.

[0117] Deep learning algorithms are used to extract features and understand semantics of heterogeneous data in converged media content, build knowledge graphs, reveal implicit associations and semantic relationships between different media types, and provide relevance and accuracy of converged media content output.

[0118] In another embodiment provided by the present invention, the step S3 specifically includes:

[0119] Perform text analysis and semantic extraction on integrated media content to obtain keywords, entities, and thematic semantic elements of the content.

[0120] The extracted semantic elements are used as nodes in the knowledge graph, and a semantic association network of integrated media content is formed through semantic association relationships.

[0121] According to the connection relationship and semantic path of nodes in the semantic association network of convergent media content, the implicit connections and recommendation paths between convergent media content are explored to obtain the relevance and recommendation strength of convergent media content.

[0122] Obtain the content nodes that the user is currently interacting with, as well as the user's personalized needs and interaction preferences, calculate the user's interest and relevance in different content nodes, and obtain a preliminary content recommendation list.

[0123] Combine the preliminary content recommendation list with the relevance and recommendation strength of the integrated media content to generate a content recommendation list based on content relevance and user preference.

[0124] Based on the user's real-time feedback and interactive behavior, the content recommendation list of the user's preference is dynamically adjusted and optimized to obtain a personalized recommendation sequence that meets the user's preferences.

[0125] In another embodiment provided by the present invention, the step S4 specifically includes:

[0126] Get the real-time geographic location information, longitude and latitude coordinates and geographic area tags of the user's device.

[0127] Get the current timestamp of the user's device and convert the timestamp to a standard date and time format to obtain a structured representation of the time information.

[0128] According to the user's geographic location coordinates, the real-time weather data of the user's current location is obtained to obtain a structured weather information representation.

[0129] The acquired geographic location, time and weather environment information are integrated to determine the user's environmental context and obtain a context label.

[0130] Obtain real-time interactive behavior data of users within the app to determine the user's current interactive intentions and behavioral preferences.

[0131] If the user's interactive behavior matches the current environmental context, the content recommendation strategy is dynamically adjusted based on the user's historical preferences and real-time feedback to generate a personalized recommendation list that meets the user's real-time needs.

[0132] Dynamically adjust the application's interaction method and display style based on the user's environmental context and interaction preferences to provide an immersive interactive experience.

[0133] By obtaining the user's real-time environmental information, including geographic location, time, and weather information, the user's environmental context is determined; based on the user's environmental context, personalized content recommendation sequence, and user interaction behavior, the virtual content recommendation strategy and interaction method are dynamically adjusted to provide an immersive interactive experience.

[0134] In another embodiment provided by the present invention, the step S5 specifically includes:

[0135] Perform semantic segmentation and 3D reconstruction on the user's real environment to obtain the 3D model and spatial structure information of the environment, providing a spatial reference for the insertion of virtual content.

[0136] Obtain the camera video stream and sensor data of the user's device, track the user's position, posture and ambient lighting conditions in real time, and use them as rendering parameters to provide parameters for rendering virtual content.

[0137] According to the user's interaction intention and environmental semantic information, matching virtual content is retrieved from the knowledge base to obtain the virtual content material to be rendered.

[0138] The virtual content material is fused and rendered in real time with the video frames of the real environment, and the virtual content is embedded into the real scene based on the spatial mapping relationship and occlusion relationship.

[0139] Using augmented reality and virtual reality technologies, virtual content is embedded into real scenes to provide users with an immersive interactive experience.

[0140] In another embodiment provided by the present invention, the process of controlling the integrated media content according to the user's interactive behavior specifically includes:

[0141] Based on the acquired voice commands and gesture action information, the user's interaction intention is analyzed and understood to determine the user's specific interaction needs.

[0142] When the intent understanding result indicates that the user needs to perform operational control on the integrated media content, the identified keywords and semantic information are structured; the extracted keywords and semantic information of the structured representation are matched with predefined operation instructions to obtain the integrated media content operation control instructions corresponding to the user's intent.

[0143] When the mapped operation instruction matches the state and attribute of the current convergent media content, the operation instruction is passed to the convergent media content control module to adjust the convergent media content.

[0144] By using voice recognition and gesture recognition technologies to judge users' interaction intentions and converting their interactive behaviors into operation and control instructions for integrated media content, the personalized and interactive user experience is improved, allowing users to enjoy richer, more intuitive and more personalized integrated media content, thereby improving the accessibility and interactive effects of the content.

[0145] See also Figure 2 , is a schematic diagram of a structure of a media content sharing device based on IVVR provided by an embodiment of the present invention, the device comprising:

[0146] The graph construction module is used to extract features and understand semantics of heterogeneous data in converged media content and construct a knowledge graph;

[0147] The portrait analysis module is used to analyze the user's historical behavior data and build a user portrait;

[0148] A preference generation module, used to generate a personalized recommendation sequence based on the knowledge graph and the user portrait;

[0149] An adjustment module, used to dynamically adjust the recommendation strategy and interaction mode of the converged media content in the personalized recommendation sequence according to the user's real-time environmental information;

[0150] The output module is used to output corresponding integrated media content according to the adjusted content recommendation strategy and interaction method.

[0151] The IVVR-based integrated media content sharing device provided in this embodiment can execute all the steps and functions of the IVVR-based integrated media content sharing method provided in any of the above embodiments, and the specific functions of the device will not be described in detail here.

[0152] An embodiment of the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the IVVR-based integrated media content sharing method described in any one of the above embodiments is implemented.

[0153] See also Figure 3 , is a structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The IVVR-based media content sharing device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, and the like.

[0154] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0155] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the IVVR-based media content sharing device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0156] Wherein, if the module integrated in the IVVR-based media content sharing device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0157] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of any method described in the above embodiments when executed by a processor.

[0158] It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for sharing media content based on IVVR, characterized in that: The method comprises: Extract features and understand semantics of heterogeneous data in converged media content to build a knowledge graph; Conduct behavioral analysis on users’ historical behavior data and build user portraits; Generate a personalized recommendation sequence based on the knowledge graph and the user portrait; Dynamically adjust the recommendation strategy and interaction mode of the converged media content in the personalized recommendation sequence according to the user's real-time environmental information; The corresponding converged media content is output according to the adjusted converged media content recommendation strategy and interaction method.

2. The method for sharing media content based on IVVR according to claim 1, characterized in that: Extract features and understand semantics of heterogeneous data in converged media content to build a knowledge graph, including: Acquire heterogeneous data of different media types in the converged media content; Standardizing the heterogeneous data to obtain structured data; Conduct semantic understanding and association analysis on structured data of different media types to determine the implicit semantic relationship between structured data of different media types; Mapping semantically related associations into the semantic space to form a semantic relationship graph; By using the semantic relationship graph, the user's interaction needs are understood and analyzed, and the converged media content related to the user's intention and the semantic relationship between the converged media content are obtained; The knowledge graph is constructed according to the semantic relationship between the integrated media contents.

3. The method for sharing media content based on IVVR according to claim 2, characterized in that: The method further comprises: Convert the audio data in the audio and video heterogeneous data into text format, obtain the speech transcription text of the video, and obtain the speech recognition result; Performing text analysis on the speech transcription text to obtain a text analysis result; Performing scene switching detection and shot boundary recognition on the audio and video heterogeneous data to obtain scene conversion information of the video; Extracting key frames from the audio and video heterogeneous data; Positioning and identifying the face in the key frame, and obtaining the appearance position and identity information of the person in the video as a person recognition result; The speech recognition result, the text analysis result, the scene conversion information, the key frame and the character recognition result are integrated and associated to determine the multimodal description of data of different media types, and to perform multimodal understanding and annotation on heterogeneous audio and video data.

4. The method for sharing media content based on IVVR according to claim 1, characterized in that: Conduct behavioral analysis on the user's historical behavior data and build user portraits, including: Obtain user's historical behavior data; Preprocessing the historical behavior data to obtain a structured user behavior feature vector; Obtaining the user's interest and preference distribution for different contents according to the user behavior feature vector; Cluster the user's interaction behavior sequences to determine the user's interaction habits and behavior patterns; Determine the user's social relationship network and calculate the user's social influence and communication ability based on the user's social relationship network; The user portrait is constructed based on the interests, preference distribution, interaction habits, behavior patterns, social influence and communication capabilities of different users.

5. The method for sharing media content based on IVVR according to claim 1, characterized in that: The generating a personalized recommendation sequence according to the knowledge graph and in combination with the user portrait includes: Performing text analysis and semantic extraction on the integrated media content to obtain the subject semantic elements of the content; Taking the subject semantic elements as nodes in the knowledge graph to construct the semantic association network; Obtaining the relevance and recommendation strength of the integrated media content according to the connection relationship and semantic path of the nodes in the semantic association network; Obtain the content nodes that the user is currently interacting with, as well as the user's personalized needs and interaction preferences, calculate the user's interest and relevance in different content nodes, and obtain a preliminary content recommendation list; Combining the relevance and recommendation strength of the preliminary content recommendation list with the integrated media content, obtaining a content recommendation list preferred by the user; The personalized recommendation sequence that meets the user's preference is generated according to the content recommendation list of the user's preference.

6. The method for sharing media content based on IVVR according to claim 1, characterized in that: The method of dynamically adjusting the recommendation strategy and interaction mode of the converged media content in the personalized recommendation sequence according to the real-time environment information of the user includes: Obtain real-time geographic location information of user devices; Get the current timestamp of the user's device and convert the timestamp into a date and time in a standard format; Acquire real-time weather data of the user's current location according to the geographic location coordinates to determine structured weather environment information; The geographical location information, the date and time, and the weather environment information are integrated as the real-time environment information to determine the environment context of the user; Obtain users' real-time interactive behavior data and determine their behavioral preferences; The interaction mode is dynamically adjusted according to the environmental context and the behavioral preference.

7. The method for sharing media content based on IVVR according to claim 1, characterized in that: Outputting corresponding integrated media content according to the adjusted content recommendation strategy and interaction method includes: According to the user's interaction intention and real-time environment information, the converged media content is matched from the preset knowledge base as the virtual content material to be rendered; Determine the spatial mapping relationship and occlusion relationship according to the user's posture data; The virtual content material is embedded into the real scene according to the mapping relationship and the occlusion relationship, and the integrated media content is output.

8. An electronic device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the IVVR-based media content sharing method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the IVVR-based integrated media content sharing method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of any method described in claims 1 to 7 are implemented.

Images