Information display method and device

Abstract

The present application discloses an information display method and device, which relates to the field of information display technology. The method includes: in the process of playing a multimedia file through a first screen, collecting audio and video data of a target user when watching the multimedia file; determining the emotional characteristic value of the target user based on the audio and video data; obtaining at least one emotional dynamic micro-video based on the emotional characteristic value, the at least one emotional dynamic micro-video being an emotional dynamic micro-video of at least one user watching the multimedia file stored in a dynamic micro-video database; and displaying the at least one emotional dynamic micro-video on the first screen. The above scheme improves the user's three-dimensional viewing experience of watching multimedia files by displaying an emotional dynamic micro-video when the user watches the multimedia file.

Description

Technical Field

[0001] The present application belongs to the field of information display technology, and in particular relates to an information display method and device. Background Art

[0002] Currently, the Internet is dominated by text comments. When users watch videos, they feel other people's emotional reactions mainly by reading text descriptions, which has problems such as slow timeliness and weak three-dimensional viewing experience. Summary of the Invention

[0003] The embodiments of the present application provide an information display method and device, which can solve the problems of slow timeliness and weak three-dimensional viewing experience in the form of text barrage.

[0004] In order to solve the above technical problems, the present invention provides an information display method, including:

[0005] During the process of playing the multimedia file on the first screen, collecting audio and video data of the target user watching the multimedia file;

[0006] Determining the emotional characteristic value of the target user based on the audio and video data;

[0007] Acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who watches the multimedia file and is stored in a dynamic micro-video database;

[0008] The at least one emotional dynamic micro-video is displayed on the first screen.

[0009] Optionally, determining the emotional feature value of the target user based on the audio and video data includes:

[0010] Determining the emotional state of the audio and video information data based on the audio and video data;

[0011] Determine the emotional characteristic value of the target user according to the emotional state.

[0012] Optionally, obtaining at least one emotional dynamic micro-video according to the emotional feature value includes:

[0013] Obtaining a segment identifier of the multimedia file corresponding to the audio and video data;

[0014] In a dynamic micro-video database, obtaining first emotional states of a plurality of users matching the segment identifiers;

[0015] respectively obtaining the difference between the emotional feature value of the target user and the feature values ​​corresponding to the first emotional states of the plurality of users;

[0016] Selecting at least one first user from the multiple users, whose absolute value of the difference is less than or equal to a first preset value;

[0017] Obtain an emotional dynamic micro-video of the at least one first user.

[0018] Optionally, the method for acquiring the dynamic micro-video database includes:

[0019] Segmenting the multimedia file;

[0020] When the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment;

[0021] Determine the emotional state corresponding to each emotional dynamic micro-video;

[0022] The user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state are associated and stored to generate a dynamic micro-video database.

[0023] Optionally, when the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment includes:

[0024] When the user watches the multimedia file, collecting the micro-video of the user corresponding to each segment of the multimedia file;

[0025] The micro-video is processed to obtain an emotional dynamic micro-video including body movements, facial expression changes and sounds.

[0026] Optionally, after associating and storing the user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state to generate a dynamic micro-video database, the method further includes:

[0027] Obtaining an emotional mean of characteristic values ​​corresponding to the emotional states of N users corresponding to the multimedia file segment;

[0028] Determine a default emotional state corresponding to each multimedia file segment according to the emotional mean.

[0029] Optionally, determining the default emotional state corresponding to the multimedia file segment according to the emotional mean value includes:

[0030] Selecting a second emotional state from a plurality of emotional states according to the emotional mean, wherein an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is less than or equal to a second preset value, or an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to a third preset value;

[0031] The second affective state is determined as the default affective state.

[0032] Optionally, after obtaining at least one emotional dynamic micro-video according to the emotional feature value, the method further includes:

[0033] According to the default emotional state corresponding to the multimedia file, selecting one or more emotional dynamic micro-videos corresponding to the default emotional state in the dynamic micro-video database;

[0034] One or more emotional dynamic micro-videos corresponding to the default emotional state are displayed on the first screen.

[0035] The present invention also provides an information display device, including:

[0036] a collection module, configured to collect audio and video data of a target user viewing a multimedia file during the process of playing the multimedia file on the first screen;

[0037] A first determining module is used to determine the emotional feature value of the target user based on the audio and video data;

[0038] A first acquisition module is configured to acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who views the multimedia file and is stored in a dynamic micro-video database;

[0039] The first display module is used to display the at least one emotional dynamic micro-video on the first screen.

[0040] The embodiment of the present application further provides an information display device, including a transceiver and a processor;

[0041] The processor is configured to: during the process of playing the multimedia file through the first screen, collect audio and video data of the target user viewing the multimedia file;

[0042] Determining the emotional characteristic value of the target user based on the audio and video data;

[0043] Acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who watches the multimedia file and is stored in a dynamic micro-video database;

[0044] The at least one emotional dynamic micro-video is displayed on the first screen.

[0045] An embodiment of the present application also provides an information display device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned information display method when executing the program.

[0046] An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0047] The beneficial effects of this application are:

[0048] The above scheme collects audio and video data of the target user when watching the multimedia file through the first screen during the process of playing the multimedia file through the first screen, determines the emotional characteristic value of the target user based on the audio and video data, obtains at least one emotional dynamic micro-video based on the emotional characteristic value, and displays the at least one emotional dynamic micro-video on the first screen; the above scheme enhances the user's three-dimensional viewing experience of watching multimedia files by displaying emotional dynamic micro-videos when the user watches the multimedia file. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of the information display method according to an embodiment of the present application;

[0050] Figure 2 This is a diagram showing the corresponding relationship between the movie length and the micro-video length;

[0051] Figure 3 It is a schematic diagram of the correspondence between the plot of the movie and the micro-video;

[0052] Figure 4 This is a schematic diagram of the positional relationship between the bullet screen playback area and the movie playback area;

[0053] Figure 5 This is a schematic diagram of a module of an information display device according to an embodiment of the present application;

[0054] Figure 6 A structural diagram showing an information display device according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0056] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0057] The information display method and device provided in the embodiments of the present application are described in detail below with reference to specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0058] like Figure 1 As shown, at least one embodiment of the present application provides an information display method, which is applied to an information display device, including:

[0059] Step 101: While a multimedia file is being played on a first screen, audio and video data of a target user viewing the multimedia file is collected;

[0060] It should be noted that this step is the real-time collection of audio and video of the target user when watching the multimedia file. The audio and video data refers to the audio and video information such as body movements, facial expressions, and sounds generated by the target user when watching the multimedia file.

[0061] Step 102: determining the emotional feature value of the target user based on the audio and video data;

[0062] Step 103: obtaining at least one emotional dynamic micro-video according to the emotional feature value;

[0063] Among them, the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user watching the multimedia file stored in the dynamic micro-video database. The emotional dynamic micro-video can be understood as a dynamic video containing body movements, facial expression changes and sounds.

[0064] Optionally, the dynamic micro-video database can be understood as a barrage library of emotional dynamic micro-videos, and each emotional dynamic micro-video can be regarded as a barrage.

[0065] Step 104: display the at least one emotional dynamic micro-video on the first screen.

[0066] It should be noted that, in the process of playing multimedia files through the first screen, the embodiment of the present application collects audio and video data of the target user when watching the multimedia file, determines the emotional characteristic value of the target user based on the audio and video data, obtains at least one emotional dynamic micro-video based on the emotional characteristic value, and displays the at least one emotional dynamic micro-video on the first screen; in the above scheme, when the user watches the multimedia file, the displayed barrage is set to an emotional dynamic micro-video, and the user does not need to read the barrage in the form of text descriptions, thereby enhancing the user's three-dimensional viewing experience of watching multimedia files.

[0067] Optionally, in at least one embodiment of the present application, the implementation process of step 102 includes:

[0068] Step 1021, determining the emotional state of the audio and video information data based on the audio and video data;

[0069] It should be noted that an optional way to determine the emotional state is: by inputting audio and video data into an emotional state analysis model, the model output result is the emotional state.

[0070] Optionally, the emotional state determination model can use machine learning and video understanding technology to divide the expressions and sounds in the user's emotional video into multiple micro-videos according to emotional changes, collect historical human expression data, manually annotate them according to David R Hawkins' emotions, and use machine learning to train the model to ensure that the model can recognize changes in character expressions according to emotional states, and ultimately be able to divide the emotional state for each audio and video data.

[0071] It should be noted that David R Hawkins's sentiment classification is shown in Table 1:

[0072] Table 1 David R Hawkins sentiment classification

[0073]

[0074]

[0075] It should be noted that the scores in the above emotion classification are the emotion feature values ​​corresponding to the emotion states.

[0076] Step 1022: Determine the emotional characteristic value of the target user according to the emotional state.

[0077] It should be noted that, by obtaining the emotional state, the emotional feature value corresponding to the user's emotional state can be obtained by searching the above Table 1.

[0078] Optionally, in at least one embodiment of the present application, the implementation process of step 103 includes:

[0079] Step 1031, obtaining a segment identifier of the multimedia file corresponding to the audio and video data;

[0080] It should be noted that, since the user's audio and video data is recorded in real time, after obtaining a segment of the user's audio and video data (which may include one or more frames of data), it is possible to determine which segment in the multimedia file the audio and video data corresponds to.

[0081] It should be noted here that segmentation refers to dividing the multimedia file into plots according to the context of the multimedia file, that is, one segment corresponds to one plot.

[0082] Step 1032: Obtain first emotional states of multiple users matching the segment identifiers from a dynamic micro-video database;

[0083] It should be noted that the dynamic micro-video database mentioned in the embodiment of the present application stores segment identifiers of multimedia file segments, emotional dynamic micro-videos of multiple users corresponding to each segment identifier, and the emotional state corresponding to each emotional dynamic micro-video.

[0084] That is to say, based on the segment identifiers, the emotional states of multiple users can be obtained.

[0085] Step 1033, respectively obtaining the difference between the emotional feature value of the target user and the feature values ​​corresponding to the first emotional states of the multiple users;

[0086] It should be noted that, since a characteristic value (ie, an emotional characteristic value) can be determined based on the emotional state of each user, a difference value can be obtained by subtracting these characteristic values ​​from the emotional characteristic value corresponding to the target user.

[0087] Step 1034: Select at least one first user from the plurality of users, whose absolute value of the difference is less than or equal to a first preset value;

[0088] Step 1035: Obtain the emotional dynamic micro-video of the at least one first user.

[0089] In the embodiment of the present application, usually only the emotional dynamic micro-videos of some users are selected, that is, the emotional dynamic micro-videos that are relatively close to the emotional state of the target user are selected for display.

[0090] Optionally, in at least one embodiment of the present application, the method for acquiring the dynamic micro-video database includes:

[0091] S11, segmenting the multimedia file;

[0092] It should be noted that the segmentation here may be to divide the multimedia file into plots according to the context of the multimedia file.

[0093] S12. When the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment;

[0094] Optionally, in at least one embodiment of the present application, this step is implemented as follows:

[0095] S121. When a user watches the multimedia file, collecting a micro video of the user corresponding to each segment of the multimedia file;

[0096] S122: Process the micro-video to obtain an emotional dynamic micro-video including body movements, facial expression changes, and sounds.

[0097] It should be noted that the processing of micro-videos here refers to removing invalid images and sounds in micro-videos through machine learning and video understanding technology, leaving only the user's body movements, facial expressions and sounds.

[0098] S13, determining the emotional state corresponding to each emotional dynamic micro-video;

[0099] It should be noted that the method for obtaining the emotional state here can refer to the method for obtaining the emotional state in the above-mentioned step 1021, and will not be repeated here.

[0100] S14: Associate and store the user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state to generate a dynamic micro-video database.

[0101] Optionally, in at least one embodiment of the present application, after associating and storing the user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state to generate a dynamic micro-video database, the process further includes:

[0102] Obtain an emotional mean of characteristic values ​​corresponding to the emotional states of N users corresponding to the multimedia file segment; and determine a default emotional state corresponding to the multimedia file segment based on the emotional mean.

[0103] Optionally, in at least one embodiment of the present application, determining the default emotional state corresponding to the multimedia file segment according to the emotional mean is implemented as follows:

[0104] Selecting a second emotional state from a plurality of emotional states according to the emotional mean, wherein an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is less than or equal to a second preset value, or an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to a third preset value;

[0105] The second affective state is determined as the default affective state.

[0106] It should be noted that the default emotional state determined when the absolute value of the difference between the emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to the third preset value can be understood as the general public's conventional emotional state; and the default emotional state determined when the absolute value of the difference between the emotional characteristic value of the second emotional state and the emotional mean is less than or equal to the second preset value can be understood as a non-general public differentiated emotional state.

[0107] Optionally, in at least one embodiment of the present application, after obtaining at least one emotional dynamic micro-video according to the emotional feature value, the method further includes:

[0108] According to the default emotional state corresponding to the multimedia file, one or more emotional dynamic micro-videos corresponding to the default emotional state are selected from the dynamic micro-video database; and one or more emotional dynamic micro-videos corresponding to the default emotional state are displayed on the first screen.

[0109] Taking a user watching a video as an example, the specific implementation method of at least one embodiment of the present application is described in detail as follows.

[0110] Step 1: When a user watches a video, the user's emotional expressions are recorded in real time. The video is divided into N episodes. Each episode must have a micro-video of the user's emotional dynamics (that is, the user's emotional feedback is converted into a micro-video). The video contains the emotional changes of the user's expression and sound;

[0111] It should be noted that after obtaining the user's permission, the computer camera is used to record the user's facial expressions and voice changes while watching the video; it should be noted that because the computer camera is real-time recording, and starts when the user starts watching the video and ends when the video ends, the length of the recorded video is the same as the length of the played video. Figure 2 As shown, when the video watched by the user is a movie, and the movie duration is 90 minutes, the duration of the corresponding recorded user emotional feedback video is also 90 minutes.

[0112] Furthermore, the movie is divided into N plots, and the user's expressions and sounds (such as laughing expressions and sounds, crying expressions and sounds) at each plot are obtained to obtain a micro-video; and the micro-video is stored in a corresponding relationship with the movie plot. The specific corresponding relationship is as follows: Figure 3 shown.

[0113] Step 2: Filter out noise and invalid images in the micro-video;

[0114] Through machine learning and video understanding technology, invalid images and sounds in micro-videos are removed, leaving only emotional dynamic micro-videos that include the user's body movements, facial expressions and sounds.

[0115] Step 3: Classify the user's emotional dynamic micro-videos to obtain the emotional state corresponding to each emotional dynamic micro-video;

[0116] The specific method of obtaining the emotional state is described above and will not be repeated here.

[0117] Step 4: Repeat steps 1, 2, and 3 to accumulate a dynamic micro-video database (i.e., a barrage library) of emotional dynamic micro-videos for the video;

[0118] That is, when K users watch the video, emotional dynamic micro-videos of K users are generated and stored (K>=0).

[0119] Optionally, the barrage library is composed as follows:

[0120] K1 user's micro video commentary:

[0121] K1 Emotional dynamic micro-video 1 – Joy - video plot 1;

[0122] K1 Emotional dynamic micro-video 2-sadness-video plot 2;

[0123] …

[0124] K1 Emotional dynamic micro video N-Pride-Video plot N.

[0125] K2 user's emotional dynamic micro video:

[0126] K2 Emotional dynamic micro-video 1-Love-Video plot 1;

[0127] K2 Emotional dynamic micro-video 2-Fear-Video plot 2;

[0128] …

[0129] K2 Emotional dynamic micro-video N-Anger-Video plot N.

[0130] And so on, the storage, emotional association and plot mapping of K users' emotional dynamic micro-videos are completed.

[0131] Step 5: Generate the corresponding general public emotional state for each video plot.

[0132] For example, suppose the Mth episode of the video corresponds to Kn natural persons’ expression feature values ​​K1M, K2M…KnM;

[0133] Then the emotional mean weight Pm (M>0) corresponding to the Mth episode of the video is:

[0134] P1=(K11+K21+…+KnM) / n;

[0135] P2=(K1M+K2M+…+KnM) / n;

[0136] …

[0137] Then, the emotional state corresponding to the emotional feature value whose difference with the emotional mean weight corresponding to a plot is less than or equal to a specific value is obtained as the general public emotional state corresponding to the plot. In this way, a general public emotional state is determined for each plot; assuming that most people feel "joy" after seeing a certain plot, then the general public emotional state corresponding to the plot is joy.

[0138] Step 6: Based on the real-time feedback of the current user's emotions, play the emotional dynamic micro-video that matches the current user's emotional state in the video barrage area;

[0139] Assume that when user A watches a video, after obtaining user A's permission, his expression changes are recorded. At video episode N, the emotional feature value of user A is determined to be AM.

[0140] The emotional feature value of user A is compared with the emotional feature value corresponding to the emotional state of historical users in the barrage database, and the difference is expressed as AM-KnM, that is:

[0141] In the first scenario, the facial expression feature value of user A is AM1. The specific values ​​of the difference between the emotional feature values ​​corresponding to the emotional states of historical users in the barrage library are:

[0142] Diff1 = Math.abs(AM1-K11);

[0143] Diff2 = Math.abs(AM1-K21);

[0144] …

[0145] DiffN = Math.abs(AM1-KnM);

[0146] For example, the DiffN threshold can be set to 50. Among K historical users, only those emotional dynamic micro-videos whose emotional feature values ​​differ from those of user A by 50 (DiffN < 50) are played. Furthermore, based on step 5, some emotional dynamic micro-videos corresponding to the general public's emotional state at that particular point in the episode can be selected for playback.

[0147] For example, Figure 4 As shown, the screen is divided into a video playback area 41 and a micro video barrage player 42 (it should be noted that Figure 4 The size of the area division and the position of the two areas are only examples and do not constitute a limitation to the present application), the video screen is displayed in the video playback area 41, and the micro-video barrage player 42 plays the barrage of the emotional dynamic micro-video; for example, for the current video plot 1, the emotional state of user A is joy, then the emotional dynamic micro-videos of the four historical users 1, 3, 7, and 13 who are also in plot 1 and whose emotional state is joy are selected from the barrage library for playback, so that user A can see the expressions of these four users and hear their laughter, thereby strengthening emotional resonance and achieving the effect of watching a movie in a cinema. It should be noted here that the sound control of the emotional dynamic micro-video is preferably smaller than the volume of the video itself.

[0148] In summary, at least one embodiment of the present application can adjust the corresponding barrage display according to the user's emotional feedback on the video plot, allowing the user to experience the dynamic micro-video barrage in terms of sound and picture, and resonate with the group emotional feedback generated by multiple people watching a movie in a cinema.

[0149] like Figure 5 As shown, at least one embodiment of the present application further provides an information display device 500, including:

[0150] The collection module 501 is used to collect audio and video data of a target user watching a multimedia file during the process of playing the multimedia file on the first screen;

[0151] A first determining module 502 is configured to determine an emotional feature value of the target user based on the audio and video data;

[0152] A first acquisition module 503 is configured to acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who views the multimedia file and is stored in a dynamic micro-video database;

[0153] The first display module 504 is configured to display the at least one emotional dynamic micro-video on the first screen.

[0154] Optionally, the first determining module 502 includes:

[0155] a first determining unit, configured to determine an emotional state of the audio and video information data based on the audio and video data;

[0156] The second determining unit is configured to determine an emotional characteristic value of the target user according to the emotional state.

[0157] Optionally, the first acquisition module 503 includes:

[0158] A first acquiring unit, configured to acquire a segment identifier of the multimedia file corresponding to the audio and video data;

[0159] A second acquiring unit is configured to acquire, from a dynamic micro-video database, first emotional states of a plurality of users that match the segment identifier;

[0160] a third acquiring unit, configured to respectively acquire the difference between the emotional feature value of the target user and the feature values ​​corresponding to the first emotional states of the plurality of users;

[0161] A selection unit, configured to select at least one first user from the plurality of users, the first user having an absolute value of the difference less than or equal to a first preset value;

[0162] The fourth acquisition unit is used to acquire the emotional dynamic micro-video of the at least one first user.

[0163] Optionally, the method for acquiring the dynamic micro-video database includes:

[0164] Segmenting the multimedia file;

[0165] When the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment;

[0166] Determine the emotional state corresponding to each emotional dynamic micro-video;

[0167] The user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state are associated and stored to generate a dynamic micro-video database.

[0168] Optionally, when the user watches the multimedia file, the method for collecting the user's emotional dynamic micro-video for each multimedia file segment includes:

[0169] When the user watches the multimedia file, collecting the micro-video of the user corresponding to each segment of the multimedia file;

[0170] The micro-video is processed to obtain an emotional dynamic micro-video including body movements, facial expression changes and sounds.

[0171] Optionally, after associating and storing the user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state to generate a dynamic micro-video database, the method further includes:

[0172] A second acquisition module is used to obtain the emotional mean of the characteristic values ​​corresponding to the emotional states of N users corresponding to the multimedia file segment;

[0173] The second determining module is configured to determine a default emotional state corresponding to the multimedia file segment according to the emotional mean.

[0174] Optionally, the second determining module includes:

[0175] a selection unit, configured to select a second emotional state from a plurality of emotional states according to the emotional mean, wherein an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is less than or equal to a second preset value, or an absolute value of a difference between the emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to a third preset value;

[0176] A third determining unit is configured to determine the second emotional state as the default emotional state.

[0177] Optionally, after the first acquisition module 503 acquires at least one emotional dynamic micro-video according to the emotional feature value, the method further includes:

[0178] A selection module is configured to select, from the dynamic micro-video database, one or more emotional dynamic micro-videos corresponding to the default emotional state corresponding to the multimedia file;

[0179] The second display module is used to display one or more emotional dynamic micro-videos corresponding to the default emotional state on the first screen.

[0180] It should be noted that the device provided in at least one embodiment of the present application is a device capable of executing the above-mentioned information display method, and all embodiments of the above-mentioned information display method are applicable to the device and can achieve the same or similar beneficial effects.

[0181] At least one embodiment of the present application further provides an information display device, comprising a transceiver and a processor;

[0182] The processor is configured to:

[0183] During the process of playing the multimedia file on the first screen, collecting audio and video data of the target user watching the multimedia file;

[0184] Determining the emotional characteristic value of the target user based on the audio and video data;

[0185] Acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who watches the multimedia file and is stored in a dynamic micro-video database;

[0186] The at least one emotional dynamic micro-video is displayed on the first screen.

[0187] Optionally, the processor is configured to:

[0188] Determining the emotional state of the audio and video information data based on the audio and video data;

[0189] Determine the emotional characteristic value of the target user according to the emotional state.

[0190] Optionally, the processor is configured to:

[0191] Obtaining a segment identifier of the multimedia file corresponding to the audio and video data;

[0192] In a dynamic micro-video database, obtaining first emotional states of a plurality of users matching the segment identifiers;

[0193] respectively obtaining the difference between the emotional feature value of the target user and the feature values ​​corresponding to the first emotional states of the plurality of users;

[0194] Selecting at least one first user from the multiple users, whose absolute value of the difference is less than or equal to a first preset value;

[0195] Obtain an emotional dynamic micro-video of the at least one first user.

[0196] Optionally, the processor is configured to:

[0197] Segmenting the multimedia file;

[0198] When the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment;

[0199] Determine the emotional state corresponding to each emotional dynamic micro-video;

[0200] The user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state are associated and stored to generate a dynamic micro-video database.

[0201] Optionally, the processor is configured to:

[0202] When the user watches the multimedia file, collecting the micro-video of the user corresponding to each segment of the multimedia file;

[0203] The micro-video is processed to obtain an emotional dynamic micro-video including body movements, facial expression changes and sounds.

[0204] Optionally, the processor is further configured to:

[0205] Obtaining an emotional mean of characteristic values ​​corresponding to the emotional states of N users corresponding to the multimedia file segment;

[0206] Determine a default emotional state corresponding to each multimedia file segment according to the emotional mean.

[0207] Optionally, the processor is configured to:

[0208] Selecting a second emotional state from a plurality of emotional states according to the emotional mean, wherein an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is less than or equal to a second preset value, or an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to a third preset value;

[0209] The second affective state is determined as the default affective state.

[0210] Optionally, the processor is further configured to:

[0211] According to the default emotional state corresponding to the multimedia file, selecting one or more emotional dynamic micro-videos corresponding to the default emotional state in the dynamic micro-video database;

[0212] One or more emotional dynamic micro-videos corresponding to the default emotional state are displayed on the first screen.

[0213] like Figure 6 As shown, an embodiment of the present invention further provides an information display device, including a processor 600, a transceiver 610, a memory 620, and a program stored in the memory 620 and executable on the processor 600; wherein the transceiver 610 is connected to the processor 600 and the memory 620 via a bus interface, wherein the processor 600 is configured to read the program in the memory and execute the following process:

[0214] During the process of playing the multimedia file on the first screen, collecting audio and video data of the target user watching the multimedia file;

[0215] Determining the emotional characteristic value of the target user based on the audio and video data;

[0216] Acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who watches the multimedia file and is stored in a dynamic micro-video database;

[0217] The at least one emotional dynamic micro-video is displayed on the first screen.

[0218] The transceiver 610 is configured to receive and send data under the control of the processor 600 .

[0219] Among them, Figure 6 In the embodiment of the present invention, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits such as one or more processors represented by processor 600 and memory represented by memory 620. The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be further described herein. The bus interface provides an interface. The transceiver 610 can be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium, such as a wireless channel, a wired channel, an optical cable, and the like.

[0220] The processor 600 is responsible for managing the bus architecture and general processing, and the memory 620 can store data used by the processor 600 when performing operations.

[0221] Optionally, the processor 600 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), and the processor may also adopt a multi-core architecture.

[0222] The processor calls the computer program stored in the memory to execute any of the methods provided in the embodiments of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0223] Furthermore, when the processor 600 executes the program, the following steps are implemented:

[0224] Determining the emotional state of the audio and video information data based on the audio and video data;

[0225] Determine the emotional characteristic value of the target user according to the emotional state.

[0226] Furthermore, when the processor 600 executes the program, the following steps are implemented:

[0227] Obtaining a segment identifier of the multimedia file corresponding to the audio and video data;

[0228] In a dynamic micro-video database, obtaining first emotional states of a plurality of users matching the segment identifiers;

[0229] respectively obtaining the difference between the emotional feature value of the target user and the feature values ​​corresponding to the first emotional states of the plurality of users;

[0230] Selecting at least one first user from the multiple users, whose absolute value of the difference is less than or equal to a first preset value;

[0231] Obtain an emotional dynamic micro-video of the at least one first user.

[0232] Furthermore, when the processor 600 executes the program, the following steps are implemented:

[0233] Segmenting the multimedia file;

[0234] When the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment;

[0235] Determine the emotional state corresponding to each emotional dynamic micro-video;

[0236] The user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state are associated and stored to generate a dynamic micro-video database.

[0237] Furthermore, when the processor 600 executes the program, the following steps are implemented:

[0238] When the user watches the multimedia file, collecting the micro-video of the user corresponding to each segment of the multimedia file;

[0239] The micro-video is processed to obtain an emotional dynamic micro-video including body movements, facial expression changes and sounds.

[0240] Furthermore, when the processor 600 executes the program, the following steps are further implemented:

[0241] Obtaining an emotional mean of characteristic values ​​corresponding to the emotional states of N users corresponding to the multimedia file segment;

[0242] Determine a default emotional state corresponding to each multimedia file segment according to the emotional mean.

[0243] Furthermore, when the processor 600 executes the program, the following steps are implemented:

[0244] Selecting a second emotional state from a plurality of emotional states according to the emotional mean, wherein an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is less than or equal to a second preset value, or an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to a third preset value;

[0245] The second affective state is determined as the default affective state.

[0246] Furthermore, when the processor 600 executes the program, the following steps are implemented:

[0247] According to the default emotional state corresponding to the multimedia file, selecting one or more emotional dynamic micro-videos corresponding to the default emotional state in the dynamic micro-video database;

[0248] One or more emotional dynamic micro-videos corresponding to the default emotional state are displayed on the first screen.

[0249] At least one embodiment of the present application also provides an information display device, including a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the processor executes the program, each process in the information display method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0250] It can be understood that the above-mentioned information display device is located in a network device (for example, a forwarding device).

[0251] At least one embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program implements the various processes in the above-described information display method embodiment and can achieve the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0252] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0253] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0254] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. An information display method, characterized in that: include: During playback of the multimedia file on the first screen, collecting audio and video data of the target user viewing the multimedia file, the audio and video data including at least one of body movements, facial expressions, and sounds produced by the target user while viewing the multimedia file; Determining the emotional characteristic value of the target user based on the audio and video data; Acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who watches the multimedia file and is stored in a dynamic micro-video database; Displaying the at least one emotional dynamic micro-video on the first screen; The method for obtaining the dynamic micro-video database includes: Segmenting the multimedia file, wherein segmentation is to divide the multimedia file into plots according to the scenes of the multimedia file, with one segment corresponding to one plot; When the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment; Determine the emotional state corresponding to each emotional dynamic micro-video; The user's emotional dynamic micro-video, the segment identifier of the multimedia file segment, and the emotional state are associated and stored to generate a dynamic micro-video database; After the user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state are associated and stored to generate a dynamic micro-video database, the method further includes: Obtaining an emotional mean of characteristic values ​​corresponding to the emotional states of N users corresponding to the multimedia file segment; Determining a default emotional state corresponding to each multimedia file segment according to the emotional mean; The step of determining the default emotional state corresponding to the multimedia file segment according to the emotional mean value includes: Selecting a second emotional state from a plurality of emotional states according to the emotional mean, wherein an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is less than or equal to a second preset value, or an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to a third preset value; The second affective state is determined as the default affective state.

2. The method according to claim 1, characterized in that Determining the emotional feature value of the target user based on the audio and video data includes: Determining the emotional state of the audio and video information data based on the audio and video data; Determine the emotional characteristic value of the target user according to the emotional state.

3. The method according to claim 1, characterized in that The step of obtaining at least one emotional dynamic micro-video according to the emotional feature value includes: Obtaining a segment identifier of the multimedia file corresponding to the audio and video data; In a dynamic micro-video database, obtaining first emotional states of a plurality of users matching the segment identifiers; respectively obtaining the difference between the emotional feature value of the target user and the feature values ​​corresponding to the first emotional states of the plurality of users; Selecting at least one first user from the multiple users, whose absolute value of the difference is less than or equal to a first preset value; Obtain an emotional dynamic micro-video of the at least one first user.

4. The method according to claim 1, wherein The method of collecting the user's emotional dynamic micro-video relative to each multimedia file segment when the user watches the multimedia file includes: When the user watches the multimedia file, collecting the micro-video of the user corresponding to each segment of the multimedia file; The micro-video is processed to obtain an emotional dynamic micro-video including body movements, facial expression changes and sounds.

5. The method according to claim 1, wherein After obtaining at least one emotional dynamic micro-video according to the emotional feature value, the method further includes: According to the default emotional state corresponding to the multimedia file, selecting one or more emotional dynamic micro-videos corresponding to the default emotional state in the dynamic micro-video database; One or more emotional dynamic micro-videos corresponding to the default emotional state are displayed on the first screen.

6. An information display device, characterized in that: include: a collection module, configured to collect audio and video data of a target user viewing the multimedia file while the multimedia file is being played on the first screen, the audio and video data including at least one of body movements, facial expressions, and sounds produced by the target user while viewing the multimedia file; A first determining module is used to determine the emotional feature value of the target user based on the audio and video data; A first acquisition module is configured to acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who views the multimedia file and is stored in a dynamic micro-video database; A first display module, configured to display the at least one emotional dynamic micro-video on the first screen; The method for obtaining the dynamic micro-video database includes: Segmenting the multimedia file; When the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment; Determine the emotional state corresponding to each emotional dynamic micro-video; The user's emotional dynamic micro-video, the segment identifier of the multimedia file segment, and the emotional state are associated and stored to generate a dynamic micro-video database; After the user's emotional dynamic micro-video, the segment identifiers of the multimedia file segments, and the emotional state are associated and stored to generate a dynamic micro-video database, the method further includes: A second acquisition module is used to obtain the emotional mean of the characteristic values ​​corresponding to the emotional states of N users corresponding to the multimedia file segment; A second determining module is configured to determine a default emotional state corresponding to each multimedia file segment based on the emotional mean; The second determining module includes: a selection unit, configured to select a second emotional state from a plurality of emotional states according to the emotional mean, wherein an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is less than or equal to a second preset value, or an absolute value of a difference between the emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to a third preset value; A third determining unit is configured to determine the second emotional state as the default emotional state.

7. An information display device, characterized in that: including a transceiver and a processor; The processor is configured to: during the process of playing the multimedia file on the first screen, collect audio and video data of the target user when viewing the multimedia file, wherein the audio and video data includes at least one of body movements, facial expressions, and sounds generated by the target user when viewing the multimedia file; Determining the emotional characteristic value of the target user based on the audio and video data; Acquire at least one emotional dynamic micro-video according to the emotional feature value, wherein the at least one emotional dynamic micro-video is an emotional dynamic micro-video of at least one user who watches the multimedia file and is stored in a dynamic micro-video database; Displaying the at least one emotional dynamic micro-video on the first screen; Wherein, the processor is used to: Segmenting the multimedia file; When the user watches the multimedia file, collecting the user's emotional dynamic micro-video relative to each multimedia file segment; Determine the emotional state corresponding to each emotional dynamic micro-video; The user's emotional dynamic micro-video, the segment identifier of the multimedia file segment, and the emotional state are associated and stored to generate a dynamic micro-video database; The processor is further configured to: Obtaining an emotional mean of characteristic values ​​corresponding to the emotional states of N users corresponding to the multimedia file segment; Determining a default emotional state corresponding to each multimedia file segment according to the emotional mean; Wherein, the processor is used to: Selecting a second emotional state from a plurality of emotional states according to the emotional mean, wherein an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is less than or equal to a second preset value, or an absolute value of a difference between an emotional characteristic value of the second emotional state and the emotional mean is greater than or equal to a third preset value; The second affective state is determined as the default affective state.

8. An information display device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the information display method according to any one of claims 1 to 5 are implemented.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the information display method according to any one of claims 1 to 5 are implemented.

Images