Multimedia Report Generation Method, Device, Storage Medium and Electronic Device

Abstract

The present disclosure provides a multimedia report generation method, device, storage medium and electronic device, which relate to the field of artificial intelligence. The method includes: obtaining a user's browsing record of multimedia resources within a query time period, and determining the user's interest tags based on the browsing record; obtaining user comments generated when the user browses the multimedia resources in the browsing record, determining the user's emotional information based on the user comments, and determining the user's emotional diary based on the emotional information; obtaining the user's multimedia social information within the query time period, and determining a list of friends who interact with the user based on the multimedia social information; forming a multimedia report based on interest tags, emotional diaries and friend lists. The present disclosure can obtain the user's emotional information based on text recognition, thereby improving the accuracy of the emotional diary; and can automatically generate multimedia reports, thereby improving the efficiency of generating multimedia reports, and improving the fun of the multimedia platform and the user's stickiness to the multimedia platform.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more particularly, to a method for generating a multimedia report, a device for generating a multimedia report, a computer-readable storage medium, and an electronic device. Background Art

[0002] A report is a declarative document that summarizes the results of a certain work or business. It has the characteristics of content reporting, language statement, unidirectional writing, post-factum writing, and two-way communication. With the development of electronic business, summary documents such as electronic bills and electronic business reports have gradually emerged, which facilitate users to understand their dynamics within a certain period of time. For example, users are accustomed to using mobile payment, mobile transfer, etc. To help users understand their expenditure situation, the payment platform will generate a financial report based on the income and expenditure of the user's account in the past period of time and feedback it to the user at a specific time point.

[0003] Currently, the application of electronic reports mainly involves the financial field and is rarely applied to the multimedia field. With the enrichment of users' spiritual life, users' interests and moods are usually reflected through multimedia resources. If a multimedia report related to the user can be obtained based on the user's operations on the multimedia platform, on the one hand, an automatically generated multimedia report can be obtained, and this multimedia report can accurately and comprehensively record the user's interests and moods in the past period of time, avoiding the user from manually organizing and generating the multimedia report. On the other hand, it can improve the interest of the multimedia platform and thus increase user stickiness.

[0004] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0005] Embodiments of the present disclosure provide a method for generating a multimedia report, a device for generating a multimedia report, a computer-readable storage medium, and an electronic device, which can, to at least a certain extent, analyze the user's interests and moods based on the user's behavior on the multimedia platform and quickly generate a multimedia report, improving the interest of the multimedia platform and increasing user stickiness to the platform.

[0006] Other features and advantages of the present disclosure will become apparent through the following detailed description, or will be partially learned through the practice of the present disclosure.

[0007] According to one aspect of the embodiments of the present disclosure, a method for generating a multimedia report is provided, including: obtaining the browsing records of a user for multimedia resources during a query time period, and determining the interest tags of the user according to the browsing records; obtaining the user comments generated when the user browses the multimedia resources in the browsing records, determining the emotional information of the user according to the user comments, and determining the emotional diary of the user based on the emotional information; obtaining the multimedia social information of the user during the query time period, and determining a friend list of the friends who interact with the user according to the multimedia social information; and forming a multimedia report according to the interest tags, the emotional diary, and the friend list.

[0008] According to one aspect of the embodiments of the present disclosure, a device for generating a multimedia report is provided, including: an interest tag obtaining module, configured to obtain the browsing records of a user for multimedia resources during a query time period, and determine the interest tags of the user according to the browsing records; an emotional diary obtaining module, configured to obtain the user comments generated when the user browses the multimedia resources in the browsing records, determine the emotional information of the user according to the user comments, and determine the emotional diary of the user based on the emotional information; a friend list obtaining module, configured to obtain the multimedia social information of the user during the query time period, and determine a friend list of the friends who interact with the user according to the multimedia social information; and a multimedia report generating module, configured to form a multimedia report according to the interest tags, the emotional diary, and the friend list.

[0009] In some embodiments of the present disclosure, based on the foregoing solution, the interest tag obtaining module includes: a time obtaining unit, configured to obtain a query start time and a query end time, and determine the query time period according to the query start time and the query end time; a browsing record obtaining unit, configured to obtain the browsing records from a database according to the query time period and the user account of the user; and an interest tag obtaining unit, configured to extract the classification tags corresponding to each multimedia resource in the browsing records, and determine the interest tags according to the classification tags.

[0010] In some embodiments of the present disclosure, the classification tags include multimedia type tags and creator tags; based on the foregoing solution, the interest tag obtaining unit is configured to: respectively perform combined statistics on the multimedia type tags and the creator tags, and respectively sort the multimedia type tags and the creator tags from large to small according to the statistical results to obtain a first sequence and a second sequence; obtain target multimedia type tags from the first sequence according to a first preset quantity, and obtain target creator tags from the second sequence according to a second preset quantity; and determine the interest tags according to the target multimedia type tags and the target creator tags.

[0011] In some embodiments of the present disclosure, based on the foregoing solution, the merging and statistical analysis of the creator tags is configured to: obtain the creator information searched by the user during the query time period, and perform merging and statistical analysis on the creator information and the creator tags.

[0012] In some embodiments of the present disclosure, based on the foregoing solution, the emotional diary acquisition module is configured to: preprocess the user comments, and encode the preprocessed user comments to obtain comment vectors; extract features and classify the comment vectors to obtain emotional information corresponding to the user comments.

[0013] In some embodiments of the present disclosure, the number of multimedia resources in the browsing record is multiple; based on the foregoing solution, the emotional diary acquisition module is further configured to: obtain the user comments corresponding to each of the multimedia resources, and determine the emotional information according to each of the user comments; perform merging and statistical analysis on all the emotional information, sort the emotional information according to the statistical results, and obtain the emotional information corresponding to the query time period from the sorted emotional sequence according to a third preset number.

[0014] In some embodiments of the present disclosure, based on the foregoing solution, the emotional diary acquisition module is further configured to: fill the name of the multimedia resource, the browsing time and emotional information corresponding to the multimedia resource, and the text extracted from the multimedia resource into a diary template to obtain the emotional diary; or obtain the multimedia resources to be statistically analyzed corresponding to the same browsing time, and obtain the target multimedia resource with the largest number of the same multimedia classification tags from the multimedia resources to be statistically analyzed; fill the quantity, browsing time, multimedia classification tag and emotional information corresponding to the target multimedia resource into a diary template to obtain the emotional diary.

[0015] In some embodiments of the present disclosure, the multimedia social information includes custom multimedia interaction information and friend interaction information; based on the foregoing solution, the friend list acquisition module includes: a friend interaction information acquisition unit, configured to obtain all friend interaction information during the query time period; a first friend list construction unit, configured to obtain first friend information in the friend interaction information and construct a first friend list according to the first friend information; a second friend list construction unit, configured to obtain second friend information in the custom multimedia interaction information and construct a second friend list according to the second friend information; and a friend list determination unit, configured to determine the friend list according to the first friend list and the second friend list.

[0016] In some embodiments of the present disclosure, the friend interaction information includes interaction comment information and co-browsing information, and the first friend information includes commenting friends and browsing friends; based on the foregoing solution, the first friend list construction unit is configured to: extract the commenting friends related to the user from the interaction comment information, and at the same time extract the browsing friends related to the user from the co-browsing information; perform a duplicate removal process on the commenting friends and the browsing friends, and construct the first friend list according to the friend information after duplicate removal.

[0017] In some embodiments of the present disclosure, the custom multimedia interaction information is interaction information related to the user's custom video; based on the foregoing solution, the second friend list construction unit is configured to: obtain the custom video corresponding to the custom multimedia interaction information; perform face recognition on the images in the custom video, and obtain the second friend information corresponding to the recognized faces; perform a duplicate removal process on the second friend information, and construct the second friend list according to the friend information after duplicate removal.

[0018] In some embodiments of the present disclosure, based on the foregoing solution, the multimedia report generation device is further configured to: obtain first target friend information from the commenting friends and browsing friends after duplicate removal according to a fourth preset quantity, and construct the first friend list according to the first target friend information; obtain second target friend information from the second friend information after duplicate removal according to a fifth preset quantity, and construct the second friend list according to the second target friend information.

[0019] In some embodiments of the present disclosure, based on the foregoing solution, the multimedia report generation module is configured to: fill the interest tags, the emotional diary, and the friend list into the corresponding tags in the report template to form the multimedia report.

[0020] According to one aspect of the embodiments of the present disclosure, there is provided a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the foregoing optional implementation manners.

[0021] According to one aspect of the embodiments of the present disclosure, there is provided an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method provided in the foregoing optional implementation manners.

[0022] In the technical solutions provided by some embodiments of the present disclosure, by analyzing the browsing records of users on multimedia resources, interest tags of users can be obtained, and based on the comments of users on the browsed multimedia resources, the emotional information of users can be analyzed. Furthermore, based on the emotional information, the emotional diary of users can be obtained. At the same time, according to the multimedia social information of users, the friend list of users can be determined. Finally, based on the interest tags, emotional diaries and friend lists of users, a multimedia report can be formed. On the one hand, the technical solution of the present disclosure can obtain the emotional information of users based on text recognition, improving the accuracy of the emotional diary. On the other hand, it can automatically form a multimedia report according to the extracted interest tags, emotional diaries and friend lists of users, avoiding the need for users to manually organize and generate a multimedia report and improving the generation efficiency of the multimedia report. On the other hand, the multimedia report accurately and comprehensively records the interests and moods of users in the past period of time, which can help users understand their historical interests and historical moods, and thus improve the interestingness of the multimedia platform and the stickiness of users to the multimedia platform.

[0023] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts. In the drawings:

[0025] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiment of the present disclosure can be applied;

[0026] Figure 2 A flowchart schematically showing a method for generating a multimedia report according to an embodiment of the present disclosure;

[0027] Figure 3 A schematic diagram showing an interface for inputting a query time period according to an embodiment of the present disclosure;

[0028] Figure 4 A flowchart schematically showing a process of determining interest tags according to classification tags according to an embodiment of the present disclosure;

[0029] Figure 5 A schematic diagram showing an interface for customizing a video according to an embodiment of the present disclosure;

[0030] Figure 6Schematically shows a schematic diagram of an interface for a user to invite friends to appreciate multimedia resources according to an embodiment of the present disclosure;

[0031] Figure 7 Schematically shows a schematic diagram of a process for obtaining a friend list according to an embodiment of the present disclosure;

[0032] Figure 8 Schematically shows a schematic diagram of an interface for obtaining second friend information according to an embodiment of the present disclosure;

[0033] Figure 9 Schematically shows a schematic diagram of an interface for a video report according to an embodiment of the present disclosure;

[0034] Figure 10 Schematically shows an interaction flowchart for generating a video report according to an embodiment of the present disclosure;

[0035] Figure 11 Schematically shows a block diagram of a multimedia report generation device according to an embodiment of the present disclosure;

[0036] Figure 12 Shows a schematic diagram of the structure of a computer system suitable for implementing the multimedia report generation device of the embodiments of the present disclosure. Detailed implementation manners

[0037] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

[0038] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0039] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the accompanying drawings are only illustrative and not necessarily include all the content and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be further decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0041] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present disclosure can be applied is shown.

[0042] As Figure 1 shown, the system architecture 100 may include a terminal device 101, a network 102, and a server 103. Among them, the above terminal device 101 may be a terminal device with a display screen such as a mobile phone, a portable computer, a tablet computer, a desktop computer, etc. Further, the terminal device 101 may be a terminal device with a display screen and a sound playback unit; the network 102 is a medium for providing a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as a wired communication link, a wireless communication link, etc. In the embodiments of the present disclosure, the network 102 between the terminal device 101 and the server 103 may be a wireless communication link, specifically a mobile network.

[0043] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0044] In one embodiment of the present disclosure, users can view various types of multimedia resources through the multimedia platform installed in the terminal device 101. Users can view them alone or invite friends to browse together in the playback room. At the same time, after browsing the multimedia resources, users can share their post-view feelings by posting comments. It should be noted that all types of multimedia resources browsed by users are multimedia resources processed through regular industrial processes, such as movies, TV dramas, variety shows, music collections, and so on. At the same time, users can also upload custom multimedia resources to the multimedia platform for social interaction. Records of users browsing multimedia resources, uploading custom multimedia resources, commenting, and interacting through the terminal device 101 will be uploaded to the database of the server through the network 102. When receiving a user's trigger instruction or meeting a preset time condition, the server is triggered to generate a multimedia report based on the information in the database. Specifically, first, obtain the browsing records of the user on multimedia resources during the query time period, and determine the user's interest tags according to the browsing records; then obtain the user comments corresponding to the multimedia resources in the browsing records, perform sentiment recognition on the user comments to obtain sentiment information, and obtain the user's sentiment diary based on the sentiment information; then obtain the multimedia social information of the user during the query time period, and determine the user's friend list according to the multimedia social information; finally, form a multimedia report based on the interest tags, sentiment diary, and friend list. Among them, the multimedia social information specifically includes custom multimedia interaction information and friend interaction information.

[0045] It should be noted that the multimedia report generation method provided by the embodiments of the present disclosure is generally executed by the server. Correspondingly, the multimedia report generation device is generally set in the server. However, in other embodiments of the present disclosure, the multimedia report generation method provided by the embodiments of the present disclosure can also be executed by the terminal device.

[0046] The embodiments of the present disclosure provide a multimedia report generation method, which is implemented based on machine learning. Machine learning belongs to a type of artificial intelligence. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making.

[0047] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0048] Computer Vision Technology (CV) Computer vision is a science that studies how to enable machines to "see". More specifically, it refers to using cameras and computers to replace human eyes to perform machine vision such as object recognition, tracking, and measurement on targets, and further performing graphics processing to make the images processed by the computer more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to build artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0049] Machine Learning (ML) is an interdisciplinary field that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rote learning.

[0050] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0051] The solution provided by the embodiments of this disclosure relates to the natural language processing technology of artificial intelligence and also involves image recognition technology, which will be specifically described through the following embodiments:

[0052] Figure 2A flowchart of a multimedia report generation method according to an embodiment of the present disclosure is schematically shown. The multimedia report generation method can be executed by a server, which can be the Figure 1 server 103 shown in Figure 2 As shown, the multimedia report generation method at least includes steps S210 to S240, which are introduced in detail as follows:

[0053] In step S210, obtain the browsing records of the user on multimedia resources during the query time period, and determine the interest tags of the user according to the browsing records.

[0054] In an embodiment of the present disclosure, a multimedia report is a report generated based on the user's behavior on a multimedia platform for a period of time. Specifically, it can include information such as which types of multimedia resources the user is interested in, which creators the user is interested in, interactive friend information, and the user's mood after viewing multimedia resources on the multimedia platform. The query time period can be a time period specified by the user or a time period set by the system. For example, it can be the last half month, a certain month, several months, the last year, and so on. When it is a time period specified by the user, the user can click a button on the main page of the multimedia platform to enter the multimedia report generation page. After entering this page, the user can manually enter the time or obtain time options through a drop-down arrow, and then select the corresponding date from them to determine the query time period. Figure 3 The interface schematic diagram for setting the query time period is shown. As Figure 3 shown, there are two time input boxes in the interface. The first time input box is the query start time, and the second time input box is the query end time. There is a drop-down arrow on the right side of the two time input boxes. After clicking the drop-down arrow, time options will appear. The user can place the input cursor in the time input box to enter, or click the drop-down arrow to select the time. After determining the query start time and the query end time, click the submit button below the interface. In the embodiment of the present disclosure, the input time can be accurate to the month, or specific to the day, and of course, it can also be specific to a certain point and minute of a certain day. The embodiment of the present disclosure does not make specific limitations on this. When it is a time period set by the system, the query time period can be determined according to the query start time and the query end time set by the system. For example, if the system is set to generate a multimedia report on December 31st every year, then the query start time can be determined as January 1st of that year, and the query end time can be determined as December 31st of that year.

[0055] After obtaining the query start time and the query end time, the query time period can be determined according to the query start time and the query end time, and the browsing records of the user on the multimedia resources during the query time period can be queried in the database according to the query start time and the query end time. It should be noted that the browsing records include but are not limited to viewing records and appreciation records, and also include records of operations such as clicking to favorite, sharing, and posting comments. The multimedia platform includes various types of resources, such as videos, music, broadcasts, etc. And for each type of resource, it is also divided into different types. For example, videos can be divided into movies, TV dramas, variety shows, etc., music can be divided into classical, pop, jazz, etc., and broadcasts are divided into literature, language, life, etc. Taking the video platform as an example, there are usually long and short videos. Long videos are video resources processed according to industrial standards, such as movies, TV dramas, etc. Short videos are custom videos formed by users taking random shots or cropping and splicing video resources. Of course, there are also similar resource forms in other types of multimedia platforms. In the embodiments of the present disclosure, the multimedia resources browsed by the user on the multimedia platform are multimedia resources processed according to industrial standards, and the multimedia resources created by the user are referred to as custom multimedia resources in the embodiments of the present disclosure.

[0056] Usually, users will select corresponding multimedia resources to browse according to their interests and hobbies. Therefore, in the embodiments of the present disclosure, the interest tags of the user can be obtained by analyzing the browsing records of the user on the multimedia resources during the query time period. Specifically, the classification tags corresponding to each multimedia resource in the browsing records can be extracted, and then the interest tags can be determined according to the classification tags.

[0057] In an embodiment of the present disclosure, the classification tags include multimedia type tags and creator tags. The multimedia type tags are the lowest-level classifications of the multimedia resources. For example, if the multimedia resources are TV dramas and movies, then the multimedia type tags can be divided into comedy, love drama, tragedy, suspense drama, etc. The creator tags are the creators of the multimedia resources, such as the main creative members of movies and TV dramas, the performers of music, etc. Figure 4 The flow diagram showing the determination of the interest tags according to the classification tags is as Figure 4 shown. In step S401, the multimedia type tags and the creator tags are respectively counted, and the multimedia type tags and the creator tags are respectively sorted from large to small according to the statistical results to obtain a first sequence and a second sequence; in step S402, target multimedia type tags are obtained from the first sequence according to a first preset quantity, and target creator tags are obtained from the second sequence according to a second preset quantity; in step S403, the interest tags are determined according to the target multimedia type tags and the target creator tags.

[0058] Among them, the first preset quantity and the second preset quantity may be the same or different. For example, the first preset quantity can be set to 8, and the second preset quantity can be set to 5. After obtaining the first sequence and the second sequence, the first 8 multimedia type tags are sequentially obtained from the first sequence as the target multimedia type tags, and the first 5 creator tags are sequentially obtained from the second sequence as the target creator tags. These target multimedia type tags and target creator tags can be used as interest tags. Further, the proportion of each target multimedia type tag among all target multimedia type tags and the proportion of each target creator tag among all target creator tags can also be calculated to facilitate the generation of subsequent reports.

[0059] In an embodiment of the present disclosure, the user can also retrieve a creator to obtain multimedia resources related to the creator for browsing. Therefore, when aggregating and counting the creator tags, the creator information searched by the user during the query time period can also be obtained, and then the creator information and the creator tags are merged, counted, and sorted to obtain the target creator tags.

[0060] Taking videos as an example, by counting and sorting the type tags of all standard videos (long videos) browsed by the user in 2019, the top 3 movie and TV tags are obtained. For example, love appears 160 times, second dimension appears 123 times, suspense appears 111 times, etc. The corresponding proportions are 41% for love, 31% for second dimension, and 28% for suspense. At the same time, by counting and sorting the creator tags of all standard videos browsed by the user in 2019 and the creator information searched, the top 3 favorite actors are obtained. For example, Zhou appears 45 times, Liu appears 23 times, and Xu appears 18 times. The corresponding proportions are 52% for Zhou, 27% for Liu, and 21% for Xu. Then, in the generated multimedia report, it will be reflected that the user's favorite video types in 2019 are love, second dimension, and suspense, and the favorite actors are Zhou, Liu, and Xu.

[0061] In step S220, obtain the user comments generated when the user browses the multimedia resources in the browsing record, determine the emotional information of the user according to the user comments, and determine the emotional diary of the user based on the emotional information.

[0062] In an embodiment of the present disclosure, the multimedia report may also include the user's emotional information, so the multimedia report can also be regarded as the user's interest and mood diary. Usually, the user's emotions are expressed through words. In the scenario of browsing multimedia resources, the user usually incorporates emotions into the comments posted. For example, after watching the movie "The Legend of Little A", the user posted a comment "The Legend of Little A is good-looking. It made me laugh to death. It swept away the haze of my day. So happy~". By performing emotion recognition on this comment, the user's emotional information can be determined to be happy. Then, the user's emotion diary can be generated based on the emotional information and the movie information. That is to say, in the embodiment of the present disclosure, the user comments generated by the user when browsing the multimedia resources in the browsing record can be obtained, and then the user's emotional information can be determined according to the user comments, and the user's emotion diary can be generated based on the emotional information.

[0063] In an embodiment of the present disclosure, when determining the user's emotional information according to the user comments, first, preprocess the user comments, and encode the preprocessed user comments to obtain a comment vector; then, extract features and classify the comment vector to obtain the emotional information corresponding to the user comments.

[0064] The preprocessing specifically includes processing procedures such as word segmentation, stop word removal, and simplified-traditional conversion of the user comments. A word is the smallest meaningful language component that can be obtained independently. Through word segmentation, the user comments can be converted into a word representation. When performing word segmentation, it can be carried out based on rules, based on statistics, or a combination of the two. Taking the rule-based word segmentation method as an example, the sentences corresponding to the event descriptions can be divided by means of dictionary matching. Specifically, the forward maximum matching method, the backward maximum matching method, or the bidirectional maximum matching method can be used for word segmentation; stop words refer to certain words or characters that are automatically filtered out before or after processing natural language data (or text) in information retrieval to save storage space and improve search efficiency. These words or characters are called stop words. Usually, stop words are manually input and not automatically generated. A stop word list can be formed according to common stop words, and the stop word removal process can be performed on the results of word segmentation to retain only the meaningful words in the event description; simplified-traditional conversion is the conversion between simplified Chinese characters and traditional Chinese characters.

[0065] After obtaining the preprocessed user comments, each word segment can be encoded and converted into a word vector, and then a comment vector corresponding to the user comment can be obtained. When encoding and converting the word segments, the conversion can be performed by the word2vec method. Of course, other methods can also be used for word vector encoding. After obtaining the comment vector, a recurrent neural network such as a long short-term memory network or GRU can be used to extract features from the comment vector, and then the extracted feature vector can be classified through a fully connected layer and a normalization layer to obtain the probability of each sentiment information corresponding to the user comment, and then the sentiment information corresponding to the user comment can be determined according to the probability corresponding to each sentiment information.

[0066] In an embodiment of the present disclosure, when the number of multimedia resources in the browsing record is multiple, after obtaining the sentiment information of the user comments corresponding to each multimedia resource according to the above method, all the obtained sentiment information can be merged and statistically analyzed, and the sentiment information can be sorted according to the statistical result, and then the sentiment information corresponding to the query time period can be obtained from the sorted sentiment sequence according to the third preset quantity. For example, the query time period set by the user is the last month, and the user has watched 20 movies in the last month. Then, the comments posted by the user after watching 20 movies are obtained, and the sentiment of the comments posted by the user is recognized to obtain the sentiment information corresponding to each comment. Then, all the sentiment information is merged and statistically analyzed, and the sentiment information is sorted according to the result of the merged statistics. For example, the sorted sentiment information is touched, frightened, happy... If the third preset quantity is 3, then the first three sentiment information can be obtained, and a sentiment diary can be generated according to the obtained sentiment information and the corresponding movie information.

[0067] After obtaining the sentiment information corresponding to the user's comment, a sentiment diary can be generated based on the diary template and relevant information. The diary template can be [browsing time + multimedia resource name + classic quote + sentiment information]. Then, the name of the multimedia resource included in the browsing record within the query time period, the browsing time corresponding to the multimedia resource, the sentiment information, and the text extracted from the multimedia resource can be filled into the diary template in the order of each label of the diary template to form a sentiment diary. For example, if the user watched the movie "The Legend of Little B" on March 7, 2020 and left a comment, the generated sentiment diary could be "2020.3.7 (browsing time) Watched The Legend of Little B (movie title) Perhaps you were touched by his life is like a box of chocolates, you never know what you're gonna get (classic quote) (sentiment information)". The diary template can also be [browsing time + quantity + multimedia classification label + sentiment information]. When generating the sentiment diary, first obtain the multimedia resources to be counted corresponding to the same browsing time, and obtain the target multimedia resource with the largest quantity and the same multimedia classification label from the multimedia resources to be counted; then fill the quantity, browsing time, multimedia classification label, and sentiment information corresponding to the target multimedia resource into the template to obtain the sentiment diary. For example, if the user watched a total of 5 movies on August 8, including 3 comedy movies, 1 emotional drama movie, and 1 suspense drama movie, and the number of comedies is the largest, then obtain the user comments corresponding to the 3 comedies, such as "So stress-relieving", "So relaxing", "So funny, feeling great all over". By performing sentiment recognition on each user comment, it can be obtained that the sentiment information corresponding to the user comments of the 3 comedy movies is all stress-relieving. Then, the sentiment information of stress-relieving can be obtained, and a sentiment diary can be generated based on the browsing time [August 8], quantity [3], multimedia classification label [comedy], and sentiment information [stress-relieving], such as "Watched 3 comedies on August 8, it should be a day for stress relief". Of course, other diary templates can also be set. Corresponding content is generated according to the label types in the diary template, and then the content is added under the corresponding labels to generate a sentiment diary.

[0068] In step S230, obtain the multimedia social information of the user within the query time period, and determine the friend list that interacts with the user according to the multimedia social information.

[0069] In an embodiment of the present disclosure, the multimedia social information includes custom multimedia interaction information and friend interaction information, where the custom multimedia interaction information is information about the user and friends jointly participating in the production of custom multimedia resources. For example, a short video jointly shot and recorded by the user and friends, such as Figure 5As shown, for the short video taken by the user together with friends, the user's friend information can be obtained by identifying the people in the custom multimedia resource; the friend interaction information includes the friend information of the friends with whom the user interacts in comments when browsing the multimedia resource, and the friend information of the friends invited by the user to browse the multimedia resource. For example, the user creates a viewing hall on the video platform, selects the movie to watch and the viewing time, and then invites friends to join the viewing hall to watch the movie together, as Figure 6 As shown, user Zita creates Zita's private viewing hall and invites friends Xie and Xindejia to watch "The Legend of Little C" at 9:10 on February 6th.

[0070] By analyzing the custom multimedia interaction information and the friend interaction information, the user's friend list, that is, the user's relationship chain, can be obtained. According to the different forms of friend interaction, friends can be divided into two categories. The first category of friends are those who enjoy the same multimedia resource with the user in the same playback session and those who interact with the user in comments. The second category of friends are those who shoot short videos with the user. Figure 7 The flowchart of obtaining the friend list is shown, as Figure 7 As shown, in step S701, all friend interaction information within the query time period is obtained; in step S702, the first friend information in the friend interaction information is obtained, and the first friend list is constructed based on the first friend information; in step S703, the second friend information is determined according to the custom multimedia interaction information, and the second friend list is constructed based on the second friend information; in step S704, the friend list is determined according to the first friend list and the second friend list.

[0071] Among them, in step S702, the friend interaction information may include interaction comment information and co-browsing information. For the interaction comment information, the user relationship chain can be determined by the method of @-mentioning a friend. For example, if a user @-mentions user A after posting a comment, then it can be determined that user A is a friend of the current user. Correspondingly, the first friend information may include comment friends and browsing friends. After obtaining the friend interaction information, the comment friends related to the user can be extracted from the interaction comment information, and at the same time, the browsing friends related to the user can be extracted from the co-browsing information. Then, duplicate removal processing is performed on the comment friends and browsing friends, and the first friend list can be constructed based on the de-duplicated friends. In step S703, when the custom multimedia interaction information is the interaction information related to the user's custom video, first, the custom video corresponding to the custom multimedia interaction information can be obtained, then face recognition is performed on the images in the custom video, and the second friend information corresponding to the recognized face is obtained. Finally, duplicate removal processing is performed on the second friend information, and the second friend list can be constructed based on the de-duplicated friend information. When performing face recognition on the images in the custom multimedia resource, a machine learning model can be used to analyze and recognize the faces in the images, and an interest box is displayed in the area where a face exists according to the analysis result. At the same time, a dialog box is displayed next to the interest box, prompting the user to input the information corresponding to the face. Figure 8 shows a schematic interface diagram for obtaining the second friend information, as Figure 8 shown. A frame image of a short video jointly recorded by the user and the friend is displayed on the display interface. When the background recognizes the face existing in the image, the face is marked by an interest box. At the same time, a dialog box "Please enter the user information corresponding to this face" is displayed next to the interest box. The user can place the input cursor in the dialog box and input the information corresponding to the face in the interest box. Specifically, the user can input the friend's name, nickname, online name, etc. If the user does not input the user information corresponding to the face in the dialog box, then the face is ignored. If the custom multimedia resource is a non-video type of multimedia resource such as custom music, then the custom multimedia interaction information is mainly the user's comment on the custom multimedia resource. Then, the friend information of the user is determined according to the content of the comment. For example, the user posts a piano piece played by himself and his friend, and there is only an audio signal without image information. Then the user usually @-mentions the friend in the comment, and thus the friend information of the user can be obtained according to the user's comment.

[0072] In one embodiment of the present disclosure, when the number of friends included in the first friend list and the second friend list exceeds a preset number, it is unrealistic to display all friends in the multimedia report. Therefore, to improve the readability of the multimedia report, the first target friend information can be obtained from the friend information after deduplication of the commented friends and the browsed friends according to a fourth preset number, and the first friend list can be constructed based on the first target friend information. At the same time, the second target friend information can be obtained from the deduplicated second friend information according to a fifth preset number, and the second friend list can be constructed based on the second target friend information. Finally, the friend list is formed based on the first friend list and the second friend list, and the multimedia report is generated based on the friend list.

[0073] In step S240, a multimedia report is formed according to the interest tags, the emotional diary, and the friend list.

[0074] In one embodiment of the present disclosure, the user's interest tags, emotional diary, and friend list are obtained respectively through steps S210-S230, and the multimedia report can be generated by combining these information. Specifically, first, a report template can be set, and the report template can include tags regarding the user's interests, the user's emotional diary, and the friend list. Then, the user's interest tags, emotional diary, and friend list are filled in the corresponding tags in the report template to obtain the multimedia report. Figure 9 The interface schematic diagram of the video report is shown, as Figure 9 shown, in the video report, tags "Favorite Video Type", "Favorite Actor", "My Friends", and "My Mood Diary" are set, where "Favorite Video Type" and "Favorite Actor" correspond to the user's interest tags, "My Friends" corresponds to the user's friend list, and "My Mood Diary" corresponds to the user's emotional diary.

[0075] In the multimedia report generation method according to the embodiments of the present disclosure, by analyzing the browsing records of users on multimedia resources on the multimedia platform, interest tags of users can be obtained. At the same time, by performing sentiment analysis on the comments published by users after browsing multimedia resources, sentiment diaries of users can be generated according to the obtained sentiment information. Further, according to the multimedia social information generated by users through multimedia resources for social interaction on the multimedia platform, a friend list of users can be obtained. Then, a multimedia report can be generated according to the interest tags, sentiment diaries, and friend list of users. On the one hand, the technical solution of the present disclosure can accurately locate the sentiment information of user comments through sentiment recognition, and then generate accurate sentiment diaries. On the other hand, it can automatically form a multimedia report according to the extracted interest tags, sentiment diaries, and friend list of users, avoiding the need for users to manually organize and generate multimedia reports, and improving the generation efficiency of multimedia reports. On the third hand, the multimedia report accurately and comprehensively records the interests, emotions, and life of users in the past period of time, helps users record their lives, improves the user experience, and also improves the interestingness of the multimedia platform, thereby improving the stickiness of users to the multimedia platform.

[0076] As described in the above embodiments, the multimedia platform may include various types of resources, such as videos, music, broadcasts, and so on. Next, the technical solution of the present disclosure will be specifically described by taking a video platform as an example. Figure 10 The interaction flowchart for generating a video report is shown. As Figure 10 shown, in step S1001, the user performs operations such as watching movies, commenting, and posting short videos on the video platform; in step S1002, the video platform sends all the operation information of the user to the background; in step S1003, the user submits a query start time and a query end time on the video platform; in step S1004, the video platform sends the query start time and the query end time to the background; in step S1005, the background determines the query time period according to the query start time and the query end time; in step S1006, the viewing records of the user watching standard videos during the query time period are obtained, and the interest tags of the user are determined according to the viewing records; in step S1007, the user comments after the user watches videos during the query time period are obtained, sentiment recognition is performed on the user comments, and a sentiment diary is generated; in step S1008, the video social information of the user during the query time period is obtained, and the friend list of the user is determined according to the video social information; in step S1009, a video report is generated according to the interest tags, sentiment diaries, and friend list; in step S1010, the video report is returned to the terminal device for the user to view. Further, after step S1007, it can also be determined whether the number of sentiment diaries exceeds a preset number. If it exceeds the preset number, target sentiments can be obtained from the recognized sentiments according to the preset number, and sentiment diaries can be generated according to the target sentiments.

[0077] The following introduces the device embodiments of the present disclosure, which can be used to execute the multimedia report generation method in the above embodiments of the present disclosure. For details not disclosed in the device embodiments of the present disclosure, please refer to the embodiments of the above multimedia report generation method of the present disclosure.

[0078] Figure 11 A block diagram of a multimedia report generation device according to an embodiment of the present disclosure is schematically shown.

[0079] Referring to Figure 11 As shown, a multimedia report generation device 1100 according to an embodiment of the present disclosure includes: an interest tag acquisition module 1101, an emotional diary acquisition module 1102, a friend list acquisition module 1103, and a multimedia report generation module 1104.

[0080] Among them, the interest tag acquisition module 1101 is configured to acquire the browsing records of the user on multimedia resources during the query time period, and determine the interest tags of the user according to the browsing records; the emotional diary acquisition module 1102 is configured to acquire the user comments generated by the user when browsing the multimedia resources in the browsing records, determine the emotional information of the user according to the user comments, and determine the emotional diary of the user based on the emotional information; the friend list acquisition module 1103 is configured to acquire the multimedia social information of the user during the query time period, and determine the friend list that interacts with the user according to the multimedia social information; the multimedia report generation module 1104 is configured to form a multimedia report according to the interest tags, the emotional diary, and the friend list.

[0081] In an embodiment of the present disclosure, the interest tag acquisition module 1101 includes: a time acquisition unit, configured to acquire a query start time and a query end time, and determine the query time period according to the query start time and the query end time; a browsing record acquisition unit, configured to acquire the browsing records from the database according to the query time period and the user account of the user; an interest tag acquisition unit, configured to extract the classification tags corresponding to each multimedia resource in the browsing records, and determine the interest tags according to the classification tags.

[0082] In an embodiment of the present disclosure, the classification tags include multimedia type tags and creator tags; the interest tag acquisition unit is configured to: respectively perform combined statistics on the multimedia type tags and the creator tags, and sort the multimedia type tags and the creator tags respectively from largest to smallest according to the statistical results to obtain a first sequence and a second sequence; obtain target multimedia type tags from the first sequence according to a first preset quantity, and obtain target creator tags from the second sequence according to a second preset quantity; determine the interest tags according to the target multimedia type tags and the target creator tags.

[0083] In an embodiment of the present disclosure, the combined statistics on the creator tags is configured to: obtain the creator information searched by the user during the query time period, and perform combined statistics on the creator information and the creator tags.

[0084] In an embodiment of the present disclosure, the emotional diary acquisition module 1102 is configured to: preprocess the user comments, and encode the preprocessed user comments to obtain comment vectors; extract features and classify the comment vectors to obtain emotional information corresponding to the user comments.

[0085] In an embodiment of the present disclosure, the number of multimedia resources in the browsing record is multiple; the emotional diary acquisition module 1102 is further configured to: obtain the user comments corresponding to each of the multimedia resources, and determine the emotional information according to each of the user comments; perform combined statistics on all the emotional information, sort the emotional information according to the statistical results, and obtain the emotional information corresponding to the query time period from the sorted emotional sequence according to a third preset quantity.

[0086] In an embodiment of the present disclosure, based on the foregoing solution, the emotional diary acquisition module 1102 is further configured to: fill the name of the multimedia resource, the browsing time corresponding to the multimedia resource, the emotional information, and the text extracted from the multimedia resource into a diary template to obtain the emotional diary; or obtain the multimedia resources to be statistically analyzed corresponding to the same browsing time, and obtain the target multimedia resources with the largest number of the same multimedia classification tags from the multimedia resources to be statistically analyzed; fill the quantity, browsing time, multimedia classification tags, and emotional information corresponding to the target multimedia resources into a diary template to obtain the emotional diary.

[0087] In an embodiment of the present disclosure, the multimedia social information includes custom multimedia interaction information and friend interaction information; the friend list acquisition module 1103 includes: a friend interaction information acquisition unit configured to acquire all friend interaction information during the query time period; a first friend list construction unit configured to acquire first friend information from the friend interaction information and construct a first friend list according to the first friend information; a second friend list construction unit configured to acquire second friend information from the custom multimedia interaction information and construct a second friend list according to the second friend information; and a friend list determination unit configured to determine the friend list according to the first friend list and the second friend list.

[0088] In an embodiment of the present disclosure, the friend interaction information includes interaction comment information and co-browsing information, and the first friend information includes commenting friends and browsing friends; the first friend list construction unit is configured to: extract the commenting friends related to the user from the interaction comment information, and at the same time extract the browsing friends related to the user from the co-browsing information; perform a duplicate removal process on the commenting friends and the browsing friends, and construct the first friend list according to the friend information after duplicate removal.

[0089] In an embodiment of the present disclosure, the custom multimedia interaction information is interaction information related to a custom video of the user; the second friend list construction unit is configured to: acquire the custom video corresponding to the custom multimedia interaction information; perform face recognition on the images in the custom video, and acquire second friend information corresponding to the recognized faces; perform a duplicate removal process on the second friend information, and construct the second friend list according to the friend information after duplicate removal.

[0090] In an embodiment of the present disclosure, the multimedia report generation device 1100 is further configured to: acquire first target friend information from the commenting friends and browsing friends after duplicate removal according to a fourth preset quantity, and construct the first friend list according to the first target friend information; acquire second target friend information from the second friend information after duplicate removal according to a fifth preset quantity, and construct the second friend list according to the second target friend information.

[0091] In an embodiment of the present disclosure, the multimedia report generation module 1104 is configured to: fill the interest tags, the emotional diary, and the friend list into corresponding tags in a report template to form the multimedia report.

[0092] Figure 12 The structural schematic diagram of a computer system suitable for implementing the word embedding representation learning device and the text recall device of the embodiments of the present disclosure is shown.

[0093] It should be noted that Figure 12 The computer system 1200 of the word embedding representation learning device and the text recall device shown is only an example, and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0094] As Figure 12 shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage section 1208 into the random access memory (RAM) 1203, and implement the search string processing method described in the above embodiments. In the RAM 1203, various programs and data required for system operation are also stored. The CPU 1201, ROM 1202, and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0095] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read from it can be installed into the storage section 1208 as needed.

[0096] Specifically, according to the embodiments of the present disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 1209, and / or installed from the removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, various functions defined in the system of the present disclosure are executed.

[0097] It should be noted that the computer-readable medium shown in the embodiments of the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0099] The units described in the embodiments of the present disclosure can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation on the units themselves in some cases.

[0100] On the other hand, the present disclosure also provides a computer-readable medium, which can be included in the word embedding representation learning device and the text recall device described in the above embodiments; or can exist separately without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the one or more programs are executed by an electronic device, the electronic device implements the methods described in the above embodiments.

[0101] It should be noted that although several modules or units of devices for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0102] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0103] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure.

[0104] It should be understood that the present disclosure is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for generating a multimedia report, characterized in that, Including: Obtain the browsing records of the user on multimedia resources during the query time period, and determine the interest tags of the user according to the browsing records; Obtain the user comments generated when the user browses the multimedia resources in the browsing records, determine the emotional information of the user according to the user comments, and determine the emotional diary of the user based on the emotional information, the quantity, browsing time, and multimedia classification tags of the multimedia resources corresponding to the emotional information; Obtain the multimedia social information of the user during the query time period, and determine the friend list that interacts with the user according to the multimedia social information; The friends in the friend list are divided into the first type of friends and the second type of friends; The first type of friends are the friends who jointly appreciate the same multimedia resource with the user in the same playback room and the friends who interact and comment with the user, and the second type of friends are the friends who jointly shoot short videos with the user; Form a multimedia report according to the interest tags, the emotional diary, and the friend list; The quantity of the multimedia resources in the browsing records is multiple; obtaining the user comments generated when the user browses the multimedia resources in the browsing records, and determining the emotional information of the user according to the user comments includes: Obtain the user comments corresponding to each multimedia resource, and determine the emotional information according to each user comment; Merge and count all the emotional information, sort the emotional information according to the statistical results, and obtain the emotional information corresponding to the query time period from the sorted emotional sequence according to the third preset quantity.

2. The method according to claim 1, characterized in that The obtaining the browsing records of the user on multimedia resources during the query time period, and determining the interest tags of the user according to the browsing records includes: Obtain the query start time and the query end time, and determine the query time period according to the query start time and the query end time; Obtain the browsing records from the database according to the query time period and the user account of the user; Extract the classification tags corresponding to each multimedia resource in the browsing records, and determine the interest tags according to the classification tags.

3. The method according to claim 2, wherein The classification tags include multimedia type tags and creator tags; The determining the interest tags according to the classification tags includes: Merge and count the multimedia type tags and the creator tags respectively, and sort the multimedia type tags and the creator tags respectively from large to small according to the statistical results to obtain a first sequence and a second sequence; Obtain the target multimedia type tags from the first sequence according to the first preset quantity, and obtain the target creator tags from the second sequence according to the second preset quantity; Determine the interest tags according to the target multimedia type tags and the target creator tags.

4. The method according to claim 3, wherein The merging and counting the creator tags includes: Obtain the creator information searched by the user during the query time period, and merge and count the creator information and the creator tags.

5. The method according to claim 1, characterized in that The obtaining the user comments generated when the user browses the multimedia resources in the browsing records, and determining the emotional information of the user according to the user comments includes: Preprocess the user comment and encode the preprocessed user comment to obtain a comment vector; Extract features from and classify the comment vector to obtain sentiment information corresponding to the user comment.

6. The method according to claim 5, characterized in that, The extracting features from and classifying the comment vector to obtain sentiment information corresponding to the user comment includes: Classify the extracted feature vector through a fully connected layer and a normalization layer to obtain the probability of each sentiment information corresponding to the user comment; Determine the sentiment information corresponding to the user comment according to the probabilities corresponding to each sentiment information.

7. The method according to claim 5, characterized in that, The determining the user's sentiment diary based on the sentiment information includes: Fill the name of the multimedia resource, the browsing time corresponding to the multimedia resource, the sentiment information, and the text extracted from the multimedia resource into a diary template to obtain the sentiment diary; or Obtain the multimedia resources to be counted corresponding to the same browsing time, and obtain the target multimedia resource with the largest number of the same multimedia classification labels from the multimedia resources to be counted; Fill the quantity, browsing time, multimedia classification label, and sentiment information corresponding to the target multimedia resource into a diary template to obtain the sentiment diary.

8. The method according to claim 1, wherein The multimedia social information includes custom multimedia interaction information and friend interaction information; The obtaining the multimedia social information of the user within the query time period and determining a friend list of the friends who interact with the user according to the multimedia social information includes: Obtain all the friend interaction information within the query time period; Obtain the first friend information in the friend interaction information and construct a first friend list according to the first friend information; Obtain the second friend information in the custom multimedia interaction information and construct a second friend list according to the second friend information; Determine the friend list according to the first friend list and the second friend list.

9. The method according to claim 8, wherein The friend interaction information includes interaction comment information and co-browsing information, and the first friend information includes comment friends and browsing friends; The obtaining the first friend information in the friend interaction information and constructing a first friend list according to the first friend information includes: Extract the comment friends related to the user from the interaction comment information, and at the same time extract the browsing friends related to the user from the co-browsing information; Deduplicate the comment friends and the browsing friends, and construct the first friend list according to the deduplicated friend information.

10. The method according to claim 9, wherein The custom multimedia interaction information is interaction information related to the user's custom video; The obtaining the second friend information in the custom multimedia interaction information and constructing a second friend list according to the second friend information includes: Obtain the custom video corresponding to the custom multimedia interaction information; Perform face recognition on the images in the custom video, and obtain the second friend information corresponding to the recognized face; Deduplicate the second friend information, and construct the second friend list according to the deduplicated friend information.

11. The method according to claim 10, wherein The method further includes: Obtain the first target friend information from the deduplicated comment friends and viewed friends according to the fourth preset quantity, and construct the first friend list according to the first target friend information; Obtain the second target friend information from the deduplicated second friend information according to the fifth preset quantity, and construct the second friend list according to the second target friend information.

12. The method according to claim 1, 8 or 11, characterized in that Forming the multimedia report according to the interest tags, the emotional diary and the friend list includes: Fill the interest tags, the emotional diary and the friend list under the corresponding tags in the report template to form the multimedia report.

13. A multimedia report generation device, characterized in that, Including: An interest tag acquisition module, configured to obtain the browsing records of the user on multimedia resources during the query time period, and determine the interest tags of the user according to the browsing records; An emotional diary acquisition module, configured to obtain the user comments generated by the user when browsing the multimedia resources in the browsing records, determine the emotional information of the user according to the user comments, and determine the emotional diary of the user based on the emotional information, the quantity, browsing time, and multimedia classification tags of the multimedia resources corresponding to the emotional information; A friend list acquisition module, configured to obtain the multimedia social information of the user during the query time period, and determine the friend list that interacts with the user according to the multimedia social information; The friends in the friend list are divided into the first type of friends and the second type of friends; The first type of friends are the friends who appreciate the same multimedia resource with the user in the same playback room and the friends who interact and comment with the user, and the second type of friends are the friends who jointly shoot short videos with the user; A multimedia report generation module, configured to form a multimedia report according to the interest tags, the emotional diary and the friend list; The quantity of the multimedia resources in the browsing records is multiple; the emotional diary acquisition module is further configured to: obtain the user comments corresponding to each of the multimedia resources, and determine the emotional information according to each of the user comments; merge and count all the emotional information, sort the emotional information according to the statistical results, and obtain the emotional information corresponding to the query time period from the sorted emotional sequence according to the third preset quantity.

14. The device according to claim 13, characterized in that The interest tag acquisition module includes: a time acquisition unit, configured to obtain the query start time and the query end time, and determine the query time period according to the query start time and the query end time; a browsing record acquisition unit, configured to obtain the browsing records from the database according to the query time period and the user account of the user; an interest tag acquisition unit, configured to extract the classification tags corresponding to each multimedia resource in the browsing records, and determine the interest tags according to the classification tags.

15. The device according to claim 14, characterized in that, The classification tags include multimedia type tags and creator tags; the interest tag acquisition unit is configured to: respectively perform combined statistics on the multimedia type tags and the creator tags, and sort the multimedia type tags and the creator tags from largest to smallest according to the statistical results to obtain a first sequence and a second sequence; obtain target multimedia type tags from the first sequence according to a first preset quantity, and obtain target creator tags from the second sequence according to a second preset quantity; determine the interest tags according to the target multimedia type tags and the target creator tags.

16. The device according to claim 15, characterized in that, The performing combined statistics on the creator tags is configured to: obtain the creator information searched by the user during the query time period, and perform combined statistics on the creator information and the creator tags.

17. The device according to claim 13, characterized in that, The emotional diary acquisition module is configured to: preprocess the user comments, and encode the preprocessed user comments to obtain comment vectors; extract features and classify the comment vectors to obtain emotional information corresponding to the user comments.

18. The device according to claim 17, characterized in that, The extracting features and classifying the comment vectors to obtain emotional information corresponding to the user comments includes: classifying the extracted feature vectors through a fully connected layer and a normalization layer to obtain the probabilities of user comments corresponding to each emotional information; determining the emotional information corresponding to the user comments according to the probabilities corresponding to each emotional information.

19. The device according to claim 17, characterized in that, The emotional diary acquisition module is further configured to: fill the name of the multimedia resource, the browsing time and emotional information corresponding to the multimedia resource, and the text extracted from the multimedia resource into a diary template to obtain the emotional diary; or obtain the multimedia resources to be counted corresponding to the same browsing time, and obtain the target multimedia resources with the largest number of the same multimedia classification tags from the multimedia resources to be counted; fill the quantity, browsing time, multimedia classification tags and emotional information corresponding to the target multimedia resources into a diary template to obtain the emotional diary.

20. The device according to claim 13, characterized in that, The multimedia social information includes custom multimedia interaction information and friend interaction information; Based on the foregoing solution, the friend list acquisition module includes: a friend interaction information acquisition unit for acquiring all friend interaction information during the query time period; a first friend list construction unit for acquiring first friend information from the friend interaction information and constructing a first friend list according to the first friend information; a second friend list construction unit for acquiring second friend information from the custom multimedia interaction information and constructing a second friend list according to the second friend information; a friend list determination unit for determining the friend list according to the first friend list and the second friend list.

21. The device according to claim 20, wherein The friend interaction information includes interaction comment information and co-browsing information, and the first friend information includes comment friends and browsing friends; Based on the foregoing solution, the first friend list building unit is configured to: extract the commenting friends related to the user from the interactive comment information, and at the same time extract the browsing friends related to the user from the co-browsing information; Perform deduplication processing on the commenting friends and the browsing friends, and build the first friend list according to the deduplicated friend information.

22. The device according to claim 21, characterized in that, The custom multimedia interaction information is interaction information related to the user's custom video; based on the foregoing solution, the second friend list building unit is configured to: obtain the custom video corresponding to the custom multimedia interaction information; perform face recognition on the images in the custom video, and obtain second friend information corresponding to the recognized faces; Perform deduplication processing on the second friend information, and build the second friend list according to the deduplicated friend information.

23. The device according to claim 22, characterized in that, The multimedia report generating device is further configured to: obtain first target friend information from the deduplicated commenting friends and browsing friends according to a fourth preset quantity, and build the first friend list according to the first target friend information; Obtain second target friend information from the deduplicated second friend information according to a fifth preset quantity, and build the second friend list according to the second target friend information.

24. The device according to claim 13, 20 or 23, characterized in that, The multimedia report generating module is configured to: fill the interest tags, the emotional diary, and the friend list into the corresponding tags in the report template to form the multimedia report.

25. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the multimedia report generating method according to any one of claims 1 to 12.

26. An electronic device, characterized in that, Comprising: One or more processors; A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the multimedia report generating method according to any one of claims 1 to 12.

Images