Bullet screen processing method and device and computing equipment

Abstract

The invention discloses a bullet screen processing method and apparatus, and a computing device. The method comprises the steps of obtaining a bullet screen sent by a user in a multimedia content playing process; performing content analysis on the bullet screen to obtain a content dimension label; performing interactive environment analysis according to the multimedia content to obtain an environment dimension label; inputting the content dimension label and the environment dimension label into a drawing model, so that the drawing model outputs a bullet screen display element containing a bullet screen; and synchronizing the bullet screen display element to a client playing the multimedia content, so that the client renders and displays the bullet screen display element. Through the above mode, the bullet screen display element can adapt to the change of the personalized content of the user and the interaction environment, and the bullet screen display effect which is personalized and accords with the real-time interaction environment is realized.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and specifically to a bullet comment processing method, device, computing equipment, computer storage medium and computer program product. Background Art

[0002] Danmu is a real-time comment display method that allows users to send short text comments during the playback of multimedia content. With the popularity of video platforms and live broadcast platforms, Danmu has become an important interactive method, enhancing the real-time connection between viewers and content creators.

[0003] After the user sends a barrage, it will be displayed in real time on the client interface. In the existing barrage display method, the display effects of barrages sent by different users are consistent or there are only slight differences. For example, some platforms provide configuration options for basic formats such as barrage fonts and colors. The overall visual effect of the barrage display is unified, lacking flexibility and interest. Summary of the invention

[0004] In view of the above problems, the present application is proposed to provide a bullet screen processing method, apparatus, computing device, computer storage medium and computer program product that overcome the above problems or at least partially solve the above problems.

[0005] According to one aspect of the present application, a bullet screen processing method is provided, which is used on a server, and includes:

[0006] Obtain the bullet comments sent by the user during the playback of multimedia content;

[0007] Conduct content analysis on the bullet comments to obtain content dimension labels;

[0008] Conduct interactive environment analysis based on multimedia content to obtain environment dimension labels;

[0009] Inputting the content dimension tag and the environment dimension tag into the drawing model, so that the drawing model outputs a bullet comment display element including the bullet comment;

[0010] The bullet screen display elements are synchronized to the client that plays the multimedia content, so that the client can render and display the bullet screen display elements.

[0011] Optionally, content analysis is performed on the bullet comment to obtain a content dimension label, which specifically includes at least one of the following processes:

[0012] Perform sentiment analysis on the comments and obtain sentiment labels;

[0013] Extract keywords from the bullet comments and obtain keyword tags;

[0014] Conduct topic analysis on the barrage and obtain content topic tags.

[0015] Optionally, the environmental dimension tag includes: environmental theme tag, atmosphere tag and / or style tag; performing interactive environment analysis based on multimedia content to obtain environmental dimension tags further includes:

[0016] The video content and / or audio content of the multimedia content is analyzed to determine an environmental dimension label.

[0017] Optionally, the method further comprises:

[0018] Configure tag weights for content dimension tags and environment dimension tags respectively;

[0019] The step of inputting the content dimension tag and the environment dimension tag into the drawing model further comprises:

[0020] The content dimension labels and their label weights and the environment dimension labels and their label weights are input into the drawing model.

[0021] Optionally, the barrage display elements also include: color elements, background frame elements, motion effect elements and / or accessory elements.

[0022] According to another aspect of the present application, a bullet screen processing method is provided, which is used for a client, including:

[0023] Receive comments sent by users during the playback of multimedia content and send them to the server;

[0024] The server performs content analysis on the bullet screen to obtain content dimension tags; and performs interactive environment analysis based on the multimedia content to obtain environment dimension tags; and inputs the content dimension tags and environment dimension tags into the drawing model so that the drawing model outputs bullet screen display elements containing the bullet screen;

[0025] Receive the bullet screen display elements synchronized by the server, and render and display the bullet screen display elements.

[0026] Optionally, rendering and displaying the bullet screen display element further includes:

[0027] Rendering bullet screen display elements to generate content to be displayed; displaying the content to be displayed in a bullet screen layer; wherein the bullet screen layer is located at an upper layer of a player of multimedia content.

[0028] According to another aspect of the present application, a bullet screen processing device is provided, including:

[0029] An acquisition module, adapted to acquire bullet comments sent by a user during the playback of multimedia content;

[0030] The tag analysis module is suitable for performing content analysis on the bullet comments to obtain content dimension tags; and performing interactive environment analysis based on the multimedia content to obtain environment dimension tags;

[0031] A drawing module, adapted to input the content dimension tag and the environment dimension tag into the drawing model, so that the drawing model outputs a bullet screen display element including the bullet screen;

[0032] The processing module synchronizes the bullet screen display elements to the client that plays the multimedia content, so that the client can render and display the bullet screen display elements.

[0033] Optionally, the tag analysis module is further adapted to:

[0034] Perform sentiment analysis on the comments and obtain sentiment labels;

[0035] Extract keywords from the bullet comments and obtain keyword tags;

[0036] Conduct topic analysis on the barrage and obtain content topic tags.

[0037] Optionally, the environmental dimension tag includes: environmental theme tag, atmosphere tag and / or style tag; the tag analysis module is further adapted to:

[0038] The video content and / or audio content of the multimedia content is analyzed to determine an environmental dimension label.

[0039] Optionally, the device further comprises:

[0040] A weight configuration module, adapted to configure tag weights for content dimension tags and environment dimension tags respectively;

[0041] The drawing module is further adapted to: input content dimension labels and their label weights and environment dimension labels and their label weights into the drawing model.

[0042] Optionally, the barrage display elements also include: color elements, background frame elements, motion effect elements and / or accessory elements.

[0043] According to another aspect of the present application, a bullet screen processing device is provided, including:

[0044] A first receiving module receives a bullet comment sent by a user during the playback of multimedia content;

[0045] A sending module is adapted to send the bullet screen to a server; wherein the server performs content analysis on the bullet screen to obtain a content dimension tag; and performs interactive environment analysis based on the multimedia content to obtain an environment dimension tag; and inputs the content dimension tag and the environment dimension tag into a drawing model, so that the drawing model outputs a bullet screen display element containing the bullet screen;

[0046] A second receiving module, adapted to receive bullet screen display elements synchronized by the server;

[0047] The display module is suitable for rendering and displaying bullet screen display elements.

[0048] Optionally, the display module is further adapted to: render bullet screen display elements to generate content to be displayed; display the content to be displayed in a bullet screen layer; wherein the bullet screen layer is located at an upper layer of a player of multimedia content.

[0049] According to another aspect of the present application, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0050] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned bullet screen processing method.

[0051] According to another aspect of the present application, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned barrage processing method.

[0052] According to another aspect of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned barrage processing method.

[0053] According to the barrage processing method, device, computing device, computer storage medium and computer program product provided in the embodiment of the present application, the barrage sent by the user during the playback of multimedia content is analyzed to obtain content dimension labels; the interactive environment analysis is performed based on the multimedia content to obtain environment dimension labels; the content dimension labels and environment dimension labels are input into the drawing model so that the drawing model can output barrage display elements containing barrages; the barrage display elements are synchronized to the client that plays the multimedia content so that the client can render and display the barrage display elements. Through the above methods, a personalized barrage display effect that fits the real-time interactive environment is achieved, so that the barrage display elements can adapt to changes in the user's personalized content and interactive environment. Different designs of barrage display elements can enhance the visual interest of the barrage, thereby enhancing the user's interactive experience; further, the use of AI drawing technology to automatically generate barrage display elements can reduce the time and cost of manual design.

[0054] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0056] Figure 1 A flowchart of a bullet screen processing method provided by an embodiment of the present application is shown;

[0057] Figure 2 A flowchart of a bullet screen processing method provided by another embodiment of the present application is shown;

[0058] Figure 3 A flowchart of a bullet screen processing method provided by another embodiment of the present application is shown;

[0059] Figure 4 A flowchart of a bullet screen processing method provided by another embodiment of the present application is shown;

[0060] Figure 5 A functional structure diagram of a bullet screen processing device provided in an embodiment of the present application is shown;

[0061] Figure 6 A functional structure diagram of a bullet screen processing device provided in an embodiment of the present application is shown;

[0062] Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0063] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0064] First, the terms involved in one or more embodiments of the present application are explained.

[0065] Danmu: It is a real-time comment display method that allows viewers to send short text comments while watching videos. These comments will fly across the screen at high speed like bullets, hence the name "danmu". This comment system was originally popular in animation, comics and game-related video content, but has now expanded to various types of platforms.

[0066] Drawing models: refers to large neural network models that use deep learning and artificial intelligence techniques to generate, edit or understand images. These models usually have a large number of parameters, can handle complex image tasks, and use large image datasets during training.

[0067] Barrage display elements: Its basic components are barrage text, and also include other visual effect elements, which not only affect the viewer's visual experience, but also enhance interactivity and entertainment. By carefully designing barrage display elements, it can provide users with an interactive environment that is both beautiful and practical.

[0068] NLP technology: Natural Language Processing, is a branch of artificial intelligence technology that aims to enable computers to understand, process, and generate natural language. NLP technology includes text analysis, sentiment recognition, topic modeling, keyword extraction and other functions.

[0069] TF-IDF algorithm: the term frequency-inverse document frequency algorithm, is a statistical method used to evaluate the importance of a word to a document or multiple documents in a corpus.

[0070] BERT model: is a pre-trained language model based on the Transformer architecture that can understand the contextual meaning of words in a text.

[0071] Topic model LDA: Latent Dirichlet Allocation, or latent Dirichlet allocation, is a generative statistical model for discovering abstract topics from a text collection. It assumes that each document is a mixture of multiple topics, and each topic is a probability distribution of a series of words. By analyzing the vocabulary usage patterns in a large number of documents, LDA can reveal the topic structure hidden behind these documents.

[0072] Figure 1 The flowchart of the bullet screen processing method provided by an embodiment of the present application is shown, and the method is applied to the server. Figure 1 As shown, the method comprises the following steps:

[0073] Step S110, obtaining the bullet comments sent by the user during the playback of the multimedia content.

[0074] Among them, the multimedia content can be live content or on-demand content. Specifically, when the multimedia content is live content, the client pulls the real-time transmitted live stream from the server, renders and displays the video screen based on the video stream in the live stream, and plays the sound based on the audio stream in the live stream. When the multimedia content is on-demand content, the client obtains the pre-recorded on-demand resources from the server, displays the video screen and plays the audio according to the on-demand resources. A barrage entrance is set in the client for users to send barrages, and the barrages sent by users are received through the barrage entrance. In the live broadcast scenario, in addition to the viewing end users (i.e., audience users) who can send barrages, the live broadcast end users (i.e., anchor users) can also send barrages. The client receives the barrages sent by the user and sends them to the server.

[0075] In the method of the embodiment of the present application, during the playback of multimedia content, real-time analysis is performed when the barrage sent by the user is obtained to generate real-time barrage display elements.

[0076] Step S120, perform content analysis on the barrage to obtain content dimension labels.

[0077] Content dimension labels are used to guide the drawing model to design barrage display elements. Barrage content is the personalized output of users. By analyzing the barrage content itself, such as the emotions expressed in the barrage content, the topics involved, etc., content dimension labels that match the user's personalized content are generated.

[0078] Step S130, performing interactive environment analysis based on the multimedia content to obtain an environment dimension label.

[0079] Environmental dimension labels are also used to guide the drawing model in designing barrage display elements. By analyzing the multimedia content being played, the real-time interactive environment when the user sends the barrage is determined, and environmental dimension labels that match the real-time interactive environment are generated.

[0080] Step S140, inputting the content dimension tag and the environment dimension tag into the drawing model, so that the drawing model outputs a bullet comment display element including the bullet comment.

[0081] The content dimension labels and environment dimension labels are input into the drawing model. The drawing model (i.e., the AI ​​drawing model) understands the content dimension labels and environment dimension labels and designs and generates barrage display elements. The characteristics of the barrage itself and the environmental characteristics when the user sends the barrage are reflected in the barrage display elements, thereby realizing a flexible, personalized and real-time interactive environment-compatible barrage display element design.

[0082] Step S150, synchronizing the bullet screen display elements to the client that plays the multimedia content, so that the client can render and display the bullet screen display elements.

[0083] The barrage display elements are synchronized to all clients that are playing multimedia content. After the client receives the barrage display elements, it renders and displays the barrage display elements. In addition to displaying basic barrage text, other display effect elements can also be presented. The overall display effect is personalized and fits the real-time interactive environment.

[0084] In summary, according to the barrage processing method provided in this embodiment, the barrage sent by the user during the playback of multimedia content is obtained; by analyzing the barrage content itself, a content dimension label matching the user's personalized content is generated; by analyzing the multimedia content being played, the interactive environment when the user sends the barrage is determined, and an environmental dimension label matching the real-time interactive environment is generated; using a drawing model, a barrage display element is generated according to the content dimension label and the environmental dimension label, and the characteristics of the barrage itself and the environmental characteristics when the user sends the barrage are reflected in the barrage display element, thereby realizing a personalized design of the barrage display element that matches the real-time interactive environment; finally, the barrage display element is displayed in the client that plays the multimedia content. In the above manner, a personalized barrage display effect that matches the real-time interactive environment is realized, so that the barrage display element can adapt to changes in the user's personalized content and interactive environment, and different barrage display element designs can enhance the visual interest of the barrage, thereby enhancing the user's interactive experience; further, the use of AI drawing technology to automatically generate barrage display elements can reduce the time and cost of manual design.

[0085] Figure 2 FIG. 2 shows a flow chart of a bullet screen processing method provided by another embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0086] Step S210, obtaining the bullet comments sent by the user during the playback of the multimedia content.

[0087] The client obtains multimedia content for playback, and the user sends barrages through the barrage entrance in the client. The barrages sent by the user will be sent to the server, and the server determines the real-time content dimension tags and environment dimension tags.

[0088] Step S220, perform content analysis on the barrage to obtain content dimension labels.

[0089] Content dimension tags include: emotional tags, keyword tags and / or content theme tags. Among them, emotional tags are determined by analyzing the emotions expressed by the barrage, providing emotional guidance for the design of barrage display elements; keyword tags and content theme tags are both barrage content tags, which are determined by analyzing the keywords and themes involved in the barrage content, providing semantic guidance for the design of barrage display elements.

[0090] On the one hand, the implementation method of analyzing and obtaining emotional tags is: perform emotional analysis on the barrage to obtain emotional tags. Specifically, the emotional analysis of the barrage content is performed through NLP technology to automatically identify and classify the emotional tendencies expressed by the barrage content, such as positive, negative, neutral, etc.; further analyze the specific emotional states, such as happy, angry, surprised, etc. Correspondingly, the emotional tags are determined according to the emotional tendencies and / or emotional states, and the emotional tags can be used to generate barrage display elements that match the user's real-time emotions. The design style of the barrage display elements is adjusted according to the user's emotions, so that the barrage display elements can be consistent with the user's emotions.

[0091] For example, the content of the barrage is: "It's so exciting, so cool! I like it so much!", the analysis determines that the emotional label that matches the emotional tendency is "pleasure", and the emotional labels that match the emotional state are "excited" and "surprised"; for another example, the content of the barrage is: "Why is it like this, this is too bad!", the analysis determines that the emotional label that matches the emotional tendency is "anger", and the emotional labels that match the emotional state are "disappointed" and "angry".

[0092] On the other hand, the implementation method of analyzing and obtaining keyword tags is: extracting keywords from the bullet screen to obtain keyword tags. Specifically, the keyword extraction of the bullet screen is performed by means of TF-IDF (term frequency-inverse document frequency) algorithm or BERT pre-training model to identify the most representative words in the bullet screen.

[0093] For example, the content of the barrage is: "This game is so exciting!", and the recognized keywords include "game" and "exciting"; for another example, the content of the barrage is: "This character is so cute and adorable!", and the recognized keywords include: "character", "cute", and "adorable".

[0094] By extracting keywords, we can understand the keywords discussed in the bullet screen, and further guide the design of the bullet screen display elements. For example, if the keyword is "match" or "competition", a concise and fierce design style can be generated; if the keyword is "cute" or "cute", the design style may be warmer and softer.

[0095] On the other hand, the implementation method of analyzing and obtaining content theme tags is: performing theme analysis on the bullet comments to obtain content theme tags. Specifically, through machine learning (such as topic model LDA) or deep learning (such as BERT text classification) technology, the theme category of the bullet comment content is automatically identified to obtain content theme tags, and the content theme tags are used to guide the design of the bullet comment display elements.

[0096] For example, if the content theme tag is "hot-blooded battle", the barrage display elements can be designed to have intense and contrasting tones; if the content theme tag is "funny", the barrage display elements can be designed to have light-hearted and lively tones.

[0097] Content dimension tags may also include preference tags, so that barrage display elements can be designed in combination with user preferences so that the display effect of the barrage display elements is consistent with user preferences.

[0098] Step S230, performing interactive environment analysis based on the multimedia content to obtain an environment dimension label.

[0099] The interactive environment presented by multimedia content, including program types, live topics, current events, etc., has an important impact on the user's emotions and behaviors. The method of the embodiment of the present application combines the user's interactive behavior with the interactive environment presented by the current multimedia content, further improving the design flexibility and diversity of the bullet screen display elements.

[0100] Specifically, the video content and / or audio content of the multimedia content is analyzed to determine the environmental dimension label. For video content, computer vision technology can be used to analyze the video frame, and text analysis technology can be used to analyze the text in the video frame (such as subtitles, text contained in the video image frame, etc.); for audio content, it can be converted into text content and then analyzed using text analysis technology.

[0101] Among them, the environmental dimension labels specifically include: environmental theme labels, atmosphere labels and / or style labels. That is, the analysis of the interactive environment mainly includes: analysis of the style, theme and / or atmosphere of the program presented by the multimedia content. For example, if the multimedia content is an animation video, environmental theme labels such as "two-dimensional style", "youth campus", "fantasy adventure", and "healing animation" can be determined. If the multimedia content is a live broadcast of a sports event, atmosphere labels such as "exciting", "competitive competition", and "real-time scores" can be determined; if the multimedia content is a live news broadcast, style labels such as "formal" and "serious" can be determined.

[0102] Based on the environmental dimension label, the drawing model can add visual elements that match the real-time interactive environment to the barrage display elements. For example, in game live broadcasts, the barrage display elements can add more game-related elements; in news live broadcasts or talk shows, the barrage display elements can be dynamically adjusted according to the seriousness and emotional atmosphere of the topic.

[0103] Step S240, respectively configure label weights for content dimension labels and environment dimension labels, and input the content dimension labels and their label weights and the environment dimension labels and their label weights into the drawing model, so that the drawing model can output a barrage display element containing barrage.

[0104] Each label corresponds to one or more components of the bullet screen display element, and the drawing model designs one or more different components of the bullet screen display element according to the label. For example, the emotional label is used to guide the drawing model to design the emotional tone and pattern of the bullet screen display element, using bright colors for pleasant emotions and high-contrast colors for intense emotions; the environmental dimension label is used to guide the drawing model to design the light, font, animation, accessories, etc. of the bullet screen display element. For example, when the environmental theme label of the multimedia content is an animation label, add cartoon character accessories or design the bullet screen display element to be high brightness.

[0105] Tag weights are used to determine the priority order of tags that the drawing model refers to when designing bullet screen display elements. The larger the tag weight, the higher the priority the drawing model will give to generating the corresponding components of the tag.

[0106] In an optional manner, information is configured according to the priority order of the dimensions, and tag weights are configured for content dimension tags and environment dimension tags. For example, if the content dimension tag is set to be prioritized, a higher tag weight is configured for the content dimension tag, and if the environment dimension tag is set to be prioritized, a higher tag weight is set for the environment dimension tag.

[0107] Furthermore, if the content dimension tag contains multiple tags, the sum of the tag weights of the multiple tags is equal to the tag weight of the content dimension tag, and the tag weights of the multiple tags contained in the content dimension tag can be allocated according to the specified allocation strategy; accordingly, if the environment dimension tag contains multiple tags, the sum of the tag weights of the multiple tags is equal to the tag weight of the environment dimension tag, and the tag weights of the multiple tags contained in the environment dimension tag can be allocated according to the specified allocation strategy.

[0108] In another optional manner, according to the configuration information of the priority order of the tags, a tag weight is configured for each tag. In this manner, the configuration of the priority is refined to each tag, which can achieve a more refined weight configuration.

[0109] For example, if the emotional tag priority is set, the emotional tag has a larger label weight. The emotional tag is used to determine the color, pattern and other design elements of the barrage display elements to ensure that it is in line with the user's current emotional state; if the environmental theme tag priority is set, the environmental theme tag has a larger weight. If the user is watching a video on a specific theme (such as anime, sports), the drawing model will prioritize generating the basic design of the barrage display element based on that theme; if the barrage content tag priority is set, the barrage content tag has a larger label weight. If the keyword of the barrage content is "match" or "competition", the drawing model can prioritize generating a concise and intense design style.

[0110] It should be noted that the dimension priority order configuration information and the label priority order configuration information can be the default configuration information or can be flexibly configured by the configuration party according to actual needs, and this application does not limit this.

[0111] The drawing model uses large-scale data training to generate prediction and creation capabilities for images, and can automatically design styles, colors, dynamic effects, etc. In the embodiment of the present application, the drawing model is used to generate personalized bullet screen display elements that match the real-time interactive environment based on the input tags.

[0112] The acquired content dimension tags and environment dimension tags are integrated to form a comprehensive tag set, and then a weight is set for each tag in the tag set. Based on the tag set carrying the tag weights, the drawing model outputs barrage display elements that conform to the real-time situation.

[0113] Among them, in addition to the barrage content itself, the barrage display elements also include: color elements, background frame elements, motion effect elements and / or accessory elements. Among them, the color element determines the overall tone of the barrage display elements; the background frame element is the background of the barrage text, which can also include: background image, user avatar, timestamp information, etc.; the motion effect element is an element with dynamic display effects, which can enhance interactivity, make the barrage display elements more vivid, and attract user attention. The motion effect elements include flashing effect elements, gradient effect elements, ripple effect elements, etc., which are used to indicate the dynamic display effect of the barrage display elements; the accessory elements can be pendants, bubbles, stickers, etc.

[0114] Step S250, synchronizing the bullet screen display elements to the client that plays the multimedia content, so that the client can render and display the bullet screen display elements.

[0115] The bullet screen display elements generated by the drawing model will be rendered in real time and displayed in real time in all clients that play the multimedia content. Specifically, the client renders the bullet screen display elements to generate the content to be displayed; the content to be displayed is displayed in the bullet screen layer; wherein the bullet screen layer is located in the upper layer of the multimedia content player. During the playback of multimedia content, as the user's emotions fluctuate and the content of the audio or video being played changes, the bullet screen display elements are dynamically designed. In addition to the bullet screen content itself, the display effect of the bullet screen display elements will change dynamically, realizing a flexible and dynamic bullet screen display method.

[0116] To sum up, according to the barrage processing method provided by the present embodiment, by performing sentiment analysis and content analysis on the barrage content, sentiment tags, keyword tags and / or content theme tags can be obtained; by performing interactive environment analysis on the multimedia content being played, environmental theme tags, atmosphere tags and / or style tags can be obtained; further, it is possible to generate barrage display elements by combining multi-dimensional tags, and it is possible to integrate sentiment analysis, barrage content analysis and interactive environment analysis to generate barrage display elements for each user that are consistent with their personalized content and real-time interactive environment, so that the visual effect of the barrage display elements can not only conform to the user's emotions, but also be consistent with the theme atmosphere style of the currently playing video or audio, thereby achieving differentiation, personalization and consistency. The barrage display effect is suitable for the real-time interactive environment. It can adjust the display effect of the barrage display elements in real time according to the changes of the user's personalized content and multimedia content, increase the diversity and randomness of the barrage visual effects, and enhance the fun of the barrage display. It helps to enhance the interaction between users and the interaction between users and the creators of multimedia content, which helps to improve the sociality and user stickiness of the platform; further, by utilizing AI drawing technology and adding random elements to automatically generate barrage display elements, it is ensured that the barrage display elements of each user are unique and creative, which can save manual design costs and effectively enhance the user's interactive experience; further, by configuring weights for different tags, the drawing model can determine the priority order of each tag according to the tag weight.

[0117] Figure 3 A flowchart of a bullet screen processing method provided by another embodiment of the present application is shown. Figure 3 As shown, the method specifically includes:

[0118] The bullet comment 301 is processed in step 303 and step 304. Step 303 is specifically sentiment analysis, thereby obtaining a sentiment tag list 306; step 304 is specifically a bullet comment content analysis, thereby obtaining a bullet comment content tag list 307. The multimedia content 302 is processed in step 305, which is specifically an interactive environment analysis, thereby obtaining an environment dimension tag list 308.

[0119] The emotional tag list 306, the bullet comment content tag list 307 and the environmental dimension tag list 308 are summarized to obtain a tag set 309, and a tag weight is assigned to each tag in the tag set. Finally, the tag set is input into the AI ​​drawing model 310, and the AI ​​drawing model 310 generates random bullet comment display elements.

[0120] Figure 4 A flowchart of a bullet screen processing method provided by another embodiment of the present application is shown, and the method is applied to a client, such as Figure 4 As shown, the method includes:

[0121] Step S410, receiving the bullet comments sent by the user during the playback of the multimedia content, and sending the bullet comments to the server.

[0122] Specifically, when the multimedia content is live content, the client pulls the live stream transmitted in real time from the server, renders and displays the video screen based on the video stream in the live stream, and plays the sound based on the audio stream in the live stream. When the multimedia content is on-demand content, the client obtains the pre-recorded on-demand resources from the server, displays the video screen and plays the audio based on the on-demand resources. A barrage entrance is set in the client for users to send barrages, and the barrages sent by users are received through the barrage entrance. In the live broadcast scenario, in addition to the viewing end users (i.e., audience users) who can send barrages, the live broadcast end users (i.e., anchor users) can also send barrages.

[0123] The client obtains multimedia content for playback, and the user sends barrages through the barrage entrance in the client. The barrages sent by the user will be sent to the server, and the server determines the real-time content dimension tags and environment dimension tags.

[0124] Specifically, the server performs content analysis on the barrage to obtain content dimension tags; and performs interactive environment analysis based on the multimedia content to obtain environment dimension tags; and inputs the content dimension tags and environment dimension tags into the drawing model so that the drawing model can output barrage display elements containing the barrage.

[0125] Content dimension tags are used to guide the drawing model to design bullet screen display elements. Bullet screen content is the personalized output of users. By analyzing the bullet screen content itself, such as analyzing the emotions expressed by the bullet screen content, the topics involved, etc., content dimension tags that match the user's personalized content are generated. Environment dimension tags are also used to guide the drawing model to design bullet screen display elements. By analyzing the multimedia content being played, the real-time interactive environment when the user sends the bullet screen is determined, and an environment dimension tag that matches the real-time interactive environment is generated.

[0126] The content dimension labels and environment dimension labels are input into the drawing model. The drawing model (i.e., the AI ​​drawing model) understands the content dimension labels and environment dimension labels and designs and generates barrage display elements. The characteristics of the barrage itself and the environmental characteristics when the user sends the barrage are reflected in the barrage display elements, thereby realizing a flexible, personalized and real-time interactive environment-compatible barrage display element design.

[0127] In an optional manner, content dimension tags include: emotional tags, keyword tags and / or content theme tags. Among them, emotional tags are determined by analyzing the emotions expressed by the barrage, providing emotional guidance for the design of barrage display elements; keyword tags and content theme tags are both barrage content tags, which are determined by analyzing the keywords and themes involved in the barrage content, providing semantic guidance for the design of barrage display elements.

[0128] Specifically, the sentiment tag is obtained by performing sentiment analysis on the bullet screen; the keyword tag is obtained by extracting keywords from the bullet screen; and the content theme tag is obtained by performing theme analysis on the bullet screen. The specific implementation method can be found in the description of the above embodiment, which will not be repeated here.

[0129] In an optional manner, the environmental dimension tag specifically includes: an environmental theme tag, an atmosphere tag and / or a style tag, and the environmental dimension tag is obtained by analyzing the video content and / or audio content of the multimedia content.

[0130] In another optional method, after analyzing and obtaining the content dimension tag and the environment dimension tag, the server configures label weights for the content dimension tag and the environment dimension tag respectively, and inputs the content dimension tag and its label weight and the environment dimension tag and its label weight into the drawing model so that the drawing model can output the barrage display elements.

[0131] Step S420, receiving the bullet screen display elements synchronized by the server, and rendering and displaying the bullet screen display elements.

[0132] It should be noted that, for any client, the barrage display elements it receives include the barrage display elements triggered by all clients playing the same multimedia content, that is, it not only receives the barrage display elements triggered by the local client, but also receives the barrage display elements triggered by other clients playing the same multimedia content.

[0133] Specifically, the bullet screen display element is rendered to generate content to be displayed; the content to be displayed is displayed in a bullet screen layer; wherein the bullet screen layer is located at an upper layer of a player of the multimedia content.

[0134] In an optional manner, the bullet screen display elements also include: color elements, background frame elements, motion effect elements and / or accessory elements.

[0135] In summary, according to the barrage processing method provided in this embodiment, the barrage sent by the user during the playback of multimedia content is received, and the barrage is sent to the server, so that the server can analyze the barrage content itself to generate a content dimension label that matches the user's personalized content, and, by analyzing the multimedia content being played, determine the interactive environment when the user sends the barrage, and generate an environmental dimension label that matches the real-time interactive environment, and, using a drawing model, generate barrage display elements according to the content dimension label and the environmental dimension label, and reflect the characteristics of the barrage itself and the environmental characteristics when the user sends the barrage in the barrage display elements, thereby realizing a personalized design of barrage display elements that matches the real-time interactive environment; and finally displaying the barrage display elements in the client that plays the multimedia content. In the above manner, a personalized barrage display effect that matches the real-time interactive environment is realized, so that the barrage display elements can adapt to changes in the user's personalized content and interactive environment, and different barrage display element designs can enhance the visual interest of the barrage, thereby enhancing the user's interactive experience; further, the use of AI drawing technology to automatically generate barrage display elements can reduce the time and cost of manual design.

[0136] Furthermore, by performing sentiment analysis and content analysis on the barrage content, sentiment tags, keyword tags and / or content theme tags can be obtained; by performing interactive environment analysis on the multimedia content being played, environmental theme tags, atmosphere tags and / or style tags can be obtained; further, it is possible to generate barrage display elements by combining multi-dimensional tags, and it is possible to integrate sentiment analysis, barrage content analysis and interactive environment analysis to generate barrage display elements for each user that are consistent with their personalized content and real-time interactive environment, so that the visual effect of the barrage display elements can not only conform to the user's emotions, but also be consistent with the theme atmosphere style of the currently playing video or audio, thereby achieving differentiation, personalization and compliance with the real-time interactive environment. The barrage display effect can adjust the display effect of the barrage display elements in real time as the user's personalized content and multimedia content change, increase the diversity and randomness of the barrage visual effects, enhance the fun of the barrage display, and help enhance the interaction between users and the interaction between users and the creators of multimedia content, which helps to improve the sociality and user stickiness of the platform; further, by utilizing AI drawing technology and adding random elements to automatically generate barrage display elements, it is ensured that each user's barrage display elements are unique and creative, which can save manual design costs and effectively enhance the user's interactive experience; further, by configuring weights for different tags, the drawing model can determine the priority order of each tag based on the tag weight.

[0137] Figure 5 The functional structure diagram of the bullet screen processing device provided in the embodiment of the present application is shown, and the device is applied to the server, such as Figure 5 As shown, the device comprises:

[0138] The acquisition module 501 is adapted to acquire the bullet comments sent by the user during the playback of the multimedia content;

[0139] The tag analysis module 502 is adapted to perform content analysis on the bullet comments to obtain content dimension tags; and to perform interactive environment analysis based on the multimedia content to obtain environment dimension tags;

[0140] A drawing module 503, adapted to input the content dimension tag and the environment dimension tag into the drawing model, so that the drawing model outputs a bullet-chat display element including the bullet-chat;

[0141] The processing module 504 synchronizes the bullet screen display elements to the client that plays the multimedia content, so that the client can render and display the bullet screen display elements.

[0142] In an optional manner, the tag analysis module 502 is further adapted to:

[0143] Perform sentiment analysis on the comments and obtain sentiment labels;

[0144] Extract keywords from the bullet comments and obtain keyword tags;

[0145] Conduct topic analysis on the barrage and obtain content topic tags.

[0146] In an optional manner, the environmental dimension tag includes: environmental theme tag, atmosphere tag and / or style tag; the tag analysis module 502 is further adapted to:

[0147] The video content and / or audio content of the multimedia content is analyzed to determine an environmental dimension label.

[0148] In an optional manner, the device further comprises:

[0149] A weight configuration module, adapted to configure tag weights for content dimension tags and environment dimension tags respectively;

[0150] The drawing module 503 is further adapted to: input content dimension labels and their label weights and environment dimension labels and their label weights into the drawing model.

[0151] In an optional manner, the bullet screen display elements also include: color elements, background frame elements, motion effect elements and / or accessory elements.

[0152] In an optional manner, the client is further adapted to: render bullet screen display elements to generate content to be displayed; and display the content to be displayed in a bullet screen layer; wherein the bullet screen layer is located at an upper layer of a player of multimedia content.

[0153] In summary, according to the barrage processing device provided in this embodiment, the barrage sent by the user during the playback of multimedia content is obtained; by analyzing the barrage content itself, a content dimension label matching the user's personalized content is generated; by analyzing the multimedia content being played, the interactive environment when the user sends the barrage is determined, and an environmental dimension label matching the real-time interactive environment is generated; using a drawing model, a barrage display element is generated according to the content dimension label and the environmental dimension label, and the characteristics of the barrage itself and the environmental characteristics when the user sends the barrage are reflected in the barrage display element, thereby realizing a personalized design of the barrage display element that matches the real-time interactive environment; and finally, the barrage display element is displayed in the client that plays the multimedia content. In the above manner, a personalized barrage display effect that matches the real-time interactive environment is realized, so that the barrage display element can adapt to changes in the user's personalized content and interactive environment, and different barrage display element designs can enhance the visual interest of the barrage, thereby enhancing the user's interactive experience; further, the use of AI drawing technology to automatically generate barrage display elements can reduce the time and cost of manual design.

[0154] Figure 6 The functional structure diagram of the bullet screen processing device provided in the embodiment of the present application is shown, and the device is applied to the client, such as Figure 6 As shown, the device comprises:

[0155] The first receiving module 601 receives the bullet comments sent by the user during the playback of multimedia content;

[0156] The sending module 602 is adapted to send the bullet screen to the server; wherein the server performs content analysis on the bullet screen to obtain a content dimension tag; and performs interactive environment analysis according to the multimedia content to obtain an environment dimension tag; and inputs the content dimension tag and the environment dimension tag into the drawing model, so that the drawing model outputs a bullet screen display element containing the bullet screen;

[0157] The second receiving module 603 is adapted to receive the bullet screen display elements synchronized by the server;

[0158] The display module 604 is suitable for rendering and displaying the bullet screen display elements.

[0159] In an optional manner, the content dimension tags include sentiment tags, keyword tags and / or content theme tags; the sentiment tags are obtained by performing sentiment analysis on the barrage; the keyword tags are obtained by performing keyword extraction on the barrage; and the content theme tags are obtained by performing theme analysis on the barrage.

[0160] In an optional manner, the environmental dimension tag includes: an environmental theme tag, an atmosphere tag and / or a style tag; the environmental dimension tag is obtained by analyzing the video content and / or audio content of the multimedia content.

[0161] In an optional manner, the server is further configured to perform the following steps:

[0162] Configure tag weights for content dimension tags and environment dimension tags respectively;

[0163] The content dimension labels and their label weights and the environment dimension labels and their label weights are input into the drawing model.

[0164] In an optional manner, the bullet screen display elements also include: color elements, background frame elements, motion effect elements and / or accessory elements.

[0165] In an optional manner, the display module 604 is further adapted to: render bullet screen display elements to generate content to be displayed; and display the content to be displayed in a bullet screen layer; wherein the bullet screen layer is located at an upper layer of a player of multimedia content.

[0166] In summary, according to the barrage processing device provided in this embodiment, the barrage sent by the user during the playback of multimedia content is received, and the barrage is sent to the server; the server generates a content dimension label that matches the user's personalized content by analyzing the barrage content itself; and by analyzing the multimedia content being played, the interactive environment when the user sends the barrage is determined, and an environmental dimension label that matches the real-time interactive environment is generated; the server also uses a drawing model to generate barrage display elements according to the content dimension label and the environmental dimension label, and reflects the characteristics of the barrage itself and the environmental characteristics when the user sends the barrage into the barrage display elements, thereby realizing a personalized design of barrage display elements that matches the real-time interactive environment; and finally, the barrage display elements are displayed in the client that plays the multimedia content. Through the above method, a personalized barrage display effect that matches the real-time interactive environment is realized, so that the barrage display elements can adapt to changes in the user's personalized content and interactive environment, and different barrage display element designs can enhance the visual interest of the barrage, thereby enhancing the user's interactive experience; further, the use of AI drawing technology to automatically generate barrage display elements can reduce the time and cost of manual design.

[0167] An embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction or computer program, which can enable a processor to perform operations corresponding to the barrage processing method in any of the above method embodiments.

[0168] An embodiment of the present application provides a computer program product, which includes at least one executable instruction or computer program, which can enable a processor to perform operations corresponding to the barrage processing method in any of the above-mentioned method embodiments.

[0169] Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of the present application is shown, and the specific embodiment of the present application does not limit the specific implementation of the computing device.

[0170] like Figure 7 As shown, the computing device may include: a processor (processor) 702 , a communications interface (Communications Interface) 704 , a memory (memory) 706 , and a communication bus 708 .

[0171] The processor 702, the communication interface 704, and the memory 706 communicate with each other via the communication bus 708. The communication interface 704 is used to communicate with other devices such as a client or other server network elements. The processor 702 is used to execute the program 710, which can specifically execute the relevant steps in the above-mentioned embodiment of the bullet screen processing method for a computing device.

[0172] Specifically, the program 710 may include program codes, which include computer operation instructions.

[0173] The processor 702 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0174] The memory 706 is used to store the program 710. The memory 706 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0175] Program 710 can be specifically used to enable processor 702 to execute the barrage processing method in any of the above-mentioned method embodiments. The specific implementation of each step in program 710 can refer to the corresponding description in the corresponding steps and units in the barrage processing embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the aforementioned method embodiment, which will not be repeated here.

[0176] The algorithm or display provided here are not inherently related to any specific computer, virtual system or other equipment. Various general systems can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application described here, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application.

[0177] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0178] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims, the inventive aspects lie in less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present application.

[0179] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0180] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0181] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to the embodiments of the present application. The application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0182] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be constructed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A bullet comment processing method, used on a server, comprising: Obtain the bullet comments sent by the user during the playback of multimedia content; Performing content analysis on the barrage to obtain content dimension labels; Performing interactive environment analysis based on the multimedia content to obtain an environment dimension label; Inputting the content dimension tag and the environment dimension tag into a drawing model, so that the drawing model outputs a bullet-screen display element including the bullet-screen; The bullet screen display element is synchronized to a client that plays the multimedia content, so that the client can render and display the bullet screen display element.

2. The method according to claim 1, wherein: The performing content analysis on the bullet comments to obtain content dimension labels specifically includes at least one of the following processes: Performing sentiment analysis on the barrage to obtain sentiment labels; Extract keywords from the bullet comments to obtain keyword tags; Perform a topic analysis on the barrage to obtain content topic tags.

3. The method according to claim 1, wherein: The environmental dimension tags include: environmental theme tags, atmosphere tags and / or style tags; the interactive environment analysis based on the multimedia content to obtain environmental dimension tags further includes: The video content and / or audio content of the multimedia content is analyzed to determine an environmental dimension label.

4. The method according to any one of claims 1 to 3, wherein: The method further comprises: respectively configuring tag weights for the content dimension tag and the environment dimension tag; The inputting the content dimension tag and the environment dimension tag into the drawing model further comprises: The content dimension labels and their label weights and the environment dimension labels and their label weights are input into the drawing model.

5. The method according to any one of claims 1 to 4, wherein: The barrage display elements also include: color elements, background frame elements, motion effect elements and / or accessory elements.

6. A bullet comment processing method, used in a client, comprising: Receive comments sent by users during the playback of multimedia content and send them to the server; The server performs content analysis on the bullet screen to obtain a content dimension tag; and performs interactive environment analysis based on the multimedia content to obtain an environment dimension tag; and inputs the content dimension tag and the environment dimension tag into a drawing model, so that the drawing model outputs a bullet screen display element containing the bullet screen; Receive the bullet screen display element synchronized by the server, and render and display the bullet screen display element.

7. The method according to claim 6, wherein: The rendering and displaying of the bullet screen display element further comprises: Rendering the bullet screen display element to generate content to be displayed; The content to be displayed is displayed in a bullet screen layer; wherein the bullet screen layer is located on an upper layer of a player of the multimedia content.

8. A bullet screen processing device, comprising: An acquisition module, adapted to acquire bullet comments sent by a user during the playback of multimedia content; A label analysis module, adapted to perform content analysis on the barrage to obtain content dimension labels; and, performing interactive environment analysis based on the multimedia content to obtain an environment dimension label; A drawing module, adapted to input the content dimension tag and the environment dimension tag into a drawing model, so that the drawing model outputs a bullet-screen display element including the bullet-screen; The processing module is adapted to synchronize the bullet-screen display element to a client that plays the multimedia content, so that the client can render and display the bullet-screen display element.

9. A bullet screen processing device, comprising: A first receiving module receives a bullet comment sent by a user during the playback of multimedia content; A sending module, adapted to send the bullet screen to a server; wherein the server performs content analysis on the bullet screen to obtain a content dimension tag; and performs interactive environment analysis based on the multimedia content to obtain an environment dimension tag; and inputs the content dimension tag and the environment dimension tag into a drawing model, so that the drawing model outputs a bullet screen display element containing the bullet screen; A second receiving module, adapted to receive the bullet screen display element synchronized by the server; The display module is suitable for rendering and displaying the bullet screen display elements.

10. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the barrage processing method as described in any one of claims 1-5 or any one of claims 6-7.

11. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the barrage processing method as described in any one of claims 1-5 or any one of claims 6-7.

12. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the barrage processing method as described in any one of claims 1-5 or any one of claims 6-7.

Images