Bullet screen display method and device, computer equipment, storage medium and program product

Abstract

The embodiment of the invention provides a bullet screen display method and device, computer equipment, a storage medium and a program product, and relates to the technical field of video processing, and the method comprises the steps: detecting whether there is audio data in the video content played at the current moment in a process of playing a target video; under the condition that audio data exist in the video content played at the current moment, obtaining a first emotion label of the audio data at the current moment, and obtaining a second emotion label of a target bullet screen needing to be displayed at the current moment; performing matching processing on the first emotion tag and the second emotion tag; under the condition that the first emotion tag is successfully matched with the second emotion tag, determining a bullet screen special effect of a successfully matched target bullet screen according to the first emotion tag; and displaying the successfully matched target bullet screen according to the bullet screen special effect. According to the technical scheme provided by the embodiment of the invention, the bullet screen display mode can be enriched, and the interaction experience is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video processing technology, and in particular to a bullet screen display method, device, computer equipment, storage medium and program product. Background Art

[0002] With the development of the Internet and multimedia technology, video has become a way of life and entertainment for the public. When watching videos, users like to send their opinions in the form of barrage to express their feelings. Barrage can play a role in heating up the atmosphere and increasing popularity, while giving the audience a sense of real-time interaction.

[0003] However, the current presentation form of barrage is relatively simple, generally only with a simple scrolling effect. Such barrage presentation form provides a poor user experience and is boring. Therefore, how to enrich the presentation form of barrage and improve the user experience is a problem that urgently needs to be solved.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a bullet screen display method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a bullet screen display method, the method comprising: During the process of playing the target video, detecting whether there is audio data in the video content currently being played; In the case where there is audio data in the video content being played at the current moment, obtaining a first emotion label of the audio data at the current moment, and obtaining a second emotion label of a target barrage that needs to be displayed at the current moment; Performing matching processing on the first emotion label and the second emotion label; In the case where the first emotion tag successfully matches the second emotion tag, determining a bullet screen special effect of a successfully matched target bullet screen according to the first emotion tag; The successfully matched target barrage is displayed according to the barrage special effect.

[0007] Optionally, the method further comprises: Extracting an audio stream from the target video, wherein the audio stream includes a plurality of audio data; Dividing the audio stream into a plurality of audio segments; Inputting the multiple audio clips into a pre-trained emotion recognition model respectively to obtain a third emotion label corresponding to each of the audio clips; A first association relationship is established between each of the audio clips, each of the third emotion tags, and a time interval corresponding to each of the audio clips, and the first association relationship is saved.

[0008] Optionally, dividing the audio stream into a plurality of audio segments comprises: The audio stream is divided into a plurality of audio segments with partial overlap.

[0009] Optionally, the first emotion tag includes a fifth emotion tag and a sixth emotion tag, and when the first emotion tag successfully matches the second emotion tag, determining the barrage special effects of the successfully matched target barrage according to the first emotion tag includes: When the first emotion tag and the second emotion tag are successfully matched, the barrage special effects of the successfully matched target barrage are determined according to the fifth emotion tag and the sixth emotion tag respectively.

[0010] Optionally, the method further comprises: Acquire a bullet comment stream associated with the target video, wherein the bullet comment stream includes multiple bullet comments; Inputting each barrage in the barrage stream into a pre-trained natural language processing model to obtain a fourth emotion label corresponding to each barrage; Establish a second association relationship between each of the bullet comments and the corresponding fourth emotional tag, and save the second association relationship Optionally, displaying the successfully matched target bullet chat according to the bullet chat special effect includes: Obtain the number of successfully matched target bullet comments; When the number exceeds a preset number, selecting the preset number of target bullet screens from the successfully matched target bullet screens; The selected target barrage is displayed according to the barrage special effect.

[0011] Optionally, the method further comprises: The target bullet comments that were not successfully matched are displayed using a preset display method.

[0012] Another aspect of an embodiment of the present application provides a bullet screen display device, the device comprising: A detection module, used to detect whether there is audio data in the video content currently being played during the process of playing the target video; An acquisition module, used for acquiring, when there is audio data in the video content being played at the current moment, a first emotion label of the audio data at the current moment, and acquiring a second emotion label of a target barrage that needs to be displayed at the current moment; A matching module, used for matching the first emotion label with the second emotion label; A determination module, configured to determine, when the first emotion tag successfully matches the second emotion tag, a bullet screen special effect of a successfully matched target bullet screen according to the first emotion tag; A display module is used to display the successfully matched target barrage according to the barrage special effects.

[0013] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0014] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0015] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0016] The above technical solution adopted in the embodiment of the present application may have the following advantages: During the process of playing the target video, it will be detected in real time whether there is audio data (music data) in the video content being played at the current moment. Then, when it is detected that there is audio data in the video being played at the current moment, the first emotional label of the audio data at the current moment (for example, happy, sad, etc.) will be obtained, and the second emotional label of the target barrage that needs to be displayed at the current moment (for example, happy, sad, etc.) will be obtained. After that, the second emotional label of each target barrage to be displayed will be matched with the first emotional label one by one. In the case of a successful match, the barrage special effect corresponding to the second emotional label will be determined. Finally, the successfully matched target barrage will be displayed according to the barrage special effect. It can be seen that in the embodiment of the present application, when the barrage is displayed, the barrage with the same emotion will be displayed with special effects according to the emotion of the audio data in the target video, thereby enriching the display method of the barrage. In addition, through the above-mentioned display method, the barrage display is coordinated with the emotional state of the audio and video content, enhancing the interactive experience of the audience with the video content. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 The following schematically shows an operating environment diagram of the bullet screen display method according to the first embodiment of the present application; Figure 2 The flowchart of the bullet screen display method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 Flow chart of sub-steps of step S208; Figure 4 The newly added flow chart of the bullet screen display method according to the first embodiment of the present application is schematically shown; Figure 5 The newly added flow chart of the bullet screen display method according to the first embodiment of the present application is schematically shown; Figure 6 A block diagram schematically shows a processing device for a live broadcast room according to Embodiment 2 of the present application; and Figure 7 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0021] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0022] First, the following terms are explained: Danmu is a subtitle that pops up and moves in a predetermined direction when watching a video on the Internet. There is no fixed word for Danmu in English. It is usually called: comment, danmaku, barrage, bullet screen, bullet-screen comment, etc. Danmu allows viewers to comment or feel the video, but unlike ordinary video sharing websites that only display comments in the dedicated comment area under the player, it will appear in real time on the video screen in the form of sliding subtitles to ensure that all viewers can notice it. Danmu can be managed by the Danmu system. Some Danmu systems can use CSS (Cascading StyleSheets) or JavaScript language to provide specific Danmu forms, such as the appearance or disappearance of Danmu in a specific position, control the pop-up speed of Danmu, Danmu position, etc. In addition, the Danmu that appears fixed at the bottom or top of the screen will also be used as subtitles for videos without subtitles.

[0023] For example, the data structure of a bullet comment can be as follows: { The time of emission in the video is in seconds; Types of barrages; The size of the barrage text; The color of the barrage text; Date the comment was created; The class name of the bullet comment; The user ID that created the bullet comment; Barrage ID (e.g. barrage group ID); Barrage content (e.g., text, images, controls); The duration of the barrage; Whether the bullet screen has a border; The color of the bullet screen border; The interline style of the bullet comment; }.

[0024] Audio stream: A data stream consisting of multiple audio data.

[0025] Danmu stream: Danmu data composed of multiple Danmu messages.

[0026] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: When watching videos, users like to send their opinions in the form of barrage to express their feelings.

[0027] However, the current barrage expression is relatively simple, generally only with a simple scrolling effect, and cannot be dynamically adjusted according to the emotions of the music and the barrage. This expression cannot fully utilize the emotional characteristics of music videos and barrage, and it is difficult to improve the user's interactive experience.

[0028] To this end, the embodiment of the present application provides a technical solution for displaying bullet screens. In this technical solution, by extracting the emotional features of the audio and classifying the emotions of different types of bullet screen content, when the bullet screen is displayed, the display effect of the bullet screen can be adjusted according to the emotional state of the audio content in the video, thereby enriching the display method of the bullet screen, and enhancing the interactive experience between the audience and the video content, so that the bullet screen display is coordinated with the emotional state of the audio and video content. See below for details.

[0029] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0030] like Figure 1 As shown, the environmental application schematic diagram includes: a service platform 2 and a client.

[0031] The service platform 2 can be connected to the client terminals ( 4A, 4B, ..., 4N) via the network.

[0032] The service platform 2 can provide various content and bullet screen services for the client. For example, the bullet screen is sent to the client via the network. The network includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices and / or the like. The network may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, etc. The network may include wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0033] The service platform 2 may be a single server, a server cluster or a cloud computing service center, which is used to provide various services. For example: The service platform 2 may provide content services. The content services may be configured to provide content such as video, audio, text data, or a combination thereof. The content may include content streams (e.g., video streams, audio streams, information streams), content files (e.g., video files, audio files, text files), and / or other data.

[0034] The service platform 2 can be configured to manage messages for various content items. Users can browse content and access different content items to view comments for specific content, such as comments posted by other users for the specific content. Comments from users associated with a specific content item can be output to other users who view the specific content item. For example, all users who access a content item (e.g., a video) can view comments associated with the content item. The input comment content can be output in real time or near real time.

[0035] The service platform 2 may be configured to receive a plurality of messages. The plurality of messages may include a plurality of bullet comments associated with the content.

[0036] The service platform 2 may be configured to process multiple messages, for example, various processing operations such as message storage, message filtering, and message push. Among them, message storage is used to store multiple messages in a data storage such as a database. Message filtering may include rejecting or marking messages that match the filtering criteria. Among them, the filtering criteria may specify terms and / or phrases, such as profanity, hate speech, indecent language, etc. The filtering criteria may specify characters, such as symbols, fonts, etc. The filtering criteria may specify languages, computer-readable code patterns, etc.

[0037] The service platform 2 can perform natural language processing, topic recognition, pattern recognition, artificial intelligence, etc. to automatically determine the characteristics of the message and / or group the message. As an example, frequently occurring phrases or patterns can be identified as topics. As another example, a database of topics associated with the content can be maintained. Topics can include genres (e.g., action, drama, comedy), personalities (e.g., actors, actresses, directors), languages, etc. Messages can be grouped based on the characteristics of the client device and / or the user sending the message. Demographics, interests, history, and / or the like can be stored for multiple users to determine potential groupings of messages. For example, the bullet screens uploaded by each user can be screened, filtered, sorted, and personalized push, etc.

[0038] The service platform 2 may provide a bullet screen service, which is configured to allow users to comment and / or share comments (i.e. bullet screens) associated with the content. The bullet screens are presented on the same screen together with the content. The bullet screens may be displayed in an overlay above the content and may have animation effects. For example, the bullet screens may scroll (e.g., from right to left, from left to right, from top to bottom, ...), and the animation effect may be implemented based on the transition property of CSS3.

[0039] The client (4A, 4B, ..., 4N) may be configured to access the content and services of the service platform 2. The client (4A, 4B, ..., 4N) may include an electronic device carrying or connected to a display panel, such as a mobile device, a tablet device, a laptop computer, a workstation, a virtual reality device, a gaming device, a digital streaming device, a vehicle client, a smart TV, a set-top box, etc., and may also include a virtualized computing instance. The virtualized computing instance may include a virtual machine, such as a simulation of a computer system, an operating system, a server, etc.

[0040] The client (4A, 4B, ..., 4N) may be associated with one or more users. A single user may also use one or more of the clients (4A, 4B, ..., 4N) to access the service platform 2. The client (4A, 4B, ..., 4N) may travel to various locations and use different networks to access the service platform 2.

[0041] The client (4A, 4B, ..., 4N) may include multiple client programs, such as a video APP, a live broadcast APP or a mini program with a built-in bullet screen system. In this embodiment, the video and other contents provided by the service platform 2 may be presented through the client program, and the bullet screen system may present the bullet screen contents in a specific manner.

[0042] The client (4A, 4B, ..., 4N) may include an interface, which may include a touch pad, a touch screen, a mouse, a keyboard or other sensing elements. For example, the input element may be configured to receive user instructions, which may enable the client (4A, 4B, ..., 4N) to perform various operations, such as inputting Barrage wait.

[0043] It should be noted that the above devices are exemplary, and the number and type of devices are adjustable in different scenarios or according to different needs.

[0044] The following describes the technical solution of the present application through multiple embodiments, taking the client as the execution subject. It should be noted that these embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments described here.

[0045] Embodiment 1 Figure 2 The flowchart of the bullet screen display method according to the first embodiment of the present application is schematically shown.

[0046] like Figure 2 As shown, the bullet screen display method may include steps S200 to S208, wherein: Step S200: During the process of playing the target video, it is detected whether there is audio data in the video content currently being played.

[0047] Step S202, when there is audio data in the video content being played at the current moment, obtain the first emotion label of the audio data at the current moment, and obtain the second emotion label of the target barrage that needs to be displayed at the current moment.

[0048] Step S204: matching the first emotion tag with the second emotion tag.

[0049] Step S206, when the first emotion tag successfully matches the second emotion tag, determine the barrage special effects of the successfully matched target barrage according to the first emotion tag.

[0050] Step S208: display the successfully matched target bullet comments according to the bullet comment special effects.

[0051] The barrage display method provided in this embodiment will detect in real time whether there is audio data (music data) in the video content being played at the current moment during the process of playing the target video. Then, when it is detected that there is audio data in the video being played at the current moment, the first emotional label (for example, happy, sad, etc.) of the audio data at the current moment will be obtained, and the second emotional label (for example, happy, sad, etc.) of the target barrage that needs to be displayed at the current moment will be obtained. After that, the second emotional label of each target barrage to be displayed will be matched with the first emotional label one by one, and in the case of successful matching, the barrage special effect corresponding to the second emotional label will be determined. Finally, the successfully matched target barrage will be displayed according to the barrage special effect. It can be seen that in the embodiment of the present application, when the barrage is displayed, the barrage with the same emotion will be displayed with special effects according to the emotion of the audio data in the target video, thereby enriching the display method of the barrage. In addition, through the above-mentioned display method, the barrage display is coordinated with the emotional state of the audio and video content, enhancing the interactive experience of the audience with the video content.

[0052] The following combination Figure 2 , each step in steps S200~S208 and other optional steps are explained in detail.

[0053] Step S200 , during the process of playing the target video, it is detected whether there is audio data in the video content being played at the current moment.

[0054] The target video can be a video watched online by the user or a video downloaded locally. In actual applications, the target video is preferably a video containing music, and the target video has an associated barrage stream, which is a barrage collection composed of barrages emitted by the user when watching the target video.

[0055] Exemplarily, since the target video is composed of an image data stream and an audio data stream, during the process of playing the target video, the video content currently being played may be parsed to determine whether there is audio data in the video content currently being played.

[0056] In this embodiment, the method of detecting audio data from video content is not limited, as long as audio data can be detected from video content.

[0057] Step S202 , when there is audio data in the video content being played at the current moment, obtain the first emotion label of the audio data at the current moment, and obtain the second emotion label of the target barrage that needs to be displayed at the current moment.

[0058] The first emotion tag is a tag used to reflect the emotional state of the audio data, such as a tag of the emotional state of pure music or songs, or a tag of the emotional state of characters in spoken or dialogue. The first emotion tag may be "happy", "angry", "sad", "calm", etc. The second emotion tag is a tag used to reflect the emotional state of the user who sent the target barrage. The second emotion tag may also be "happy", "angry", "sad", "calm", etc.

[0059] In this embodiment, the first emotion tag can be obtained in advance by the server by analyzing the audio data through a machine learning algorithm, and the obtained emotion tag is returned to the client for storage, so that the client can obtain it in time when needed. Similarly, the second emotion tag can also be obtained in advance by the server by analyzing the barrage data through a machine learning algorithm, and the obtained emotion tag is returned to the client for storage, so that the client can obtain it in time when needed. In this embodiment, the server processes the audio data and barrage data in advance to obtain the first emotion tag and the second emotion tag, which can reduce the resource consumption of the client device.

[0060] In another embodiment, the first emotion tag may also be obtained in advance by the client device by analyzing the audio data through a machine learning algorithm. Similarly, the second emotion tag may also be obtained in advance by the client device by analyzing the audio data through a machine learning algorithm.

[0061] For example, when there is audio data in the video content being played at the current moment, the first emotion label of the audio data will be obtained, and the second emotion label of the target barrage that needs to be displayed at the current moment will be obtained, so that the two can be matched later to achieve special effects display of the corresponding barrage according to the matching results.

[0062] Step S204 , matching the first emotion label with the second emotion label.

[0063] Exemplarily, whether the first emotion label and the second emotion label match can be determined by comparing whether the first emotion label and the second emotion label are the same, or whether the first emotion label and the second emotion label match can be determined by calculating the similarity between the first emotion label and the second emotion label.

[0064] Step S206 When the first emotion tag successfully matches the second emotion tag, the barrage special effects of the successfully matched target barrage are determined according to the first emotion tag.

[0065] Exemplarily, a corresponding bullet screen special effect is pre-set for each first emotion label. For example, when the first emotion label is "happy", the corresponding bullet screen special effect may be "jumping, rotating", etc. of the bullet screen content. When the first emotion label is "sad", the corresponding bullet screen special effect may be "slowly floating, fading in and out", etc. of the bullet screen content. When the first emotion label is "happy", the corresponding bullet screen special effect may be the color of the bullet screen content changing to color; when the first emotion label is "angry", the corresponding bullet screen special effect may be the font size of the bullet screen content becoming larger.

[0066] Step S208 , display the successfully matched target barrage according to the barrage special effects.

[0067] Exemplarily, if the barrage special effect is an increase in the font size of the barrage content, then when the target barrage is displayed, the font size of the barrage content of the target barrage will be gradually increased; if the barrage special effect is the "fade in and out" of the color of the barrage content, then when the target barrage is displayed, the color of the barrage content of the target barrage will first become lighter, then darker, and finally lighter again; if the barrage special effect is the "jumping" of the barrage content, then when the target barrage is displayed, the barrage content of the target barrage will be displayed in a "jumping" manner.

[0068] In an optional embodiment, if Figure 3 As shown, step S208 may include: Step S300, obtaining the number of successfully matched target bullet comments.

[0069] Step S302, when the number exceeds a preset number, selecting the preset number of target barrages from the successfully matched target barrages.

[0070] Step S304, displaying the selected target bullet comment according to the bullet comment special effect.

[0071] The preset number is a pre-set number, which can be set and modified according to actual conditions. For example, the preset number is 20.

[0072] In one embodiment, the preset number can also be set according to the first emotion label, for example, the preset number corresponding to "happy" is 20; the preset number corresponding to "sad" is 15; the preset number corresponding to "calm" is 25, etc.

[0073] For example, since there may be many successfully matched target bullets, in order to avoid overloading the content displayed on the screen, when displaying the successfully matched target bullets, a preset number of target bullets will be selected from all successfully matched target bullets, and then these selected target bullets will be displayed with special effects. When selecting the target bullet, it can be selected randomly or according to the length of the bullet, and the specific selection method is not limited in this embodiment.

[0074] In another embodiment, the number of special effect bullet comments may also be automatically reduced according to the density of the bullet comments currently displayed on the screen.

[0075] In this embodiment, when there are too many bullet screens, some of them are selected for special effect display, and the unselected bullet screens are not displayed with special effects, thereby avoiding overload of the content displayed on the screen.

[0076] In an optional embodiment, if Figure 4 As shown, the method may also include: Step S400: extracting an audio stream from the target video, wherein the audio stream includes a plurality of audio data.

[0077] Step S402: divide the audio stream into multiple audio segments.

[0078] Step S404: input the multiple audio clips into a pre-trained emotion recognition model respectively to obtain a third emotion label corresponding to each of the audio clips.

[0079] Step S406: establishing a first association relationship between each of the audio clips, each of the third emotion tags, and a time interval corresponding to each of the audio clips, and saving the first association relationship.

[0080] Exemplarily, the emotion recognition model can detect the user's emotional tendency based on audio features (such as rhythm, spectrum, pitch) in the audio clip, thereby outputting a third emotion tag, wherein the emotion recognition model can be a recursive neural network (RNN) model.

[0081] In one embodiment, in order to facilitate the emotion recognition model to analyze the audio clip, audio features such as rhythm, spectrum, and pitch can be first extracted from the audio clip, and then these extracted audio features are converted into feature data that can be recognized by the emotion recognition model.

[0082] In another embodiment, in order to facilitate the emotion recognition model to analyze the audio clip, the audio clip can also be first subjected to speech recognition to obtain text information, and then keywords are extracted from the text information. Finally, the extracted keywords are converted into feature data that can be recognized by the emotion recognition model, so that the emotion recognition model can output the third emotion tag corresponding to the audio clip based on the feature data.

[0083] In one embodiment, when the audio stream is divided into a plurality of audio segments, the audio stream may be divided into a plurality of audio segments according to a preset time window length. The time window length may be a fixed time length, such as 10 seconds. The time window length may also be dynamically adjusted according to the number of bullet comments contained in the bullet comment stream and the length of the target video.

[0084] It should be noted that the first emotion label is one of the third emotion labels.

[0085] In this embodiment, the audio stream is divided into multiple audio segments in advance, and then the third emotion tag of each audio segment is analyzed by the emotion recognition model. Finally, a first association relationship is established between each audio segment, each third emotion tag and the time interval corresponding to each audio segment, and the first association relationship is saved. This facilitates the subsequent playback of the target video. The emotion tag corresponding to the audio data can be accurately and real-time determined according to the first association relationship, ensuring that the target barrage can be displayed with special effects in a timely manner.

[0086] In an optional embodiment, step S402 may include: dividing the audio stream into a plurality of audio segments having partial overlap.

[0087] Exemplarily, the overlapping portion of two adjacent audio segments may be pre-set, for example, the overlapping portion may be set to be 2 seconds of audio data.

[0088] As an example, assuming that the total length of the audio stream is 20 seconds and the length of each audio segment is 5 seconds, when the audio stream is divided into multiple audio segments, the audio data of 0-5 seconds will be used as the first audio segment; the audio data of 3-8 seconds will be used as the second audio segment; the audio data of 6-11 seconds will be used as the third audio segment; the audio data of 9-14 seconds will be used as the fourth audio segment; the audio data of 12-17 seconds will be used as the fifth audio segment; and the audio data of 15-20 seconds will be used as the sixth audio segment.

[0089] In an embodiment of the present application, by dividing the audio stream into multiple audio segments with partial overlap, the audio data in the overlapping portion can have two different emotion labels. In this way, when the target barrage is subsequently displayed with special effects, the target barrage can be displayed according to the two special effects, thereby avoiding sudden changes in the special effects.

[0090] In an optional embodiment, the first emotion tag includes a fifth emotion tag and a sixth emotion tag, and when the first emotion tag successfully matches the second emotion tag, determining the barrage special effects of the successfully matched target barrage according to the first emotion tag includes: when the first emotion tag successfully matches the second emotion tag, determining the barrage special effects of the successfully matched target barrage according to the fifth emotion tag and the sixth emotion tag respectively.

[0091] In this embodiment, when the first emotional tag includes the fifth emotional tag and the sixth emotional tag, the barrage special effects of the target barrage will be determined according to each emotional tag respectively, so that when the target barrage is displayed, the special effects of the two can be combined to achieve the final special effects display, thereby avoiding sudden changes in the special effects during the display of the barrage and improving the user experience.

[0092] In an optional embodiment, if Figure 5 As shown, the method may also include: Step S500, obtaining a barrage stream associated with the target video, wherein the barrage stream includes multiple barrages.

[0093] Step S502: input each barrage in the barrage stream into a pre-trained natural language processing model to obtain a fourth emotion label corresponding to each barrage.

[0094] Step S504, establishing a second association relationship between each of the bullet comments and the corresponding fourth emotional tag, and saving the second association relationship.

[0095] Exemplarily, the natural language processing model can detect the user's emotional tendency based on the content of the barrage, thereby outputting a fourth emotional label.

[0096] In one embodiment, in order to facilitate the natural language processing model to analyze the barrage, meaningless content in the barrage, such as invalid characters, can be filtered out first, and then the filtered barrage content is input into the natural language processing model for sentiment analysis, thereby outputting the fourth sentiment label.

[0097] It should be noted that the second emotion label is one of the emotion labels among the fourth emotion labels.

[0098] In this embodiment, the fourth emotion label of each barrage is analyzed in advance through a natural language processing model, and then a second association relationship between each barrage and the corresponding fourth emotion label is established, and the second association relationship is saved. This facilitates the subsequent playback of the target video. The emotion label corresponding to each target barrage can be accurately and timely determined according to the second association relationship, ensuring that the subsequent special effects display of the target barrage can be synchronized with the audio data.

[0099] In an optional embodiment, the method further comprises: The target bullet comments that were not successfully matched are displayed using a preset display method.

[0100] Exemplarily, the preset display mode is a default display mode, such as sliding display on the screen.

[0101] In this embodiment, a preset display method is used to display unsuccessfully matched target bullet comments, thereby ensuring that all target bullet comments can be displayed without causing significant interference to the audience.

[0102] Embodiment 2 Figure 6 The block diagram of the bullet screen display device 600 according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 6 As shown, the device 600 may include a detection module 601, an acquisition module 602, a matching module 603, a determination module 604 and a display module 605, wherein: Detection module 601 is used to detect whether there is audio data in the video content being played at the current moment during the process of playing the target video. The acquisition module 602 is used to obtain the first emotion label of the audio data at the current moment when there is audio data in the video content being played at the current moment, and to obtain the second emotion label of the target barrage that needs to be displayed at the current moment.

[0103] The matching module 603 is used to match the first emotion label with the second emotion label.

[0104] The determination module 604 is used to determine the barrage special effects of the successfully matched target barrage according to the first emotion tag when the first emotion tag and the second emotion tag are successfully matched.

[0105] The display module 605 is used to display the successfully matched target bullet comments according to the bullet comment special effects.

[0106] As an optional embodiment, the root device 600 further includes an extraction module, a division module, a first input module and a first establishment module.

[0107] The extraction module is used to extract an audio stream from the target video, wherein the audio stream includes multiple audio data.

[0108] The division module is used to divide the audio stream into multiple audio segments.

[0109] The first input module is used to input the multiple audio clips into a pre-trained emotion recognition model respectively to obtain a third emotion label corresponding to each of the audio clips.

[0110] The first establishing module is used to establish a first association relationship between each of the audio clips, each of the third emotion tags and the time interval corresponding to each of the audio clips, and save the first association relationship.

[0111] As an optional embodiment, dividing the audio stream into a plurality of audio segments includes: The audio stream is divided into a plurality of audio segments with partial overlap.

[0112] As an optional embodiment, the first emotion tag includes a fifth emotion tag and a sixth emotion tag, and when the first emotion tag successfully matches the second emotion tag, determining the barrage special effects of the successfully matched target barrage according to the first emotion tag includes: When the first emotion tag and the second emotion tag are successfully matched, the barrage special effects of the successfully matched target barrage are determined according to the fifth emotion tag and the sixth emotion tag respectively.

[0113] As an optional embodiment, the device 600 also includes a bullet comment acquisition module, a second input module, and a second establishment module.

[0114] The barrage acquisition module is used to obtain a barrage stream associated with the target video, and the barrage stream includes multiple barrages.

[0115] The second input module is used to input each barrage in the barrage stream into a pre-trained natural language processing model to obtain a fourth emotion label corresponding to each barrage.

[0116] The second establishing module is used to establish a second association relationship between each of the barrages and the corresponding fourth emotional label, and save the second association relationship.

[0117] As an optional embodiment, displaying the successfully matched target bullet screen according to the bullet screen special effect includes: Obtain the number of successfully matched target bullet comments; When the number exceeds a preset number, selecting the preset number of target bullet screens from the successfully matched target bullet screens; The selected target barrage is displayed according to the barrage special effect.

[0118] As an optional embodiment, the display module 605 is also used to display the target bullet screen that has not been successfully matched using a preset display method.

[0119] Embodiment 3 Figure 7 The schematic diagram of the hardware architecture of a computer device 700 suitable for implementing the bullet screen display method according to the third embodiment of the present application is shown schematically. In some embodiments, the computer device 700 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 700 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Figure 7 As shown, the computer device 700 includes but is not limited to: a memory 701, a processor 702, and a network interface 703 that can communicate with each other through a system bus. The memory 701 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 701 can be an internal storage module of the computer device 700, such as a hard disk or memory of the computer device 700. In other embodiments, the memory 701 can also be an external storage device of the computer device 700, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 700. Of course, the memory 701 can also include both the internal storage module of the computer device 700 and its external storage device. In this embodiment, the memory 701 is generally used to store the operating system and various application software installed on the computer device 700, such as the program code of the bullet screen display method, etc. In addition, the memory 701 can also be used to temporarily store various types of data that have been output or are to be output.

[0120] In some embodiments, the processor 702 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 702 is generally used to control the overall operation of the computer device 700, such as performing control and processing related to data interaction or communication with the computer device 700. In this embodiment, the processor 702 is used to run the program code stored in the memory 701 or process data.

[0121] The network interface 703 may include a wireless network interface or a wired network interface, and the network interface 703 is generally used to establish a communication link between the computer device 700 and other computer devices. For example, the network interface 703 is used to connect the computer device 700 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 700 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0122] It should be pointed out that Figure 7 Only a computer device having components 701 - 703 is shown, but it should be understood that implementing all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0123] In this embodiment, the bullet screen display method stored in the memory 701 can also be divided into one or more program modules and executed by one or more processors (such as processor 702) to complete the embodiment of the present application.

[0124] Embodiment 4 An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the barrage display method in the embodiment are implemented.

[0125] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on a computer device, such as the program code of the bullet screen display method in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0126] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0127] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0128] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A bullet screen display method, characterized in that: The method comprises: During the process of playing the target video, detecting whether there is audio data in the video content currently being played; In the case where there is audio data in the video content being played at the current moment, obtaining a first emotion label of the audio data at the current moment, and obtaining a second emotion label of a target barrage that needs to be displayed at the current moment; Performing matching processing on the first emotion label and the second emotion label; In the case where the first emotion tag successfully matches the second emotion tag, determining a bullet screen special effect of a successfully matched target bullet screen according to the first emotion tag; The successfully matched target barrage is displayed according to the barrage special effect.

2. The method according to claim 1, characterized in that The method further comprises: Extracting an audio stream from the target video, wherein the audio stream includes a plurality of audio data; Dividing the audio stream into a plurality of audio segments; Inputting the multiple audio clips into a pre-trained emotion recognition model respectively to obtain a third emotion label corresponding to each of the audio clips; A first association relationship is established between each of the audio clips, each of the third emotion tags, and a time interval corresponding to each of the audio clips, and the first association relationship is saved.

3. The method according to claim 2, characterized in that The dividing the audio stream into a plurality of audio segments comprises: The audio stream is divided into a plurality of audio segments with partial overlap.

4. The method according to claim 3, characterized in that The first emotion tag includes a fifth emotion tag and a sixth emotion tag, and when the first emotion tag successfully matches the second emotion tag, determining the barrage special effects of the successfully matched target barrage according to the first emotion tag includes: When the first emotion tag and the second emotion tag are successfully matched, the barrage special effects of the successfully matched target barrage are determined according to the fifth emotion tag and the sixth emotion tag respectively.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a bullet comment stream associated with the target video, wherein the bullet comment stream includes multiple bullet comments; Inputting each barrage in the barrage stream into a pre-trained natural language processing model to obtain a fourth emotion label corresponding to each barrage; A second association relationship is established between each of the bullet comments and the corresponding fourth emotional tag, and the second association relationship is saved.

6. The method according to any one of claims 1 to 5, characterized in that: The displaying of the successfully matched target bullet screen according to the bullet screen special effect comprises: Obtain the number of successfully matched target bullet comments; When the number exceeds a preset number, selecting the preset number of target bullet screens from the successfully matched target bullet screens; The selected target barrage is displayed according to the barrage special effect.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The target bullet comments that were not successfully matched are displayed using a preset display method.

8. A bullet screen display device, characterized in that: The device comprises: A detection module, used to detect whether there is audio data in the video content currently being played during the process of playing the target video; An acquisition module, used for acquiring, when there is audio data in the video content being played at the current moment, a first emotion label of the audio data at the current moment, and acquiring a second emotion label of a target barrage that needs to be displayed at the current moment; A matching module, used for matching the first emotion label with the second emotion label; A determination module, configured to determine, when the first emotion tag successfully matches the second emotion tag, a bullet screen special effect of a successfully matched target bullet screen according to the first emotion tag; A display module is used to display the successfully matched target barrage according to the barrage special effects.

9. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 7 are implemented.

Images