A method, device, medium and product for generating a barrage

Abstract

This application provides a method, device, medium, and product for generating bullet comments. The bullet comment generation method includes: upon receiving a target audio stream, analyzing the target audio stream to obtain acoustic features and lyric features; determining a multi-level emotion grid based on the acoustic features and lyric features; generating bullet comment rendering instructions corresponding to the target audio stream based on the multi-level emotion grid; and generating bullet comments that match the target audio stream based on the bullet comment rendering instructions.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, device, medium and product for generating bullet comments. Background Technology

[0002] In related technologies, the rhythmic characteristics of the song being played are obtained, and the appearance rhythm of the corresponding bullet comments (such as bullet comment speed and density) is determined based on these rhythmic characteristics. The bullet comments corresponding to the song are then displayed based on these appearance rhythms. However, this reduces the matching degree between the appearance rhythms of the song and the bullet comments. Summary of the Invention

[0003] This application provides a method, device, medium, and product for generating bullet comments, which can improve the matching degree between the rhythm of song and bullet comments.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a method for generating bullet comments, the method including: Upon receiving the target audio stream, the target audio stream is analyzed to obtain acoustic features and lyric features; Based on the acoustic features and the lyric features, a multi-level emotional grid is determined; Based on the multi-level emotion grid, generate the bullet screen rendering instructions corresponding to the target audio stream; Based on the bullet screen rendering instructions, bullet screens matching the target audio stream are generated.

[0005] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the bullet screen generation method provided in the embodiments of this application.

[0006] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the bullet screen generation method provided in this application.

[0007] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the bullet screen generation method provided in this application.

[0008] The embodiments of this application have the following beneficial effects: The bullet screen generation device analyzes the target audio stream to determine the acoustic features and lyric features corresponding to the target audio stream, and then determines the multi-level emotional grid of the target audio stream based on the acoustic features and lyric features. Finally, it generates a bullet screen rendering instruction corresponding to the target audio stream based on the multi-level emotional grid. That is, the bullet screen rendering instruction is determined based on the acoustic features and lyric features of the target audio stream, not just the rhythm features of the song. This allows the bullet screen rendering instruction to render bullet screens that match the target audio stream, thereby improving the matching degree between the rhythm of the song and the bullet screen. Attached Figure Description

[0009] Figure 1 This is a flowchart of a bullet screen generation method provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the composition of an exemplary bullet screen generation structure provided in an embodiment of this application; Figure 3 This is a schematic diagram of an exemplary bullet screen generation method provided in an embodiment of this application; Figure 4 This is an exemplary schematic diagram of the bullet screen effect of a default image during the climax phase, provided in an embodiment of this application. Figure 5 This is an exemplary illustration of the bullet screen effect after switching to the "rock fan" profile provided in this application embodiment; Figure 6 This is an exemplary "rock fan" portrait provided in this application embodiment, with "enhanced" modulation enabled, as shown in the schematic diagram. Figure 7 This is a schematic diagram of the composition structure of a bullet screen generation device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application.

[0010] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] This application provides a method for generating bullet comments, which is applied to a bullet comment generation device. Figure 1A flowchart of a bullet screen generation method provided in this application embodiment is shown below. Figure 1 As shown, methods for generating bullet comments can include: S101. Upon receiving the target audio stream, analyze the target audio stream to obtain acoustic features and lyric features.

[0013] The bullet screen generation method provided in this application embodiment is applicable to scenarios where bullet screens corresponding to a target audio stream are generated.

[0014] In the embodiments of this application, the bullet screen generation device can be implemented in various forms. For example, the bullet screen generation device described in this application may include processors, controllers, servers, terminals, etc. The specific bullet screen generation device can be determined according to the actual situation, and the embodiments of this application do not limit it.

[0015] In this embodiment, the target audio stream can be information input by the user, information transmitted from other devices to the bullet screen generation device, or information obtained by the bullet screen generation device through other means. The specific way the bullet screen generation device obtains the target audio stream can be determined according to the actual situation, and this embodiment does not limit it.

[0016] It should be noted that the target audio stream can be a song, such as a traditional Chinese style song, a rock song, a pop song, or a country music song. The specific target audio stream can be determined according to the actual situation, and this application embodiment does not limit it in this regard.

[0017] It should also be noted that the target audio stream can be a complete song; or it can be a part of a song, such as the climax of the song; the specifics can be determined according to the actual situation, and this application embodiment does not limit this.

[0018] In this embodiment, a large model can be used to analyze the target audio stream to obtain acoustic features and lyric features; other network models can also be used to analyze the target audio stream to obtain acoustic features and lyric features; other methods can also be used to analyze the target audio stream to obtain acoustic features and lyric features; the specific method of analyzing the target audio stream to obtain acoustic features and lyric features can be determined according to the actual situation, and this embodiment does not limit it.

[0019] In this embodiment, acoustic features are deep-level features extracted from the target audio stream. Specifically, acoustic features include beat information, swing information, and valence information. Acoustic feature information may also include arousal information, timbre information, loudness information, etc. The specific content of acoustic feature information can be determined according to the actual situation, and this embodiment does not limit it.

[0020] In this embodiment, the lyrics features are obtained by analyzing the lyrics text in the target audio stream. Specifically, the lyrics features include emotional information and syllable stress information. The lyrics features may also include keyword information. The specific content included in the lyrics features can be determined according to the actual situation, and this embodiment does not limit it.

[0021] In this embodiment of the application, the process by which the bullet screen generation device analyzes the target audio stream to obtain acoustic features and lyric features includes: extracting musical features from the target audio stream to obtain acoustic features; and performing natural language processing on the target audio stream to obtain lyric features.

[0022] In the embodiments of this application, music features can be extracted from the target audio stream using a large model to obtain acoustic features; music features can also be extracted from the target audio stream using the Jukebox music feature extraction module to obtain acoustic features; music features can also be extracted from the target audio stream in other ways to obtain acoustic features; the specific method of extracting music features from the target audio stream to obtain acoustic features can be determined according to the actual situation, and the embodiments of this application do not limit this.

[0023] In this embodiment, a large model can be used to perform natural language processing on the target audio stream to obtain lyric features; alternatively, a Contrastive Language-Audio Pretraining (CLAP) lyric semantic analysis module can be used to perform natural language processing on the target audio stream to obtain lyric features; other methods can also be used to perform natural language processing on the target audio stream to obtain lyric features. The specific method of performing natural language processing on the target audio stream to obtain lyric features can be determined according to the actual situation, and this embodiment does not limit it.

[0024] For example, the Jukebox music feature extraction module can be used to receive the signal from the music input source (target audio stream) and is responsible for extracting deep acoustic features from the music, such as beat (BPM) information, swing information, timbre information, loudness information, arousal information, and valence information.

[0025] For example, the CLAP lyrics semantic analysis module can be used to receive the signal (if containing lyrics) from the music input source (target audio stream), and natural language processing technology can be used to analyze the emotional polarity (i.e., emotional information of the lyrics), keyword information and syllable stress information of the lyrics text to obtain lyrics features.

[0026] S102. Based on acoustic features and lyric features, determine a multi-level emotional grid.

[0027] In this embodiment of the application, the bullet screen generation device analyzes the target audio stream to obtain acoustic features and lyric features, and then determines a multi-level emotional grid based on the acoustic features and lyric features.

[0028] It should be noted that the multi-level emotional grid includes grids such as the main grid, sub-grids, and micro-grids. Among them, the main grid represents the emotional atmosphere of the song in the target audio stream; the sub-grids represent the rhythmic information of the target audio stream; and the micro-grids represent the detailed texture of the target audio stream.

[0029] In this embodiment of the application, the process by which the bullet screen generation device determines a multi-level emotional grid based on acoustic features and lyric features includes: determining a main grid based on valence information in acoustic features and lyric emotional information in lyric features; determining a sub-grid based on tempo information in acoustic features and syllable accent information in lyric features; determining a micro-grid based on swaying information in acoustic features; and determining a multi-level emotional grid based on the main grid, sub-grids, and micro-grids.

[0030] In the embodiments of this application, the emotional atmosphere of the target audio stream, i.e., the main grid, can be determined based on valence information and lyric emotional information.

[0031] It should be noted that the macro-grid defines the overall emotional tone (song sentiment information) of a song segment (target audio stream). It is generated by merging the valence information of the music with the textual sentiment polarity of the lyrics (i.e., the lyrical sentiment information).

[0032] For example, high valence and positive text will generate a “pleasant” main grid, which determines the main color tone of the paragraph’s comments (such as warm colors) and the overall feel of the animation (such as smoothness).

[0033] In this embodiment, the rhythmic information (including rhythm and style), i.e., the sub-grid, of the target audio stream can be determined based on beat information and syllable accent information. Specifically, the sub-grid velocity factor can be determined first based on the beat information, and then the sub-grid can be determined based on the sub-grid velocity factor and syllable accent information.

[0034] It should be noted that the process of determining the sub-grid velocity factor based on the beat time information (BPM) is as shown in formula (1): (1) It should also be noted that BPM is the number of beats per minute in music. The BPM is normalized using formula (1) and mapped to the (0,1) interval, serving as the basic factor for tempo. Centered on 100 BPM, faster tempos approach 1, and slower tempos approach 0.

[0035] It should be noted that the sub-grid defines the rhythm and style of the music (target audio stream). It primarily generates beat points by mapping beat (BPM) information and analyzing syllable intensity (syllable accent information).

[0036] For example, BPM=72 will generate a slower beat sequence, which determines the base frequency and movement speed of the bullet comments.

[0037] In this embodiment of the application, the process of determining the microgrid based on the swaying sensation information in the acoustic features includes determining the microgrid rhythm factor based on the swaying sensation information; and determining the microgrid based on the microgrid rhythm factor.

[0038] In this embodiment of the application, the process of determining the microgrid rhythm factor based on swing information is shown in formula (2): (2) It should be noted that "swing" refers to the swing effect coefficient in music, i.e., the swing sensation information, with a value range of [0, 1], used to add rhythmic feel to the basic beat. 0.3 is the weighting coefficient of the swing effect.

[0039] In this embodiment, a micro-grid can be used to determine the detailed texture of the target audio stream. The micro-grid defines the detailed dynamics and effect triggers of the music (target audio stream). More refined trigger points are generated using swing information and specific positions within the beat (rhythm information) to control detailed embellishments such as particle effects and lighting changes. Specifically, the rhythm information of the target audio stream is obtained; the micro-grid is determined based on the swing information and rhythm information. More specifically, the micro-grid rhythm factor is determined based on the swing information; the micro-grid is then determined based on the micro-grid rhythm factor and rhythm information.

[0040] In this embodiment of the application, after determining the main grid, sub-grids and micro-grids, the main grid, sub-grids and micro-grids can be defined as a multi-level emotional grid.

[0041] Understandably, unlike existing technologies that only use single features such as BPM, this application establishes a multi-dimensional control structure comprising a "main grid (emotion)," a "sub-grid (rhythm)," and a "micro-grid (details)." This structure integrates multimodal information such as musical valence / arousal, lyrical emotion, BPM, swing feel, and syllable accent. It achieves a comprehensive understanding of music from "macro-emotion" to "micro-details," making the generation of bullet comments no longer a simple rhythmic follow, but a deep visual expression of emotion and style, greatly enhancing the semantic relevance and emotional resonance between bullet comments and music.

[0042] S103. Generate the barrage rendering instructions corresponding to the target audio stream based on the multi-level emotion grid.

[0043] In this embodiment, after determining a multi-level emotional grid based on acoustic features and lyric features, the bullet screen generation device generates bullet screen rendering instructions corresponding to the target audio stream based on the multi-level emotional grid.

[0044] In this embodiment of the application, the process of the bullet screen generation device generating bullet screen rendering instructions corresponding to the target audio stream based on a multi-level emotion grid includes: obtaining object portrait information of the target object; adjusting the multi-level emotion grid based on the object portrait information to obtain the adjusted multi-level emotion grid; and generating bullet screen rendering instructions corresponding to the adjusted multi-level emotion grid.

[0045] It should be noted that the target object is the object used to receive the target audio stream. That is, the target object is the user, the user who listens to or watches the target audio stream.

[0046] In this embodiment of the application, the object profile information of the target object can be obtained from the database, or from other devices, or through other means. The specific method of obtaining the object profile information of the target object can be determined according to the actual situation, and this embodiment of the application does not limit it.

[0047] It should be noted that the target profile information includes information such as the target object's gender, age, historical song categories, and preferred comment response speed. The specific target profile information can be determined according to the actual situation, and this application embodiment does not limit it.

[0048] In this embodiment of the application, object preference information of the target object can be obtained from the object profile, and then the multi-level sentiment grid can be adjusted based on the object preference information to obtain the adjusted multi-level sentiment grid.

[0049] It should be noted that the object preference information includes object reaction speed information, bullet screen effect preference information, and object style information. Object preference information may also include other information; the specific content of the object preference information can be determined according to actual circumstances, and this application embodiment does not limit this.

[0050] For example, object profile information of the target object can be obtained from the object profile database. Based on this object profile information, the personalized preference information of the target object (i.e., object preference information) can be determined, such as the visual style of the favorite songs (retro, cyberpunk), the preference for bullet screen reaction speed, historical interaction data, etc.

[0051] In this embodiment, the method of generating and adjusting the barrage rendering instructions corresponding to the multi-level emotion grid can be determined according to the actual situation, and this embodiment does not limit it.

[0052] In this embodiment, the process by which the bullet screen generation device adjusts a multi-level emotion grid based on object profile information to obtain an adjusted multi-level emotion grid includes: determining object reaction speed information, bullet screen effect preference information, and object style information based on object profile information; adjusting sub-grids in the multi-level emotion grid based on object reaction speed information to obtain adjusted sub-grids; adjusting micro-grids in the multi-level emotion grid based on bullet screen effect preference information to obtain adjusted micro-grids; adjusting the main grid in the multi-level emotion grid based on object style information to obtain adjusted main grids; and determining the adjusted multi-level emotion grid based on the adjusted sub-grids, adjusted micro-grids, and adjusted main grids.

[0053] In this embodiment of the application, the object profile information includes object reaction speed information, bullet screen effect preference information and object style information, and the object reaction speed information, bullet screen effect preference information and object style information can be obtained from the object profile information.

[0054] In this embodiment, the process of adjusting sub-grids in a multi-level emotion grid based on object reaction speed information to obtain adjusted sub-grids includes adjusting the beat information in the sub-grids using object reaction speed information to obtain adjusted sub-grids. Other methods can also be used to adjust sub-grids in a multi-level emotion grid based on object reaction speed information. The specific implementation method can be determined according to the actual situation, and this embodiment does not limit this approach.

[0055] For example, the process of adjusting the beat information in the sub-grid using the object's reaction speed information to obtain the adjusted sub-grid is shown in formula (3): (3) It should be noted that, For the object's reaction speed information, This refers to the beat information within a sub-grid (i.e., the sub-grid).

[0056] In this embodiment, the process of adjusting the microgrids in a multi-level emotion grid based on bullet screen effect preference information to obtain the adjusted microgrids includes determining the product of the microgrids and the bullet screen effect preference information to obtain the adjusted microgrids. Alternatively, other methods can be used to adjust the microgrids in a multi-level emotion grid based on the object's bullet screen effect preference information. The specific implementation method can be determined according to the actual situation, and this embodiment does not limit this approach.

[0057] For example, the process of adjusting the microgrids in the multi-level emotion grid based on the bullet screen effect preference information to obtain the adjusted microgrid is shown in formula (4): (4) It should be noted that, The sensitivity preference for special effects in the object's profile information (i.e., the preference information for bullet screen special effects, such as 0 for minimalist style and 1.5 for strong visual stimulation) is the basic number of special effects, which is the microgrid.

[0058] In this embodiment, the process of adjusting the main grid in a multi-level sentiment grid based on object style information to obtain the adjusted main grid includes fusing the object style information with the main grid to obtain the adjusted main grid. Other methods can also be used to adjust the main grid in a multi-level sentiment grid based on object style information. The specific implementation method can be determined according to the actual situation, and this embodiment does not limit this approach.

[0059] In this embodiment of the application, the object style information ( ) and main grid ( The process of fusing the modulated main grid sentiment scores is shown in formula (5): (5) It should be noted that 0.6 is the base score for the main grid (i.e., the main grid). The example weighting coefficient is 0.4, which can be adjusted according to the actual situation; 0.4 is the example weighting coefficient for object style information, which can be adjusted according to the actual situation.

[0060] It should be noted that the object style information is a bias item set according to the user's style preferences, such as -0.1 for retro style and +0.2 for rock style. It is used to fine-tune the emotional tone of the main grid during the dynamic modulation stage. The base score of the main grid is an objective tone score generated after integrating the musical valence and the emotional polarity of the lyrics in the early stage.

[0061] In the embodiments of this application, the adjusted multi-level emotion grid includes adjusted sub-grids, adjusted micro-grids, and adjusted main grids.

[0062] In this embodiment, the adjusted multi-level sentiment grid can also be determined based on the adjusted sub-grids, micro-grids, and the adjusted main grid. That is, the adjusted multi-level sentiment grid includes the adjusted sub-grids, micro-grids, and the adjusted main grid. The specific method for determining the adjusted multi-level sentiment grid can be determined according to the actual situation, and this embodiment does not limit it.

[0063] Understandably, this application aims to address the problems of weak correlation between existing bullet screen systems and music content, coarse granularity of emotional expression, and low user personalization adaptability. Through multimodal feature fusion, dynamic generation of a three-level emotional grid, and real-time intervention based on user profiles, it achieves a high degree of synchronization and personalized presentation of bullet screen content with music in terms of semantics, emotion, and visual style. This application introduces a dynamic weight modulator, using user profiles (such as style preferences, bullet screen effect preferences, and reaction speed) as key variables to apply to the emotional grid in real time. For example, if a "cyberpunk" enthusiast is watching the same song, their bullet screen will automatically adjust to neon tones, fast-paced animations, and futuristic particle effects. This solves the problem of "one-size-fits-all" in existing technologies, achieving a highly personalized visual experience. Bullet screens are no longer a nuisance, but rather a "companion" that aligns with the user's aesthetics and enhances immersion, significantly reducing the probability of users turning off bullet screens.

[0064] S104. Generate bullet comments that match the target audio stream based on bullet comment rendering instructions.

[0065] In this embodiment of the application, after the bullet screen generation device generates bullet screen rendering instructions corresponding to the target audio stream based on the multi-level emotion grid, it generates bullet screens that match the target audio stream based on the bullet screen rendering instructions.

[0066] In this embodiment of the application, the process of the bullet screen generation device generating bullet screens that match the target audio stream based on bullet screen rendering instructions includes: obtaining target bullet screen information corresponding to the target audio stream; and generating bullet screens that match the target audio stream based on the bullet screen rendering instructions and the target bullet screen information.

[0067] In this embodiment, the target bullet screen information can be obtained from user input, from a database, or from other devices. The specific method of obtaining the target bullet screen information can be determined according to the actual situation, and this embodiment does not limit it.

[0068] It should be noted that the target bullet screen information can be text information, character information, emoticon information, or other forms of information. The specific target bullet screen information can be determined according to the actual situation, and this application embodiment does not limit it in this way.

[0069] In this embodiment, the adjusted sub-grid and adjusted micro-grid (or micro-grid) contain the rendering time of the target bullet screen information, and the adjusted main grid includes rendering effects. After determining the adjusted multi-level emotion grid based on the adjusted sub-grid, adjusted micro-grid, and adjusted main grid, and generating the bullet screen rendering instruction corresponding to the target audio stream based on the multi-level emotion grid, the bullet screen rendering instruction includes rendering effects and rendering time. Bullet screens matching the target audio stream can be generated based on rendering effects, rendering time, and the obtained target bullet screen information.

[0070] It should be noted that the method of generating bullet comments that match the target audio stream based on the rendering effects, rendering time, and target bullet comment information indicated by the bullet comment rendering command can be determined according to the actual situation, and this application embodiment does not limit this.

[0071] Understandably, this application ensures that the triggering of bullet comments and visual effects is strictly aligned with the key beat points of the music (such as drum beats and accents) through precise timestamps generated by the raster and optimization of the rendering engine, and can automatically enhance the visual intensity during climaxes such as the chorus. This creates an ultimate "audio-visual synchronization" experience, where the visual impact perfectly matches the rhythm of the music, bringing users an unprecedented sensory enjoyment.

[0072] In this application embodiment, a schematic diagram of an exemplary bullet screen generation structure is shown below. Figure 2As shown, it includes an input layer, a feature extraction and analysis layer, a core processing layer, and an output and rendering layer. Specifically, the input layer includes a user profile database and a music input source; the feature extraction and analysis layer includes a Jukebox music feature extraction module and a CLAP lyrics semantic analysis module; the core processing layer includes a three-level sentiment grid generation module and a dynamic weight modulator; and the output and rendering layer includes a bullet screen rendering engine and a user interface. The music input source receives the target audio stream, and the user profile database provides object profile information for the target. The Jukebox music feature extraction module obtains the target audio stream from the music input source and extracts music features from it, resulting in acoustic features (such as beat (BPM), swing, timbre, loudness, arousal, and valence). This structured data is then transmitted to the three-level sentiment grid generation module in the core processing layer. The CLAP lyrics semantic analysis module obtains the target audio stream from the music input source and performs natural language processing on it to obtain lyric features (such as the emotional polarity of the lyrics (i.e., lyric sentiment information), keyword information, and syllable stress information). This structured data is also transmitted to the three-level sentiment grid generation module in the core processing layer. The three-level sentiment grid generation module fuses acoustic and lyric features to construct a multi-layered, dynamic temporal grid (i.e., a multi-level sentiment grid), which guides the timing and style of bullet comments generation. The dynamic weight modulator serves as a bridge between technology and users. It receives a three-level sentiment grid (i.e., a multi-level sentiment grid) as a base template and retrieves user preferences (which can be obtained based on the target object's profile information) from a user profile database as modulation parameters. Based on user preferences (such as style preferences, reaction speed, etc.), it dynamically adjusts the grid's weights and parameters, generating the final rendering instructions, which are then sent to the bullet screen rendering engine. Upon receiving the rendering instructions, the bullet screen rendering engine renders the bullet screen text, visual effects (such as particles and lighting effects), and animations at the correct time points (spatiotemporal alignment). The user interface ultimately presents the visual content generated by the bullet screen rendering engine to the user and receives real-time user interactions (such as likes, comments, parameter adjustments, etc.). This interaction data can be used to update the user profile database, forming a closed loop.

[0073] In this application embodiment, an exemplary flowchart of a bullet screen generation method is shown as follows: Figure 3As shown: When a target audio stream (input audio / lyrics) is received, the feature extraction and analysis layer starts immediately. The Jukebox music feature extraction module processes the audio signal and outputs the acoustic features of the music. For example, for a lyrical folk song, the extracted features may be BPM=72, swing=0.2, valence=0.7. At the same time, the CLAP lyric semantic analysis module analyzes the lyric "this windy day", identifies that the text emotion tends to be positive (i.e., lyric emotion information), and locates the accented syllable position (i.e., syllable stress information) such as stress=[0.8, 0.3]. The three-level emotion grid generation module in the core processing layer receives the features (extracts multi-modal features) transmitted from the feature extraction and analysis layer, and generates a three-level emotion grid (i.e., multi-level emotion grid) based on the multi-modal features, including a main grid, a sub-grid, and a micro-grid. The core processing layer also acquires object preference information (user preference) based on the object portrait information (user portrait) of the target object, so that the dynamic weight modulator adjusts the multi-level emotion grid based on the object preference information (dynamic weight modulation, that is, user preference weighting, to adjust grid parameters) to obtain an adjusted multi-level emotion grid. After the modulation is completed, a corresponding bullet screen rendering instruction is generated based on the adjusted multi-level emotion grid (that is, a rendering instruction is generated), and the rendering instruction is transmitted to the bullet screen rendering engine in the rendering and presentation layer. The bullet screen rendering engine executes the instruction to ensure that the appearance timing of all visual elements (bullet screens, special effects) is strictly aligned with the music beat points (for example, the technical requirement error is within ±20ms), so as to achieve precise spatiotemporal synchronization, and finally present a personalized bullet screen experience perfectly integrated with the music on the user interface.

[0074] As an example, the process of adjusting the multi-level emotion grid based on the object portrait information to obtain the adjusted multi-level emotion grid includes: User style intervention: If the user is a "retro enthusiast", a retro style template can be applied. For example, the emotional polarity score is adjusted through the following formula to affect the final visual effect, as shown in Formula (5): where, is the weight of the retro style, which will make the bullet screen color tend to amber and the font use vinyl record texture. 0.6 is an exemplary weight coefficient of the basic score of the main grid, and 0.4 is an exemplary weight coefficient of

[0075] User reaction speed intervention: The user's reaction speed preference react_speed (for example, 0.8) will adjust the trigger timing of special effects to achieve personalized advance or delay, as shown in Formula (3): where, is the beat information in the sub-grid (that is, the sub-grid).

[0076] For example, a user with a "retro enthusiast" profile is listening to a song that has been analyzed as having "folk ballad" characteristics.

[0077] Step 1: Multimodal Feature Input Parallel reception and processing of two input signals from music and the user: Music feature input: This application does not simply classify songs as "folk ballads," but rather uses its built-in feature extraction module to deconstruct the audio into a set of quantifiable, multi-dimensional feature vectors. Specifically, it uses the Jukebox music feature extraction module to analyze acoustic features, outputting: {BPM: 72, arousal: 0.3, valence: 0.7, swing: 0.2}.

[0078] The CLAP lyrics semantic analysis module analyzes the lyrics content and outputs: {Lyrics Emotion: Positive}.

[0079] User profile input: Read the current user's personal preference parameters from the user profile database.

[0080] The profile parameters for this "retro enthusiast" user are: {style_weight: -0.3 (this value is negative, indicating a preference for retro, nostalgic, and other non-exciting styles), reaction_speed: 0.9 (less than 1, indicating a preference for slightly slower visual feedback)}.

[0081] Step 2: Generating the three-level emotion raster and determining native rendering instructions After receiving the music features, the three-level emotion grid generation module begins to construct the "native template" of the rendering instructions in layers. This template only reflects the content of the music itself and has not yet been personalized by the user.

[0082] First, determine the "overall atmosphere" based on the main grid: Grid Content: The main grid is responsible for defining the overall mood of a song segment. It combines the high valence of the music (0.7, representing positive emotions) with the positive emotions in the lyrics to create a "warm and pleasant" emotional tone.

[0083] Definite rendering effect: The emotional tone is mapped to a set of basic visual parameters that constitute the overall atmosphere of the bullet comments.

[0084] Main color scheme: defined as warm colors (e.g., off-white, pale yellow).

[0085] Animation form: defined as a smooth, fluid motion trajectory (e.g., moving along a sine curve).

[0086] Basic lighting effects: defined as a soft, non-aggressive halo effect.

[0087] Then, "rhythm synchronization" is determined based on the sub-grid: Grid content: The sub-grid is responsible for resolving the beat and rhythm of the music. Based on the parameter BPM=72, it marks a series of beat times accurate to milliseconds on the timeline.

[0088] Definite rendering effect: These beat points serve as the precise time reference for the bullet screen behavior.

[0089] Timing of appearance: The generation and appearance of each new bullet comment are strictly aligned with a beat point on the sub-grid to ensure that the visual and auditory rhythms are synchronized.

[0090] Movement speed: The slow BPM of 72 determines that the basic movement speed of the bullet comments on the screen is relatively slow.

[0091] Emphasis on accent: If the subgrid further incorporates the accent information of the lyrics, when the lyrics corresponding to the accent are played, the font size can be instantly increased by 10% or the brightness can be increased to emphasize the rhythmic accent of the music.

[0092] Finally, the "detail texture" was determined based on the microgrid. Grid Content: The microgrid focuses on more subtle dynamic changes in the music. It uses the swing parameter of 0.2 to generate fine-tuning instructions for controlling detailed dynamics between two main beat points.

[0093] Definite rendering effect: These fine-tuning instructions add vivid details to the bullet screen, making it more lifelike.

[0094] Motion details: When the bullet screen moves along the main trajectory set by the main grid, it will have an additional slight, non-linear left and right swaying dynamic controlled by the swing parameter, making its movement more "rhythmic" rather than a mechanical uniform translation.

[0095] Detailed effects triggering: The microgrid can also trigger a tiny "starlight" or "ripple" particle effect during the intervals between beats, to correspond to subtle instrumental sounds that may exist in the music (such as triangle, plucked strings, etc.), to achieve a deeper level of synesthesia between sound and image.

[0096] Step 3: User Profile Modulation and Final Rendering At this point, a set of purely music-based "native" rendering instructions has been generated. Subsequently, the dynamic weight modulator module intervenes, using the user's personalized profile parameters to perform real-time "secondary processing" on this set of native instructions.

[0097] Style modulation: Call the formula as shown in formula (5); Calculation process: Substitute the base score of the main grid (0.7) and the user's style weight (-0.3) into the calculation to obtain a final sentiment score adjusted for user preferences.

[0098] Rendering impact: This score will trigger the preset "retro" style template, which will override the main color scheme determined by the main grid from "warm colors" to a more retro "amber color, old paper color", and can add visual effects such as "vinyl record texture" or "noise" to the bullet comments.

[0099] Timing modulation: The formula is as shown in formula (3): Calculation process: baseDelay is the original beat time point in the sub-grid (i.e., the sub-grid), and react_speed is the user's reaction speed preference (0.9).

[0100] Rendering impact: The timing of all bullet comments and effects will be multiplied by a coefficient of 0.9. This means that for a user who prefers slightly slower feedback, the visual effects will have a subtle delay that is just right and matches their personal perception habits.

[0101] Finally, the final rendering instruction set, deeply customized based on the user profile, is sent to the bullet screen rendering engine. The engine strictly follows the timestamps, colors, textures, animation paths, and special effects parameters in the instruction set to render the bullet screen frame by frame onto the screen, thus presenting users with a unique bullet screen experience that is perfectly synchronized with the emotions and rhythm of the music and deeply matches their personal aesthetics.

[0102] For example, the default image's bullet screen effect during the climax is as follows: Figure 4 As shown; the bullet screen effect after switching to the "rock fan" profile picture (comparison image) is as follows. Figure 5 As shown; Figure 4 and Figure 5 Using the same music clip, but with the user profile switched to "rock fan," the color (more reddish), size (larger), and effect frequency of the bullet comments all change significantly, becoming more in line with the rock style. This image visually demonstrates the real-time intervention capability of the user profile in the "user dynamic modulation mechanism" on the rendering results. The effect of the "rock fan" profile with "enhanced" modulation enabled (comparison image) is shown below. Figure 6 As shown, Figure 6 exist Figure 5 Building upon this foundation, users further enhanced the experience, resulting in an extremely exaggerated visual impact from the bullet comments. This demonstrates that users can manually intervene in the automatic "music + profile" results in real time, achieving a final personalized closed loop.

[0103] Understandably, this application addresses the shortcomings of existing bullet screen generation technologies in terms of music relevance, emotional granularity, and user adaptability. Through multimodal music feature hierarchical modeling and real-time user feedback intervention, it achieves improved accuracy in matching bullet screens with music, more flexible emotional mapping of bullet screens, and higher adaptability to user styles. Specifically, it utilizes a three-level emotional grid algorithm (main grid / sub-grid / micro-grid) to achieve multi-dimensional decoupling and dynamic matching of musical emotions; it employs a real-time user profile modulation mechanism to dynamically adjust rendering parameters based on user behavior and preferences, improving the matching degree between bullet screen effects and user styles; and it uses spatiotemporal alignment rendering technology to ensure frame-level synchronization (±20ms) between bullet screen effects and music beats. Experiments show that this application can improve the accuracy of bullet screen-music matching by 40%, reduce the probability of users closing bullet screens by 37%, and support cross-scene personalized adaptation (such as virtual live streaming and online concerts).

[0104] Understandably, the bullet screen generation device analyzes the target audio stream to determine its acoustic and lyric features. Based on these features, it determines a multi-level emotional grid for the target audio stream and then generates a bullet screen rendering instruction based on this grid. This instruction is determined based on the acoustic and lyric features of the target audio stream, not just the rhythm of the song. This allows the device to render bullet screens that match the target audio stream, thus improving the matching between the rhythm of the song and the bullet screen.

[0105] Based on the same inventive concept as the above-mentioned bullet screen generation method, this application provides a bullet screen generation device 1, corresponding to a bullet screen generation method; Figure 7 A schematic diagram of the composition structure of a bullet screen generation device provided in this application embodiment. Figure 1 The bullet screen generation device 1 may include: Analysis unit 11 is used to analyze the target audio stream upon receiving it, and obtain acoustic features and lyric features; The determining unit 12 is used to determine a multi-level emotion grid based on the acoustic features and the lyrics features; The generation unit 13 is used to generate a barrage rendering instruction corresponding to the target audio stream based on the multi-level emotion grid; and to generate a barrage that matches the target audio stream based on the barrage rendering instruction.

[0106] In some embodiments of this application, the determining unit 12 is used to determine a main grid based on the valence information in the acoustic features and the emotional information of the lyrics in the lyrics features; to determine a sub-grid based on the beat information in the acoustic features and the syllable accent information in the lyrics features; to determine a micro-grid based on the swinging information in the acoustic features; and to determine the multi-level emotional grid based on the main grid, the sub-grid, and the micro-grid.

[0107] In some embodiments of this application, the apparatus further includes an acquisition unit and an adjustment unit; The acquisition unit is used to acquire object profile information of the target object; the target object is an object used to receive the target audio stream; The adjustment unit is used to adjust the multi-level emotion grid based on the object profile information to obtain the adjusted multi-level emotion grid. The generation unit 13 is used to generate bullet screen rendering instructions corresponding to the adjusted multi-level emotion grid.

[0108] In some embodiments of this application, the determining unit 12 is used to determine object reaction speed information, bullet screen effect preference information, and object style information based on the object profile information; and to determine the adjusted multi-level emotion grid based on the adjusted sub-grid, the adjusted micro-grid, and the adjusted main grid. The adjustment unit is used to adjust the sub-grids in the multi-level emotion grid based on the object reaction speed information to obtain the adjusted sub-grids; adjust the micro-grids in the multi-level emotion grid based on the bullet screen effect preference information to obtain the adjusted micro-grids; and adjust the main grid in the multi-level emotion grid based on the object style information to obtain the adjusted main grid.

[0109] In some embodiments of this application, the acoustic features include: beat information, swing information, and valence information; The lyric features include emotional information and syllable stress information.

[0110] In some embodiments of this application, the apparatus further includes an extraction unit and a processing unit; The extraction unit is used to extract musical features from the target audio stream to obtain the acoustic features; The processing unit is used to perform natural language processing on the target audio stream to obtain the lyrics features.

[0111] In some embodiments of this application, the acquisition unit is used to acquire target bullet screen information corresponding to the target audio stream; The generation unit 13 is used to generate a bullet screen that matches the target audio stream based on the bullet screen rendering instruction and the target bullet screen information.

[0112] It should be noted that, in practical applications, the analysis unit 11, determination unit 12 and generation unit 13 mentioned above can be implemented by the processor 21 on the electronic device, specifically by a CPU (Central Processing Unit), MPU (Microprocessor Unit), DSP (Digital Signal Processor) or Field Programmable Gate Array (FPGA), etc.; the data storage mentioned above can be implemented by the memory 22 on the electronic device.

[0113] This application also provides an electronic device, such as... Figure 8 As shown, the electronic device includes a processor 21, a memory 22, and a communication bus 23. The memory 22 communicates with the processor 21 through the communication bus 23. The memory 22 stores programs executable by the processor 21. When the program is executed, the processor 21 executes the bullet screen generation method as described above.

[0114] In practical applications, the aforementioned memory 22 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 21.

[0115] This application provides a computer-readable storage medium having a computer program thereon, which, when executed by a processor 21, implements the bullet screen generation method as described above.

[0116] This application also provides a computer program product, including a computer program that can be executed by the processor 21 in the bullet screen generation device to complete the steps described in the aforementioned bullet screen generation method.

[0117] Understandably, the bullet screen generation device analyzes the target audio stream to determine its acoustic and lyric features. Based on these features, it determines a multi-level emotional grid for the target audio stream and then generates a bullet screen rendering instruction based on this grid. This instruction is determined based on the acoustic and lyric features of the target audio stream, not just the rhythm of the song. This allows the device to render bullet screens that match the target audio stream, thus improving the matching between the rhythm of the song and the bullet screen.

[0118] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0119] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0122] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for generating bullet comments, characterized in that, The method includes: Upon receiving the target audio stream, the target audio stream is analyzed to obtain acoustic features and lyric features; Based on the acoustic features and the lyric features, a multi-level emotion grid is determined; Based on the multi-level emotion grid, generate the bullet screen rendering instructions corresponding to the target audio stream; Based on the bullet screen rendering instructions, bullet screens matching the target audio stream are generated.

2. The method according to claim 1, characterized in that, The determination of a multi-level emotion grid based on the acoustic features and the lyric features includes: The main grid is determined based on the valence information in the acoustic features and the emotional information in the lyrics features; Based on the beat information in the acoustic features and the syllable stress information in the lyrics features, a sub-grid is determined; The microgrid is determined based on the swaying sensation information in the acoustic features. The multi-level emotion grid is determined based on the main grid, the sub-grids, and the micro-grids.

3. The method according to claim 1, characterized in that, The step of generating the bullet screen rendering instructions corresponding to the target audio stream based on the multi-level emotion grid includes: Obtain object profile information of the target object; the target object is the object used to receive the target audio stream. The multi-level emotion grid is adjusted based on the object profile information to obtain the adjusted multi-level emotion grid. Generate bullet screen rendering instructions corresponding to the adjusted multi-level emotion grid.

4. The method according to claim 3, characterized in that, The step of adjusting the multi-level sentiment grid based on the object profile information to obtain the adjusted multi-level sentiment grid includes: Based on the object profile information, determine the object's reaction speed information, bullet screen effect preference information, and object style information; Based on the object's reaction speed information, the sub-grids in the multi-level emotion grid are adjusted to obtain the adjusted sub-grids; Based on the bullet screen effect preference information, the microgrids in the multi-level emotion grid are adjusted to obtain the adjusted microgrids; The main grid in the multi-level emotion grid is adjusted based on the object style information to obtain the adjusted main grid; Based on the adjusted sub-grid, the adjusted micro-grid, and the adjusted main grid, the adjusted multi-level sentiment grid is determined.

5. The method according to claim 1, characterized in that, The acoustic features include: beat information, swing information, and valence information; The lyric features include emotional information and syllable stress information.

6. The method according to claim 1, characterized in that, The analysis of the target audio stream to obtain acoustic features and lyric features includes: Musical features are extracted from the target audio stream to obtain the acoustic features; Natural language processing is performed on the target audio stream to obtain the lyrics features.

7. The method according to claim 1, characterized in that, The step of generating bullet comments that match the target audio stream based on the bullet comment rendering instructions includes: Obtain the target bullet screen information corresponding to the target audio stream; Based on the bullet screen rendering instructions and the target bullet screen information, a bullet screen matching the target audio stream is generated.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images