Bullet screen processing method
Through real-time interactive information processing and computing model screening, the barrage display in the live broadcast room is optimized, which solves the display problem caused by too many barrage, improves the user experience, and realizes efficient screening and display of barrage.
Patent Information
- Application Number
- CN202411523933.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-10-30
AI Technical Summary
In live broadcast applications, when there are too many barrages in the live broadcast room, it may not be displayed in time, which will affect the user experience. Especially when the popularity is high, it is difficult for the existing technology to effectively optimize the barrage display effect.
A barrage processing method is adopted to obtain real-time interactive information, filter and correct undisplayed barrage based on the calculation model to form an optimized barrage sequence, including similar barrage aggregation, key barrage markings and non-compliant barrage Filter to improve the information content and user interaction experience displayed on the barrage.
It realizes automatic processing of barrage in high-popular live broadcast rooms, aggregating similar content, deleting non-compliant barrage, highlighting key content, comprehensively optimizing the barrage display effect, and improving user interaction experience.
Smart Images

Figure CN119094838A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of electronic digital data processing, and in particular to a bullet screen processing method. Background Art
[0002] In live broadcast applications, users watching live broadcasts can interact with the anchor through bullet comments. However, due to the limited display interface of the live broadcast room, if the live broadcast is very popular, too many bullet comments may not be displayed in time, which may also cause the viewing experience of the live broadcast room to decline, affecting the user experience. Summary of the invention
[0003] The embodiment of the present application provides a barrage processing method, which can intelligently plan and process the barrage in the live broadcast room, adopt different processing methods for different types of barrage, optimize the barrage display effect of the live broadcast room, improve the effective information content in the barrage in the live broadcast room, and promote live broadcast room interaction based on barrage.
[0004] According to one aspect of an embodiment of the present application, a bullet screen processing method is provided, the method comprising: Obtaining the interaction information corresponding to the target live broadcast room in a preset time period, wherein the interaction information includes the number of real-time viewers, the number of real-time barrages, and the number of real-time barrage words; Performing data processing on the interaction information based on a first calculation model to determine the interaction heat; When the interaction heat reaches a preset heat threshold, the second calculation model is started, and the second calculation model is triggered to screen target bullet screens that meet the preset bullet screen requirements from the first preset number of candidate bullet screens in the first bullet screen sequence, the first bullet screen sequence is used to store bullet screens that are acquired in real time but have not yet been displayed in the target live broadcast room, and the candidate bullet screens are bullet screens that have not been screened by the second calculation model; Using the first preset number of candidate bullet screens to form a second bullet screen sequence, and modifying the second bullet screen sequence based on the target bullet screen to obtain a modified result; The second barrage sequence is replaced by the correction result and displayed in the target live broadcast room.
[0005] In an exemplary embodiment, the preset barrage requirements include: similar barrage requirements, key barrage requirements and specific barrage requirements, the second computing model includes a first embedding model, a second embedding model and a special barrage filtering model, the second embedding model has professional knowledge of the professional field corresponding to the target live broadcast room, and the second computing model performs the following operations: The first embedding model extracts common features from each of the candidate bullet comments to obtain a common feature pool, and based on the common feature pool, screens out a first bullet comment group consisting of bullet comments that meet the similar bullet comment requirements; The second embedding model extracts features from each of the candidate bullet comments to obtain a professional feature pool, and based on the professional feature pool, screens out a second bullet comment group consisting of bullet comments that meet the requirements of the key bullet comments; The special barrage filtering model extracts special words from each of the candidate barrages, and screens out a third barrage group consisting of barrages that meet the requirements of the specific barrage; Any barrage in the first barrage group, the second barrage group and the third barrage group belongs to the target barrage.
[0006] In an exemplary embodiment, the step of selecting a first barrage group consisting of barrages meeting the similar barrage requirements based on the general feature pool includes: Clustering the universal features in the universal feature pool to obtain multiple feature clusters; For any feature cluster, if the feature distance between any two common features in the feature cluster is less than a first distance threshold, and the number of common features in the feature cluster reaches a first number, then the barrages corresponding to the common features in the feature cluster respectively constitute a corresponding first barrage group; Correspondingly, the modifying the second barrage sequence based on the target barrage includes: For any of the first barrage groups, determine the first barrage with the shortest feature distance to the cluster center in the feature cluster corresponding to the first barrage group; generate a fused barrage, the barrage content of the fused barrage is the content of the first barrage, and the publishing account of the fused barrage includes the publishing account of each barrage of the first barrage group; delete each barrage of the first barrage group in the second barrage sequence, and insert the fused barrage into the head of the second barrage group.
[0007] In an exemplary embodiment, the key bullet comment requirement includes a plurality of source texts, and the method further includes: inputting each of the source texts into the second embedding model to obtain corresponding source text professional features, and screening out a second bullet comment group consisting of bullet comments that meet the key bullet comment requirement based on the professional feature pool, including: For each professional feature in the professional feature pool, the following operations are performed: calculating the feature distance between the professional feature and each of the professional features of the source text; if the minimum feature distance is less than a second distance threshold, determining that the bullet screen corresponding to the professional feature meets the key bullet screen requirement; determining the source text corresponding to the professional feature of the source text corresponding to the minimum feature distance as the associated source text of the bullet screen corresponding to the professional feature; Correspondingly, the modifying the second barrage sequence based on the target barrage includes: For any barrage in the second barrage sequence that is hit by the second barrage group, a special mark is added to the barrage.
[0008] In an exemplary embodiment, the key bullet comment requirement further includes a reply text corresponding to each of the source texts, and the method further includes performing the following operations on any bullet comment in the second bullet comment group: Based on the key bullet screen requirement, determining an associated reply text corresponding to an associated source text corresponding to the bullet screen; Displaying a first prompt message and a first control on a target client, the target client being a client corresponding to the publishing account of the barrage, the first prompt message being used to indicate that the host of the target live broadcast room has interacted with the publishing account through the barrage; When the first control is triggered, the corresponding associated reply text is displayed on the target client.
[0009] In an exemplary embodiment, the method further comprises: Displaying a second control on the target client; When the second control is triggered, a reply voice is played on the target client, where the reply voice is automatically generated by a voice generation model based on the associated reply text and conforms to the timbre of the host.
[0010] In an exemplary embodiment, the modifying the second barrage sequence based on the target barrage includes: Any barrage in the second barrage sequence that is hit by the third barrage group is deleted.
[0011] In an exemplary embodiment, the speech generation model is trained by the following method: Obtain a pre-trained speech decoder, wherein the speech decoder is trained based on a general text-speech pair, and the speech decoder includes a speech encoder and a speech decoder; Obtaining a training sample recorded by the anchor, wherein the training sample includes a sample text and a corresponding sample voice, and both the sample text and the sample voice have the professional knowledge corresponding to the anchor; Inputting the sample text into the second embedding model to obtain sample text features; Inputting the sample speech into the speech encoder to obtain sample speech features; The connector is trained based on the sample text features and the sample voice features; the step includes: inputting the sample text features into the connector to obtain predicted voice features; adjusting the parameters of the connector based on the difference between the predicted voice features and the corresponding sample voice features, so that the voice information output by the connector conforms to the host's voice; The speech generation model is composed of the second embedding model, the trained connector and the speech decoder.
[0012] In an exemplary embodiment, the method further includes performing the following operations based on the speech generation model: Reusing the second embedding model to output text features corresponding to the associated reply text; Inputting the text features into a connector in the speech generation model to obtain corresponding speech features; The speech feature is input into the speech decoder in the speech generation model to obtain the reply speech.
[0013] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above-mentioned barrage processing method.
[0014] The technical solution provided in the embodiments of the present application can bring the following beneficial effects: This embodiment proposes a barrage processing method, which can automatically process the barrage to be broadcast when the target live broadcast room is popular, aggregate similar barrages, and delete non-compliant barrages to reduce the number of barrages. By identifying key barrages, the key content of the barrage can be highlighted, thereby achieving the purpose of comprehensively optimizing the barrage display effect and enhancing the user's interactive experience based on the barrage. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0016] Figure 1 This is a flow chart of a bullet comment processing method provided by an embodiment of the present application; Figure 2 It is a schematic diagram of a first bullet chat group generation method provided by an embodiment of the present application; Figure 3 is a flow chart of a second bullet chat group generation method provided by an embodiment of the present application; Figure 4 This is a flow chart of a bullet screen interaction method provided by an embodiment of the present application; Figure 5 This is a flow chart of a speech generation model training method provided by an embodiment of the present application. DETAILED DESCRIPTION
[0017] Please refer to Figure 1 , which shows a flow chart of a bullet screen processing method provided by an embodiment of the present application. The method can be applied to a live broadcast system. The method may include the following steps: S101. Obtain the interaction information corresponding to the target live broadcast room in a preset time period, wherein the interaction information includes the real-time number of viewers, the number of real-time barrages, and the number of real-time barrage words.
[0018] The target live broadcast room is any live broadcast room in the live broadcast system that broadcasts live. The embodiment of the present application does not limit the preset time period, for example, it can be a few minutes, or more than ten minutes, and can be set according to actual conditions. The real-time number of viewers, the number of real-time barrages, and the number of real-time barrage words can be continuously sampled within the preset time period to obtain the first sequence, second sequence, and third sequence that respectively correspond to the changing trends of the real-time number of viewers, the number of real-time barrages, and the number of real-time barrage words. Specifically, the interactive information corresponding to the preset time period may include the first sequence, the second sequence, and the third sequence.
[0019] S102. Perform data processing on the interaction information based on the first calculation model to determine the interaction heat.
[0020] The method of obtaining the first calculation model does not constitute an implementation obstacle. Various models for determining interaction heat in the prior art can be used as long as the interaction heat can be fitted based on the first sequence, the second sequence and the third sequence. For example, the first sequence, the second sequence and the third sequence can be directly averaged, each average is subtracted from the corresponding average reference value, and the obtained differences are weighted and summed to obtain the interaction heat. The average reference value and the weight can be set according to the actual situation and do not constitute an implementation obstacle.
[0021] S103. When the interaction heat reaches a preset heat threshold, the second calculation model is started, triggering the second calculation model to screen target barrages that meet the preset barrage requirements from the first preset number of candidate barrages in the first barrage sequence, the first barrage sequence is used to store barrages that are acquired in real time but have not yet been displayed in the target live broadcast room, and the candidate barrages are barrages that have not been screened by the second calculation model.
[0022] The preset heat threshold is not limited in this application and can be set according to actual conditions. If the interaction heat reaches the preset heat threshold, it can be considered that when the interaction heat is too high, the user has high requirements for the display of the barrage of the target live broadcast room being watched, and there are many barrages. It is necessary to optimize the display results and interaction methods of the barrage. In this case, the second calculation model can be started.
[0023] The second calculation model can process a preset number of bullet screens each time. The embodiment of the present application does not limit the preset number and can be set according to actual conditions.
[0024] S104. Use the previously preset number of candidate barrages to form a second barrage sequence, and modify the second barrage sequence based on the target barrage to obtain a modified result; replace the second barrage sequence with the modified result and display it in the target live broadcast room.
[0025] After processing based on the second calculation model, a correction result can be obtained, and the correction result is used to replace the display of the second barrage sequence. For example, the first N (N is an integer greater than 100) candidate barrages in the first barrage sequence form the second barrage sequence. Then, when displaying, the first to N barrages in the first barrage sequence will not be displayed, but the correction result will be displayed. The correction result can merge similar barrages, delete non-compliant barrages and highlight key barrages. Therefore, the display effect of the correction result is better than the original display effect obtained by directly displaying the first to N barrages in the first barrage sequence.
[0026] In one embodiment, the preset barrage requirements include: similar barrage requirements, key barrage requirements and specific barrage requirements, which are used to screen similar barrages, key barrages and barrages that need to be filtered, respectively, and the second computing model includes a first embedding model, a second embedding model and a special barrage filtering model. The embodiment of the present application does not limit the acquisition process of the first embedding model, the second embedding model and the special barrage filtering model, and existing technologies can be used without posing an implementation obstacle.
[0027] Among them, the second embedding model is a professional model, and the second embedding model has the professional knowledge of the professional field corresponding to the target live broadcast room. For example, if the target live broadcast room is a legal education live broadcast room, the second embedding model has the professional knowledge in the legal field. If the target live broadcast room is a programming live broadcast room, the second embedding model has the professional knowledge in the computer field.
[0028] In an exemplary embodiment, a new training method for the second embedding model is proposed: (1) Obtain multiple first training samples and multiple second training samples that carry professional knowledge. In the embodiment of the present application, multiple first training samples and multiple second training samples all have professional knowledge corresponding to the target live broadcast room. Specifically, a large amount of document data in the professional field corresponding to the target live broadcast room can be obtained, and core words are extracted from the document data to form a core vocabulary. Extract multiple question-answer pairs from the large amount of document data, and for each question-answer pair, extract the target core words in the answer that are hit by the core vocabulary to form a question-core word pair. Use a part of the question-core word pairs as the first training samples, and another part of the question-core word pairs as the second training samples.
[0029] (2) Copy the first embedding model to obtain a third embedding model. Input each of the first training samples into the third embedding model to obtain a first question feature and a corresponding first core word feature, and adjust the parameters of the third embedding model to obtain a first intermediate model with the training goal of narrowing the distance between the two features. The present embodiment of the application does not elaborate on the parameter adjustment process based on the training goal, and does not constitute an implementation obstacle.
[0030] (3) Input each question in the second training sample into the first intermediate model to obtain a corresponding second question feature. Input each core word in the core word library into the first intermediate model to obtain a corresponding core word reference feature. Select the 10 target core word reference features that are closest to the second question feature from each of the core word reference features. If the core word set formed by the core words corresponding to each of the 10 target core word reference features includes the core word in the second training sample, add a corresponding preset tag to the corresponding second training sample.
[0031] (4) Calculate the total number N1 of all second training samples and the number N2 of second training samples with preset labels, use N1 / N2 as the weight corresponding to the second training samples with preset labels, and use 1 as the weight corresponding to the second training samples without preset labels.
[0032] (5) Input all the second training samples into the first intermediate model, and for each second training sample, obtain the corresponding third question feature and the corresponding second core word feature, and use the product of the feature distance between the third question feature and the corresponding second core word feature and the weight corresponding to the second training sample as the loss component of the second training sample; accumulate the loss components of each second training sample to obtain the total loss, and adjust the first intermediate model parameters based on the total loss to obtain the second intermediate model. The embodiment of the present application does not elaborate on the parameter adjustment process, and does not constitute an implementation obstacle. The embodiment of the present application does not limit the method for calculating the feature distance, and existing technologies can be used.
[0033] (6) For each core word in the core word library, the corresponding explanation text is searched in the document data to form a core word-explanation text pair. Each core word-explanation text pair is input into the second intermediate model to obtain the features corresponding to the core word and the features corresponding to the explanation text. The parameters of the second intermediate model are adjusted to obtain the second embedding model with the training goal of narrowing the distance between the two features. The embodiment of the present application does not elaborate on the parameter adjustment process based on the training goal, and it does not constitute an implementation obstacle.
[0034] In the embodiment of the present application, the second computing model performs the following operations: (1) The first embedding model extracts common features from each of the candidate bullet comments to obtain a common feature pool, and based on the common feature pool, screens out a first bullet comment group consisting of bullet comments that meet the similar bullet comment requirements.
[0035] The first embedding model is a model that is trained with general corpus to extract features from text. It can obtain the general features corresponding to each candidate barrage, thus forming a general feature pool.
[0036] Please refer to Figure 2 , which shows a schematic diagram of a method for generating a first bullet screen group. The first bullet screen group consisting of bullet screens that meet the similar bullet screen requirements is selected based on the general feature pool, including: S201. Clustering the general features in the general feature pool to obtain multiple feature clusters.
[0037] The embodiments of the present application do not limit the clustering algorithm and existing technologies may be used.
[0038] S202. For any feature cluster, if the feature distance between any two common features in the feature cluster is less than the first distance threshold, and the number of common features in the feature cluster reaches a first number, then a corresponding first barrage group is formed based on the barrages corresponding to each of the common features in the feature cluster.
[0039] The embodiment of the present application does not limit the first distance threshold and the first quantity, and they can be set according to actual conditions.
[0040] Correspondingly, the second barrage sequence is corrected based on the target barrage, including: for any of the first barrage groups, determining the first barrage with the shortest feature distance to the cluster center in the feature cluster corresponding to the first barrage group; generating a fused barrage, the barrage content of the fused barrage is the content of the first barrage, and the publishing account of the fused barrage includes the publishing account of each barrage of the first barrage group; deleting each barrage of the first barrage group in the second barrage sequence, and inserting the fused barrage into the head of the second barrage group.
[0041] (2) The second embedding model extracts features from each of the candidate bullet comments to obtain a professional feature pool, and based on the professional feature pool, screens out a second bullet comment group consisting of bullet comments that meet the requirements of the key bullet comment.
[0042] The key bullet screen requirement includes multiple source texts, and the method further includes: inputting each of the source texts into the second embedding model to obtain corresponding professional features of the source text. Specifically, the key bullet screen requirement includes multiple source texts and a reply text corresponding to each source text, which essentially records some professional communication information in the professional field corresponding to the target live broadcast room, and the source text and the corresponding reply text can form a dialogue in the professional field corresponding to the target live broadcast room.
[0043] Each source text professional feature is a feature that conforms to professional knowledge and is output by a professional model, which is significantly different from the aforementioned general features.
[0044] Please refer to Figure 3 , which shows a flow chart of a method for generating a second bullet screen group. The second bullet screen group consisting of bullet screens that meet the key bullet screen requirements based on the professional feature pool includes: S301. Perform the following operation on each professional feature in the professional feature pool: calculate the feature distance between the professional feature and each professional feature of the source text.
[0045] S302. If the minimum feature distance is less than the second distance threshold, it is determined that the barrage corresponding to the professional feature meets the key barrage requirement; the source text corresponding to the professional feature of the source text corresponding to the minimum feature distance is determined as the associated source text of the barrage corresponding to the professional feature.
[0046] If the minimum feature distance is less than the second distance threshold, it means that the barrage corresponding to the professional feature belongs to the typical interactive information in the professional field and is a very professional and high-quality barrage. Therefore, it meets the key barrage requirements. The source text closest to the actual meaning of the barrage is determined as the associated source text. The minimum feature distance is less than the second distance threshold, indicating that the barrage is highly similar to the associated source text. The embodiment of the present application does not limit the second distance threshold, and it can be set according to actual conditions.
[0047] Correspondingly, the second barrage sequence is modified based on the target barrage, including: for any barrage in the second barrage sequence that is hit by the second barrage group, a special mark is added to the barrage. Special marks are added to high-quality barrages to remind users to watch high-quality barrages and improve the display effect of barrages. The embodiment of the present application does not limit special marks, for example, they can be highlighted, or added with "&" or "*".
[0048] (3) The special barrage filtering model extracts special words from each of the candidate barrages, and selects a third barrage group consisting of barrages that meet the requirements of the specific barrage.
[0049] The special word may be some sensitive words, words that are not suitable to be released. The bullet screen containing the special word meets the specific bullet screen requirements, is included in the third bullet screen group, and should be deleted. The second bullet screen sequence is modified based on the target bullet screen, including: deleting any bullet screen hit by the third bullet screen group in the second bullet screen sequence. The special bullet screen filtering model belongs to the prior art and will not be elaborated. Delete the bullet screen that is not suitable to be sent, reduce junk bullet screens, and improve the bullet screen display effect.
[0050] Any barrage in the first barrage group, the second barrage group and the third barrage group belongs to the target barrage.
[0051] In an exemplary embodiment, the key bullet text requirement also includes the reply text corresponding to each source text, please refer to Figure 4 , which shows a flow chart of a bullet screen interaction method. The interaction method includes performing the following operations on any bullet screen in the second bullet screen group: S401. Based on the key barrage requirement, determine the associated reply text corresponding to the associated source text corresponding to the barrage.
[0052] The key barrage is required to include the reply text corresponding to each source text, so as to determine the associated reply text corresponding to the associated source text corresponding to the barrage.
[0053] S402. Display a first prompt message and a first control on a target client, wherein the target client is a client corresponding to the publishing account of the barrage, and the first prompt message is used to indicate that the host of the target live broadcast room has interacted with the publishing account through the barrage.
[0054] S403. When the first control is triggered, display the corresponding associated reply text on the target client.
[0055] For example, the first prompt information may be "The host has replied to your barrage", and then clicking the first control may show the associated reply text, which may be used as the host's automatic reply to enhance the interaction between the host and the user.
[0056] In some embodiments, the method further includes: displaying a second control on the target client; when the second control is triggered, playing a reply voice on the target client, wherein the reply voice is automatically generated by a voice generation model based on an associated reply text corresponding to the target client and matches the timbre of the host.
[0057] The first prompt information may be "The host has replied to your barrage", and then clicking the second control may allow the user to hear a reply voice played in the tone of the host in the target live broadcast room. The user may feel that the host is making a real-person targeted reply to the user, and the interactive effect is stronger.
[0058] In order to achieve that each live broadcast room can hear the reply voice played by simulating the timbre of the corresponding anchor, it is necessary to train the voice generation model corresponding to each anchor to generate the corresponding reply voice. However, the voice generation models in the prior art are obtained by fine-tuning the general end-to-end model. This fine-tuning process will cause model degradation problems and require a large amount of training data. Using this type of method to train the voice generation model corresponding to each anchor will reduce the model quality, and the cost of obtaining training data is too high. In order to solve this problem, the embodiment of the present application provides a high-quality voice generation model training method.
[0059] Please refer to Figure 5 , which shows a flow chart of a speech generation model training method. The speech generation model is trained by the following method: S501. Obtain a pre-trained speech decoder, wherein the speech decoder is trained based on a common text-speech pair, and the speech decoder includes a speech encoder and a speech decoder.
[0060] This step is prior art and needless to say.
[0061] S502. Obtain training samples recorded by the host, wherein the training samples include sample text and corresponding sample voice, and both the sample text and the sample voice have the professional knowledge corresponding to the host.
[0062] The sample voice is recorded by the anchor, and both the sample text and the sample voice have the professional knowledge corresponding to the anchor, so that the trained model can generate voice in the professional field corresponding to the anchor and have the anchor's timbre.
[0063] S503. Input the sample text into the second embedding model to obtain sample text features.
[0064] The second embedding model is a professional model in the anchor's field, and has a very good text feature extraction effect in the professional field. The second embedding model is used to improve the quality of sample text features.
[0065] S504. Input the sample speech into the speech encoder to obtain sample speech features; and train a connector based on the sample text features and the sample speech features.
[0066] The connector essentially completes the cross-modal conversion of text information to voice information. The connector can be constructed using convolutional neural networks, diffusion models, etc., and does not constitute an implementation obstacle. Specifically, high-quality sample text features are input into the connector to obtain predicted voice features; based on the difference between the predicted voice features and the corresponding sample voice features, the parameters of the connector are adjusted so that the voice information output by the connector is more and more consistent with the host's timbre. The embodiments of the present application do not limit the parameter adjustment method, the difference quantification method, and the parameter adjustment end conditions, and the existing technology can be used.
[0067] S505. The speech generation model is constructed by the second embedding model, the trained connector and the speech decoder.
[0068] The method also includes performing the following operations based on the speech generation model: reusing the second embedding model to output text features corresponding to the associated reply text; inputting the text features into a connector in the speech generation model to obtain corresponding speech features; and inputting the speech features into a speech decoder in the speech generation model to obtain the reply speech.
[0069] Reusing the second embedding model ensures the generation of highly professional and high-quality text features, and the connector can convert text features into speech features. Since the training process of the connector uses sample speech recorded by the host, the speech features output by the connector have the host's timbre. Decoding the speech features into reply speech can make the reply speech have the dual effects of high professionalism and simulated host's timbre.
[0070] Obviously, the training method of the speech generation model avoids fine-tuning and model degradation. Only one connector needs to be trained, which significantly reduces the demand for training data. Moreover, the reply voice generated by it has the dual effects of high professionalism and simulated host voice.
[0071] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. When the at least one instruction, the at least one program, the code set or the instruction set is executed by a processor, the barrage processing method is implemented.
[0072] Specifically, the method comprises: Obtaining the interaction information corresponding to the target live broadcast room in a preset time period, wherein the interaction information includes the number of real-time viewers, the number of real-time barrages, and the number of real-time barrage words; Performing data processing on the interaction information based on a first calculation model to determine the interaction heat; When the interaction heat reaches a preset heat threshold, the second calculation model is started, and the second calculation model is triggered to screen target bullet screens that meet the preset bullet screen requirements from the first preset number of candidate bullet screens in the first bullet screen sequence, the first bullet screen sequence is used to store bullet screens that are acquired in real time but have not yet been displayed in the target live broadcast room, and the candidate bullet screens are bullet screens that have not been screened by the second calculation model; Using the first preset number of candidate bullet screens to form a second bullet screen sequence, and modifying the second bullet screen sequence based on the target bullet screen to obtain a modified result; The second barrage sequence is replaced by the correction result and displayed in the target live broadcast room.
[0073] In an exemplary embodiment, the preset barrage requirements include: similar barrage requirements, key barrage requirements and specific barrage requirements, the second computing model includes a first embedding model, a second embedding model and a special barrage filtering model, the second embedding model has professional knowledge of the professional field corresponding to the target live broadcast room, and the second computing model performs the following operations: The first embedding model extracts common features from each of the candidate bullet comments to obtain a common feature pool, and based on the common feature pool, screens out a first bullet comment group consisting of bullet comments that meet the similar bullet comment requirements; The second embedding model extracts features from each of the candidate bullet comments to obtain a professional feature pool, and based on the professional feature pool, screens out a second bullet comment group consisting of bullet comments that meet the requirements of the key bullet comments; The special barrage filtering model extracts special words from each of the candidate barrages, and screens out a third barrage group consisting of barrages that meet the requirements of the specific barrage; Any barrage in the first barrage group, the second barrage group and the third barrage group belongs to the target barrage.
[0074] In an exemplary embodiment, the step of selecting a first barrage group consisting of barrages meeting the similar barrage requirements based on the general feature pool includes: Clustering the universal features in the universal feature pool to obtain multiple feature clusters; For any feature cluster, if the feature distance between any two common features in the feature cluster is less than a first distance threshold, and the number of common features in the feature cluster reaches a first number, then the barrages corresponding to the common features in the feature cluster respectively constitute a corresponding first barrage group; Correspondingly, the modifying the second barrage sequence based on the target barrage includes: For any of the first barrage groups, determine the first barrage with the shortest feature distance to the cluster center in the feature cluster corresponding to the first barrage group; generate a fused barrage, the barrage content of the fused barrage is the content of the first barrage, and the publishing account of the fused barrage includes the publishing account of each barrage of the first barrage group; delete each barrage of the first barrage group in the second barrage sequence, and insert the fused barrage into the head of the second barrage group.
[0075] In an exemplary embodiment, the key bullet comment requirement includes a plurality of source texts, and the method further includes: inputting each of the source texts into the second embedding model to obtain corresponding source text professional features, and screening out a second bullet comment group consisting of bullet comments that meet the key bullet comment requirement based on the professional feature pool, including: For each professional feature in the professional feature pool, the following operations are performed: calculating the feature distance between the professional feature and each of the professional features of the source text; if the minimum feature distance is less than a second distance threshold, determining that the bullet screen corresponding to the professional feature meets the key bullet screen requirement; determining the source text corresponding to the professional feature of the source text corresponding to the minimum feature distance as the associated source text of the bullet screen corresponding to the professional feature; Correspondingly, the modifying the second barrage sequence based on the target barrage includes: For any barrage in the second barrage sequence that is hit by the second barrage group, a special mark is added to the barrage.
[0076] In an exemplary embodiment, the key bullet comment requirement further includes a reply text corresponding to each of the source texts, and the method further includes performing the following operations on any bullet comment in the second bullet comment group: Based on the key bullet screen requirement, determining an associated reply text corresponding to an associated source text corresponding to the bullet screen; Displaying a first prompt message and a first control on a target client, the target client being a client corresponding to the publishing account of the barrage, the first prompt message being used to indicate that the host of the target live broadcast room has interacted with the publishing account through the barrage; When the first control is triggered, the corresponding associated reply text is displayed on the target client.
[0077] In an exemplary embodiment, the method further comprises: Displaying a second control on the target client; When the second control is triggered, a reply voice is played on the target client, where the reply voice is automatically generated by a voice generation model based on the associated reply text and conforms to the timbre of the host.
[0078] In an exemplary embodiment, the modifying the second barrage sequence based on the target barrage includes: Any barrage in the second barrage sequence that is hit by the third barrage group is deleted.
[0079] In an exemplary embodiment, the speech generation model is trained by the following method: Obtain a pre-trained speech decoder, wherein the speech decoder is trained based on a general text-speech pair, and the speech decoder includes a speech encoder and a speech decoder; Obtaining a training sample recorded by the anchor, wherein the training sample includes a sample text and a corresponding sample voice, and both the sample text and the sample voice have the professional knowledge corresponding to the anchor; Inputting the sample text into the second embedding model to obtain sample text features; Inputting the sample speech into the speech encoder to obtain sample speech features; The connector is trained based on the sample text features and the sample voice features; the step includes: inputting the sample text features into the connector to obtain predicted voice features; adjusting the parameters of the connector based on the difference between the predicted voice features and the corresponding sample voice features, so that the voice information output by the connector conforms to the host's voice; The speech generation model is composed of the second embedding model, the trained connector and the speech decoder.
[0080] In an exemplary embodiment, the method further includes performing the following operations based on the speech generation model: Reusing the second embedding model to output text features corresponding to the associated reply text; Inputting the text features into a connector in the speech generation model to obtain corresponding speech features; The speech feature is input into the speech decoder in the speech generation model to obtain the reply speech.
[0081] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives) or an optical disk, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).
[0082] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.
[0083] In addition, in the specific implementation of this application, account information and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, account permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0084] The above are merely exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for processing barrage, It is characterized in that The method comprises: Obtaining the interaction information corresponding to the target live broadcast room in a preset time period, wherein the interaction information includes the number of real-time viewers, the number of real-time barrages, and the number of real-time barrage words; Performing data processing on the interaction information based on a first calculation model to determine the interaction heat; When the interaction heat reaches a preset heat threshold, the second calculation model is started, and the second calculation model is triggered to screen target bullet screens that meet the preset bullet screen requirements from the first preset number of candidate bullet screens in the first bullet screen sequence, the first bullet screen sequence is used to store bullet screens that are acquired in real time but have not yet been displayed in the target live broadcast room, and the candidate bullet screens are bullet screens that have not been screened by the second calculation model; Using the first preset number of candidate bullet screens to form a second bullet screen sequence, and modifying the second bullet screen sequence based on the target bullet screen to obtain a modified result; The second barrage sequence is replaced by the correction result and displayed in the target live broadcast room.
2. The method according to claim 1, It is characterized in that The preset barrage requirements include: similar barrage requirements, key barrage requirements and specific barrage requirements. The second computing model includes a first embedding model, a second embedding model and a special barrage filtering model. The second embedding model has professional knowledge in the professional field corresponding to the target live broadcast room. The second computing model performs the following operations: The first embedding model extracts common features from each of the candidate bullet comments to obtain a common feature pool, and based on the common feature pool, screens out a first bullet comment group consisting of bullet comments that meet the similar bullet comment requirements; The second embedding model extracts features from each of the candidate bullet comments to obtain a professional feature pool, and based on the professional feature pool, screens out a second bullet comment group consisting of bullet comments that meet the requirements of the key bullet comments; The special barrage filtering model extracts special words from each of the candidate barrages, and screens out a third barrage group consisting of barrages that meet the requirements of the specific barrage; Any barrage in the first barrage group, the second barrage group and the third barrage group belongs to the target barrage.
3. The method according to claim 2, It is characterized in that The step of selecting a first barrage group consisting of barrages meeting the similar barrage requirements based on the general feature pool includes: Clustering the universal features in the universal feature pool to obtain multiple feature clusters; For any feature cluster, if the feature distance between any two common features in the feature cluster is less than a first distance threshold, and the number of common features in the feature cluster reaches a first number, then the barrages corresponding to the common features in the feature cluster respectively constitute a corresponding first barrage group; Correspondingly, the modifying the second barrage sequence based on the target barrage includes: For any of the first barrage groups, determine the first barrage with the shortest feature distance to the cluster center in the feature cluster corresponding to the first barrage group; generate a fused barrage, the barrage content of the fused barrage is the content of the first barrage, and the publishing account of the fused barrage includes the publishing account of each barrage of the first barrage group; delete each barrage of the first barrage group in the second barrage sequence, and insert the fused barrage into the head of the second barrage group.
4. The method according to claim 2, It is characterized in that The key bullet screen requirement includes a plurality of source texts, and the method further includes: inputting each of the source texts into the second embedding model to obtain corresponding professional features of the source text, and screening out a second bullet screen group consisting of bullet screens that meet the key bullet screen requirement based on the professional feature pool, including: For each professional feature in the professional feature pool, the following operations are performed: calculating the feature distance between the professional feature and each of the professional features of the source text; if the minimum feature distance is less than a second distance threshold, determining that the bullet screen corresponding to the professional feature meets the key bullet screen requirement; determining the source text corresponding to the professional feature of the source text corresponding to the minimum feature distance as the associated source text of the bullet screen corresponding to the professional feature; Correspondingly, the modifying the second barrage sequence based on the target barrage includes: For any barrage in the second barrage sequence that is hit by the second barrage group, a special mark is added to the barrage.
5. The method according to claim 4, It is characterized in that The key bullet comment requirement also includes a reply text corresponding to each source text, and the method further includes performing the following operations on any bullet comment in the second bullet comment group: Based on the key bullet screen requirement, determining an associated reply text corresponding to an associated source text corresponding to the bullet screen; Displaying a first prompt message and a first control on a target client, the target client being a client corresponding to the publishing account of the barrage, the first prompt message being used to indicate that the host of the target live broadcast room has interacted with the publishing account through the barrage; When the first control is triggered, the corresponding associated reply text is displayed on the target client.
6. The method according to claim 5, It is characterized in that The method further comprises: Displaying a second control on the target client; When the second control is triggered, a reply voice is played on the target client, where the reply voice is automatically generated by a voice generation model based on the associated reply text and conforms to the timbre of the host.
7. The method according to claim 2, It is characterized in that The modifying the second barrage sequence based on the target barrage includes: Any barrage in the second barrage sequence that is hit by the third barrage group is deleted.
8. The method according to claim 6, It is characterized in that The speech generation model is trained by the following method: Obtain a pre-trained speech decoder, wherein the speech decoder is trained based on a general text-speech pair, and the speech decoder includes a speech encoder and a speech decoder; Obtaining a training sample recorded by the anchor, wherein the training sample includes a sample text and a corresponding sample voice, and both the sample text and the sample voice have the professional knowledge corresponding to the anchor; Inputting the sample text into the second embedding model to obtain sample text features; Inputting the sample speech into the speech encoder to obtain sample speech features; Training a connector based on the sample text features and the sample speech features; This step includes: inputting the sample text feature into the connector to obtain a predicted voice feature; adjusting the parameters of the connector based on the difference between the predicted voice feature and the corresponding sample voice feature, so that the voice information output by the connector conforms to the host's voice; The speech generation model is composed of the second embedding model, the trained connector and the speech decoder.
9. The method according to claim 8, It is characterized in that The method further comprises performing the following operations based on the speech generation model: Reusing the second embedding model to output text features corresponding to the associated reply text; Inputting the text features into a connector in the speech generation model to obtain corresponding speech features; The speech feature is input into the speech decoder in the speech generation model to obtain the reply speech.
10. A computer-readable storage medium, It is characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the barrage processing method as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Information processing method and apparatus, terminal device and storage medium
CN107613392A
Bullet screen reply method, bullet screen reply device and live broadcast equipment
CN110035325A
Bullet screen publishing method, model generation method and device
CN111225227A
Bullet screen data processing method, device and equipment and storage medium
CN112417895A
Bullet screen management method and device, terminal equipment and storage medium
CN112765336A
