Video bullet screen processing method, device and equipment and computer medium

Abstract

The invention provides a video bullet screen processing method, device and equipment and a computer readable medium, and the method comprises the steps: extracting a bullet screen text from a to-be-processed video bullet screen, determining an associated video clip related to the video bullet screen according to the timestamp information of the video bullet screen, and carrying out the processing of the video bullet screen. According to the video content information of the associated video clip and the overall information formed by the bullet screen text, judging whether the video bullet screen accords with a preset screening condition, processing the video bullet screen which accords with the preset screening condition according to a preset bullet screen processing strategy, and selecting the associated video clip related to the video bullet screen. And the video content information and the bullet screen text of the associated video clip are combined for overall analysis, so that the significance of the video bullet screen in a specific video scene can be comprehensively captured, and accurate identification and effective processing of the specific bullet screen are ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video bullet screen processing method, device, equipment and computer-readable medium. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the application that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

[0003] With the rapid development of Internet technology, video barrage has been widely used on online video platforms as a new interactive method. Users can communicate with other viewers in real time by sending barrage while watching videos. This interactive method greatly enriches the user's viewing experience. However, due to the immediacy and anonymity of barrage, some inappropriate remarks and bad information have been spread, seriously affecting the health of the network environment. In this context, video barrage processing technology has also developed, aiming to maintain a healthy network environment and prevent the spread of inappropriate remarks and bad information.

[0004] Existing barrage processing systems mostly rely on text analysis, filtering out bad content in barrages through methods such as keyword matching and sentiment analysis. Although certain results have been achieved, there are still some barrages that are inappropriate in specific video scenarios and have not been discovered and processed in a timely manner.

[0005] In view of the above problems, it is necessary to provide a technical solution for more accurately identifying and processing specific barrages. Summary of the invention

[0006] Multiple aspects of the present application provide a video barrage processing method, device, equipment and computer-readable storage medium for more accurately identifying and processing specific barrages.

[0007] In one aspect of the present application, a method for processing video bullet screen is provided, wherein the method comprises:

[0008] Extract the bullet comment text from the video bullet comment to be processed;

[0009] Determine, according to the timestamp information of the video bullet screen, an associated video segment related to the video bullet screen;

[0010] Determining whether the video barrage meets a preset screening condition based on the video content information of the associated video clip and the overall information composed of the barrage text;

[0011] The video bullet comments that meet the preset screening conditions are processed according to the preset bullet comment processing strategy.

[0012] Another aspect of the present application provides a video bullet screen processing device, wherein the device comprises:

[0013] A bullet text extraction module is used to extract bullet text from the video bullet to be processed;

[0014] An associated segment determination module, used to determine an associated video segment related to the video barrage according to the timestamp information of the video barrage;

[0015] A condition compliance judgment module, used to judge whether the video barrage meets the preset screening condition according to the video content information of the associated video clip and the overall information composed of the barrage text;

[0016] The barrage screening processing module is used to process the video barrages that meet the preset screening conditions according to the preset barrage processing strategy.

[0017] Another aspect of the present application provides an electronic device, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video barrage processing method as described above.

[0021] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the video barrage processing method as described above.

[0022] In the solution provided in the embodiment of the present application, by extracting the barrage text from the video barrage to be processed, and determining the associated video clip related to the video barrage according to the timestamp information of the video barrage, and then judging whether the video barrage meets the preset screening conditions according to the video content information of the associated video clip and the overall information composed of the barrage text, and then processing the video barrage that meets the preset screening conditions according to the preset barrage processing strategy, it is possible to comprehensively analyze the relationship between the video content and the barrage text, and ensure the accurate recognition of specific barrages, wherein, by selecting the associated video clips related to the video barrage, and combining the video content information and the barrage text of the associated video clips for overall analysis, it is possible to comprehensively capture the significance of the video barrage in the specific video scene, thereby reducing the misjudgment and missed judgment caused by being separated from the video content, ensuring the accurate recognition of specific barrages, and by adopting the preset barrage processing strategy to process the video barrage that meets the preset screening conditions, it is possible to effectively purify the video playback environment, improve user experience, and maintain a good network order. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0025] Figure 1 A flowchart of a video bullet screen processing method provided in one embodiment of the present application;

[0026] Figure 2 A schematic diagram of a process for screening video comments provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of the structure of a video bullet screen processing device provided in one embodiment of the present application;

[0028] Figure 4 A schematic diagram of the structure of an electronic device suitable for implementing the solution in the embodiments of the present application;

[0029] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPU), input / output interface, network interface and memory.

[0032] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0033] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer program instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0034] In view of the problem that some barrages that appear inappropriate in specific video scenes cannot be discovered and processed in a timely manner when barrages are processed by keyword matching and other methods in the prior art, the embodiments of the present application provide a video barrage processing method, device, equipment and computer-readable storage medium for more accurately identifying and processing specific barrages. It extracts barrage text from the video barrage to be processed, and determines the associated video clip related to the video barrage based on the timestamp information of the video barrage, and then determines whether the video barrage meets the preset screening conditions based on the video content information of the associated video clip and the overall information composed of the barrage text, and then processes the barrage based on the preset The processing strategy is used to process the video barrage that meets the preset screening conditions, so as to comprehensively analyze the relationship between the video content and the barrage text, and ensure the accurate recognition of specific barrages. Among them, by selecting related video clips related to the video barrage, and combining the video content information and the barrage text of the related video clips for overall analysis, the significance of the video barrage in the specific video scene can be fully captured, thereby reducing misjudgments and missed judgments caused by being separated from the video content, ensuring the accurate recognition of specific barrages, and by adopting the preset barrage processing strategy to process the video barrage that meets the preset screening conditions, it can effectively purify the video playback environment, improve user experience, and maintain a good network order.

[0035] It should be noted that one of the purposes of the embodiment of the present application is to accurately identify and process specific bullet comments that meet the preset screening conditions. By combining the video content information with the overall analysis of the bullet comment text, the meaning of the bullet comment in the specific video scene is captured, so as to accurately identify those bullet comments that need special attention. The above-mentioned specific bullet comment can be any video bullet comment with clear processing requirements. Such processing requirements are positive and negative. For example, positive requirements include identifying and highlighting positive feedback from the audience, valuable opinions, or bullet comments that make constructive contributions to the video content; while negative requirements include identifying and filtering out inappropriate remarks, bad content, or other bullet comments that do not comply with platform regulations. By carrying out targeted processing on these specific bullet comments, the user's viewing experience can be optimized while maintaining a healthy network environment. This comprehensive and flexible processing mechanism can ensure that the video platform can not only effectively manage potential problematic content, but also actively promote high-quality interactions, thereby promoting the formation of a more harmonious and vibrant online community.

[0036] The following is an exemplary description of the video barrage processing method, device, equipment and computer-readable storage medium provided in the present application.

[0037] In actual scenarios, the execution subject of the video barrage processing method provided in the embodiment of the present application can be a user device, or a device formed by integrating a user device and a network device through a network, or an application (such as a machine review system) running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer collections. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0038] Figure 1 The following is a processing flow of a video bullet screen processing method provided by an embodiment of the present application. The method includes at least the following processing steps:

[0039] Step S101, extracting bullet comment text from the video bullet comment to be processed.

[0040] Among them, video barrage is a kind of comment or short message sent by the audience in real time and displayed on the video screen during the playback of online videos. It usually appears on the video screen in the form of floating text and interacts with the video content in real time.

[0041] In an exemplary scenario, the user enters the barrage text in the video playback interface and clicks the send button to complete the sending of the video barrage. After that, the above video barrage will be sent to the server of the video playback platform through the network. After receiving the video barrage, the server will start the barrage processing process according to the preset processing rules. The barrage processing can be instant, that is, the video barrage is processed by the automated processing system immediately after it is sent, or it can be automatically triggered by the system within a short time after the video barrage is sent. The purpose of barrage processing is to check whether the content of the video barrage complies with the regulations of the video playback platform to avoid the spread of inappropriate remarks and bad information.

[0042] The video barrage processing method provided in the embodiment of the present application can be executed by an automated processing system. The automated processing system is a software system that uses computer technology and algorithms to automatically detect and evaluate whether digital content (such as video, audio, text, etc.) meets specific standards or regulations. It can also be called a machine review system or a machine review platform.

[0043] In this step, after the automated processing system starts the processing flow for the video bullet comments to be processed, it can extract the specific text content from the video bullet comments posted by the user during the video playback process. The process of extracting bullet comment text can be achieved by parsing the bullet comment data stream of the video playback platform, and the obtained text information will serve as the basis for subsequent analysis.

[0044] Step S102: determining an associated video segment related to the video barrage according to the timestamp information of the video barrage.

[0045] The timestamp information records the exact time point when the video bullet comment is released, which can be recorded as the bullet comment release time. Based on this, video clips within a period of time related to the bullet comment release time are selected as associated video clips related to the video bullet comment. For example, video clips 5 seconds before and after the bullet comment release time are selected. The purpose of this step is to obtain the context information of the video bullet comment release, so as to more accurately understand the meaning of the video bullet comment in the scene where the video content is played.

[0046] Step S103, judging whether the video bullet comment meets a preset screening condition based on the video content information of the associated video clip and the overall information composed of the bullet comment text.

[0047] Among them, the above-mentioned video content information can be identified from associated video clips using a machine model, which may include but is not limited to at least one of text content information, voice content information and scene element content information. The above video content information can all be organized into text information and together with the barrage text, form overall information to determine whether the video barrage meets the preset screening conditions.

[0048] One of the purposes of the embodiments of the present application is to use the above-mentioned overall information to determine whether the video barrage meets the preset screening conditions. By combining the overall information of the video content and the barrage text, it is possible to more accurately identify whether the barrage meets the preset screening conditions. Traditional barrage processing methods often rely only on the barrage text itself, ignoring its contextual relationship with the video content, which may result in some barrages that are inappropriate in specific video scenes not being discovered and processed in time. However, the present application selects video clips within a preset time period before and after the release time of the video barrage as associated video clips, and combines the video content information and the barrage text for overall analysis, which can fully capture the meaning of the barrage in a specific video scene, thereby ensuring accurate recognition of specific barrages.

[0049] Step S104, processing the video bullet comments that meet the preset screening conditions according to the preset bullet comment processing strategy.

[0050] Among them, the preset barrage processing strategy may include a negative processing strategy and / or a positive processing strategy.

[0051] Among them, the negative processing strategy is used to identify and process those bullet comments that do not comply with platform regulations or may have a negative impact on user experience, which may include but is not limited to recalling specific bullet comments, that is, removing them from the video screen; setting the visibility of specific bullet comments to be invisible to specified users, that is, blocking specific bullet comments only for some users; or sending specific bullet comments to the manual processing platform for further manual processing, etc. Those skilled in the art can flexibly set them according to actual needs. These processing strategies can effectively prevent the negative impact of specific bullet comments on the video playback environment and ensure the good operation order of the video platform.

[0052] Positive processing strategies are used to identify, enhance and promote those bullet comments that contribute positively to the video viewing experience. They can include a series of processing strategies to improve the visibility and influence of bullet comments. For example, they can be displayed in a more prominent way in the video playback interface, such as enlarging the font, changing the color or adding special effects to attract the attention of other viewers. In addition, positive bullet comments can be recommended to more users and even archived as part of the video content for subsequent reference or review. Through positive processing strategies, not only can high-quality interactions be encouraged, but also a positive, healthy and vibrant community atmosphere can be created to promote more valuable discussions and exchanges.

[0053] The above-mentioned video barrage processing method provided in the embodiment of the present application extracts the barrage text from the video barrage to be processed, and determines the associated video clip related to the video barrage according to the timestamp information of the video barrage, and then judges whether the video barrage meets the preset screening conditions according to the video content information of the associated video clip and the overall information composed of the barrage text, and then processes the video barrage that meets the preset screening conditions according to the preset barrage processing strategy, so as to comprehensively analyze the relationship between the video content and the barrage text, and ensure the accurate recognition of specific barrages. Among them, by selecting the associated video clips related to the video barrage, and combining the video content information and the barrage text of the associated video clip for overall analysis, the meaning of the video barrage in the specific video scene can be fully captured, so as to reduce the misjudgment and missed judgment caused by being separated from the video content, and ensure the accurate recognition of specific barrages. By adopting the preset barrage processing strategy to process the video barrage that meets the preset screening conditions, it can effectively purify the video playback environment, improve user experience, and maintain a good network order.

[0054] Regarding how to select the above-mentioned associated video clips, in some modified implementation modes, determining the associated video clips related to the video barrage according to the timestamp information of the video barrage includes:

[0055] According to the timestamp information of the video barrage, video clips in a first preset time period before the release of the video barrage and / or a second preset time period after the release are selected as video clips related to the video barrage.

[0056] One of the purposes of this embodiment is to obtain contextual information related to the video barrage by selecting video clips within a preset time period before and after the release time of the video barrage, so as to more accurately determine whether the video barrage meets the preset screening conditions.

[0057] Among them, the above-mentioned timestamp information refers to the time point in the video playback progress, that is, the time position in the video. The timestamp information of the video barrage represents the release time of the video barrage. Based on the timestamp information, the video clip within the first preset time period before the video barrage is released can be selected as the associated video clip, or the video clip within the second preset time period after the video barrage is released can be selected as the associated video clip, or the video clip of the first preset time period and the video clip of the second preset time period can be used together as the associated video clip. All of the above can obtain the associated video clip related to the above-mentioned video barrage, so as to combine the associated video clip with the video barrage to determine the true intention of the video barrage. Among them, the above-mentioned first preset time period and the second preset time period can be the same or different, can be fixed, such as the first 5 seconds and the last 5 seconds, or can be dynamically adjusted. Technical personnel in this field can flexibly adjust and determine according to the complexity of the video content and the relevance of the barrage content, and the embodiments of the present application are not limited.

[0058] For example, assuming that the timestamp of a video barrage is the 10th minute and 33th second of the video playback progress, the system can select the video clips 5 seconds before (i.e., from 10th minute 28th second to 10th minute 33th second) and 5 seconds after (i.e., from 10th minute 33th second to 10th minute 38th second) of this time point according to preset rules as associated video clips related to the video barrage, so as to use the associated video clips to obtain the video context when the video barrage is released, so as to more accurately understand the meaning of the video barrage content and whether it is consistent with the video content.

[0059] By selecting related video clips within a preset time period before and after the video comment is released, the system can use the related video clips to accurately understand the intention of the video comment and ensure accurate recognition of specific comments. For example, if the video comment is "This is so funny", the system can confirm that there is indeed a funny scene in the video by selecting video clips within the preset time period before and after, and thus determine that the video comment is a normal comment rather than content; conversely, if there is no funny scene in the video clip, the system can further analyze whether the video comment is inappropriate speech.

[0060] Through the above implementation, not only can the accuracy and timeliness of specific bullet comments be improved, but also the misjudgment caused by relying solely on the bullet comment text can be effectively prevented, thereby maintaining the health and order of the video playback environment. By combining the video content information and the bullet comment text information for comprehensive analysis, the system can more accurately determine whether the video bullet comment meets the preset screening conditions, ensuring the good operation order of the video platform.

[0061] In some modified implementations, before judging whether the video bullet comment meets a preset screening condition based on the video content information of the associated video clip and the overall information composed of the bullet comment text, the method further includes:

[0062] Content recognition is performed on the associated video clips to obtain video content information.

[0063] One of the purposes of this embodiment is to first identify the content of the associated video clips, obtain the key information in the video, and then conduct a comprehensive analysis based on the barrage text, so as to more accurately determine whether the video barrage is a specific barrage.

[0064] By pre-identifying the content of the associated video clips to obtain the video content information, it is possible to ensure that the specific barrage can be accurately identified by combining the video content information and the barrage text information for comprehensive analysis. In addition, by identifying the content of the associated video clips, the key information in the video can be fully captured, so that the meaning of the video barrage and whether it is consistent with the video content can be more accurately understood, as well as the true intention of the video barrage under the associated video clip. This can not only improve the accuracy and timeliness of the identification of specific barrages, but also effectively prevent misjudgments caused by relying solely on barrage text, thereby maintaining the health and order of the video playback environment.

[0065] In specific implementation, a machine model can be used to realize content recognition of associated video clips. Considering that the video content is rich and diverse, it includes text, voice, scene elements (such as characters, environment, animals, etc.), and other aspects. Any aspect of the content may be associated with the barrage. Therefore, the implementation method of the present application can perform content recognition on associated video clips from multiple aspects to ensure that the key information in the video is fully captured.

[0066] Accordingly, based on the above implementation, in some modified implementations, the performing content recognition on the associated video clip to obtain video content information includes at least one of the following:

[0067] Using an optical character recognition model to perform text recognition on each video frame in the associated video segment to obtain text content information;

[0068] Using an automatic speech recognition model to perform speech recognition on the associated video clip to obtain speech content information;

[0069] A computer vision recognition model is used to recognize scene elements appearing in each video frame in the associated video clip to obtain scene element content information.

[0070] Among them, the optical character recognition (OCR) model is a model developed based on artificial intelligence technology, which is used to convert the text in the image into an editable text format. The OCR model usually uses deep learning technology, such as convolutional neural networks (CNNs), to recognize the characters in the image and convert them into text information. In this embodiment, the optical character recognition model can be used to recognize the text content such as subtitles, book covers, titles, signs, product logos, menus, posters, etc. in the video, so that the text content information in the video can be captured and comprehensively analyzed with the barrage text, and then the real intention of the video barrage under the associated video clip can be more accurately judged based on the text content in the video.

[0071] The Automatic Speech Recognition (ASR) model is a model developed by artificial intelligence technology for converting speech signals into a readable text format. ASR models are usually based on deep learning techniques, such as Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM), which can accurately convert speech into text. In this embodiment, the automatic speech recognition model can be used to recognize and convert the speech content such as dialogue content, narration, commentary, songs, etc. in the video into text information, so that the speech content information in the video can be captured and comprehensively analyzed with the bullet text, and then the real intention of the video bullet under the associated video clip can be more accurately judged based on the speech content in the video.

[0072] The Computer Vision Recognition (CVR) model is a model developed based on artificial intelligence technology, which is used to identify scene elements such as objects, people, and environments in images and convert them into understandable and processable information. CVR models are usually based on deep learning technology, such as Convolutional Neural Networks (CNNs), which can accurately identify various scene elements in images. In this embodiment, the computer vision recognition model can be used to identify the content of scene elements such as people, animals, markers, and environments in the video and convert them into text information, so that the scene element content information in the video can be captured and comprehensively analyzed with the bullet text, and then the real intention of the video bullet under the associated video clip can be more accurately judged based on the scene element content in the video.

[0073] It should be noted that the above content identification methods can be implemented one by one or in combination. The embodiments of the present application are not limited thereto, and those skilled in the art can flexibly set them according to actual needs.

[0074] Please refer to Figure 2 To understand, Figure 2 A schematic diagram of the process of screening video barrage provided for an embodiment of the present application, as shown in the figure, the OCR model is used to identify text content information such as subtitles, advertising signs, books, etc., the ASR model is used to identify voice content information such as background commentary, and the CVR model includes the CV face model and CV entity recognition model in the figure, which are respectively used to identify people and animals, scenes and other scene elements. The above models are combined to identify the video content information.

[0075] This embodiment, by utilizing a variety of recognition technologies, can comprehensively capture multi-dimensional information in related video clips, including text, voice, and visual elements, so that it can more accurately understand the true intention of the video barrage and its relevance to the video content, and improve the accuracy and timeliness of specific barrage recognition. For example, if the video barrage mentions "this shot is so beautiful", the CVR model can be used to identify the landscape in the video, thereby confirming that the true intention of the video barrage is a normal comment rather than inappropriate content. In addition, by combining the use of OCR, ASR, and CVR models in some embodiments, potential inappropriate content can be more effectively identified and processed. For example, if there are inappropriate remarks in the video barrage, but the video content itself does not show any related scenes or dialogues, the system can judge that the video barrage is inappropriate content and take corresponding processing measures.

[0076] Based on any of the above implementations, in some modified implementations, the above step S103 determines whether the video bullet comment meets the preset screening condition according to the video content information of the associated video clip and the overall information composed of the bullet comment text, which may include:

[0077] The overall information consisting of the video content information of the associated video clip and the barrage text is input into a pre-trained classification model, and the classification model is used to determine whether the video barrage meets the preset screening conditions.

[0078] One of the purposes of this embodiment is to more accurately identify specific bullet comments by integrating video content information and overall information of the bullet comment text and using a pre-trained classification model for automated judgment.

[0079] Among them, the above-mentioned classification model is a machine learning model, which is used to assign input data to one or more predefined categories, and in this embodiment is used to determine whether the video barrage is a specific barrage.

[0080] In the specific implementation, the system obtains the video content information in the associated video clips through the method described above, including text content information, voice content information and scene element content information. Then, the video content information is combined with the barrage text information to form the overall information. Then, the overall information is input into the pre-trained classification model, which can be implemented based on deep learning networks such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), and trained with a large amount of labeled data to enable it to distinguish between normal barrage and specific barrage.

[0081] For example, suppose there is a conversation in a video clip, and one of the characters says, "It's a beautiful day." If the video comment is "What a beautiful day," the system combines the voice content information in the associated video clip ("It's a beautiful day" obtained by the automatic speech recognition model) with the comment text information ("What a beautiful day") to form the overall information. Then, the system inputs this overall information into the pre-trained classification model, and uses the classification model to determine whether the video comment is a specific comment, such as a bad comment. If the classification model recognizes that the comment content of the video comment is consistent with the video content and there is no inappropriate speech, then the video comment will be judged as a non-bad comment.

[0082] This implementation method, by utilizing a pre-trained classification model for automated judgment, can more accurately identify specific barrages, which can not only improve the accuracy and timeliness of specific barrage identification, but also effectively prevent misjudgments caused by relying solely on barrage text, thereby maintaining the health and order of the video playback environment and improving user experience.

[0083] In one example, a classification model can be pre-built and trained through the following steps: first, a large amount of barrage data is collected and labeled, normal and specific barrages are clearly distinguished, and the data is cleaned and divided to form a training set, a validation set, and a test set; then, video content information is extracted from associated video clips, including text content information, voice content information, and scene element content information, and barrage text information is extracted from the barrage data, and this information is integrated to form overall information; then, a suitable deep learning model architecture is selected, such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs), and model parameters are initialized; by inputting the overall information into the model, the calculation Calculate the loss value and update the model parameters through back propagation until the model converges or reaches the predetermined number of training rounds; evaluate and adjust the model parameters on the validation set, and evaluate the final performance of the model on the test set; use grid search or random search and other methods to find the optimal hyperparameter combination, and use early stopping to prevent overfitting; finally, deploy the trained model to the production environment to realize real-time or batch processing of barrage data, so that it can automatically judge by integrating the overall information of the video content and the barrage text, and use the pre-trained classification model to more accurately identify specific barrages, improve the accuracy and timeliness of specific barrage identification, and maintain the health and order of the video playback environment.

[0084] In some other modified implementations, the above step S103, judging whether the video bullet comment meets the preset screening condition according to the video content information of the associated video clip and the overall information composed of the bullet comment text, may include:

[0085] Using a pre-trained semantic recognition model to perform semantic recognition on the overall information, obtaining the situational semantic information of the video barrage corresponding to the associated video segment, wherein the overall information is composed of the video content information of the associated video segment and the barrage text;

[0086] Determine whether the video commentary meets preset screening conditions based on the situational semantic information.

[0087] One of the purposes of this embodiment is to perform semantic recognition by integrating the overall information of the video content information and the barrage text, using a pre-trained semantic recognition model, so as to more accurately judge the true intention and situational semantic information of the video barrage in the associated video clip, so as to identify specific barrage.

[0088] Among them, the semantic recognition model is a model developed based on deep learning technology, which is used to understand and analyze the meaning of text content and its relationship with the context. In this embodiment, it is used to identify the situational semantic information of video barrage in associated video clips.

[0089] This embodiment can use the BERT (Bidirectional Encoder Representations from Transformers) model to construct the above-mentioned semantic recognition model. Among them, BERT is a model based on the Transformer architecture, which has achieved very good results in various natural language processing (NLP) tasks through bidirectional training. The core advantage of the BERT model lies in its bidirectional encoder design, which can consider contextual information while understanding the text, so as to more accurately capture the relationship between words. In this embodiment, the BERT model can be used to analyze the semantic relationship between video content information and barrage text, and obtain the situational semantic information of the video barrage corresponding to the associated video clip.

[0090] In some examples, the semantic recognition model based on BERT in this application can be trained through the following steps: first, prepare a large amount of annotated data containing video content information (such as text, voice, scene elements, etc.) and barrage text; in the pre-training stage, through Masked Language Modeling (MLM) and NextSentence Prediction (MLM), the semantic recognition model based on BERT in this application can be trained through the following steps: first, prepare a large amount of annotated data containing video content information (such as text, voice, scene elements, etc.) and barrage text; in the pre-training stage, The internal semantic structure of the text is learned through the task of automatic speech recognition (NSP); subsequently, in the feature extraction stage, the video content information is extracted through optical character recognition (OCR), automatic speech recognition (ASR) and computer vision recognition (CVR) models, and combined with the bullet text information to form the overall information; in the fine-tuning stage, the BERT model is further trained with the labeled data to adapt it to the specific task of judging whether the video bullet is a specific bullet; after the fine-tuning is completed, the semantic recognition model can perform semantic recognition on the overall information and obtain the situational semantic information of the video bullet corresponding to the associated video clip; finally, the model performance is evaluated and optimized on the validation set to ensure that the model can accurately judge whether the video bullet is a specific bullet in actual application, thereby maintaining the health and order of the video playback environment. The above is only an exemplary principle description, and those skilled in the art can flexibly implement it based on the above description to train a semantic recognition model that meets the requirements of the embodiments of the present application.

[0091] When implementing the implementation method of this application, please refer to Figure 2 To understand, Figure 2 A schematic diagram of a process for screening video comments provided in an embodiment of the present application is shown in FIG. Figure 2As shown, when it is necessary to process the video barrage, it is necessary to determine the associated video clips and query the video content information corresponding to the associated video clips, wherein the video content information can be identified based on the aforementioned OCR model, CVR model and ASR model, etc., and the barrage text and the above video content information are combined into overall information and input into the semantic recognition model, and then the semantic recognition model is used to identify the situational semantic information of the video barrage corresponding to the associated video clip, and then the situational semantic information is used to judge whether the above video barrage meets the preset screening conditions.

[0092] This embodiment, by integrating the overall information of video content and barrage text, uses a pre-trained semantic recognition model to perform semantic recognition, so as to more accurately judge the true intention and situational semantic information of video barrage in associated video clips, improve the accuracy and timeliness of specific barrage recognition, and effectively prevent misjudgment caused by relying solely on barrage text, thereby maintaining the health and order of the video playback environment and improving user experience.

[0093] On the basis of the above implementation, in some modified implementations, semantic recognition and keyword matching may be combined to determine whether the video barrage is a specific barrage. Accordingly, the step of determining whether the video barrage meets the preset screening condition according to the scenario semantic information may include:

[0094] Perform keyword matching on the overall information and the preset screening condition keywords to obtain keyword matching results;

[0095] Whether the video bullet screen meets the preset screening conditions is determined based on the keyword matching result and the situational semantic information.

[0096] For example, if the overall information contains preset filtering condition keywords and the semantics match, it is determined that the video commentary is a specific commentary.

[0097] For another example, suppose there is a scene in a video clip showing a character saying in a movie: "This is a dangerous action." If the content of the video barrage is "Too dangerous, be careful!", the system combines the voice content information in the associated video clip ("This is a dangerous action" obtained by the automatic speech recognition model) with the barrage text information ("Too dangerous, be careful!") to form the overall information. Then, the system uses the pre-trained semantic recognition model to perform semantic recognition on the overall information and obtain the situational semantic information of the video barrage corresponding to the associated video clip. At the same time, the system will also perform keyword matching based on the overall information to obtain keyword matching results.

[0098] Taking the preset filtering condition keywords as bad keywords and specific barrages as bad barrages as an example, assuming that some bad keywords are defined in the system, such as "dangerous", "vulgar words", etc. In this case, the system will first check whether the barrage text contains these bad keywords. If the barrage text contains the word "dangerous", the system will further determine whether the word constitutes a bad meaning in this context based on the situational semantic information. If the semantic recognition model recognizes that the barrage content of the video barrage is consistent with the video content, and expresses understanding and support for the plot, and has no bad meaning, and the keyword matching results show that the word does not constitute bad behavior in this context, then the video barrage will be judged as a non-bad barrage.

[0099] Through the above implementation mode, not only can the semantic relationship between the video content and the barrage text be captured through the semantic recognition model, but also the presence of potential specific content can be further confirmed through keyword matching. Thus, by combining the video content information and the overall information of the barrage text, semantic recognition can be performed using a pre-trained semantic recognition model, and combined with keyword matching, it is possible to more accurately judge the true intention and situational semantic information of the video barrage in the associated video clip, improve the accuracy and timeliness of specific barrage recognition, and effectively prevent misjudgment caused by relying solely on barrage text, thereby maintaining the health and order of the video playback environment and improving user experience.

[0100] In some modified implementations, the processing of the video bullet comments that meet the preset screening conditions according to the preset bullet comment processing strategy includes at least one of the following:

[0101] Recalling the video comments that meet the preset screening conditions;

[0102] Setting the visible status of the video bullet screen that meets the preset screening conditions to be invisible to the specified user;

[0103] The video comments that meet the preset screening conditions are sent to the manual processing platform for manual processing.

[0104] In the specific implementation, first determine whether the video barrage is a specific barrage through the method described above. Once it is determined that a certain video barrage is a specific barrage, the system will process it according to the preset barrage processing strategy. For the situation where a specific barrage is a bad barrage, the processing strategy includes but is not limited to the following methods: First, recall the video barrage that meets the preset filtering conditions, that is, immediately remove the video barrage from the video playback and no longer display it; second, set the visible state of the video barrage that meets the preset filtering conditions to be invisible to the specified user, that is, although the video barrage still exists in the system, it is hidden from certain specific user groups (such as minors, etc.); third, send the video barrage that meets the preset filtering conditions to the manual processing platform for manual processing, that is, the system temporarily hides the video barrage and submits it to the manual processing team for further judgment to determine the final processing result.

[0105] Through this implementation, specific barrages can be processed through different processing strategies, so that the barrage content in the video playback environment can be more effectively managed, the accuracy and timeliness of specific barrage processing can be improved, the adverse effects of bad content on users can be effectively prevented, the health and order of the video playback environment can be maintained, and the user experience can be enhanced.

[0106] Based on the same inventive concept, a video barrage processing device is also provided in an embodiment of the present application. The method corresponding to the device may be the video barrage processing method in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The video barrage processing device provided in an embodiment of the present application can implement the aforementioned video barrage processing method, and the video barrage processing device can be implemented by software, hardware, or a combination of software and hardware. For example, the video barrage processing device may include integrated or separate functional modules or units to perform the corresponding steps in the aforementioned methods. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts may refer to the partial description of the method embodiment. The device embodiment described below is merely illustrative. Please refer to Figure 3 , the video barrage processing device may include:

[0107] A bullet text extraction module 101 is used to extract bullet text from the video bullet to be processed;

[0108] The associated segment determination module 102 is used to determine the associated video segment related to the video barrage according to the timestamp information of the video barrage;

[0109] A condition compliance judgment module 103 is used to judge whether the video barrage meets a preset screening condition based on the video content information of the associated video clip and the overall information composed of the barrage text;

[0110] The bullet comment screening processing module 104 is used to process the video bullet comments that meet the preset screening conditions according to the preset bullet comment processing strategy.

[0111] In some modified implementations, the associated fragment determination module 102 includes:

[0112] An associated segment selection unit is used to select video segments in a first preset time period before the release of the video barrage and / or a second preset time period after the release of the video barrage according to the timestamp information of the video barrage as video segments related to the video barrage.

[0113] In some variations, the device further comprises:

[0114] The video content recognition module is used to perform content recognition on the associated video clips to obtain video content information.

[0115] In some modified implementations, the video content identification module includes at least one of the following units:

[0116] A text content recognition unit, configured to perform text recognition on each video frame in the associated video segment using an optical character recognition model to obtain text content information;

[0117] A speech content recognition unit, configured to perform speech recognition on the associated video clip using an automatic speech recognition model to obtain speech content information;

[0118] The element content recognition unit is used to use a computer vision recognition model to recognize the scene elements appearing in each video frame in the associated video segment to obtain scene element content information.

[0119] In some modified implementations, the condition compliance determination module 103 includes:

[0120] The classification model judgment unit is used to input the overall information consisting of the video content information of the associated video clip and the barrage text into a pre-trained classification model, and use the classification model to judge whether the video barrage meets the preset screening conditions.

[0121] In some modified implementations, the condition compliance determination module 103 includes:

[0122] A semantic recognition unit, configured to perform semantic recognition on the overall information using a pre-trained semantic recognition model to obtain the situational semantic information of the video barrage corresponding to the associated video segment, wherein the overall information is composed of the video content information of the associated video segment and the barrage text;

[0123] The semantic screening judgment unit is used to judge whether the video barrage meets the preset screening conditions according to the situational semantic information.

[0124] In some modified implementations, the semantic screening and judgment unit includes:

[0125] A keyword matching subunit, used for keyword matching the overall information with preset screening condition keywords to obtain keyword matching results;

[0126] Combined with the judgment subunit, it is used to judge whether the video barrage meets the preset screening conditions based on the keyword matching results and the situational semantic information.

[0127] In some modified implementations, the barrage screening processing module 104 includes at least one of the following units:

[0128] A bullet screen recall unit, used to recall the video bullet screens that meet the preset screening conditions;

[0129] A visible status setting unit, used to set the visible status of the video bullet screen that meets the preset screening condition to be invisible to the designated user;

[0130] The manual processing unit is transferred to send the video barrage that meets the preset screening conditions to the manual processing platform for manual processing.

[0131] The video barrage processing device provided in the embodiment of the present application is based on the same inventive concept as the video barrage processing method provided in the aforementioned embodiment of the present application and has the same beneficial effects.

[0132] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application, and the method corresponding to the electronic device may be the video barrage processing method in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video barrage processing methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0133] The electronic device may be a user device, or a device formed by integrating a user device and a network device through a network, or may be an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0134] Figure 4 The structure of an electronic device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown, and the electronic device 1200 includes a central processing unit (CPU, Central Processing Unit) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM, Random Access Memory) 1203. In RAM 1203, various programs and data required for system operation are also stored. CPU 1201, ROM 1202 and RAM1203 are connected to each other through bus 1204. Input / output (I / O, Input / Output) interface 1205 is also connected to bus 1204.

[0135] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc., and a speaker, etc.; a storage section 1208 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet.

[0136] In particular, the methods and / or embodiments in the embodiments of the present application may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 1201, the above functions defined in the method of the present application are executed.

[0137] Another embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application described above.

[0138] Specifically, the present embodiment may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination therewith.

[0139] Computer readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0140] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0141] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0142] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0144] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0147] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0149] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A video bullet screen processing method, wherein: The method comprises: Extract the bullet comment text from the video bullet comment to be processed; Determine, according to the timestamp information of the video bullet screen, an associated video segment related to the video bullet screen; Determining whether the video barrage meets a preset screening condition based on the video content information of the associated video clip and the overall information composed of the barrage text; The video bullet comments that meet the preset screening conditions are processed according to the preset bullet comment processing strategy.

2. The video barrage processing method according to claim 1, wherein: The determining, according to the timestamp information of the video bullet screen, an associated video segment related to the video bullet screen includes: According to the timestamp information of the video barrage, video clips in a first preset time period before the release of the video barrage and / or a second preset time period after the release are selected as video clips related to the video barrage.

3. The video barrage processing method according to claim 1, wherein: Before judging whether the video bullet screen meets a preset screening condition based on the video content information of the associated video clip and the overall information composed of the bullet screen text, the method further includes: Content recognition is performed on the associated video clips to obtain video content information.

4. The video barrage processing method according to claim 3, wherein: The performing content recognition on the associated video clip to obtain video content information includes at least one of the following: Using an optical character recognition model to perform text recognition on each video frame in the associated video segment to obtain text content information; Using an automatic speech recognition model to perform speech recognition on the associated video clip to obtain speech content information; A computer vision recognition model is used to recognize scene elements appearing in each video frame in the associated video clip to obtain scene element content information.

5. The video barrage processing method according to claim 1, wherein: The determining, based on the video content information of the associated video clip and the overall information composed of the barrage text, whether the video barrage meets the preset screening condition includes: The overall information consisting of the video content information of the associated video clip and the barrage text is input into a pre-trained classification model, and the classification model is used to determine whether the video barrage meets the preset screening conditions.

6. The video barrage processing method according to claim 1, wherein: The determining, based on the video content information of the associated video clip and the overall information composed of the barrage text, whether the video barrage meets the preset screening condition includes: Using a pre-trained semantic recognition model to perform semantic recognition on the overall information, obtaining the situational semantic information of the video barrage corresponding to the associated video segment, wherein the overall information is composed of the video content information of the associated video segment and the barrage text; Determine whether the video commentary meets preset screening conditions based on the situational semantic information.

7. The video barrage processing method according to claim 6, wherein: The determining, according to the scenario semantic information, whether the video bullet screen meets a preset screening condition includes: Perform keyword matching on the overall information and the preset screening condition keywords to obtain keyword matching results; Whether the video bullet screen meets the preset screening conditions is determined based on the keyword matching result and the situational semantic information.

8. The video bullet screen processing method according to claim 1, wherein: The processing of the video bullet comments that meet the preset screening conditions according to the preset bullet comment processing strategy includes at least one of the following: Recalling the video comments that meet the preset screening conditions; Setting the visible status of the video bullet screen that meets the preset screening conditions to be invisible to the specified user; The video comments that meet the preset screening conditions are sent to the manual processing platform for manual processing.

9. A video bullet screen processing device, wherein: The device comprises: A bullet text extraction module is used to extract bullet text from the video bullet to be processed; An associated segment determination module, used to determine an associated video segment related to the video barrage according to the timestamp information of the video barrage; A condition compliance judgment module, used to judge whether the video barrage meets the preset screening condition according to the video content information of the associated video clip and the overall information composed of the barrage text; The barrage screening processing module is used to process the video barrages that meet the preset screening conditions according to the preset barrage processing strategy.

10. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

11. A computer readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 8.

Images