A method and system for intelligently polishing audio and video bullet screens
Through the intelligent barrage polishing system, combined with semantic understanding and keyword matching technology, the problem of inaccurate barrage emotional understanding in the existing technology is solved, the accurate matching of barrage content and polishing effect is achieved, and the barrage system is improved match and real-time.
Patent Information
- Application Number
- CN202410193896.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-21
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-02-21
AI Technical Summary
The existing barrage system fails to effectively consider the impact of the video playback environment and date on barrage content, resulting in inaccurate understanding of barrage emotions edited by users, and the problem of mismatch between barrage content and polishing.
An intelligent barrage polishing system for audio and video videos was designed. Through barrage acquisition, processing, environment acquisition and result analysis units, combined with semantic understanding and keyword matching technology, accurate semantic understanding is carried out according to the character type and length of barrage data. , and determine the output color in the keyword color matrix and database matching to ensure the matching degree of the barrage content and the polishing effect.
The degree of matching between barrage content and polishing effects is improved, the accuracy of semantic understanding and feature extraction of barrage data is ensured, error matching is avoided, and the complexity processing ability of barrage content and real-time video content matching ability are enhanced.
Smart Images

Figure CN118264843B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of audio and video technology, and in particular to an audio and video intelligent bullet screen polishing method and system. Background Art
[0002] Video barrage refers to the scrolling text comments that appear in audio and video. Some barrage systems use scripting languages to provide specific barrage forms, such as the appearance or disappearance of barrages at specific locations, control of barrage pop-up speed, barrage position, etc., and barrage demonstration performances can be achieved in conjunction with videos with solid colors or little screen changes and specific background music. In addition, barrages that appear fixedly at the bottom or top of the screen will also be used as subtitles for non-localized videos without subtitles. With the development of barrage systems, the demonstration forms of barrages are becoming more and more diverse, especially showing different demonstration forms according to the text content of the barrage.
[0003] Chinese patent publication number: CN105979338A, discloses a system and method for matching colors according to the emotions of barrage content; it is to assign emotions to each word in the emotional vocabulary, and then divide the emotions into different levels, select the corresponding emotional level according to the barrage edited by the user and display it in the corresponding color; it can be seen that in the existing barrage system, the impact of the video playback environment, playback date, etc. on the barrage is not considered. At the same time, the corresponding emotional vocabulary is selected from the stored vocabulary, which leads to the system's inaccurate understanding of the barrage emotions edited by the user, resulting in the problem of mismatch between the barrage content and the polishing. Summary of the invention
[0004] To this end, the present invention provides an intelligent audio and video bullet comment polishing method and system to overcome the problem of inaccurate understanding of the emotions of the bullet comment content in the prior art, which easily leads to the mismatch between the bullet comment content and the polishing effect.
[0005] To achieve the above-mentioned purpose, the present invention provides an intelligent audio and video bullet screen polishing system, comprising:
[0006] A bullet chat collection unit, which is used to collect the bullet chat data input by the client and determine the sent bullet chat data;
[0007] A bullet-screen processing unit connected to the bullet-screen collection unit, configured to obtain the character type and character length of the collected bullet-screen data, and perform semantic understanding according to the character type and character length of the bullet-screen data, and determine whether to output a number of keywords of the semantic understanding corresponding to the bullet-screen data;
[0008] An environment collection unit, which is used to collect video time node data of the real-time video playback of the client;
[0009] A result analysis unit, which is connected to the barrage processing unit and the environment acquisition unit respectively, and a keyword color matrix is set in the result analysis unit. When there is only one corresponding keyword in the barrage data, the result analysis unit selects the corresponding preset keyword in the keyword color matrix, and uses the color corresponding to the selected preset keyword as the output color of the barrage data. When there is more than one corresponding keyword in the barrage data, the result analysis unit matches each keyword corresponding to the barrage data with each stored barrage data in the barrage database, selects the stored barrage data with the keyword as a matching item, and determines the output color of the barrage data according to the selected matching item;
[0010] The bullet screen playback unit is connected to the result analysis unit and is used to play the bullet screen data according to the output color determined by the result analysis unit.
[0011] Furthermore, a first preset character length L1 and a second preset character length L2 are set in the barrage processing unit, wherein L1<L2. When the barrage acquisition unit sends the collected barrage data Dm to the barrage processing unit, the barrage processing unit obtains the character length Ld of the barrage data Dm, and compares the character length Ld of the barrage data Dm with the first preset character length L1 and the second preset character length L2.
[0012] When Ld<L1, the barrage processing unit determines that the character length Ld of the barrage data Dm is less than the first preset character length L1, the barrage processing unit will perform semantic understanding on the barrage data Dm, generate keywords of the barrage data Dm, and send the barrage data Dm and the keywords to the result analysis unit;
[0013] When L1≤Ld≤L2, the bullet screen processing unit determines that the character length Ld of the bullet screen data Dm is between the first preset character length L1 and the second preset character length L2, and the bullet screen processing unit obtains the character type included in the bullet screen data Dm;
[0014] When Ld>L2, the barrage processing unit determines that the character length Ld of the barrage data Dm is higher than the second preset character length L2, the barrage processing unit does not perform semantic understanding on the barrage data Dm, and the barrage processing unit sends the barrage data Dm to the result analysis unit.
[0015] Furthermore, when determining that the character length Ld of the bullet-screen data Dm is between the first preset character length L1 and the second preset character length L2, the bullet-screen processing unit obtains the character type included in the bullet-screen data Dm and makes a determination.
[0016] If there is only one character type in the bullet-screen data Dm, the bullet-screen processing unit determines the output mode of the keyword according to the character type of the bullet-screen data Dm, and sends the bullet-screen data Dm and the output keyword to the result analysis unit;
[0017] If there is more than one character type in the bullet-screen data Dm, the bullet-screen processing unit will segment the bullet-screen data Dm according to each character type of the bullet-screen data Dm.
[0018] Furthermore, when there is more than one character type in the barrage data Dm, the barrage processing unit segments the barrage data Dm, selects data segments with the same character type and continuous as the segmentation results in the barrage data Dm, until the selected data segments completely cover the barrage data Dm, generates several barrage data segments, determines the output method of the keyword according to the character type of any barrage data segment, and sends the barrage data Dm and the output keyword to the result analysis unit.
[0019] Furthermore, when determining the output mode of the keyword according to the character type, the bullet screen processing unit performs a matching judgment on the character type of the bullet screen data Dm or the bullet screen data segment.
[0020] If the character type of the bullet-screen data Dm or the bullet-screen data segment is a text character, the bullet-screen processing unit will perform semantic understanding on the bullet-screen data Dm or the bullet-screen data segment to generate keywords;
[0021] If the character type of the bullet screen data Dm or the bullet screen data segment is a numeric character, the bullet screen processing unit outputs the bullet screen data Dm or the bullet screen data segment as a keyword;
[0022] If the character type of the bullet screen data Dm or the bullet screen data segment is a punctuation character, the bullet screen processing unit will not output keywords for the bullet screen data Dm or the bullet screen data segment;
[0023] If the character type of the barrage data Dm or the barrage data segment is other characters, the barrage processing unit outputs the barrage data Dm or the barrage data segment as a keyword.
[0024] Furthermore, a keyword color matrix WC is provided in the result analysis unit, WC=(W1-C1, W2-C2, W3-C3, ..., Wn-Cn), wherein W1 is the first preset keyword, W2 is the second preset keyword, W3 is the third preset keyword, ..., Wn is the nth preset keyword, C1 is the color corresponding to the first preset keyword, C2 is the color corresponding to the second preset keyword, C3 is the color corresponding to the third preset keyword, ..., Cn is the color corresponding to the nth preset keyword. When the result analysis unit receives the barrage data Dm sent by the barrage processing unit and a number of keywords corresponding to the barrage data Dm, the result analysis unit will determine the number of the several keywords corresponding to the barrage data Dm.
[0025] If the bullet-screen data Dm does not contain the corresponding keyword, the result analysis unit does not perform polishing on the bullet-screen data Dm, and the result analysis unit outputs the bullet-screen data Dm to the bullet-screen playback unit, which plays the bullet-screen data Dm output by the result analysis unit in a default color;
[0026] If the barrage data Dm has only one corresponding keyword, the result analysis unit selects the corresponding Wi in the keyword color matrix WC as the i-th preset keyword, and determines the color Ci corresponding to the i-th preset keyword as the output color of the barrage data Dm. The result analysis unit outputs the barrage data Dm and the color Ci to the barrage playback unit, and the barrage playback unit plays the barrage data Dm with the color Ci, where i=1, 2, 3, ..., n;
[0027] If the bullet-screen data Dm has more than one corresponding keyword, the result analysis unit will judge each keyword corresponding to the bullet-screen data Dm to determine the output color of the bullet-screen data Dm.
[0028] Furthermore, when there is more than one corresponding keyword in the barrage data Dm, the result analysis unit determines the number of keywords M corresponding to the barrage data Dm, and matches each keyword corresponding to the barrage data Dm with each stored barrage data in the barrage database. For any keyword We corresponding to Dm, the stored barrage data with the keyword We is selected as a matching item. When the matching items of each keyword are selected, the matching items selected by each keyword are integrated, the number of selection repetitions of each matching item is determined, and the matching items whose selection repetitions are equal to M are marked.
[0029] If there is no marked matching item among the matching items, the result analysis unit will determine the output color of the barrage data Dm according to the video time node data collected by the environment collection unit and each stored barrage data in the barrage database;
[0030] If there is only one marked matching item among the matching items, the result analysis unit will use the playback color of the stored bullet screen data corresponding to the matching item as the output color of the bullet screen data Dm, and send it to the bullet screen playback unit for playback;
[0031] If there is more than one marked match item in each match item, the result analysis unit will obtain the playback color of the stored barrage data corresponding to each marked match item, and calculate the RGB average value of each playback color, and use the calculated RGB average value as the output color of the barrage data Dm, and send it to the barrage playback unit for playback.
[0032] Furthermore, when there is no marked matching item in each matching item, the result analysis unit will obtain the video time node data Tm collected by the environment acquisition unit, and select each stored barrage data overlapping with the video time node data Tm in the barrage database, and calculate the RGB average value of the playback color corresponding to each stored barrage data, and use the calculated RGB average value as the output color of the barrage data Dm, and send it to the barrage playback unit for playback.
[0033] Furthermore, a preset barrage playback duration t is provided in the result analysis unit, and the result analysis unit can store the barrage data Dm with a character length between the first preset character length L1 and the second preset character length L2, the output color of the barrage data Dm and the playback time period of the barrage data Dm into the barrage database, wherein the playback time period of the barrage data Dm takes the video time node data Tm collected by the environment acquisition unit as the starting time node, and the preset barrage playback duration t as the duration of the playback time period of the barrage data Dm.
[0034] The present invention also provides an audio and video intelligent bullet screen polishing method, which is applied to any of the above-mentioned audio and video intelligent bullet screen polishing systems, comprising:
[0035] Step S1, obtaining the character type and character length of the collected bullet-screen data, performing semantic understanding according to the character type and character length of the bullet-screen data, and outputting several keywords of the semantic understanding corresponding to the bullet-screen data;
[0036] Step S2, determining a method for selecting an output color according to the number of keywords corresponding to the bullet screen data, and selecting an output color corresponding to the bullet screen data;
[0037] Step S3, playing the bullet screen data according to the output color, and storing the bullet screen data whose character length is between the first preset character length and the second preset character length.
[0038] Compared with the prior art, the beneficial effect of the present invention lies in that a barrage processing unit is set to obtain the character type and character length of the barrage data, and a semantic understanding is performed on the barrage data according to the semantic understanding method of the character length, and the result of the semantic understanding is output in the form of keywords, which can ensure the accuracy of the semantic understanding output result, and can quickly extract the characteristics of the barrage data, and further determine the keywords output by the barrage processing unit by setting a result analysis unit, determine the selection method of the output color of the barrage data according to the number of keywords, and set a keyword color matrix in the result analysis unit, directly perform color matching on a single output keyword, and quickly and accurately perform barrage polishing. When there are multiple output keywords, it indicates that the content of the barrage data is relatively complex. Therefore, by matching the stored barrage data in the barrage database, accurate matching items are selected, and the barrage content is accurately matched and understood, thereby improving the matching degree between the barrage content and the polishing effect.
[0039] In particular, by setting a first preset character length and a second preset character length in the barrage processing unit, the character length of the collected barrage data is determined. When the character length of the barrage data is lower than the first preset character length, it indicates that the input content of the barrage is relatively simple. Therefore, by directly performing semantic understanding processing on it, the barrage content features can be accurately extracted. When the character length of the barrage data is higher than the second preset character length, it indicates that the input content of the barrage is relatively complex, and multiple emotions expressed therein may coexist. Therefore, there is no need to polish the barrage data, and no semantic understanding is performed on it, so as to avoid incorrect matching of barrage polishing.
[0040] Furthermore, when determining that the character length of the barrage data is between the first preset character length and the second preset character length, the character type of the barrage data is determined. When there is only one character type, it indicates that the barrage content is simple to express. Therefore, the output method of the keyword can be directly determined according to the character type, so that the barrage content features can be accurately extracted to ensure the matching degree between the barrage polishing effect and the content.
[0041] Furthermore, when there is more than one character type in the barrage data, it means that the barrage input content is relatively complex. Therefore, it is segmented according to the continuous same character types, and keywords are determined for each barrage data segment respectively. Multiple keywords are generated to accurately extract features of the barrage content, thereby improving the matching degree between the barrage content and the polishing effect.
[0042] In particular, a result analysis unit is set up to make a preliminary judgment on the number of keywords corresponding to the barrage data to determine the complexity of the barrage content. When there is no corresponding keyword in the barrage data, it is played with the default color to avoid large deviations in the barrage matching. When there is only one corresponding keyword, the color output is directly selected according to the keyword color matrix for precise matching and polishing. When there is more than one corresponding keyword in the barrage data, it indicates that the barrage content is more complex, so the barrage database will be accurately matched to ensure the matching degree between the barrage polishing effect and the content.
[0043] Furthermore, by selecting corresponding matching items for each keyword and then integrating the matching items, it is possible to quickly select the stored barrage data with a high degree of overlap with the keywords in the barrage data and mark them, and then polish the barrage data according to the playback color of the marked matching items, thereby ensuring a high degree of matching of the polishing effect.
[0044] Furthermore, when there are no marked matching items among the matching items, the playback color of each stored barrage data under the video content at that moment is selected according to the video time node data, and the average value is selected as the output color of the barrage data, thereby ensuring that the barrage content and polishing can be effectively matched with the real-time video content, thereby improving the playback effect of the barrage polishing.
[0045] Furthermore, by storing the barrage data with character length between the first preset character length and the second preset character length, as well as its output color and playback time period, the basic data volume of the barrage database can be increased, thereby ensuring the accuracy of the matching results when matching the barrage data using the barrage database. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 Schematic diagram of the intelligent audio and video comment polishing system according to the present embodiment;
[0047] Figure 2 This is a flow chart of the method for intelligent barrage polishing of audio and video described in this embodiment. DETAILED DESCRIPTION
[0048] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0049] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.
[0050] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.
[0051] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0052] See also Figure 1 As shown, it is a schematic diagram of the intelligent bullet-screening polishing system for audio and video according to this embodiment. This embodiment discloses an intelligent bullet-screening polishing system for audio and video, including:
[0053] A bullet chat collection unit, which is used to collect the bullet chat data input by the client and determine the sent bullet chat data;
[0054] A bullet-screen processing unit connected to the bullet-screen collection unit, configured to obtain the character type and character length of the collected bullet-screen data, and perform semantic understanding according to the character type and character length of the bullet-screen data, and determine whether to output a number of keywords of the semantic understanding corresponding to the bullet-screen data;
[0055] An environment collection unit, which is used to collect video time node data of the real-time video playback of the client;
[0056] A result analysis unit, which is connected to the barrage processing unit and the environment acquisition unit respectively, and a keyword color matrix is set in the result analysis unit. When there is only one corresponding keyword in the barrage data, the result analysis unit selects the corresponding preset keyword in the keyword color matrix, and uses the color corresponding to the selected preset keyword as the output color of the barrage data. When there is more than one corresponding keyword in the barrage data, the result analysis unit matches each keyword corresponding to the barrage data with each stored barrage data in the barrage database, selects the stored barrage data with the keyword as a matching item, and determines the output color of the barrage data according to the selected matching item;
[0057] The bullet screen playback unit is connected to the result analysis unit and is used to play the bullet screen data according to the output color determined by the result analysis unit.
[0058] By setting a barrage processing unit to obtain the character type and character length of the barrage data, semantic understanding is performed on the barrage data according to the semantic understanding method of the character length, and the result of the semantic understanding is output in the form of keywords. This can ensure the accuracy of the semantic understanding output result, and can quickly extract the characteristics of the barrage data. By setting a result analysis unit, the keywords output by the barrage processing unit are further determined, and the selection method of the output color of the barrage data is determined according to the number of keywords. A keyword color matrix is set in the result analysis unit, and a single output keyword is directly matched in color to quickly and accurately perform barrage polishing. When there are multiple output keywords, it indicates that the content of the barrage data is relatively complex. Therefore, by matching the stored barrage data in the barrage database, accurate matching items are selected, and the barrage content is accurately matched and understood, thereby improving the matching degree between the barrage content and the polishing effect.
[0059] Specifically, a first preset character length L1 and a second preset character length L2 are set in the barrage processing unit, wherein L1<L2. When the barrage collection unit sends the collected barrage data Dm to the barrage processing unit, the barrage processing unit obtains the character length Ld of the barrage data Dm, and compares the character length Ld of the barrage data Dm with the first preset character length L1 and the second preset character length L2.
[0060] When Ld<L1, the barrage processing unit determines that the character length Ld of the barrage data Dm is less than the first preset character length L1, the barrage processing unit will perform semantic understanding on the barrage data Dm, generate keywords of the barrage data Dm, and send the barrage data Dm and the keywords to the result analysis unit;
[0061] When L1≤Ld≤L2, the bullet screen processing unit determines that the character length Ld of the bullet screen data Dm is between the first preset character length L1 and the second preset character length L2, and the bullet screen processing unit obtains the character type included in the bullet screen data Dm;
[0062] When Ld>L2, the barrage processing unit determines that the character length Ld of the barrage data Dm is higher than the second preset character length L2, the barrage processing unit does not perform semantic understanding on the barrage data Dm, and the barrage processing unit sends the barrage data Dm to the result analysis unit.
[0063] By setting a first preset character length and a second preset character length in the barrage processing unit, the character length of the collected barrage data is determined. When the character length of the barrage data is lower than the first preset character length, it indicates that the input content of the barrage is relatively simple. Therefore, by directly performing semantic understanding processing on it, the barrage content features can be accurately extracted. When the character length of the barrage data is higher than the second preset character length, it indicates that the input content of the barrage is relatively complex, and multiple emotions expressed may coexist. Therefore, there is no need to polish the barrage data, and no semantic understanding is performed on it, so as to avoid incorrect matching of barrage polishing.
[0064] Specifically, when determining that the character length Ld of the bullet-screen data Dm is between the first preset character length L1 and the second preset character length L2, the bullet-screen processing unit obtains the character type included in the bullet-screen data Dm and makes a determination.
[0065] If there is only one character type in the bullet-screen data Dm, the bullet-screen processing unit determines the output mode of the keyword according to the character type of the bullet-screen data Dm, and sends the bullet-screen data Dm and the output keyword to the result analysis unit;
[0066] If there is more than one character type in the bullet-screen data Dm, the bullet-screen processing unit will segment the bullet-screen data Dm according to each character type of the bullet-screen data Dm.
[0067] When determining that the character length of the barrage data is between the first preset character length and the second preset character length, the character type of the barrage data is determined. When there is only one character type, it means that the barrage content is simple to express. Therefore, the output method of the keyword can be directly determined according to the character type, so as to accurately extract the barrage content features and ensure the matching degree between the barrage polishing effect and the content.
[0068] Specifically, when there is more than one character type in the barrage data Dm, the barrage processing unit segments the barrage data Dm, selects data segments with the same character type and continuous as the segmentation results in the barrage data Dm, until the selected data segments completely cover the barrage data Dm, generates several barrage data segments, determines the output method of the keyword according to the character type of any barrage data segment, and sends the barrage data Dm and the output keyword to the result analysis unit.
[0069] When there is more than one character type in the barrage data, it means that the barrage input content is relatively complex. Therefore, it is segmented according to the continuous same character types, and keywords are determined for each barrage data segment. Multiple keywords are generated to accurately extract features of the barrage content, thereby improving the matching degree between the barrage content and the polishing effect.
[0070] Specifically, when determining the output mode of the keyword according to the character type, the bullet screen processing unit performs a matching judgment on the character type of the bullet screen data Dm or the bullet screen data segment.
[0071] If the character type of the bullet-screen data Dm or the bullet-screen data segment is a text character, the bullet-screen processing unit will perform semantic understanding on the bullet-screen data Dm or the bullet-screen data segment to generate keywords;
[0072] If the character type of the bullet screen data Dm or the bullet screen data segment is a numeric character, the bullet screen processing unit outputs the bullet screen data Dm or the bullet screen data segment as a keyword;
[0073] If the character type of the bullet screen data Dm or the bullet screen data segment is a punctuation character, the bullet screen processing unit will not output keywords for the bullet screen data Dm or the bullet screen data segment;
[0074] If the character type of the barrage data Dm or the barrage data segment is other characters, the barrage processing unit outputs the barrage data Dm or the barrage data segment as a keyword.
[0075] Specifically, a keyword color matrix WC is set in the result analysis unit, WC = (W1-C1, W2-C2, W3-C3, ..., Wn-Cn), wherein W1 is the first preset keyword, W2 is the second preset keyword, W3 is the third preset keyword, ..., Wn is the nth preset keyword, C1 is the color corresponding to the first preset keyword, C2 is the color corresponding to the second preset keyword, C3 is the color corresponding to the third preset keyword, ..., Cn is the color corresponding to the nth preset keyword. When the result analysis unit receives the barrage data Dm sent by the barrage processing unit and several keywords corresponding to the barrage data Dm, the result analysis unit will determine the number of several keywords corresponding to the barrage data Dm.
[0076] If the bullet-screen data Dm does not contain the corresponding keyword, the result analysis unit does not perform polishing on the bullet-screen data Dm, and the result analysis unit outputs the bullet-screen data Dm to the bullet-screen playback unit, which plays the bullet-screen data Dm output by the result analysis unit in a default color;
[0077] If the barrage data Dm has only one corresponding keyword, the result analysis unit selects the corresponding Wi in the keyword color matrix WC as the i-th preset keyword, and determines the color Ci corresponding to the i-th preset keyword as the output color of the barrage data Dm. The result analysis unit outputs the barrage data Dm and the color Ci to the barrage playback unit, and the barrage playback unit plays the barrage data Dm with the color Ci, where i=1, 2, 3, ..., n;
[0078] If the bullet-screen data Dm has more than one corresponding keyword, the result analysis unit will judge each keyword corresponding to the bullet-screen data Dm to determine the output color of the bullet-screen data Dm.
[0079] By setting the result analysis unit, a preliminary judgment is made on the number of keywords corresponding to the barrage data to determine the complexity of the barrage content. When there is no corresponding keyword in the barrage data, it is played with the default color to avoid large deviations in the barrage matching. When there is only one corresponding keyword, the color output is directly selected according to the keyword color matrix for precise matching and polishing. When there is more than one corresponding keyword in the barrage data, it indicates that the barrage content is more complex, so the barrage database will be accurately matched to ensure the matching degree between the barrage polishing effect and the content.
[0080] Specifically, when there is more than one corresponding keyword in the barrage data Dm, the result analysis unit determines the number of keywords M corresponding to the barrage data Dm, and matches each keyword corresponding to the barrage data Dm with each stored barrage data in the barrage database. For any keyword We corresponding to Dm, the stored barrage data with the keyword We is selected as a matching item. When the matching items of each keyword are selected, the matching items selected by each keyword are integrated, the number of selection repetitions of each matching item is determined, and the matching items whose selection repetitions are equal to M are marked.
[0081] If there is no marked matching item among the matching items, the result analysis unit will determine the output color of the barrage data Dm according to the video time node data collected by the environment collection unit and each stored barrage data in the barrage database;
[0082] If there is only one marked matching item among the matching items, the result analysis unit will use the playback color of the stored bullet screen data corresponding to the matching item as the output color of the bullet screen data Dm, and send it to the bullet screen playback unit for playback;
[0083] If there is more than one marked match item in each match item, the result analysis unit will obtain the playback color of the stored barrage data corresponding to each marked match item, and calculate the RGB average value of each playback color, and use the calculated RGB average value as the output color of the barrage data Dm, and send it to the barrage playback unit for playback.
[0084] By selecting the corresponding matching items for each keyword and then integrating the matching items, it is possible to quickly select the stored barrage data with a high degree of overlap with the keywords in the barrage data and mark them, and then polish the barrage data according to the playback color of the marked matching items, ensuring a high degree of matching of the polishing effect.
[0085] Specifically, when there is no marked matching item in each matching item, the result analysis unit will obtain the video time node data Tm collected by the environment acquisition unit, and select each stored barrage data overlapping with the video time node data Tm in the barrage database, and calculate the RGB average value of the playback color corresponding to each stored barrage data, and use the calculated RGB average value as the output color of the barrage data Dm, and send it to the barrage playback unit for playback.
[0086] When there are no marked matching items among the matching items, the playback color of each stored barrage data under the video content at that moment is selected according to the video time node data, and the average value is selected as the output color of the barrage data to ensure that the barrage content and polishing can be effectively matched with the real-time video content, thereby improving the playback effect of the barrage polishing.
[0087] Specifically, a preset barrage playback duration t is set in the result analysis unit. The result analysis unit can store the barrage data Dm with a character length between the first preset character length L1 and the second preset character length L2, the output color of the barrage data Dm and the playback time period of the barrage data Dm into the barrage database, wherein the playback time period of the barrage data Dm takes the video time node data Tm collected by the environment acquisition unit as the starting time node, and the preset barrage playback duration t is the duration of the playback time period of the barrage data Dm.
[0088] By storing the barrage data with a character length between the first preset character length and the second preset character length, as well as its output color and playback time period, the basic data volume of the barrage database can be increased, thereby ensuring the accuracy of the matching results when matching the barrage data using the barrage database.
[0089] Please continue reading Figure 2 As shown, it is a flow chart of the method for intelligently polishing audio and video bullet comments according to this embodiment. This embodiment discloses a method for intelligently polishing audio and video bullet comments, including:
[0090] Step S1, obtaining the character type and character length of the collected bullet-screen data, performing semantic understanding according to the character type and character length of the bullet-screen data, and outputting several keywords of the semantic understanding corresponding to the bullet-screen data;
[0091] Step S2, determining a method for selecting an output color according to the number of keywords corresponding to the bullet screen data, and selecting an output color corresponding to the bullet screen data;
[0092] Step S3, playing the bullet screen data according to the output color, and storing the bullet screen data whose character length is between the first preset character length and the second preset character length.
[0093] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.
[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. An intelligent video and audio comment polishing system, characterized in that: include, A bullet chat collection unit, which is used to collect the bullet chat data input by the client and determine the sent bullet chat data; A bullet-screen processing unit connected to the bullet-screen collection unit, configured to obtain the character type and character length of the collected bullet-screen data, and perform semantic understanding according to the character type and character length of the bullet-screen data, and determine whether to output a number of keywords of the semantic understanding corresponding to the bullet-screen data; An environment collection unit, which is used to collect video time node data of the real-time video playback of the client; A result analysis unit, which is connected to the barrage processing unit and the environment acquisition unit respectively, and a keyword color matrix is set in the result analysis unit. When there is only one corresponding keyword in the barrage data, the result analysis unit selects the corresponding preset keyword in the keyword color matrix, and uses the color corresponding to the selected preset keyword as the output color of the barrage data. When there is more than one corresponding keyword in the barrage data, the result analysis unit matches each keyword corresponding to the barrage data with each stored barrage data in the barrage database, selects the stored barrage data with the keyword as a matching item, and determines the output color of the barrage data according to the selected matching item; A bullet screen playback unit, connected to the result analysis unit, for playing the bullet screen data according to the output color determined by the result analysis unit; When there is more than one corresponding keyword in the bullet-screen data Dm, the result analysis unit determines the number of keywords M corresponding to the bullet-screen data Dm, and matches each keyword corresponding to the bullet-screen data Dm with each stored bullet-screen data in the bullet-screen database. For any keyword We corresponding to Dm, the stored bullet-screen data with the keyword We is selected as a matching item. When the matching items of each keyword are selected, the matching items selected by each keyword are integrated, the number of selection repetitions of each matching item is determined, and the matching items whose selection repetitions are equal to M are marked. If there is no marked matching item among the matching items, the result analysis unit will determine the output color of the barrage data Dm according to the video time node data collected by the environment collection unit and each stored barrage data in the barrage database; If there is only one marked matching item among the matching items, the result analysis unit will use the playback color of the stored bullet screen data corresponding to the matching item as the output color of the bullet screen data Dm, and send it to the bullet screen playback unit for playback; If there is more than one marked match item in each match item, the result analysis unit will obtain the playback color of the stored barrage data corresponding to each marked match item, and calculate the RGB average value of each playback color, and use the calculated RGB average value as the output color of the barrage data Dm, and send it to the barrage playback unit for playback.
2. The intelligent video and audio comment polishing system according to claim 1, characterized in that: The barrage processing unit is provided with a first preset character length L1 and a second preset character length L2, wherein L1<L2. When the barrage acquisition unit sends the collected barrage data Dm to the barrage processing unit, the barrage processing unit obtains the character length Ld of the barrage data Dm, and compares the character length Ld of the barrage data Dm with the first preset character length L1 and the second preset character length L2. When Ld<L1, the barrage processing unit determines that the character length Ld of the barrage data Dm is less than the first preset character length L1, the barrage processing unit will perform semantic understanding on the barrage data Dm, generate keywords of the barrage data Dm, and send the barrage data Dm and the keywords to the result analysis unit; When L1≤Ld≤L2, the bullet screen processing unit determines that the character length Ld of the bullet screen data Dm is between the first preset character length L1 and the second preset character length L2, and the bullet screen processing unit obtains the character type included in the bullet screen data Dm; When Ld>L2, the barrage processing unit determines that the character length Ld of the barrage data Dm is higher than the second preset character length L2, the barrage processing unit does not perform semantic understanding on the barrage data Dm, and the barrage processing unit sends the barrage data Dm to the result analysis unit.
3. The intelligent video and audio comment polishing system according to claim 2, characterized in that: When determining that the character length Ld of the bullet-screen data Dm is between the first preset character length L1 and the second preset character length L2, the bullet-screen processing unit obtains the character type included in the bullet-screen data Dm and makes a determination. If there is only one character type in the bullet-screen data Dm, the bullet-screen processing unit determines the output mode of the keyword according to the character type of the bullet-screen data Dm, and sends the bullet-screen data Dm and the output keyword to the result analysis unit; If there is more than one character type in the bullet-screen data Dm, the bullet-screen processing unit will segment the bullet-screen data Dm according to each character type of the bullet-screen data Dm.
4. The intelligent video and audio comment polishing system according to claim 3, characterized in that: When there is more than one character type in the barrage data Dm, the barrage processing unit segments the barrage data Dm, selects data segments with the same character type and continuous as the segmentation results in the barrage data Dm, until the selected data segments completely cover the barrage data Dm, generates several barrage data segments, determines the output method of the keyword according to the character type of any barrage data segment, and sends the barrage data Dm and the output keyword to the result analysis unit.
5. The intelligent video and audio comment polishing system according to claim 4, characterized in that: When determining the output mode of the keyword according to the character type, the bullet screen processing unit performs matching judgment on the character type of the bullet screen data Dm or the bullet screen data segment. If the character type of the bullet-screen data Dm or the bullet-screen data segment is a text character, the bullet-screen processing unit will perform semantic understanding on the bullet-screen data Dm or the bullet-screen data segment to generate keywords; If the character type of the bullet screen data Dm or the bullet screen data segment is a numeric character, the bullet screen processing unit outputs the bullet screen data Dm or the bullet screen data segment as a keyword; If the character type of the bullet screen data Dm or the bullet screen data segment is a punctuation character, the bullet screen processing unit will not output keywords for the bullet screen data Dm or the bullet screen data segment; If the character type of the barrage data Dm or the barrage data segment is other characters, the barrage processing unit outputs the barrage data Dm or the barrage data segment as a keyword.
6. The intelligent video and audio comment polishing system according to claim 5, characterized in that: The result analysis unit is provided with a keyword color matrix WC, WC=(W1-C1, W2-C2, W3-C3, ..., Wn-Cn), wherein W1 is the first preset keyword, W2 is the second preset keyword, W3 is the third preset keyword, ..., Wn is the nth preset keyword, C1 is the color corresponding to the first preset keyword, C2 is the color corresponding to the second preset keyword, C3 is the color corresponding to the third preset keyword, ..., Cn is the color corresponding to the nth preset keyword. When the result analysis unit receives the barrage data Dm sent by the barrage processing unit and a number of keywords corresponding to the barrage data Dm, the result analysis unit will determine the number of the keywords corresponding to the barrage data Dm. If the bullet-screen data Dm does not contain the corresponding keyword, the result analysis unit does not perform polishing on the bullet-screen data Dm, and the result analysis unit outputs the bullet-screen data Dm to the bullet-screen playback unit, which plays the bullet-screen data Dm output by the result analysis unit in a default color; If the barrage data Dm has only one corresponding keyword, the result analysis unit selects the corresponding Wi in the keyword color matrix WC as the i-th preset keyword, and determines the color Ci corresponding to the i-th preset keyword as the output color of the barrage data Dm. The result analysis unit outputs the barrage data Dm and the color Ci to the barrage playback unit, and the barrage playback unit plays the barrage data Dm with the color Ci, where i=1, 2, 3, ..., n; If the bullet-screen data Dm has more than one corresponding keyword, the result analysis unit will judge each keyword corresponding to the bullet-screen data Dm to determine the output color of the bullet-screen data Dm.
7. The intelligent video and audio comment polishing system according to claim 6, characterized in that: When there is no marked matching item in each matching item, the result analysis unit will obtain the video time node data Tm collected by the environment acquisition unit, and select each stored barrage data overlapping with the video time node data Tm in the barrage database, and calculate the RGB average value of the playback color corresponding to each stored barrage data, and use the calculated RGB average value as the output color of the barrage data Dm, and send it to the barrage playback unit for playback.
8. The intelligent video and audio comment polishing system according to claim 7, characterized in that: The result analysis unit is provided with a preset barrage playback duration t, and the result analysis unit can store the barrage data Dm with a character length between the first preset character length L1 and the second preset character length L2, the output color of the barrage data Dm and the playback time period of the barrage data Dm into the barrage database, wherein the playback time period of the barrage data Dm takes the video time node data Tm collected by the environment acquisition unit as the starting time node, and the preset barrage playback duration t as the duration of the playback time period of the barrage data Dm.
9. An intelligent video bullet comment polishing method, applied to the intelligent video bullet comment polishing system according to any one of claims 1 to 8, characterized in that: include, Step S1, obtaining the character type and character length of the collected bullet-screen data, performing semantic understanding according to the character type and character length of the bullet-screen data, and outputting several keywords of the semantic understanding corresponding to the bullet-screen data; Step S2, determining a method for selecting an output color according to the number of keywords corresponding to the bullet screen data, and selecting an output color corresponding to the bullet screen data; Step S3, playing the bullet screen data according to the output color, and storing the bullet screen data whose character length is between the first preset character length and the second preset character length.
