Data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a data processing method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining a video feature corresponding to a to-be-projected video; acquiring first information matched with the video features; and determining a to-be-pushed bullet screen based on the first information, wherein the to-be-pushed bullet screen is used for being pushed when the video is played. Therefore, the information of multiple different data sources matched with the video features of the video can be displayed and pushed in the screen projection process, so that the content of the bullet screen is richer and more dimensional, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of data processing technology, and in particular to a data processing method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Screen projection technology is mainly used to transmit content on mobile devices or computers to large-screen devices in real time to better display the content. However, existing screen projection technology can usually only display and transmit video content, which is relatively simple and provides poor user experience. Summary of the invention

[0003] The embodiments of the present disclosure provide a data processing method, an apparatus, an electronic device, and a computer-readable storage medium, which aim to solve one of the technical problems in the related art at least to a certain extent.

[0004] In a first aspect, an embodiment of the present disclosure provides a data processing method, the method comprising:

[0005] Get the video features corresponding to the video to be projected;

[0006] Acquire first information matching the video feature;

[0007] The barrage to be pushed is determined based on the first information, and the barrage to be pushed is used to push when playing the video.

[0008] In a second aspect, an embodiment of the present disclosure further provides a data processing device, the device comprising:

[0009] The first acquisition module is used to acquire video features corresponding to the video to be projected;

[0010] A second acquisition module, used to acquire first information matching the video feature;

[0011] A determination module is used to determine the barrage to be pushed based on the first information, and the barrage to be pushed is used to push when playing the video.

[0012] In a third aspect, an embodiment of the present disclosure further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and which implements the steps in the above-mentioned data processing method when the computer program is executed by the processor.

[0013] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned data processing method are implemented.

[0014] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations of the embodiments of the present disclosure.

[0015] In the disclosed embodiment, the video features corresponding to the video to be projected are first obtained, and then the first information matching the video features is obtained, and then the bullet screen to be pushed is determined based on the first information, and the bullet screen to be pushed is used to push when the video is played. In this way, during the projection process, information from multiple different data sources matching the video features of the video can be displayed and pushed, thereby making the content of the bullet screen richer and more multi-dimensional, and improving the user experience.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a flowchart of a data processing method provided by the first embodiment of the present disclosure;

[0019] Figure 2 is a flowchart of a data processing method provided by the second embodiment of the present disclosure;

[0020] Figure 3 is a flowchart of a data processing method provided by the third embodiment of the present disclosure;

[0021] Figure 4 is a schematic diagram of feature extraction of key frame picture data provided by an embodiment of the present disclosure;

[0022] Figure 5 is a flowchart of a data processing method provided by the fourth embodiment of the present disclosure;

[0023] Figure 6 is a flowchart of an exemplary data processing method provided by an embodiment of the present disclosure;

[0024] Figure 7 is a structural schematic diagram of a data processing device provided by an embodiment of the present disclosure;

[0025] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for clarity and brevity, descriptions of features known in the art may be omitted.

[0027] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0028] It should be noted that the executor of the data processing method of this embodiment may be a data processing device, which may be configured in any type of electronic device, such as a television, a projector, a computer, etc., and is not limited here.

[0029] In the embodiments of the present disclosure, the data processing method is described by taking the "data processing device" as the execution subject, and no limitation is made here. It should be noted that the description order of the following embodiments is not used as a limitation on the priority order of the embodiments.

[0030] Figure 1 It is a flowchart of a data processing method provided according to the first embodiment of the present disclosure.

[0031] like Figure 1 As shown, the method includes:

[0032] Step 101, obtaining video features corresponding to the video to be projected.

[0033] The video to be projected can be a video selected by the user in any way for projection.

[0034] For example, the user can select through a mobile device, or a non-mobile device (such as a desktop computer). Alternatively, the user can also select the video to be cast through voice control, which is not limited here.

[0035] For example, the user can select a football match video as the video to be projected by touching the mobile phone. Alternatively, the user can also say "project movie M" to the tablet computer, and the tablet computer can select movie M as the video to be projected, and the tablet computer establishes a connection with the projection device to project movie M, etc., which is not limited here.

[0036] Among them, video features can be specific information that describes video content or attributes, including but not limited to one or more of visual features, content features, character features, scene features, time features, audio features, etc., which are not limited here.

[0037] As a possible implementation method, features of the video to be projected may be first extracted to obtain target feature data, and then the target feature data may be analyzed to obtain video features.

[0038] The target feature data may include at least one of the following: video title, key frame timestamp, key frame picture data, key frame-related audio data, and key frame-related subtitle data.

[0039] The video title may reflect the main theme or content type of the video and is used to reflect the content characteristics of the video.

[0040] Among them, the key frame timestamp can be the time label information corresponding to the key frame picture in the video. The timestamp information can help analyze the time structure and rhythm of the video. For example, if the key frame timestamps are evenly distributed, it means that the rhythm of the video may be relatively stable; if the timestamps appear in a certain time period, it may mean that the content of this part of the video is relatively rich or an important event has occurred.

[0041] The key frame picture data may be image data of the key frame picture.

[0042] The audio data associated with the key frame may be audio data within a period of time before and after the key frame, for example, audio track data within 5 seconds before and after the key frame, which is not limited here.

[0043] The subtitle data associated with the key frame may be subtitle data within a period of time before and after the key frame, for example, subtitle text within 5 seconds before and after the key frame, which is not limited here.

[0044] It should be noted that for different types of target feature data, the feature extraction methods used may be different. For example, for video titles, they can be directly obtained from the metadata of the video. For key frame timestamps, video encoding technology can be used to determine the key frames in the video, and then the timestamps of the key frames appearing can be recorded. For each key frame, its picture data can be extracted, and image processing technology, such as color histogram, texture features, shape features, etc., can be used to describe the picture content of the key frame. The color histogram can reflect the color distribution of the key frame; the texture features can describe the texture details of the key frame; the shape features can identify the shape of the object in the key frame. While extracting the key frame, the audio data and subtitle data associated with the key frame can be obtained.

[0045] Furthermore, by analyzing the key frame data, video features such as the theme, scene, and object of the video can be determined. For example, if a large number of natural scenery images appear in the key frame, it can be determined that the theme of the video may be related to nature; if a specific person or object appears in the key frame, the person's characteristics or object characteristics can be identified, which are not limited here.

[0046] Optionally, the time structure of the video can be analyzed by using the key frame timestamps. For example, the video features such as the rhythm, climax, and plot development of the video can be determined. If the time interval between key frames is short, it means that the rhythm of the video is fast; if the key frames are concentrated in a certain time period, it may mean that the climax of the video appears in this time period, which is not limited here.

[0047] Optionally, by analyzing the audio data associated with the key frames, the video features such as the atmosphere and emotion of the video can be understood. For example, passionate music may indicate that the atmosphere of the video is tense and exciting; soft music may indicate that the atmosphere of the video is warm and peaceful, which is not limited here.

[0048] Optionally, by analyzing the subtitle data associated with the keyframes, video features such as the theme and emotional tendency of the video can be further confirmed. Keyword extraction can help determine the main topic of the video, and sentiment analysis can determine whether the emotional tone of the video is positive or negative, which is not limited here.

[0049] It should be noted that target feature data can be of various types, and video features can also be of various types, which are not limited here.

[0050] Step 102: Acquire first information matching the video feature.

[0051] The first information may be information matching the video feature.

[0052] In the disclosed embodiment, the first information may be comment information, which is not limited herein.

[0053] Optionally, the corresponding first information may be matched from a pre-built comment database based on video features.

[0054] The comment database stores user comment data from one or more data sources. Preferably, the data source can refer to different information platforms, such as forums, video playback applications, social media applications, etc. The data sources of the comment database cover a variety of different types of platforms, and the user comment data on these platforms are rich and diverse, and can provide information from different angles and viewpoints. For example, in a forum, users may have an in-depth discussion on the plot of a specific TV series; in a forum, professionals may analyze and comment on the shooting technology of a movie; and the comments in the video playback application focus more on aspects such as the viewing experience.

[0055] Optionally, when constructing a comment database, public comment data can be obtained from external websites through crawler technology. The crawler program can automatically access the web pages of various platforms, grab user comment content, and store it in the comment database. The information in the comment database can be updated in real time, can be updated periodically, and can also be updated based on specific trigger conditions, for example, triggering the update of the comment database based on a video playback instruction, etc., which is not limited here.

[0056] For example, by setting specific keywords or target websites, the crawler can search and capture comments related to these keywords, such as "popular movie reviews", "discussions on a certain TV series forum", etc. Then, the captured data can be cleaned and sorted, and then stored in the comment database for subsequent matching and use.

[0057] It is understandable that by integrating review data from multiple platforms, more comprehensive and diverse information can be obtained, providing more options for determining the first information.

[0058] As a possible implementation method, the matching degree between each comment information in the comment database and the video feature may be calculated first, and then the comment information with a matching degree greater than a preset threshold is used as the first information associated with the video feature.

[0059] For example, if there are 10 comments in the comment database, and the matching degree between comment 1 and comment 2 and the video feature is 0.9, which is greater than the preset threshold of 0.8, then comment 1 and comment 2 are considered to be associated with the video feature, and comment 1 and comment 2 can be used as the first information, which is not limited here.

[0060] It should be noted that when calculating the matching degree between each comment information in the comment database and the video feature, the matching degree calculation methods corresponding to different video features may be the same or different, which is not limited here.

[0061] For example, if the video features only include subtitle location features and subtitle character features, then a text matching algorithm (such as string matching, regular expression matching, etc.) can be used to compare the location feature set in the video with the location information in the comments to calculate the degree of match. The degree of match can be defined as the ratio of the number of matched locations to the size of the video location feature set. Similarly, a text matching algorithm can be used to compare the character feature set in the video with the character information in the comments to calculate the degree of match. The degree of match can also be defined as the ratio of the number of matched characters to the size of the video character feature set. Finally, the location feature matching degree and the character feature matching degree can be weighted averaged or other combination strategies can be used to obtain the final matching degree. The choice of weights can be determined based on the actual application scenario and requirements.

[0062] For example, if the video features include person mentions, location mentions, and picture mentions corresponding to the video frames.

[0063] Optionally, a text matching algorithm or a semantic matching algorithm (such as word vector, TF-IDF, etc.) can be used to compare the character mention feature set in the video with the character information in the comment to calculate the matching degree. The matching degree can be defined as the ratio or similarity score between the matched character mention information and the information amount in the video character mention feature set. Then, a text matching algorithm or a geographic information system (GIS) related technology can be used to compare the place mention feature set in the video with the place information in the comment to calculate the matching degree. The matching degree can also be defined as the ratio or similarity score between the matched place mention information and the information amount in the video place mention feature set. Optionally, an image matching algorithm or a text and image joint matching algorithm can be used to compare the picture feature set of the video frame with the picture mention information in the comment to calculate the matching degree. The matching degree can be defined as the ratio or similarity score between the number of matched picture features and the size of the picture feature set of the video frame. Afterwards, the character mention matching degree, the place mention matching degree and the picture mention matching degree corresponding to the video frame can be weighted averaged or other combination strategies (such as machine learning model prediction, etc.) can be adopted to obtain the final matching degree. The choice of weights can be determined according to the actual application scenario and requirements.

[0064] As a possible implementation method, the device can also send the target feature data to the cloud after obtaining the target feature data. The cloud can process the target feature data to obtain first information matching the video feature, and then the device can receive the first information sent by the cloud.

[0065] Step 103, determining the bullet screen to be pushed based on the first information, and the bullet screen to be pushed is used to push when playing the video.

[0066] The bullet screen to be pushed may be a bullet screen to be displayed to the user. The bullet screen may be in the form of voice bullet screen, text bullet screen, expression bullet screen, symbol bullet screen, etc., which are not limited here.

[0067] Optionally, the first information may be filtered based on preset filtering rules to obtain the bullet comments to be pushed.

[0068] The screening rule is used to screen the first information to provide viewers with a better and more personalized viewing experience. Optionally, the screening rule includes at least one of the following: platform screening rule, emotion screening rule, keyword screening rule, and popularity screening rule.

[0069] It should be noted that only one screening rule or multiple screening rules may be used, and there are many types of screening rules, which are not limited here. The above-mentioned multiple screening rules will be schematically described below.

[0070] Among them, the platform screening rule is a rule for screening the first information according to the platform type. For example, only the first information of the video playback application and the social media platform can be selected from the first information. In addition, the first information of a specific platform can also be selected from multiple platforms of the same type. For example, if the first information contains first information from video playback application A, video playback application B, and video playback application C, only the first information from video playback application A can be retained according to the preset platform screening rule, which is not limited here.

[0071] Among them, the keyword screening rule is a rule for screening the first information based on keywords. For example, if the video is about food, words such as "delicious", "yummy", and "cooking" can be used as keywords to screen out the first information containing these keywords. The keyword screening rule can be adjusted according to different video types. For example, a movie can set keywords such as actor names and movie types, and a sports game can set keywords such as athlete names and competition events. There is no limitation here.

[0072] Among them, the emotional screening rule is a rule for screening based on the emotional tendency of the first information. For example, the first information can be divided into three emotional types: positive, negative and neutral. For example, positive barrages may enhance the audience's viewing experience, and negative barrages may affect the audience's emotions. Therefore, a suitable emotional tendency can be pre-selected as a screening criterion, such as selecting the emotional tendency as "positive" or "neutral" as a screening criterion, and then the first information is screened based on the screening criterion.

[0073] Among them, the heat screening rule refers to screening based on the heat value of the first information. The heat value can be calculated based on user behavior data, including the number and frequency of interactive behaviors such as reading, liking, commenting, and sharing. These behavioral data can reflect the user's attention and interest in the information, and thus serve as an important indicator for measuring the heat of the information. Different first information corresponds to different heat values. For example, the first information with a heat value greater than a preset heat value threshold can be used as a barrage to be pushed, which is not limited here.

[0074] As a possible implementation method, after obtaining the first information, the first information may be preliminarily screened, for example, first information containing sensitive words, advertising information, and meaningless characters may be removed, which is not limited here.

[0075] As another possible implementation method, the first information can also be filtered based on the user's habitual preference settings. For example, the user can pre-set personal filtering rules, such as filtering out information containing keyword X1 (such as a character name, a movie name), filtering out information from platform X2, and filtering out information with negative emotional tendencies, which are not limited here. In this way, the filtered first information can be more in line with the user's preferences.

[0076] As another possible implementation method, when filtering the first information, it can also be filtered according to the preset number of bullet comments. For example, if the preset number of bullet comments is 200, the device can filter out 200 first information from the first information as bullet comments to be pushed, which is not limited here. In this way, it can prevent the number of bullet comments from being too many or too few, affecting the user's viewing experience.

[0077] As another possible implementation method, the first information input by other users when they cast the video and watch it can also be used as the barrage to be pushed, which is not limited here. It can be understood that the first information input by other users can also be filtered according to the above filtering rules and then used as the barrage to be pushed, which is not limited here.

[0078] Specifically, when playing a video, the barrage can be displayed on the video playback interface, so that users can see the barrage while watching the video.

[0079] Optionally, the barrage can be rendered first, and then the rendered barrage can be added to the video to be projected.

[0080] Specifically, the font and color of the barrage can be rendered.

[0081] The choice of font will affect the readability and aesthetics of the bullet screen. You can choose a clear and easy-to-read font to ensure that the bullet screen content can be easily recognized by users at different screen sizes and resolutions. In addition, you can also choose a font style that matches the style and atmosphere of the video, such as cute, serious, and technological, to enhance the overall visual effect.

[0082] Optionally, you can also choose a suitable location to place the bullet screen according to the layout and content characteristics of the video. For example, the bullet screen can be displayed at the top or bottom edge of the video to avoid blocking the key content of the video. At the same time, you can also provide different location options for users to choose according to their habits and preferences.

[0083] Optionally, the device can synthesize the video frame and the bullet screen data through a synthesis rendering module, and then accurately superimpose the bullet screen on the video frame, ensuring that the position, font and color of the bullet screen are consistent with the preset configuration. Then, the device can play and display the synthesized video frame and bullet screen data.

[0084] In the disclosed embodiment, the video features corresponding to the video to be projected are first obtained, and then the first information matching the video features is obtained, and then the bullet screen to be pushed is determined based on the first information, and the bullet screen to be pushed is used to push when the video is played. In this way, during the projection process, information from multiple different data sources matching the video features of the video can be displayed and pushed, thereby making the content of the bullet screen richer and more multi-dimensional, and improving the user experience.

[0085] based on Figure 1 In the embodiment, the data processing device can not only determine the bullet screen for pushing when the video is played based on the first information, but also obtain the second information input by the user during the video viewing process, thereby increasing the interactive relationship between the user and other viewers. Figure 2 As shown, it is the second embodiment of the data processing method provided by the present application. Figure 2 The second embodiment of the data processing method provided specifically includes:

[0086] 201. Obtain video features corresponding to the video to be projected.

[0087] Among them, step 201 can refer to Figure 1 The step 101 in FIG. 1 is understood and will not be described in detail here.

[0088] 202. Obtain first information matching the video feature.

[0089] Among them, step 202 can refer to Figure 1 The step 102 in FIG. 1 is understood and will not be described in detail here.

[0090] 203. Acquire second information input by the user in real time.

[0091] The second information may be comment information input by the user in real time, such as one or more of text information, expression information, symbol information, etc., which are not limited here. Optionally, the user may generate the second information through any device such as a mobile device or a remote controller, and then the device sends the second information to the data processing device, so that the second information input by the user in real time can be obtained.

[0092] It should be noted that the embodiment of the present application does not limit the order of step 202 and step 203.

[0093] 204. Determine the bullet screen to be pushed based on the first information and the second information.

[0094] In an optional implementation, the first information and the second information may be combined to form a bullet screen to be pushed.

[0095] In another optional implementation, since the first information and the second information may contain a large amount of information after being combined, the first information and the second information may be filtered, and then the filtered information is used as the bullet screen to be pushed.

[0096] For example, the first information and the second information may be filtered based on a preset filtering rule to obtain the bullet screen to be pushed. It should be noted that the method of filtering the first information and the second information based on the filtering rule may refer to the above-mentioned process of filtering the first information, which will not be described in detail here.

[0097] In the disclosed embodiment, the video features corresponding to the video to be projected are first obtained, and then the first information matching the video features is obtained. Then, based on the first information and the second information input by the user in real time, the bullet screen to be pushed is determined, and then the bullet screen is displayed when the video is played. Thus, while watching the projected video, the user can not only input his own thoughts and feelings as the second information, but also see the bullet screen determined by the first information and the second information determined according to the video features, which not only enriches the content of the bullet screen, but also enables the user to establish a multi-dimensional interactive relationship with the video content and other viewers, thereby better improving the user's viewing experience.

[0098] Figure 3 It is a flowchart of a data processing method provided according to the third embodiment of the present disclosure.

[0099] like Figure 3 As shown, the method includes:

[0100] Step 301: decode the video to be projected to obtain multiple first video frames.

[0101] The first video frame may be a video frame obtained after decoding the video.

[0102] Among them, video decoding is the process of converting compressed video data into a displayable image sequence. By decoding the video to be projected, multiple first video frames can be obtained, which contain the image information of the video. Among them, the decoding operation can be completed by a video decoder, and the decoder can restore the compressed data to the original image data according to the encoding format of the video.

[0103] Step 302: extract a second video frame from a plurality of first video frames based on a preset frame extraction method.

[0104] The frame extraction method may be a method of extracting specific frames from each video frame.

[0105] The second video frame may be a video frame extracted from a plurality of first video frames by a preset frame extraction method.

[0106] As a possible implementation method, frame extraction may be performed at a key frame interval or a fixed time interval, which is not limited here.

[0107] For extracting frames at fixed time intervals, it is necessary to first set a fixed time interval, which can be determined according to the video playback speed and the required data volume. For example, if the video is played at a speed of 30 frames per second, 1 frame, 5 frames or more frames can be selected per second as the fixed time interval, which is not limited here. Then, the second video frame can be extracted from each consecutive first video frame according to the set time interval.

[0108] For key frame interval extraction, key frames may be extracted from the video stream at a certain interval. For example, if the key frame interval is set to 10, a second video frame is extracted from each first video frame every 10 frames, which is not limited here.

[0109] As another possible implementation method, the key frame interval frame extraction method and the fixed time interval frame extraction method can be combined, and then the minimum frame interval of the two methods is used for frame extraction.

[0110] That is to say, after selecting the two frame extraction methods of "key frame interval extraction" and "fixed time interval extraction", the frame intervals obtained by the two methods are compared, and the smaller interval is selected as the final frame extraction interval. Therefore, by selecting a smaller interval, it can be ensured that the extracted frames contain key information while minimizing redundancy.

[0111] For example, if the fixed time interval frame extraction is to extract 1 frame every 1 second, and the video is played at 30 frames per second, then the frame interval is 30 frames. If the key frame interval is 10 frames, which is less than the frame interval of 30 in the fixed time interval frame extraction method, the key frame interval frame extraction method can be used, which is not limited here.

[0112] Step 303: determine whether the second video frame is a key frame.

[0113] Optionally, whether the second video frame is a key frame may be determined by video coding information. Alternatively, whether the second video frame is a key frame may be determined by image content analysis. Alternatively, whether the second video frame is a key frame may be determined by combining video coding information and image content analysis, which is not limited here.

[0114] For example, the type of the current frame can be determined by parsing the frame header information of the second video frame. If the frame type is identified as an I frame (Intra-coded picture), it can be determined that the second video frame is a key frame. For example, in the H.264 encoding format, the NAL (Network Abstraction Layer) unit type of the second video frame can be analyzed to determine whether the frame is a key frame. The NAL unit type of an I frame is usually "5" or "7".

[0115] It is understandable that a key frame usually has a large difference from the previous and next frames, because the key frame contains complete image information, while other frames are encoded by the difference from the key frame. Therefore, the image difference between the second video frame and the previous and next adjacent frames can also be calculated. If the difference is large, it may be a key frame.

[0116] It should be noted that there are many ways to detect whether the second video frame is a key frame, and the present disclosure does not limit this.

[0117] Step 304: when the second video frame is a key frame, obtain key frame picture data corresponding to the second video frame.

[0118] Optionally, when the second video frame is a key frame, the device can obtain key frame picture data corresponding to the second video frame, and then the device can perform feature extraction based on the key frame picture data. When the second video frame is not a key frame, the device does not use the data of the second video frame.

[0119] Optionally, when the second video frame is a key frame, it can be determined first whether the data amount of the second video frame is greater than a first preset threshold value, and then, when the data amount of the second video frame is less than the first preset threshold value, the pre-decoded data corresponding to the second video frame is used as the key frame picture data to be adopted, and then, when the data amount of the second video frame is greater than or equal to the first preset threshold value, the second video frame is re-encoded, and the re-encoded data of the second video frame is used as the key frame picture data to be adopted.

[0120] The first preset threshold may be a data volume threshold of the second video frame.

[0121] If the data amount of the second video frame is greater than or equal to the first preset threshold, it means that the data amount of the second video frame is large. If the data amount of the second video frame is less than the first preset threshold, it means that the data amount of the second video frame is small.

[0122] It is understandable that if the data volume of the second video frame is greater than or equal to the first preset threshold, it means that the data volume of the second video frame is relatively large and may contain relatively complex image content. Therefore, the second video frame needs to be re-encoded and the re-encoded data is used as the key frame picture data to be adopted.

[0123] Specifically, re-encoding can adopt a more efficient encoding method to reduce the data volume of the second video frame while maintaining the image quality as much as possible, making it more suitable for subsequent feature analysis processing. Through re-encoding, the data of the key frame can be adjusted to a suitable range to better extract the picture features.

[0124] It is understandable that if the data amount of the second video frame is less than the first preset threshold, it means that the data amount of the second video frame is relatively small. Therefore, the video data of the second video frame before decoding can be used. The second video frame is obtained through a decoding operation. The data of the second video frame before decoding contains the encoding parameters and structural information of the video, which is more valuable for certain specific feature analysis. In addition, the decoding operation may also cause some errors or changes in the video frame data, resulting in some potential information deformation. Therefore, when the data amount of the second video frame is small, the pre-decoding data corresponding to the second video frame can be used as the key frame picture data to be adopted.

[0125] Step 305: Analyze the target feature data to obtain video features, wherein the target feature data is key frame picture data.

[0126] Step 306: Acquire first information matching the video feature.

[0127] Step 307, determining the bullet screen to be pushed based on the first information, and the bullet screen to be pushed is used to push when playing the video.

[0128] It should be noted that the specific implementation of steps 305-307 can refer to the above embodiment and will not be described in detail here.

[0129] In the disclosed embodiment, the video to be projected is first decoded to obtain a plurality of first video frames, and then a second video frame is extracted from the plurality of first video frames based on a preset frame extraction method, and then it is determined whether the second video frame is a key frame, and then the key frame picture data corresponding to the second video frame is obtained when the second video frame is a key frame, and then the target feature data is analyzed to obtain the video feature, wherein the target feature data is the key frame picture data, and then the first information matching the video feature is obtained, and then the barrage to be pushed is determined based on the first information, and the barrage to be pushed is used to push when playing the video. Thus, by extracting the second video frame and judging the key frame by the preset frame extraction method, important picture information in the video can be efficiently extracted. Analyzing the key frame picture data to obtain the video feature helps to grasp the core content and characteristics of the video more accurately, and provides a reliable basis for subsequent information matching and barrage generation. And it can display information from a variety of different data sources matching the video features of the video in the process of projecting, so that the content of the barrage is richer and more multi-dimensional, and the user experience is improved.

[0130] Figure 4 FIG. 1 is a schematic diagram of feature extraction of key frame image data provided by an embodiment of the present disclosure. Figure 4 As shown, the frame extraction module first determines the frame extraction mode, which has two frame extraction modes: key frame interval extraction and fixed time interval extraction. Then, the minimum interval extraction is selected from the two frame extraction modes, and then it is determined whether the extracted video frame is a key frame (I frame), and then the data size can be determined. If the data is less than the threshold (the first preset threshold), the data before decoding can be used. If the data is greater than the threshold, the decoded data can be re-encoded, and then the feature value extraction module is used for decoding and feature extraction.

[0131] Figure 5 It is a flowchart of a data processing method provided according to the fourth embodiment of the present disclosure.

[0132] like Figure 5 As shown, the method includes:

[0133] Step 401, obtaining video features corresponding to the video to be projected.

[0134] Among them, video features may include but are not limited to video mentions, voiceprint features, subtitle location features, subtitle character features, character mentions, location mentions, and screen mentions, screen location features, and screen portrait features corresponding to the video frames.

[0135] The video mention rate can be the frequency or degree of a video being mentioned within a specific range, such as the number of times it is shared and discussed on social media platforms, or the number of times it is rated on professional film and television review websites. A higher video mention rate means that the video has a higher level of attention and influence.

[0136] The voiceprint feature may be a feature obtained by analyzing the audio data in the video, such as the timbre, pitch, rhythm, etc. of the audio, which is not limited here.

[0137] The subtitle location feature may be location information mentioned in the subtitle, which can help determine the scene or geographical location that appears in the video. For example, if the video is a natural scenery video, "Jiuzhaigou" may appear in the subtitle. Then, the device can obtain the associated first information based on the subtitle location feature "Jiuzhaigou".

[0138] The character information appearing in the subtitles may constitute the subtitle character features. For example, if the video is a video introducing a historical figure, the historical figure "Zhu Yuanzhang" may appear in the video. Then, the device obtains the first information associated with the subtitle character feature "Zhu Yuanzhang".

[0139] Among them, the character mention rate refers to the frequency of a specific character being mentioned in a video. If a character appears frequently or is discussed in detail in a video, then the character mention rate will be relatively high. For example, there are 108 characters mentioned in the video, among which "Zhang San" and "Li Si" are mentioned relatively frequently. The device can then obtain the first information associated with "Zhang San" and "Li Si".

[0140] Among them, the picture mention degree can indicate the degree to which the picture content of the video frame is paid attention to or discussed. For example, some pictures with shocking visual effects may have a higher picture mention degree. Alternatively, some classic pictures are also paid attention to relatively more.

[0141] The location features of the picture may be elements such as geographical landmarks, architectural styles, and natural landscapes that appear in the video frame, which can help identify the location of the video frame or the scene where the story takes place. For example, if the Eiffel Tower appears in the picture, it can be inferred that the location feature of the video frame is related to Paris, France.

[0142] The image portrait features may be elements such as the appearance, expression, and action of the person in the video frame. By analyzing the image portrait features, the identity, emotional state, and possible behavioral intention of the person can be determined. For example, if a certain video frame shows a female actress crying, the image portrait features can be determined to be "crying" or "sad", which are not limited here.

[0143] In the disclosed embodiment, the video features may include screen mentions, screen location features, and screen portrait features corresponding to the video frames.

[0144] It should be noted that the specific implementation of step 301 can refer to the above embodiment and will not be described in detail here.

[0145] Step 402 , obtaining target comment data from a comment database based on a time interval between a first historical time and a current time, wherein the time interval is less than a second preset threshold.

[0146] The second preset threshold may be a time interval threshold, such as one week, five days, or one month, which is not limited here. In the disclosed embodiment, the second preset threshold may be a smaller value, such as one week. Since the time interval between the first historical time and the current time is less than the second preset threshold, it indicates that the first historical time is closer to the current time.

[0147] The target comment data may be comment data generated in a time interval between the first historical time and the current time.

[0148] The first historical time may represent a historical time that is closer to the current time. The time interval between the first historical time and the current time is less than the second preset threshold, indicating that the first historical time is closer to the current time. Therefore, the target comment data obtained by the device according to the time interval is recent comment data, which is more timely.

[0149] Step 403, when the picture mention degree corresponding to any video frame is greater than the third preset threshold, based on the picture location features and picture portrait features corresponding to any video frame, obtain the first information corresponding to any video frame from the target comment data.

[0150] Among them, the third preset threshold is the threshold of the screen mention degree. If the screen mention degree corresponding to any video frame is greater than the third preset threshold, it means that the screen mention degree of the video frame is relatively high, indicating that the screen content of the video frame may be highly concerned or discussed. If the screen mention degree corresponding to any video frame is less than or equal to the third preset threshold, it means that the screen mention degree of the video frame is relatively low, indicating that the screen content of the video frame may be less concerned or discussed.

[0151] For example, if the video frames acquired by the device include A1, A2, and A3. Since A3 is a classic scene in the video, the scene mention degree is relatively high. Therefore, the device can obtain data associated with the scene location feature and the scene portrait feature of the video frame A3 from the target comment data as the first information.

[0152] Optionally, the first information may be filtered from the target comment data using techniques such as keyword matching and semantic analysis, which are not limited here.

[0153] For example, if the location feature of the video frame is "seaside", then the target comment data can be searched for comment data related to keywords such as "seaside", "beach", "ocean", and "seawater" as the first information. If the portrait feature of the video frame is "smile", then the comment data related to emotions such as "happy" and "joyful" can be searched for as the first information.

[0154] It should be noted that the above examples are only illustrative and are not intended to limit the present disclosure.

[0155] Step 404, determining the bullet screen to be pushed based on the first information, and the bullet screen to be pushed is used to push when playing the video.

[0156] It should be noted that the specific implementation of step 404 can refer to the above embodiment and will not be described in detail here.

[0157] In the disclosed embodiment, firstly, the video features corresponding to the video to be projected are obtained, and then the target comment data is obtained from the comment database based on the time interval between the first historical time and the current time, and the time interval is less than the second preset threshold. Then, when the screen mention degree corresponding to any video frame is greater than the third preset threshold, the first information corresponding to any video frame is obtained from the target comment data based on the screen location features and screen portrait features corresponding to any video frame, and then the barrage to be pushed is determined based on the first information, and the barrage to be pushed is used to push when playing the video. Therefore, by setting the time interval between the first historical time and the current time to be less than the second preset threshold to obtain the target comment data, it is ensured that the comment information used has a high timeliness. The user can see comments related to the recent status of the video, making the viewing experience closer to the present. The device obtains the first information based on the screen location features and screen portrait features of the video frame with a high screen mention degree, so that the barrage content is more personalized, in line with the user's focus and interest in specific video content, and greatly enhances the user's viewing experience.

[0158] Figure 6 FIG. 1 is a flow chart of an exemplary data processing method provided by an embodiment of the present disclosure. Figure 6As shown, the user first determines the video to be projected through the video source, and then the device can decode and extract features from the video through the feature value extraction module to obtain feature parameters, such as key frame timestamp, video title, key frame data, audio track data within 5 seconds before and after the key frame, and subtitle text within 5 seconds before and after the key frame. After that, a request for barrage data can be sent to the barrage matching system. The barrage matching system obtains the video / screen mention degree, portrait recognition in the screen, location matching, voiceprint recognition of the audio track, and the names and locations of the characters mentioned in the subtitle text / audio track text through the feature analysis system and feature parameters. Then the comment extraction module is used to extract comments. For example, the recent hot comments about the content of the video clip and the recent hot comments about the characters / places identified in the video clip can then form an external barrage to feedback to the device. Users can input comments through mobile devices or remote controls, and the device can combine real-time comments with external barrages. After that, the barrage and video can be synthesized and rendered and displayed.

[0159] In order to better implement the data processing method of the present disclosure, the present disclosure also provides a data processing device based on the above data processing method. The meanings of the terms are the same as those in the above data processing method, and the specific implementation details can refer to the description in the method embodiment.

[0160] See also Figure 7 , Figure 7 is a structural diagram of a data processing device provided in an embodiment of the present disclosure, the data processing device 600 includes:

[0161] A first acquisition module 610 is used to acquire video features corresponding to the video to be projected;

[0162] A second acquisition module 620, configured to acquire first information matching the video feature;

[0163] The determination module 630 is used to determine the bullet comments to be pushed based on the first information, and the bullet comments to be pushed are used to push when playing the video.

[0164] Optionally, determine the module, also used to:

[0165] Obtaining second information input by the user;

[0166] Optionally, the determining module is specifically used to:

[0167] Based on the first information and the second information, the bullet screen to be pushed is determined.

[0168] Optionally, the acquisition module includes:

[0169] An extraction unit, used for extracting features of the video to be projected to obtain target feature data;

[0170] an analysis unit, configured to analyze the target feature data to obtain video features,

[0171] The target feature data includes at least one of the following: video title, key frame timestamp, key frame picture data, key frame-related audio data, and key frame-related subtitle data.

[0172] Optionally, the target feature data is key frame picture data, and the extraction unit includes:

[0173] A decoding subunit, configured to decode the video to be projected to obtain a plurality of first video frames;

[0174] An extraction subunit, configured to extract a second video frame from the plurality of first video frames based on a preset frame extraction method;

[0175] A judging subunit, used to judge whether the second video frame is a key frame;

[0176] The acquisition subunit is used to acquire key frame picture data corresponding to the second video frame when the second video frame is a key frame.

[0177] Optionally, the acquisition subunit is specifically used to:

[0178] Determining whether the data amount of the second video frame is greater than a first preset threshold;

[0179] When the data amount of the second video frame is less than the first preset threshold, using the pre-decoded data corresponding to the second video frame as the key frame picture data to be adopted;

[0180] When the data amount of the second video frame is greater than or equal to the first preset threshold, the second video frame is re-encoded, and the re-encoded data of the second video frame is used as the key frame picture data to be adopted.

[0181] Optionally, the acquisition module includes:

[0182] a matching unit, configured to match corresponding first information from a pre-built comment database based on the video feature,

[0183] The video feature includes at least one of the following:

[0184] Video mentions, voiceprint features, subtitle location features, subtitle character features, character mentions, location mentions, and the screen mentions, screen location features, and screen portrait features corresponding to the video frame.

[0185] The comment database stores user comment data from different types of platforms.

[0186] Optionally, the video features include screen mentions, screen location features, and screen portrait features corresponding to the video frames, and the matching unit is specifically configured to:

[0187] acquiring target comment data from the comment database based on a time interval between the first historical time and the current time, the time interval being less than a second preset threshold;

[0188] When the picture mention degree corresponding to any video frame is greater than a third preset threshold,

[0189] Based on the picture location features and the picture portrait features corresponding to the any video frame, first information corresponding to the any video frame is obtained from the target comment data.

[0190] Optionally, the determining module is specifically used to:

[0191] Based on the preset filtering rules, the first information is filtered to obtain the bullet comments to be pushed.

[0192] Optionally, the determining module is specifically used to:

[0193] Based on preset filtering rules, the first information and the second information are filtered to obtain the barrage to be pushed, wherein the filtering rules include at least one of the following: platform filtering rules, emotion filtering rules, keyword filtering rules, and popularity filtering rules.

[0194] Optionally, determine the module, also used to:

[0195] Rendering the bullet comments;

[0196] Adding the rendered bullet comment to the video to be projected;

[0197] Play the video to be projected.

[0198] The data processing device provided by the embodiment of the present disclosure can first obtain the video features corresponding to the video to be projected, then obtain the first information matching the video features, and then determine the bullet screen to be pushed based on the first information, and the bullet screen to be pushed is used to push when playing the video. Therefore, during the projection process, information from multiple different data sources matching the video features of the video can be displayed and pushed, thereby making the content of the bullet screen richer and more multi-dimensional, and improving the user experience.

[0199] In addition, the present disclosure also provides an electronic device, such as Figure 8 As shown, it shows a schematic diagram of the structure of the electronic device involved in the present disclosure, specifically:

[0200] The electronic device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will appreciate that Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0201] The processor 701 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 702, and calling data stored in the memory 702, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701.

[0202] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0203] The electronic device also includes a power supply 703 for supplying power to each component. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 703 can also include any components such as one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, and power status indicators.

[0204] The electronic device may further include an input unit 704, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0205] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 701 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the application programs stored in the memory 702, thereby implementing the steps in any data processing method provided in the embodiments of the present disclosure.

[0206] In the disclosed embodiment, the video features corresponding to the video to be projected are first obtained, and then the first information matching the video features is obtained, and then the bullet screen to be pushed is determined based on the first information, and the bullet screen to be pushed is used to push when the video is played. In this way, during the projection process, information from multiple different data sources matching the video features of the video can be displayed and pushed, thereby making the content of the bullet screen richer and more multi-dimensional, and improving the user experience.

[0207] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0208] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0209] To this end, the present disclosure provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any data processing method provided by the present disclosure.

[0210] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0211] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0212] Since the instructions stored in the computer-readable storage medium can execute the steps in any data processing method provided in the present disclosure, the beneficial effects that can be achieved by any data processing method provided in the present disclosure can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0213] The data processing method, device, electronic device and computer-readable storage medium provided by the present disclosure are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A data processing method, characterized in that: include: Get the video features corresponding to the video to be projected; Acquire first information matching the video feature; The barrage to be pushed is determined based on the first information, and the barrage to be pushed is used to push when playing the video.

2. The method according to claim 1, characterized in that Before determining the bullet screen to be pushed based on the first information, the method further includes: Obtaining second information input by the user; Correspondingly, determining the bullet screen to be pushed based on the first information includes: Based on the first information and the second information, the bullet screen to be pushed is determined.

3. The method according to claim 1, characterized in that: The obtaining of video features corresponding to the video to be projected includes: Extracting features of the video to be projected to obtain target feature data; Analyzing the target feature data to obtain video features, The target feature data includes at least one of the following: video title, key frame timestamp, key frame picture data, key frame-related audio data, and key frame-related subtitle data.

4. The method according to claim 3, characterized in that The target feature data is key frame picture data, and the feature extraction of the video to be projected to obtain the target feature data includes: Decoding the video to be projected to obtain a plurality of first video frames; Extracting a second video frame from the plurality of first video frames based on a preset frame extraction method; Determining whether the second video frame is a key frame; When the second video frame is a key frame, key frame picture data corresponding to the second video frame is acquired.

5. The method according to claim 4, characterized in that The acquiring key frame picture data corresponding to the second video frame includes: Determining whether the data amount of the second video frame is greater than a first preset threshold; When the data amount of the second video frame is less than the first preset threshold, using the pre-decoded data corresponding to the second video frame as the key frame picture data to be adopted; When the data amount of the second video frame is greater than or equal to the first preset threshold, the second video frame is re-encoded, and the re-encoded data of the second video frame is used as the key frame picture data to be adopted.

6. The method according to claim 1, characterized in that The acquiring first information matching the video feature includes: Based on the video feature, matching the corresponding first information from a pre-built comment database, The video feature includes at least one of the following: Video mentions, voiceprint features, subtitle location features, subtitle character features, character mentions, location mentions, and the screen mentions, screen location features, and screen portrait features corresponding to the video frame. The comment database stores user comment data from different types of platforms.

7. The method according to claim 6, characterized in that The video features include screen mentions, screen location features, and screen portrait features corresponding to the video frame, and matching the corresponding first information from a pre-built comment database based on the video features includes: acquiring target comment data from the comment database based on a time interval between the first historical time and the current time, the time interval being less than a second preset threshold; When the picture mention degree corresponding to any video frame is greater than a third preset threshold, Based on the picture location features and the picture portrait features corresponding to the any video frame, first information corresponding to the any video frame is obtained from the target comment data.

8. The method according to claim 2, characterized in that: The determining the bullet screen to be pushed based on the first information includes: Based on the preset screening rules, the first information is screened to obtain the bullet comments to be pushed, Correspondingly, determining the bullet screen to be pushed based on the first information and the second information includes: Based on a preset screening rule, the first information and the second information are screened to obtain the bullet screen to be pushed, Among them, the screening rules include at least one of the following: platform screening rules, emotion screening rules, keyword screening rules, and popularity screening rules.

9. The method according to claim 1, characterized in that: After determining the bullet screen to be pushed based on the first information, the method further includes: Rendering the bullet comments; Adding the rendered bullet comment to the video to be projected; Play the video to be projected.

10. A data processing device, characterized in that: include: The first acquisition module is used to acquire video features corresponding to the video to be projected; A second acquisition module, used to acquire first information matching the video feature; A determination module is used to determine the barrage to be pushed based on the first information, and the barrage to be pushed is used to push when playing the video.

11. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the method according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 9 are implemented.

Images