Video content processing method and device, electronic device and storage medium

Abstract

The present application provides a method and device, electronic device and storage medium for processing video content, wherein the method comprises: obtaining the transition time point of the video to be processed; obtaining the first video frame sequence and the second video frame sequence corresponding to the transition time point; obtaining a sequence group composed of the first video frame sequence and the second video frame sequence, calculating the brightness feature distance between the two video frames in the sequence group, and determining the feature maximum value; comparing the numerical value relationship between the feature maximum value and the first preset threshold value, and merging the first video frame sequence and the second video frame sequence based on the comparison result. Through the present application, the problems existing in the related art of inaccurate determination of the start and end time points of the video and unreasonable splitting of the video content are solved.

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to a method and device for processing video content, an electronic device, and a storage medium. Background Art

[0002] At present, the segmentation results of video shots are often used in video platforms, and they are put into use after further processing. For example: it is necessary to detect all the shot segments in a long video, cut the segments, and after further detecting the tags, store the start and end frames of the shots together with the tags in the video material library. When making materials later, you can directly call the complete shot segments with a certain tag from the library, and then perform operations such as mixing and synthesis, and finally distribute the video segments on the site for promotion or distribute them outside the site for diversion.

[0003] In the existing video splitting scheme, only the pixel value characteristics of the picture frame are considered, such as a completely black screen. However, in special cases, the content adjacent to the left and right of the black screen may be related to the plot. Simply relying on the segmentation of the existing algorithm cannot form a segment with coherent content well, resulting in unreasonable start and end time points of the split segments. Summary of the invention

[0004] The present application provides a method and device for processing video content, an electronic device and a storage medium, so as to at least solve the problems in the related art of inaccurate determination of the start and end time points of a video and unreasonable segmentation of video content.

[0005] According to one aspect of an embodiment of the present application, a method for processing video content is provided, the method comprising:

[0006] Get the transition time point of the video to be processed;

[0007] Acquire a first video frame sequence and a second video frame sequence corresponding to the transition time point, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point;

[0008] Acquire a sequence group consisting of the first video frame sequence and the second video frame sequence, calculate the brightness feature distance between any two video frames in the sequence group, and determine a feature maximum value, wherein the feature maximum value includes a maximum value of the brightness feature distance and a minimum value of the brightness feature distance;

[0009] The numerical value relationship between the feature maximum value and a first preset threshold is compared, and the first video frame sequence and the second video frame sequence are merged based on the comparison result.

[0010] According to another aspect of the embodiment of the present application, a method for generating video material is also provided, the method comprising:

[0011] Obtaining a location identifier of the transition time point in the video to be processed;

[0012] The video frame sequence before the positioning mark and the video frame sequence after the positioning mark are merged to generate a merged video material.

[0013] According to another aspect of the embodiment of the present application, a device for processing video content is also provided, the device comprising:

[0014] A first acquisition unit, used to acquire a transition time point of a video to be processed;

[0015] A second acquisition unit is used to acquire a first video frame sequence and a second video frame sequence corresponding to the transition time point, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point sequence, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point sequence;

[0016] a third acquisition unit, configured to acquire a sequence group consisting of the first video frame sequence and the second video frame sequence, calculate a brightness feature distance between any two video frames in the sequence group, and determine a feature maximum value, wherein the feature maximum value includes a maximum value of the brightness feature distance and a minimum value of the brightness feature distance;

[0017] The first merging unit is used to compare the numerical value relationship between the feature maximum value and a first preset threshold, and merge the first video frame sequence and the second video frame sequence based on the comparison result.

[0018] According to another aspect of the embodiment of the present application, a device for generating video material is also provided. The device uses various units and modules of the device for processing video content to complete the generation of video material. The device includes:

[0019] A seventh acquisition unit, used to acquire a location identifier of the transition time point in the video to be processed;

[0020] The second merging unit is used to merge the video frame sequence located before the positioning mark and the video frame sequence located after the positioning mark to generate a merged video material.

[0021] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein the memory is used to store a computer program; and the processor is used to execute the method steps in any of the above embodiments by running the computer program stored in the memory.

[0022] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the method steps in any of the above embodiments when executed.

[0023] In an embodiment of the present application, a transition time point of a video to be processed is obtained; a first video frame sequence and a second video frame sequence corresponding to the transition time point are obtained, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point sequence, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point sequence; a sequence group consisting of the first video frame sequence and the second video frame sequence is obtained, the brightness feature distance between each two video frames in the sequence group is calculated, and a feature maximum value is determined, wherein the feature maximum value includes a maximum value of the brightness feature distance and a minimum value of the brightness feature distance; a numerical value relationship between the feature maximum value and a first preset threshold value is compared, and the first video frame sequence and the second video frame sequence are merged based on the comparison result. Since the embodiment of the present application calculates the brightness feature distance between each pair of video frames in the sequence group of the left adjacent video frame sequence and the right adjacent video frame sequence at the transition time point to obtain the feature maximum value, and uses the feature maximum value to perform numerical comparison with the first preset threshold value to determine whether the left adjacent video frame sequence and the right adjacent video frame sequence are to be merged, this can avoid the situation where the video content continuity is poor due to unreasonable start and end time points of the video split segments, thereby providing more available complete shot segments, achieving the purpose of providing more available materials for mixed cutting, and further solving the problems of inaccurate determination of the start and end time points of the video and unreasonable video content splitting existing in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 is a schematic diagram of a hardware environment of an optional video content processing method according to an embodiment of the present invention;

[0027] Figure 2 is a flowchart of an optional method for processing video content according to an embodiment of the present application;

[0028] Figure 3 is a schematic diagram of an optional overall process of determining a lens change point according to an embodiment of the present application;

[0029] Figure 4 is a structural block diagram of an optional video content processing device according to an embodiment of the present application;

[0030] Figure 5 It is a structural block diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] According to one aspect of an embodiment of the present application, a method for processing video content is provided. Optionally, in this embodiment, the method for processing video content can be applied to Figure 1 In the hardware environment shown in the figure. Figure 1As shown, the terminal 102 may include a memory 104, a processor 106, and a display 108 (optional component). The terminal 102 may be connected to a server 112 via a network 110 for communication. The server 112 may be used to provide services (such as game services, application services, etc.) for the terminal or a client installed on the terminal. A database 114 may be set on the server 112 or independently of the server 112 to provide data storage services for the server 112. In addition, a processing engine 116 may be running in the server 112, and the processing engine 116 may be used to execute the steps executed by the server 112.

[0034] Optionally, the terminal 102 may be, but is not limited to, a terminal that can calculate data, such as a mobile terminal (e.g., a mobile phone, a tablet computer), a laptop, a PC (Personal Computer), etc. The above network may include, but is not limited to, a wireless network or a wired network. The wireless network includes: Bluetooth, WIFI (Wireless Fidelity) and other networks that implement wireless communication. The above wired network may include, but is not limited to: a wide area network, a metropolitan area network, and a local area network. The above server 112 may include, but is not limited to, any hardware device that can perform calculations.

[0035] In addition, in this embodiment, the above-mentioned video content processing method can also be applied to, but not limited to, an independent processing device with a relatively powerful processing capability without data interaction. For example, the processing device can be, but not limited to, a terminal device with a relatively powerful processing capability, that is, each operation in the above-mentioned video content processing method can be integrated in an independent processing device. The above is only an example, and this embodiment does not make any limitation to this.

[0036] Optionally, in this embodiment, the above-mentioned video content processing method can be executed by the server 112, can be executed by the terminal 102, or can be executed by the server 112 and the terminal 102. Among them, the terminal 102 executes the video content processing method of the embodiment of the present application, or it can be executed by a client installed thereon.

[0037] Take running on the server as an example, Figure 2 is a flow chart of an optional method for processing video content according to an embodiment of the present application, such as Figure 2 As shown, the process of the method may include the following steps:

[0038] Step S201, obtaining the transition time point of the video to be processed;

[0039] Step S202, obtaining a first video frame sequence and a second video frame sequence corresponding to the transition time point, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point sequence, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point sequence;

[0040] Step S203, obtaining a sequence group consisting of a first video frame sequence and a second video frame sequence, calculating a brightness feature distance between two video frames in the sequence group, and determining a feature maximum value, wherein the feature maximum value includes a maximum value of the brightness feature distance and a minimum value of the brightness feature distance;

[0041] Step S204, comparing the numerical value relationship between the maximum feature value and the first preset threshold, and merging the first video frame sequence and the second video frame sequence based on the comparison result.

[0042] Optionally, in an embodiment of the present application, all video frames of a video are first extracted according to the frame rate of the video to obtain the transition time point of the video. The time point of the transition is usually defined as the time point corresponding to the video frame where a black screen appears. Then, a video frame sequence that is adjacent to the transition time point on the left when arranged in time sequence is obtained as the first video frame sequence, and a video frame sequence that is adjacent to the transition time point on the right is obtained as the second video frame sequence.

[0043] A sequence group consisting of a first video frame sequence and a second video frame sequence is obtained, and then histogram features are extracted for each video frame in the sequence group, and timestamps of all video frames and histogram features of the video frames {'1': [timestamp, frame_feature]} are stored in seconds, and then the brightness feature distance between each two video frames is calculated according to the histogram features to obtain feature maximum values, wherein the feature maximum values ​​include the maximum value of the brightness feature distance and the minimum value of the brightness feature distance.

[0044] In the embodiment of the present application, a first preset threshold is set, and the first preset threshold is a critical value for judging the content coherence between the first video frame sequence and the second video frame sequence. At this time, the obtained feature maximum value and the first preset threshold are compared in numerical value. Based on the comparison result, when the comparison result meets the merging condition, it is determined to merge the first video frame sequence and the second video frame sequence.

[0045] In an embodiment of the present application, a transition time point of a video to be processed is obtained; a first video frame sequence and a second video frame sequence corresponding to the transition time point are obtained, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point sequence, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point sequence; a sequence group consisting of the first video frame sequence and the second video frame sequence is obtained, the brightness feature distance between each two video frames in the sequence group is calculated, and a feature maximum value is determined, wherein the feature maximum value includes a maximum value of the brightness feature distance and a minimum value of the brightness feature distance; a numerical value relationship between the feature maximum value and a first preset threshold value is compared, and the first video frame sequence and the second video frame sequence are merged based on the comparison result. Since the embodiment of the present application calculates the brightness feature distance between each pair of video frames in the sequence group of the left adjacent video frame sequence and the right adjacent video frame sequence at the transition time point to obtain the feature maximum value, and uses the feature maximum value to perform numerical comparison with the first preset threshold value to determine whether the left adjacent video frame sequence and the right adjacent video frame sequence are to be merged, this can avoid the situation where the video content continuity is poor due to unreasonable start and end time points of the video split segments, thereby providing more available complete shot segments, achieving the purpose of providing more available materials for mixed cutting, and further solving the problems of inaccurate determination of the start and end time points of the video and unreasonable video content splitting existing in the related art.

[0046] As an optional embodiment, comparing the numerical value relationship between the feature maximum value and the first preset threshold, and merging the first video frame sequence and the second video frame sequence based on the comparison result includes:

[0047] Compare the maximum value of the brightness feature distance with the first sub-threshold to obtain a first comparison result, wherein the first preset threshold includes the first sub-threshold;

[0048] Compare the minimum value of the brightness feature distance with the second sub-threshold to obtain a second comparison result, wherein the first preset threshold includes the second sub-threshold;

[0049] According to the first comparison result and the second comparison result, it is determined to merge the first video frame sequence and the second video frame sequence.

[0050] Optionally, two sub-thresholds are included in the first preset threshold, namely, a first sub-threshold and a second sub-threshold. The maximum value of the brightness feature distance is compared with the first sub-threshold. If the maximum value of the brightness feature distance is less than the first sub-threshold, a first comparison result is obtained; the minimum value of the brightness feature distance is compared with the second sub-threshold. If the minimum value of the brightness feature distance is less than the second sub-threshold, a second comparison result is obtained. At this time, after the first comparison result and the second comparison result are satisfied at the same time, it is determined that the first video frame sequence and the second video frame sequence are merged and processed.

[0051] In an embodiment of the present application, by comparing the maximum value of the brightness feature distance with the first sub-threshold, and comparing the minimum value of the brightness feature distance with the second sub-threshold, and then determining to merge the first video frame sequence and the second video frame sequence based on the comparison result, it is possible to recognize the video content at the frame level and obtain more available complete shot clips.

[0052] As an optional embodiment, before comparing the numerical value relationship between the characteristic maximum value and the first preset threshold, the method further includes:

[0053] Determine the channel type to which the current scene image belongs;

[0054] The first sub-threshold and the second sub-threshold are determined according to the channel type, wherein the first sub-threshold and / or the second sub-threshold are obtained by a channel coefficient corresponding to the channel type.

[0055] Optionally, it is possible to first determine based on experience which channel type the scene picture in the current sequence group belongs to, such as variety shows, animation, TV series, etc., and then set the values ​​of the first sub-threshold and the second sub-threshold according to the channel type to which the current picture belongs. The first sub-threshold and / or the second sub-threshold are obtained by the channel coefficient corresponding to the channel type, preferably, they can also be obtained by multiplying the channel coefficient corresponding to the channel type by the average value of the brightness feature distance between two video frames in the sequence group, such as setting the first sub-threshold to 0.65, etc., and setting the second sub-threshold to 0.2, 0.2*average value, etc.

[0056] At the same time, the following code can show the association and judgment conditions between the first sub-threshold, the second sub-threshold and the channel type, and determine whether the first video frame sequence needs to be merged with the second video frame sequence through the judgment conditions:

[0057]

[0058]

[0059] In an embodiment of the present application, the first sub-threshold and the second sub-threshold are obtained according to the channel type to which the current scene image belongs, so that the merging judgment condition between the video frames determined in this way is more accurate and more conducive to obtaining complete and coherent shot content.

[0060] As an optional embodiment, the method further includes:

[0061] The time point at which a black screen frame exists in all the video frames of the video to be processed is taken as the transition time point, wherein, when the first video frame sequence corresponding to the transition time point includes a gradual change shot, the second video frame sequence corresponding to the transition time point includes a sudden change shot; or, when the first video frame sequence corresponding to the transition time point includes a sudden change shot, the second video frame sequence corresponding to the transition time point includes a gradual change shot.

[0062] Optionally, in an embodiment of the present application, the time point when a black screen frame exists in all the video frames of the video to be processed can be used as the transition time point. Since the appearance of the black screen frame is usually due to the switching of the lens, especially at the time point of switching between a sudden shot and a gradual shot, when the first video frame sequence corresponding to the transition time point includes a gradual shot, the second video frame sequence corresponding to the transition time point will include a sudden shot; or, when the first video frame sequence corresponding to the transition time point includes a sudden shot, the second video frame sequence corresponding to the transition time point will include a gradual shot.

[0063] As an optional embodiment, the method of determining the gradient lens includes:

[0064] Acquire a first characteristic distance between a start frame and an end frame of the first video frame sequence or the second video frame sequence;

[0065] In the case where the first feature distance is indeed greater than the second preset threshold, obtaining any two video frames in the first video frame sequence or the second video frame sequence, wherein the second preset threshold is obtained by calculating the feature distance by combining each two of the video frames in the first video frame sequence or the second video frame sequence, and the distance between the two video frames in the first video frame sequence or the second video frame sequence is less than the preset value;

[0066] Determine a second characteristic distance between two video frames, and when the second characteristic distance is less than a third preset threshold, determine that the first video frame sequence or the second video frame sequence contains a gradient shot, wherein the third preset threshold is the product of the first characteristic distance and a first preset coefficient, and the first preset coefficient is a numerical value for quantitatively adjusting the first characteristic distance.

[0067] Optionally, in an embodiment of the present application, a first feature distance between a start frame and an end frame of a first video frame sequence or a second video frame sequence is obtained, wherein the first feature distance is a histogram feature distance. It can be understood that the histogram is used to characterize the brightness of the current frame of the video, and calculating the histogram feature distance is also to calculate the brightness value between the two video frames.

[0068] Among them, the embodiment of the present application can store the timestamps of all video frames in the first video frame sequence or the second video frame sequence and the histogram features of the video frame corresponding to the current timestamp in seconds, represented by {'video frame 1': [timestamp, frame_feature], ...}, and then calculate the histogram feature distance between the two video frames, D = {d(1, 2), d(k+1, k+2), ..., d(n-1, n)}, where n represents the total number of frames in the first video frame sequence or the second video frame sequence. Then the embodiment of the present application sets a second preset threshold in advance, and the second preset threshold can be the value obtained by calculating the average value of all the histogram feature distances after all the video frames in the set D are combined in pairs.

[0069] When it is true that the first feature distance is greater than the second preset threshold, it is considered that a shot change occurs in the first video frame sequence or the second video frame sequence. At this time, any two video frames in the first video frame sequence or the second video frame sequence are obtained. It should be noted that the spacing between the two video frames in the first video frame sequence or the second video frame sequence should be less than a preset value, such as the preset value is 3 frames, etc., so that the two obtained video frames will not be too far apart.

[0070] The second characteristic distance between the two selected video frames is calculated, and then the second characteristic distance is numerically compared with a third preset threshold.

[0071] Since two video frames are selected in the embodiment of the present application with a distance less than the preset value between each two video frames, the range determined by the first characteristic distance alone will change in magnitude. Therefore, in order to numerically quantify the first characteristic distance, a first preset coefficient can be set to adjust the range determined by the first characteristic distance. For example, the first preset coefficient is set to 0.5, etc. At this time, the product of the first characteristic distance multiplied by the first preset coefficient can be used as the third preset threshold, and then the second characteristic distance and the third preset threshold are compared in numerical value. When it is determined that the second characteristic distance is less than the third preset threshold, it means that the shot between the two video frames is a gradual shot, that is, the first video frame sequence or the second video frame sequence contains a gradual shot.

[0072] The embodiment of the present application determines whether the first video frame sequence or the second video frame sequence includes a gradual change shot according to the characteristics of the distance value, thereby achieving the purpose of quickly identifying the gradual change shot.

[0073] As an optional embodiment, acquiring any two video frames in the first video frame sequence or the second video frame sequence includes:

[0074] Two adjacent video frames in the first video frame sequence or the second video frame sequence are obtained.

[0075] Optionally, in the embodiment of the present application, the positional relationship between two video frames in the first video frame sequence or the second video frame sequence is further restricted, that is, two adjacent video frames are selected as the two video frames finally obtained.

[0076] In the embodiment of the present application, by selecting two adjacent video frames to calculate the second characteristic distance, the specific positions of the two video frames when the lens picture changes are more accurately defined.

[0077] As an optional embodiment, when the first feature distance is indeed greater than the second preset threshold, the second preset threshold is obtained by obtaining any two video frames in the first video frame sequence or the second video frame sequence:

[0078] After acquiring each video frame in the first video frame sequence or the second video frame sequence and combining them in pairs, a plurality of third feature distances are calculated;

[0079] averaging the plurality of third characteristic distances to obtain a second preset threshold;

[0080] or,

[0081] Obtaining a second preset coefficient, wherein the second preset coefficient is used to represent the weight of the change shot included in the first video frame sequence or the second video frame sequence;

[0082] A product of the mean value and a second preset coefficient is determined as the second preset threshold.

[0083] Optionally, when obtaining the second preset threshold of the above embodiment, each video frame in the first video frame sequence or the second video frame sequence can be obtained and combined in pairs to calculate multiple third feature distances; the multiple third feature distances are averaged to obtain the second preset threshold.

[0084] A second preset coefficient can also be set based on historical experience or by the video shot tester himself. The second preset coefficient is used to characterize the weight of the change shots contained in the first video frame sequence or the second video frame sequence, for example, 0.5. The above-mentioned mean is then multiplied by the second preset coefficient 0.5 to obtain the second preset threshold.

[0085] As an optional embodiment, the method of determining the sudden change shot includes:

[0086] Obtaining a fourth characteristic distance between a key video frame in the first video frame sequence or the second video frame sequence and a frame behind or before the key video frame, wherein the key video frame is the last frame of a video segment when a scene picture changes in the first video frame sequence or the second video frame sequence, or a black screen frame in which the scene picture no longer exists in the first video frame sequence or the second video frame sequence;

[0087] Comparing the fourth characteristic distance with a third preset threshold, wherein the third preset threshold is the product of the second preset threshold and a channel coefficient corresponding to the current scene image;

[0088] When the fourth characteristic distance is greater than the third preset threshold, it is determined that the first video frame sequence or the second video frame sequence contains a sudden shot.

[0089] Optionally, extract the key video frame in the first video frame sequence or the second video frame sequence, and simultaneously obtain the frame behind the key video frame or the frame in front of the key video frame, calculate the characteristic distance between the key video frame and the frame behind the key video frame or the key video frame and the frame in front of the key video frame, as the fourth characteristic distance, and then compare the fourth characteristic distance with the third preset threshold, wherein the third preset threshold is the product of the second preset threshold obtained in the above embodiment and the channel coefficient corresponding to the current scene picture. It should be noted that the channel coefficient can be: 0.6 for TV series, 0.8 for movies, 1 for variety shows, 0.9 for animation, etc.

[0090] In addition, the key frame may be the last frame of a video segment when a scene changes in the first video frame sequence or the second video frame sequence, or a black screen frame in which the scene no longer exists in the first video frame sequence or the second video frame sequence.

[0091] If the fourth characteristic distance is greater than the third preset threshold, it is determined that the first video frame sequence or the second video frame sequence contains a sudden change shot, and then a sudden change shot set is obtained; if the fourth characteristic distance is less than the third preset threshold, it is considered that there is no shot change.

[0092] The embodiment of the present application determines whether the first video frame sequence or the second video frame sequence contains a sudden shot according to the characteristics of the distance value, thereby achieving the purpose of quickly identifying the sudden shot.

[0093] As an optional embodiment, the embodiment of the present application further proposes a method for generating video material, which utilizes the video content processing method of each of the above embodiments, including:

[0094] Get the location mark of the transition time point in the video to be processed;

[0095] The video frame sequence before the positioning mark and the video frame sequence after the positioning mark are merged to generate a merged video material.

[0096] Optionally, determine the positioning mark of the transition time point in the video to be processed, such as represented by the character x, and then merge the video frame sequence before the positioning mark and the video frame sequence after the positioning mark to generate a merged video material, thereby obtaining multiple complete video materials.

[0097] In an embodiment of the present application, by utilizing the positioning marker to merge the video frame sequence before the positioning marker and the video frame sequence after the positioning marker, the video material inventory of each channel can be greatly improved, providing more material for subsequent video mixing and other operations.

[0098] As an optional embodiment, Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an optional overall process of determining a lens change point according to an embodiment of the present application. The process is as follows:

[0099] Get the original video and channel information; determine the gradient lens;

[0100] Extract video key frames and calculate feature distance D key ;Get the mutation lens;

[0101] Get a set of gradient lenses and mutation lenses;

[0102] Then, according to the corresponding files of all video frame features and timestamps, determine whether there is a moving shot in the two shot sets; if so, there is a shot change point.

[0103] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0105] According to another aspect of the embodiments of the present application, a method device for video content processing for implementing the above-mentioned video content processing method is also provided. Figure 4 is a structural block diagram of an optional video content processing method and device according to an embodiment of the present application, such as Figure 4 As shown, the device may include:

[0106] The first acquisition unit 401 is used to acquire the transition time point of the video to be processed;

[0107] The second acquisition unit 402 is connected to the first acquisition unit 401, and is used to acquire a first video frame sequence and a second video frame sequence corresponding to the transition time point, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point sequence, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point sequence;

[0108] The third acquisition unit 403 is connected to the second acquisition unit 402, and is used to acquire a sequence group consisting of the first video frame sequence and the second video frame sequence, calculate the brightness feature distance between any two video frames in the sequence group, and determine the feature maximum value, wherein the feature maximum value includes the maximum value of the brightness feature distance and the minimum value of the brightness feature distance;

[0109] The first merging unit 404 is connected to the third acquiring unit 403, and is used to compare the numerical value relationship between the feature maximum value and the first preset threshold, and merge the first video frame sequence and the second video frame sequence based on the comparison result.

[0110] It should be noted that the first acquisition unit 401 in this embodiment can be used to execute the above step S201, the second acquisition unit 402 in this embodiment can be used to execute the above step S202, the third acquisition unit 403 in this embodiment can be used to execute the above step S203, and the first merging unit 404 in this embodiment can be used to execute the above step S204.

[0111] Through the above module, the brightness feature distance between each two video frames in the sequence group is calculated for the left adjacent video frame sequence and the right adjacent video frame sequence at the transition time point, and the feature maximum value is obtained. The feature maximum value is compared with the first preset threshold value to determine whether the left adjacent video frame sequence and the right adjacent video frame sequence are to be merged. In this way, the situation in which the continuity of the video content is poor due to the unreasonable start and end time points of the video split segments can be avoided, thereby providing more available complete shot segments, achieving the purpose of providing more available materials for mixed cutting, and thus solving the problems of inaccurate determination of the start and end time points of the video and unreasonable video content splitting existing in the related technology.

[0112] As an optional embodiment, the first merging unit includes: a first comparison module, used to compare the maximum value of the brightness feature distance with the first sub-threshold to obtain a first comparison result, wherein the first preset threshold includes the first sub-threshold; a second comparison module, used to compare the minimum value of the brightness feature distance with the second sub-threshold to obtain a second comparison result, wherein the first preset threshold includes the second sub-threshold; a first determination module, used to determine whether to merge the first video frame sequence and the second video frame sequence based on the first comparison result and the second comparison result.

[0113] As an optional embodiment, the device also includes: an obtaining unit, which is used to obtain a first comparison result when the maximum value of the brightness feature distance is less than a first sub-threshold before determining to merge the first video frame sequence and the second video frame sequence based on the first comparison result and the second comparison result; and to obtain a second comparison result when the minimum value of the brightness feature distance is less than a second sub-threshold.

[0114] As an optional embodiment, the device also includes: a first determination unit, used to determine the channel type to which the current scene picture belongs before comparing the numerical size relationship between the feature maximum value and the first preset threshold; a second determination unit, used to determine the first sub-threshold and the second sub-threshold according to the channel type, wherein the first sub-threshold and / or the second sub-threshold are obtained by the channel coefficient corresponding to the channel type.

[0115] As an optional embodiment, the device also includes: a setting unit, used to use the time point where there is a black screen frame in all video frames of the video to be processed as the transition time point, wherein, when the first video frame sequence corresponding to the transition time point includes a gradual change shot, the second video frame sequence corresponding to the transition time point includes a sudden change shot; or, when the first video frame sequence corresponding to the transition time point includes a sudden change shot, the second video frame sequence corresponding to the transition time point includes a gradual change shot.

[0116] As an optional embodiment, the method for determining a gradual shot includes: a fourth acquisition unit, used to acquire a first characteristic distance between a start frame and an end frame of a first video frame sequence or a second video frame sequence; a fifth acquisition unit, used to acquire any two video frames in the first video frame sequence or the second video frame sequence when the first characteristic distance is indeed greater than a second preset threshold, wherein the second preset threshold is obtained by calculating the characteristic distance by combining two video frames in the first video frame sequence or the second video frame sequence, and the spacing between the two video frames in the first video frame sequence or the second video frame sequence is less than a preset value; a third determination unit, used to determine the second characteristic distance between the two video frames, and when the second characteristic distance is less than a third preset threshold, determine that the first video frame sequence or the second video frame sequence contains a gradual shot, wherein the third preset threshold is the product of the first characteristic distance and a first preset coefficient, and the first preset coefficient is a numerical value for quantitatively adjusting the first characteristic distance.

[0117] As an optional embodiment, the fifth acquisition unit includes: a first acquisition module, configured to acquire two adjacent video frames in the first video frame sequence or the second video frame sequence.

[0118] As an optional embodiment, in the fifth acquisition unit, the method for obtaining the second preset threshold includes: a second acquisition module, used to obtain each video frame in the first video frame sequence or the second video frame sequence and combine them in pairs to calculate multiple third feature distances; an acquisition module, used to average the multiple third feature distances to obtain the second preset threshold; or, a third acquisition module, used to obtain a second preset coefficient, wherein the second preset coefficient is used to characterize the weight of the change shot contained in the first video frame sequence or the second video frame sequence; a second determination module, used to determine the product of the mean and the second preset coefficient as the second preset threshold.

[0119] As an optional embodiment, the method for determining a sudden shot includes: a sixth acquisition unit, used to acquire a fourth characteristic distance between a key video frame in the first video frame sequence or the second video frame sequence and a frame behind or before the key video frame, wherein the key video frame is the last frame of a video clip when a scene picture changes in the first video frame sequence or the second video frame sequence, or a black screen frame in which the scene picture no longer exists in the first video frame sequence or the second video frame sequence; a comparison unit, used to numerically compare the fourth characteristic distance with a third preset threshold, wherein the third preset threshold is the product of the second preset threshold and a channel coefficient corresponding to the current scene picture; a fourth determination unit, used to determine that the first video frame sequence or the second video frame sequence contains a sudden shot when the fourth characteristic distance is greater than the third preset threshold.

[0120] According to another aspect of the embodiment of the present application, a device for generating video material is also provided. The device for generating video material utilizes various units and modules in the device for processing video content. The device for generating video material includes: a seventh acquisition unit, used to obtain a positioning identifier of a transition time point in the video to be processed; a second merging unit, used to merge a video frame sequence located before the positioning identifier and a video frame sequence located after the positioning identifier to generate a merged video material.

[0121] According to another aspect of the embodiments of the present application, an electronic device for implementing the above-mentioned video content processing method is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0122] Figure 5 is a structural block diagram of an optional electronic device according to an embodiment of the present application, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 communicate with each other through the communication bus 504, wherein,

[0123] Memory 503, used for storing computer programs;

[0124] The processor 501 is used to execute the computer program stored in the memory 503 to implement the following steps:

[0125] Get the transition time point of the video to be processed;

[0126] Acquire a first video frame sequence and a second video frame sequence corresponding to a transition time point, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point;

[0127] Obtain a sequence group consisting of a first video frame sequence and a second video frame sequence, calculate the brightness feature distance between any two video frames in the sequence group, and determine the feature maximum value, wherein the feature maximum value includes the maximum value of the brightness feature distance and the minimum value of the brightness feature distance;

[0128] The numerical value relationship between the feature maximum value and the first preset threshold is compared, and the first video frame sequence and the second video frame sequence are merged based on the comparison result.

[0129] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0130] The communication interface is used for communication between the above electronic device and other devices.

[0131] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0132] As an example, Figure 5 As shown, the memory 503 may include, but is not limited to, the first acquisition unit 401, the second acquisition unit 402, the third acquisition unit 403, and the first merging unit 404 in the video content processing device. In addition, other module units in the video content processing device may also be included but are not limited to, which will not be repeated in this example.

[0133] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0134] In addition, the electronic device mentioned above further includes: a display for displaying the result of video content processing.

[0135] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0136] It can be understood by those skilled in the art that Figure 5The structure shown is for illustration only. The device for implementing the above-mentioned video content processing method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 5 It does not limit the structure of the above electronic device. For example, the terminal device may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations shown.

[0137] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0138] According to another aspect of the embodiment of the present application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute the program code of the method for processing video content.

[0139] Optionally, in this embodiment, the storage medium may be located on at least one network device among a plurality of network devices in the network shown in the above embodiment.

[0140] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps:

[0141] Get the transition time point of the video to be processed;

[0142] Acquire a first video frame sequence and a second video frame sequence corresponding to a transition time point, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point;

[0143] Obtain a sequence group consisting of a first video frame sequence and a second video frame sequence, calculate the brightness feature distance between any two video frames in the sequence group, and determine the feature maximum value, wherein the feature maximum value includes the maximum value of the brightness feature distance and the minimum value of the brightness feature distance;

[0144] The numerical value relationship between the feature maximum value and the first preset threshold is compared, and the first video frame sequence and the second video frame sequence are merged based on the comparison result.

[0145] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, which will not be described in detail in this embodiment.

[0146] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

[0147] According to another aspect of the embodiments of the present application, a computer program product or a computer program is also provided, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method steps of video content processing in any of the above embodiments.

[0148] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0149] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the method for processing video content in each embodiment of the present application.

[0150] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution provided in this embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0154] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for video content processing, characterized in that: The method comprises: Obtaining a transition time point of a video to be processed, wherein the transition time point is a time point at which a black screen frame exists in all video frames of the video to be processed; Acquire a first video frame sequence and a second video frame sequence corresponding to the transition time point, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point; Acquire a sequence group consisting of the first video frame sequence and the second video frame sequence, calculate the brightness feature distance between any two video frames in the sequence group, and determine a feature maximum value, wherein the feature maximum value includes a maximum value of the brightness feature distance and a minimum value of the brightness feature distance; The numerical value relationship between the feature maximum value and the first preset threshold is compared, and the first video frame sequence and the second video frame sequence are merged based on the comparison result, wherein the start and end time points of the merged video segments are reasonable.

2. The method according to claim 1, characterized in that: The comparing the numerical value relationship between the characteristic maximum value and the first preset threshold value, and merging the first video frame sequence and the second video frame sequence based on the comparison result includes: Compare the maximum value of the brightness feature distance with a first sub-threshold to obtain a first comparison result, wherein the first preset threshold includes the first sub-threshold; Compare the minimum value of the brightness feature distance with a second sub-threshold to obtain a second comparison result, wherein the first preset threshold includes the second sub-threshold; According to the first comparison result and the second comparison result, it is determined to merge the first video frame sequence and the second video frame sequence.

3. The method according to claim 2, characterized in that Before determining to merge the first video frame sequence and the second video frame sequence according to the first comparison result and the second comparison result, the method further includes: When the maximum value of the brightness feature distance is less than the first sub-threshold, the first comparison result is obtained; when the minimum value of the brightness feature distance is less than the second sub-threshold, the second comparison result is obtained.

4. The method according to claim 2, characterized in that Before comparing the numerical value relationship between the characteristic maximum value and the first preset threshold, the method further includes: Determine the channel type to which the current scene image belongs; The first sub-threshold and the second sub-threshold are determined according to the channel type, wherein the first sub-threshold and / or the second sub-threshold are obtained by a channel coefficient corresponding to the channel type.

5. The method according to claim 1, characterized in that: The method further comprises: The time point at which a black screen frame exists in all the video frames of the video to be processed is taken as the transition time point, wherein, when the first video frame sequence corresponding to the transition time point includes a gradual change shot, the second video frame sequence corresponding to the transition time point includes a sudden change shot; or, when the first video frame sequence corresponding to the transition time point includes a sudden change shot, the second video frame sequence corresponding to the transition time point includes a gradual change shot.

6. The method according to claim 5, characterized in that Methods for determining gradient lenses include: Acquire a first characteristic distance between a start frame and an end frame of the first video frame sequence or the second video frame sequence; In the case that the first feature distance is indeed greater than the second preset threshold, obtaining any two video frames in the first video frame sequence or the second video frame sequence, wherein the second preset threshold is obtained by calculating the feature distance of each video frame in the first video frame sequence or the second video frame sequence in pairs, and the distance between the two video frames in the first video frame sequence or the second video frame sequence is less than a preset value; Determine a second characteristic distance between the two video frames, and when the second characteristic distance is less than a third preset threshold, determine that the first video frame sequence or the second video frame sequence contains the gradient shot, wherein the third preset threshold is the product of the first characteristic distance and a first preset coefficient, and the first preset coefficient is a numerical value for quantitatively adjusting the first characteristic distance.

7. The method according to claim 6, characterized in that The acquiring any two video frames in the first video frame sequence or the second video frame sequence comprises: The two adjacent video frames in the first video frame sequence or the second video frame sequence are acquired.

8. The method according to claim 6, characterized in that In the case where the first feature distance is indeed greater than the second preset threshold, the method of obtaining the second preset threshold in any two video frames in the first video frame sequence or the second video frame sequence is: After acquiring each video frame in the first video frame sequence or the second video frame sequence and combining them in pairs, a plurality of third feature distances are calculated; averaging the plurality of third characteristic distances to obtain the second preset threshold; or, Acquire a second preset coefficient, wherein the second preset coefficient is used to represent the weight of a change shot included in the first video frame sequence or the second video frame sequence; A product of the mean value and the second preset coefficient is determined as the second preset threshold.

9. The method according to claim 8, characterized in that Methods for determining mutation shots include: Obtaining a fourth characteristic distance between a key video frame in the first video frame sequence or the second video frame sequence and a frame behind or before the key video frame, wherein the key video frame is the last frame of a video segment when a scene picture changes in the first video frame sequence or the second video frame sequence, or a black screen frame in which a scene picture no longer exists in the first video frame sequence or the second video frame sequence; Comparing the fourth characteristic distance with a third preset threshold, wherein the third preset threshold is the product of the second preset threshold and a channel coefficient corresponding to the current scene image; When the fourth characteristic distance is greater than the third preset threshold, it is determined that the first video frame sequence or the second video frame sequence contains the sudden shot.

10. A method for generating video material, characterized in that: The video material is generated by using the video content processing method according to any one of claims 1 to 9, the method comprising: Obtaining a location identifier of the transition time point in the video to be processed; The video frame sequence before the positioning mark and the video frame sequence after the positioning mark are merged to generate a merged video material.

11. A device for processing video content, characterized in that: The device comprises: A first acquisition unit is used to acquire a transition time point of a video to be processed, wherein the transition time point is a time point at which a black screen frame exists in all video frames of the video to be processed; A second acquisition unit is used to acquire a first video frame sequence and a second video frame sequence corresponding to the transition time point, wherein the first video frame sequence includes a video frame sequence adjacent to the left of the transition time point sequence, and the second video frame sequence includes a video frame sequence adjacent to the right of the transition time point sequence; a third acquisition unit, configured to acquire a sequence group consisting of the first video frame sequence and the second video frame sequence, calculate a brightness feature distance between any two video frames in the sequence group, and determine a feature maximum value, wherein the feature maximum value includes a maximum value of the brightness feature distance and a minimum value of the brightness feature distance; The first merging unit is used to compare the numerical value relationship between the feature maximum value and the first preset threshold, and merge the first video frame sequence and the second video frame sequence based on the comparison result, wherein the start and end time points of the merged video segments are reasonable.

12. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, the communication interface and the memory communicate with each other via the communication bus, wherein: The memory is used to store computer programs; The processor is configured to execute the method steps of any one of claims 1 to 10 by running the computer program stored in the memory.

13. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method steps described in any one of claims 1 to 10 when executed.

Images