Video shot segmentation method and device, equipment and storage medium

Abstract

The invention provides a video shot segmentation method and device, equipment and a storage medium. The method comprises the following steps: splitting a target video to obtain a plurality of video frames, obtaining the similarity of two adjacent video frames, and storing the similarity and a corresponding video frame pair into a similarity list; obtaining N target difference values in the similarity list, wherein the difference value between two adjacent similarities does not exceed a preset threshold value, and obtaining two adjacent video frame pairs corresponding to each target difference value; obtaining a segmentation threshold value according to the proportion of 2N video frame pairs corresponding to the N target difference values to the number of the video frame pairs in the similarity list; and identifying a plurality of video frames in the target video according to the segmentation threshold to obtain at least one shot switching point. According to the method, the accuracy of video shot segmentation is improved.

Description

Technical Field

[0001] The present application relates to the field of video recognition, and particularly to a video shot segmentation method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of multimedia information technology, a vast amount of video data has begun to pour into people's daily lives. The emergence of a vast amount of video data has greatly promoted the progress of video archiving, cataloging and indexing technologies. As the basis of the above technologies, video shot segmentation technology has been widely applied in recent years. Video shot segmentation technology is used to identify shot transition scenes in videos. In traditional video shot segmentation methods, it is determined whether there is a shot transition in the video according to a preset threshold. By judging whether the similarity between video frames in the video is high enough, it is further determined whether they are under the same shot. If the similarity between two video frames exceeds the set threshold, it is determined that the above two video frames are video frames under the same shot, otherwise it is determined that the above two video frames are shot transition video frames.

[0003] However, there are some problems in determining shot transition scenes by the set threshold, that is, when setting the threshold, the particularity of different video samples is not fully considered. Different video samples require different thresholds, and it is easy to be affected by subjective factors when setting the threshold, resulting in relatively low judgment accuracy.

[0004] Therefore, there is an urgent need for a method to effectively improve the accuracy of video shot segmentation. Summary of the Invention

[0005] The present application provides a video shot segmentation method, apparatus, device and storage medium to solve the technical problem of low accuracy of video shot segmentation.

[0006] In a first aspect, the present application provides a video shot segmentation method, including:

[0007] Splitting a target video into multiple video frames, obtaining the similarity between two adjacent video frames, and storing the similarity and the corresponding video frame pair in a similarity list;

[0008] Obtaining N target differences whose differences between two adjacent similarities in the similarity list do not exceed a preset threshold, and obtaining two adjacent video frame pairs corresponding to each of the target differences; where N is an integer greater than or equal to 1;

[0009] Obtaining a segmentation threshold according to the proportion of the 2N video frame pairs corresponding to the N target differences in the number of video frame pairs in the similarity list, where the segmentation threshold is used to identify shot transition points in the target video;

[0010] Identify multiple video frames in the target video according to the segmentation threshold to obtain at least one shot transition point.

[0011] In a possible design, the obtaining the similarity between two adjacent video frames includes:

[0012] Sort the video frames in the target video in chronological order to obtain a video frame sequence;

[0013] For each video frame in the video frame sequence, calculate the similarity between the color histogram of this video frame and the color histogram of the next video frame.

[0014] In a possible design, the obtaining the segmentation threshold according to the proportion of the 2N video frame pairs corresponding to N target differences in the number of video frame pairs in the similarity list includes:

[0015] Obtain the minimum similarity in the similarity list;

[0016] Obtain the segmentation threshold according to the proportion and the minimum similarity, where the proportion is negatively correlated with the segmentation threshold, and the minimum similarity is positively correlated with the segmentation threshold.

[0017] In a possible design, the obtaining the segmentation threshold according to the proportion and the minimum similarity includes:

[0018] Obtain the difference between the preset constant and the proportion and the sum of the preset constant and the minimum similarity;

[0019] According to the difference and the sum, obtain a parameter adjustment value, and according to the parameter adjustment value and the proportion, obtain the segmentation threshold.

[0020] In a possible design, the identifying multiple video frames in the target video according to the segmentation threshold to obtain at least one shot transition point includes:

[0021] Obtain the at least one shot transition point through the following shot transition steps:

[0022] For each video frame, determine whether the similarity corresponding to it and the next video frame is less than the segmentation threshold;

[0023] If the similarity is less than the segmentation threshold, then determine whether the interval between this video frame and the previous shot transition point is greater than the preset number of frames. If so, determine this video frame as a shot transition point.

[0024] In a possible design, if the similarity is greater than or equal to the segmentation threshold, or if the similarity is less than the segmentation threshold and the interval between the video frame and the previous shot transition point is less than or equal to a preset number of frames;

[0025] Continue to execute the shot transition step until the shot transition step is completed for the second-to-last video frame of the target video.

[0026] In a possible design, after identifying at least one shot transition point by identifying multiple video frames in the target video according to the segmentation threshold, it further includes:

[0027] Group the video frame corresponding to the shot transition point, the previous video frame, and the next video frame of this video frame and save them to the database;

[0028] Send at least one group of video frames in the database to the client.

[0029] In a second aspect, the present application provides a video shot segmentation device, including:

[0030] A splitting module, configured to split a target video into multiple video frames, obtain the similarity between adjacent two video frames, and store the similarity and the corresponding video frame pair in a similarity list;

[0031] An obtaining module, configured to obtain N target differences whose difference between adjacent two similarities in the similarity list does not exceed a preset threshold, and obtain two adjacent video frame pairs corresponding to each of the target differences; where N is an integer greater than or equal to 1;

[0032] An operation module, configured to obtain a segmentation threshold according to the proportion of the 2N video frame pairs corresponding to the N target differences in the number of video frame pairs in the similarity list, where the segmentation threshold is used to identify shot transition points in the target video;

[0033] An identification module, configured to identify at least one shot transition point by identifying multiple video frames in the target video according to the segmentation threshold.

[0034] In a third aspect, the present application provides a video shot segmentation device, including: a processor and a memory communicatively connected to the processor;

[0035] The memory stores computer execution instructions;

[0036] The processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the video shot segmentation method as described in the first aspect and various possible designs of the first aspect above.

[0037] Fourthly, the present application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the video shot segmentation method described in the first aspect and various possible designs of the first aspect as above.

[0038] A video shot segmentation method, apparatus, device and storage medium provided by the present application split a target video to obtain a plurality of video frames, establish a similarity list according to the similarity between two adjacent video frames, and obtain two adjacent video frame pairs corresponding to the above target difference according to a plurality of target differences whose difference between two adjacent similarities in the similarity list does not exceed a preset threshold, and then obtain the proportion of the plurality of video frame pairs corresponding to the plurality of target differences in the video frame pairs in the similarity list, and obtain a segmentation threshold according to the above proportion, so that the segmentation threshold obtained through the above proportion can be adaptively adjusted for different videos, so as to achieve the effect of improving the accuracy of video shot segmentation. Description of the Drawings

[0039] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0040] Figure 1 It is the flowchart of the video shot segmentation method provided by the embodiment of the present application Figure 1 ;

[0041] Figure 2 It is the flowchart of the video shot segmentation method provided by the embodiment of the present application Figure 2 ;

[0042] Figure 3 It is the flowchart of the video shot segmentation method provided by the embodiment of the present application Figure 3 ;

[0043] Figure 4 It is the structural schematic diagram of the video shot segmentation apparatus provided by the embodiment of the present application Figure 1 ;

[0044] Figure 5 It is the hardware schematic diagram of the video shot segmentation apparatus provided by the embodiment of the present application.

[0045] Through the above drawings, the specific embodiments of the present application have been shown, and there will be more detailed descriptions later. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to explain the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0046] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0047] Video shot segmentation technology is used to identify shot transition scenes in videos. In traditional video shot segmentation methods, it is determined whether there is a shot transition in the video according to a preset threshold. By judging whether the similarity between video frames in the video is high enough, it is further determined whether they are under the same shot. If the similarity between two video frames exceeds the set threshold, it is determined that the above two video frames are video frames under the same shot, otherwise it is determined that the above two video frames are shot transition video frames. However, there are some problems in determining shot transition scenes through the set threshold, that is, when setting the threshold, the specificities of different video samples are not fully considered. Different video samples require different thresholds, and it is easy to be affected by subjective factors when setting the threshold, which leads to low accuracy of judgment.

[0048] In order to effectively improve the accuracy of video shot segmentation, the present invention designs a video shot segmentation method. The terminal calculates the similarity of multiple adjacent video frames in the target video to obtain a similarity list, and then adaptively adjusts the segmentation threshold of different target videos according to the difference degree between adjacent similarities in the similarity list, so as to perform more accurate segmentation on different target videos.

[0049] The video shot segmentation method provided by the present application aims to solve the above technical problems of the prior art.

[0050] The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0051] Figure 1 For the video shot segmentation method flow provided by the embodiments of the present application Figure 1 . As Figure 1 shown, the method includes:

[0052] S101. Split the target video to obtain multiple video frames, obtain the similarity between two adjacent video frames, and store the similarity and the corresponding video frame pair in a similarity list.

[0053] Specifically, multiple video frames are displayed frame by frame according to a set frame rate to form a target video. The terminal traverses each frame of the target video and splits the target video frame by frame to obtain multiple video frames. Adjacent video frames refer to two video frames that are temporally adjacent to each other. For example, the first video frame and the second video frame, the second video frame and the third video frame, etc. The similarity of adjacent video frames refers to the similarity of different video frames obtained through the similarity of the color histograms of different video frames. The similarity list stores the similarity of two adjacent video frames and the video pair corresponding to the similarity. For example, splitting the target video obtains video frame 1, video frame 2, video frame 3, video frame 4, and video frame 5. Calculate the similarity of adjacent video frames among the above multiple video frames. The similarity between video frame 1 and video frame 2 is A1, the similarity between video frame 2 and video frame 3 is A2, the similarity between video frame 3 and video frame 4 is A3, and the similarity between video frame 4 and video frame 5 is A4. Furthermore, the video frame pair corresponding to A1 is the video frame pair composed of video frame 1 and video frame 2, the video frame pair corresponding to A2 is the video frame pair composed of video frame 2 and video frame 3, the video frame pair corresponding to A3 is the video frame pair composed of video frame 3 and video frame 4, and the video frame pair corresponding to A4 is the video frame pair composed of video frame 4 and video frame 5.

[0054] In a possible design, the similarity of the color histograms of video frames is obtained through the following steps:

[0055] Video frame preprocessing: Video frame preprocessing includes extracting the color histogram of the video frame and normalizing the color histogram of the video frame. The color histogram of the video frame is the distribution of each color component in the video frame image. The distribution of each color component is statistically calculated into a histogram to obtain the color histogram of the video frame. Normalizing the color histogram of the video frame performs a normalization process on the color histogram of each video frame to eliminate the influence of brightness and contrast. The normalized color histogram represents the relative relationship of color distribution.

[0056] Calculating the similarity of video frames: The similarity of video frames is represented by the Euclidean distance between color histograms. The smaller the difference in the Euclidean distance between the color histograms of two video frames, the higher the similarity of the two video frames. Specifically, when calculating the similarity of video frames, first define multiple key regions in the video frame, and obtain the color matrix of the key regions through color averaging. The key matrix is obtained according to the color matrix. Then store the key matrix in the video feature database, generate the color curve of the key region according to the color matrix, and filter the color curve. Detect the inflection points of the color curve of the key region to obtain the inflection point matrix. Then analyze the adjacent frames at the inflection points, calculate the Euclidean distance between the adjacent frames, and obtain the similarity between the adjacent frames by analyzing the difference in the Euclidean distance between the adjacent frames.

[0057] In the specific implementation process, the terminal splits the target video to obtain multiple video frames, obtains the similarity between two adjacent video frames, and stores the similarity and the corresponding video frame pair in a similarity list.

[0058] S102. Obtain N target differences where the difference between two adjacent similarities in the similarity list does not exceed a preset threshold, and obtain the two adjacent video frame pairs corresponding to each target difference; where N is an integer greater than or equal to 1.

[0059] In a possible design, the preset threshold is 0.001. Combining the above embodiments, obtain N target differences where the difference between two adjacent similarities in the similarity list does not exceed the preset threshold. Specifically, to obtain the differences between two adjacent similarities among the four similarities A1, A2, A3, and A4 in the similarity list, that is, the difference A12 between A1 and A2, the difference A23 between A2 and A3, and the difference A34 between A3 and A4. Take the differences that do not exceed 0.001 among A12, A23, and A34 as the target differences. Obtain the two adjacent video frame pairs corresponding to each target difference. Specifically, if A34 is the target difference, then obtain the two adjacent video frame pairs corresponding to A34. A34 is the difference between A3 and A4. A3 is the similarity between video frame 3 and video frame 4, and A4 is the similarity between video frame 4 and video frame 5. Therefore, the two adjacent video frame pairs corresponding to A34 are the video frame pair composed of video frame 3 and video frame 4 and the video frame pair composed of video frame 4 and video frame 5.

[0060] In the specific implementation process, the terminal obtains N target differences where the difference between two adjacent similarities in the similarity list does not exceed 0.001, and obtains the two adjacent video frame pairs corresponding to each target difference; where N is an integer greater than or equal to 1.

[0061] S103. Obtain a segmentation threshold according to the proportion of the 2N video frame pairs corresponding to the N target differences in the number of video frame pairs in the similarity list. The segmentation threshold is used to identify the shot transition points in the target video.

[0062] In a possible design, in combination with the above embodiments, if A34 is the target difference value, then N is 1. It is the proportion of the number of video frame pairs corresponding to N target difference values in the video frame pairs in the similarity list. Specifically, according to the proportion of the number of two video frame pairs corresponding to the target difference value A34 (the video frame pair composed of video frame 3 and video frame 4 and the video frame pair composed of video frame 4 and video frame 5) in the number of video frame pairs in the similarity list, the segmentation threshold is obtained. In combination with the above embodiments, the number of video frame pairs in the similarity list is 4 pairs, namely video frame pair 1 composed of video frame 1 and video frame 2, video frame pair 2 composed of video frame 2 and video frame 3, video frame pair 3 composed of video frame 3 and video frame 4, and video frame pair 4 composed of video frame 4 and video frame 5. Furthermore, according to the proportion of the two video frame pairs corresponding to the above target difference value A34 in the four video frame pairs in the similarity list, the segmentation threshold is obtained.

[0063] In the specific implementation process, the terminal obtains the segmentation threshold according to the proportion of the number of two video frame pairs corresponding to 1 target difference value in the number of video frame pairs in the similarity list, and the segmentation threshold is used to identify the shot transition points in the target video.

[0064] S104. Identify multiple video frames in the target video according to the segmentation threshold to obtain at least one shot transition point.

[0065] Specifically, by judging whether the similarity of the color histograms of adjacent video frames among multiple video frames is less than the segmentation threshold, multiple video frames in the target video are identified to obtain at least one shot transition point. If the similarity of the color histograms of adjacent video frames is less than the segmentation threshold, it is considered that the similarity between the above two adjacent video frames is relatively high, and then the probability that the above two adjacent video frames are used as shot transition points is relatively low.

[0066] The video shot segmentation method provided in this embodiment splits the target video to obtain multiple video frames, establishes a similarity list according to the similarity of adjacent two video frames, and obtains adjacent two video frame pairs corresponding to the above target difference values according to multiple target difference values whose difference between adjacent two similarities in the similarity list does not exceed a preset threshold. Furthermore, the proportion of the multiple video frame pairs corresponding to multiple target difference values in the video frame pairs in the similarity list is obtained, and the segmentation threshold is obtained according to the above proportion, so that the segmentation threshold obtained through the above proportion can be adaptively adjusted for different videos to achieve the effect of improving the accuracy of video shot segmentation.

[0067] Figure 2 is the flow of the video shot segmentation method provided in the embodiments of the present application Figure 2 。This embodiment further elaborates on the process of obtaining the segmentation threshold on the basis of the above Figure 1 embodiment. As Figure 2As shown, the method includes:

[0068] S201. Split the target video to obtain multiple video frames.

[0069] S202. Sort the video frames in the target video in chronological order to obtain a video frame sequence.

[0070] In a possible design, the terminal reads video frames from the target video frame by frame, and when reading each video frame, records the frame index and timestamp. The frame index indicates the position of the video frame in the target video. The timestamp represents the timestamp of each video frame, which can be obtained by calling a library function, and the index value and timestamp are stored in a list. Use a sorting algorithm to sort the index values according to the timestamp, and then obtain the video frame sequence. For example, the target video includes video frame 1, video frame 2, video frame 3, video frame 4, and the timestamps and index values corresponding to the above video frames are as follows: Index values: [1, 2, 3, 4], Timestamps: [0.2 seconds, 0.5 seconds, 0.1 seconds, 0.3 seconds]. Sort the video frame index values according to the timestamp, and the sorted video frame index values are: [3, 1, 4, 2]. According to the sorted video frame index values, re-sort the video frames to obtain the video frame sequence as video frame 3, video frame 1, video frame 4, video frame 2.

[0071] In the specific implementation process, the terminal sorts the video frames in the target video in chronological order to obtain a video frame sequence.

[0072] S203. For each video frame in the video frame sequence, calculate the similarity of the color histogram between the video frame and the subsequent video frame, and store the similarity and the corresponding video frame pair in a similarity list.

[0073] S204. Obtain N target differences whose differences between adjacent similarities in the similarity list do not exceed a preset threshold, and obtain the adjacent two video frame pairs corresponding to each target difference and the proportion of the 2N video frame pairs corresponding to the N target differences in the number of video frame pairs in the similarity list.

[0074] Steps S203 - S204 are similar to Figure 1 Steps S101 - S102 in the embodiment, and this embodiment will not be elaborated here.

[0075] S205. Obtain the minimum similarity in the similarity list.

[0076] S206. Obtain the difference between the preset constant and the proportion and the sum value of the preset constant and the minimum similarity.

[0077] S207. According to the difference and the sum value, obtain a parameter adjustment value, and according to the parameter adjustment value and the proportion, obtain a segmentation threshold.

[0078] In a possible design, the obtained preset constant can be 1. The difference between the preset constant and the ratio and the sum of the preset constant and the minimum similarity can be expressed as (1 - rate) and (1.0 + min). Specifically, according to the difference and the sum, the parameter adjustment value can be obtained through the following formula: According to the parameter adjustment value and the ratio, the segmentation threshold can be obtained through the following formula:

[0079]

[0080] Where X is the segmentation threshold, rate is the ratio of the number of 2N video frame pairs corresponding to N target differences to the number of video frame pairs in the similarity list, N is an integer greater than 1, and min is the minimum similarity in the similarity list.

[0081] In the specific implementation process, the terminal obtains the difference between the preset constant and the ratio and the sum of the preset constant and the minimum similarity, and according to the difference and the sum, obtains the parameter adjustment value, and according to the parameter adjustment value and the ratio, obtains the segmentation threshold.

[0082] S208. Identify multiple video frames in the target video according to the segmentation threshold to obtain at least one shot transition point.

[0083] The video shot segmentation method provided in this embodiment can obtain the parameter adjustment value by obtaining the minimum similarity and the preset constant in the similarity list. The parameter adjustment value can adjust the weight of the parameter adjustment value according to the number of adjacent video frames with high similarity, so that the segmentation threshold obtained according to the parameter adjustment value can be adaptively adjusted according to different target videos, thereby achieving the effect of optimizing the segmentation result and improving the accuracy of video shot segmentation.

[0084] Figure 3 is the flow of the video shot segmentation method provided in the embodiments of the present application Figure 3 . Based on the above Figure 1 embodiment, the process of identifying at least one shot transition point by identifying multiple video frames in the target video according to the segmentation threshold is described in detail. As Figure 3 shown, the method includes:

[0085] S301. Split the target video into multiple video frames, obtain the similarity between adjacent two video frames, and store the similarity and the corresponding video frame pair in the similarity list.

[0086] S302. Obtain N target differences whose differences between adjacent two similarities in the similarity list do not exceed a preset threshold, and obtain two adjacent video frame pairs corresponding to each target difference; where N is an integer greater than or equal to 1.

[0087] S303. Obtain a segmentation threshold according to the ratio of the number of video frame pairs corresponding to N target differences to the number of video frame pairs in the similarity list, where the segmentation threshold is used to identify shot transition points in the target video.

[0088] Steps S301 - S303 are similar to Figure 1 Steps S101 - S103 in the embodiment, and details are not described herein again in this embodiment.

[0089] S304. For each video frame, determine whether the similarity corresponding to it and the next video frame is less than the segmentation threshold; if so, execute S305, if not, execute S304.

[0090] In a possible design, the minimum similarity in the similarity list is 0.5, and the ratio of the number of 2N video frame pairs corresponding to N target differences to the number of video frame pairs in the similarity list is 90%, then the segmentation threshold X = 0.975 is obtained according to the above segmentation threshold formula.

[0091] S305. Determine whether the interval between this video frame and the previous shot transition point is greater than a preset number of frames; if so, execute S306, if not, execute S304.

[0092] In a possible design, the preset number of frames is 5 frames.

[0093] S306. Determine this video frame as a shot transition point.

[0094] In the specific implementation process, the terminal determines for each video frame whether the similarity corresponding to it and the next video frame is less than 0.975. If the terminal determines that the similarity between the current video frame and its next video frame is less than 0.975, then it continues to determine whether the interval between this video frame and the previous shot transition point is greater than the preset number of frames. If the similarity between the current video frame and its next video frame is less than 0.975 and the interval between this video frame and the previous shot transition point is greater than 5 frames, then this video frame is determined as a shot transition point. If the terminal determines that the similarity between the current video frame and the next video frame is greater than or equal to 0.975, or the terminal determines that the similarity between the current video frame and the next video frame is less than 0.975, and the interval between this video frame and the previous shot transition point is less than or equal to 5 frames, then continue to execute the shot transition steps of S305 and S306 until the shot transition steps are completed for the second-to-last frame of the target video.

[0095] S307. Group the video frame corresponding to the shot transition point, the previous video frame of this video frame, and the next video frame, and save them to the database.

[0096] S308. Send at least one set of video frames in the database to the client.

[0097] In a specific implementation process, the terminal saves the obtained shot transition points, their corresponding video frames, the previous video frame and the next video frame of the video frame as a group in the database, and sends at least one set of video frames in the database to the client.

[0098] The video shot segmentation method provided in this embodiment can identify multiple video frames in the target video by setting a segmentation threshold to obtain shot transition points. By setting a preset interval that the current video frame and the previous shot transition point satisfy during the judgment process, it can effectively avoid misjudgment of shot transition points, and can effectively avoid misjudgment caused by too low similarity between adjacent video frames due to flash points between adjacent video frames under the same video shot, thereby effectively improving the accuracy of video shot segmentation.

[0099] Figure 4 Schematic structure of the video shot segmentation device provided in the embodiment of the present application Figure 1 As Figure 4 shown, the signal problem diagnosis device 40 includes: a splitting module 401, an acquisition module 402, an operation module 403, and an identification module 404.

[0100] The splitting module 401 is configured to split the target video into multiple video frames, obtain the similarity between two adjacent video frames, and store the similarity and the corresponding video frame pair in a similarity list.

[0101] The acquisition module 402 is configured to obtain N target differences whose differences between two adjacent similarities in the similarity list do not exceed a preset threshold, and obtain two adjacent video frame pairs corresponding to each target difference; where N is an integer greater than or equal to 1.

[0102] The operation module 403 is configured to obtain a segmentation threshold according to the proportion of the 2N video frame pairs corresponding to the N target differences in the number of video frame pairs in the similarity list, and the segmentation threshold is used to identify shot transition points in the target video.

[0103] The identification module 404 is configured to identify multiple video frames in the target video according to the segmentation threshold to obtain at least one shot transition point.

[0104] In a possible design, the splitting module 401 is further configured to:

[0105] Sort the video frames in the target video in chronological order to obtain a video frame sequence;

[0106] For each video frame in the video frame sequence, calculate the similarity of the color histogram of the video frame and the subsequent video frame.

[0107] In a possible design, the operation module 403 is further configured to:

[0108] Obtain the minimum similarity in the similarity list;

[0109] Obtain a segmentation threshold according to the ratio and the minimum similarity, where the ratio is negatively correlated with the segmentation threshold, and the minimum similarity is positively correlated with the segmentation threshold.

[0110] In a possible design, the operation module 403 is further configured to:

[0111] Obtain the difference between the preset constant and the ratio and the sum value of the preset constant and the minimum similarity;

[0112] Obtain a parameter adjustment value according to the difference and the sum value, and obtain a segmentation threshold according to the parameter adjustment value and the ratio.

[0113] In a possible design, the recognition module 404 is further configured to:

[0114] Obtain at least one shot transition point through the following shot transition steps:

[0115] For each video frame, determine whether the similarity corresponding to it and the next video frame is less than the segmentation threshold;

[0116] If the similarity is less than the segmentation threshold, determine whether the interval between this video frame and the previous shot transition point is greater than the preset number of frames. If so, determine this video frame as a shot transition point.

[0117] In a possible design, the recognition module 404 is further configured to:

[0118] If the similarity is greater than or equal to the segmentation threshold, or if the similarity is less than the segmentation threshold and the interval between this video frame and the previous shot transition point is less than or equal to the preset number of frames;

[0119] Continue to execute the shot transition steps until the shot transition steps are completed for the penultimate frame of the target video.

[0120] In a possible design, the recognition module 404 is further configured to:

[0121] Group the video frame corresponding to the shot transition point, the previous video frame and the next video frame of this video frame and save them to the database;

[0122] Send at least one group of video frames in the database to the client.

[0123] Figure 5 This is the hardware schematic diagram of the video shot segmentation device provided by the embodiments of the present application. As Figure 5As shown in the figure, the video shot segmentation device 50 provided in this embodiment includes at least one processor 501 and a memory 502. The device 50 also includes a communication component 503. Among them, the processor 501, the memory 502, and the communication component 503 are connected through a bus 504.

[0124] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above video shot segmentation method.

[0125] For the specific implementation process of the processor 501, reference may be made to the above method embodiment. The implementation principle and technical effects are similar, and will not be elaborated here in this embodiment.

[0126] This application also provides an electronic device, including: a processor, and a memory communicatively connected to the processor. The memory stores computer-executable instructions, and the processor executes the computer-executable instructions stored in the memory to implement the above video shot segmentation method.

[0127] This application also provides a computer-readable storage medium. Computer-executable instructions are stored in the computer-readable storage medium. When the processor executes the computer-executable instructions, the above video shot segmentation method is implemented.

[0128] For the above computer-readable storage medium, the above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0129] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0130] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless specifically stated in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential either, but can be executed alternately or in rotation with at least a part of other steps or sub-steps or stages of other steps.

[0131] It should be understood that the above device embodiments are merely illustrative, and the devices of the present application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0132] In addition, without special instructions, in each embodiment of the present application, the functional units / modules can be integrated into one unit / module, or each unit / module can exist physically alone, or two or more units / modules can be integrated together. The above integrated unit / module can be implemented in the form of hardware or in the form of a software program module.

[0133] When the integrated unit / module is implemented in the form of hardware, the hardware can be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Without special instructions, the processor can be any suitable hardware processor, such as CPU, GPU, FPGA, DSP, and ASIC, etc. Without special instructions, the storage unit can be any suitable magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM (Resistive Random Access Memory), dynamic random access memory DRAM (Dynamic Random Access Memory), static random access memory SRAM (Static Random-Access Memory), enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), high-bandwidth memory HBM (High-Bandwidth Memory), hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0134] When the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs.

[0135] In the above embodiments, the descriptions of each embodiment have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should all be considered as within the scope described in this specification.

[0136] Those skilled in the art will readily think of other implementation manners of this application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptive changes of this application, and these variations, uses, or adaptive changes follow the general principles of this application and include the common general knowledge or conventional technical means in the technical field not disclosed in this application. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of this application are pointed out by the following claims.

[0137] It should be understood that this application is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is only limited by the appended claims.

Claims

1. A video shot segmentation method, characterized in that, The method includes: Splitting a target video to obtain a plurality of video frames, obtaining the similarity between two adjacent video frames, and storing the similarity and the corresponding video frame pair in a similarity list; Obtaining N target differences whose differences between two adjacent similarities in the similarity list do not exceed a preset threshold, and obtaining two adjacent video frame pairs corresponding to each of the target differences; wherein, N is an integer greater than or equal to 1; Obtaining a segmentation threshold according to the proportion of the 2N video frame pairs corresponding to the N target differences in the number of video frame pairs in the similarity list, and the segmentation threshold is used to identify shot transition points in the target video; Identifying at least one shot transition point for the plurality of video frames in the target video according to the segmentation threshold.

2. The method according to claim 1, wherein The obtaining the similarity between two adjacent video frames includes: Sorting the video frames in the target video in chronological order to obtain a video frame sequence; For each video frame in the video frame sequence, calculating the similarity between the color histogram of this video frame and the color histogram of the next video frame.

3. The method according to claim 2, wherein The obtaining a segmentation threshold according to the proportion of the 2N video frame pairs corresponding to the N target differences in the number of video frame pairs in the similarity list includes: Obtaining the minimum similarity in the similarity list; Obtaining a segmentation threshold according to the proportion and the minimum similarity, wherein the proportion is negatively correlated with the segmentation threshold, and the minimum similarity is positively correlated with the segmentation threshold.

4. The method according to claim 3, wherein The obtaining a segmentation threshold according to the proportion and the minimum similarity includes: Obtaining the difference between a preset constant and the proportion and the sum value of the preset constant and the minimum similarity; Obtaining a parameter adjustment value according to the difference and the sum value, and obtaining the segmentation threshold according to the parameter adjustment value and the proportion.

5. The method according to claim 1, characterized in that The identifying at least one shot transition point for the plurality of video frames in the target video according to the segmentation threshold includes: Obtaining the at least one shot transition point through the following shot transition steps: For each video frame, determining whether the similarity corresponding to it and the next video frame is less than the segmentation threshold; If the similarity is less than the segmentation threshold, determining whether the interval between this video frame and the previous shot transition point is greater than a preset number of frames, and if so, determining this video frame as a shot transition point.

6. The method according to claim 5, wherein The method further includes: If the similarity is greater than or equal to the segmentation threshold, or if the similarity is less than the segmentation threshold, and the interval between this video frame and the previous shot transition point is less than or equal to a preset number of frames; Continuing to execute the shot transition steps until the shot transition steps are completed for the penultimate frame of the target video.

7. The method according to claim 1, characterized in that, After identifying at least one shot transition point for the plurality of video frames in the target video according to the segmentation threshold, it further includes: Dividing the video frame corresponding to the shot transition point and the previous video frame and the next video frame of this video frame into a group and saving them to a database; Sending at least one group of video frames in the database to a client.

8. A video shot segmentation device, characterized in that, Includes: A splitting module, configured to split a target video into multiple video frames, obtain the similarity between two adjacent video frames, and store the similarity and the corresponding video frame pair into a similarity list; An obtaining module, configured to obtain N target differences whose differences between two adjacent similarities in the similarity list do not exceed a preset threshold, and obtain two adjacent video frame pairs corresponding to each of the target differences; wherein, N is an integer greater than or equal to 1; An operation module, configured to obtain a segmentation threshold according to the proportion of the 2N video frame pairs corresponding to the N target differences in the number of video frame pairs in the similarity list, and the segmentation threshold is used to identify shot transition points in the target video; An identification module, configured to identify at least one shot transition point from the multiple video frames in the target video according to the segmentation threshold.

9. An electronic device, characterized in that, Comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 7.

Images