A method and device for detecting the transition state of a surveillance video shot and a method for shot segmentation

Abstract

An embodiment of the present invention provides a method and device for detecting the transitional state of a monitoring video lens and a method for segmenting a lens. First, calculate the similarity of adjacent video frames in a video file in sequence. After normalizing the similarity, determine the inter-frame difference value, as well as the rising and falling segments of the inter-frame difference value. Then, compare the inter-frame difference value with a mutation threshold to obtain the index of the mutation transition in the video. Finally, according to specific comparison rules, compare the first and last inter-frame difference values of the candidate gradual transition segments with the gradual change threshold respectively. Among all the candidate gradual transition segments, screen out the true lens gradual transition state and obtain its index, completing the process of detecting the transitional state of the monitoring video lens and segmenting the lens. This solution solves the problem that the maximum number of frames in the gradual transition state in the dual-threshold video lens detection algorithm needs to be determined manually, reduces the usage limitations of the algorithm and the requirements for prior experience, and improves the applicability and flexibility of the algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and particularly to a method for detecting the transitional state of surveillance video shots, a method for segmenting surveillance video shots, a device for detecting the transitional state of surveillance video shots, and a device for segmenting surveillance video shots. Background Art

[0002] With the wide penetration of advanced technologies such as communication technology, artificial intelligence, and blockchain into current information dissemination, the "video shift" trend in China's content industry has become increasingly prominent. As of December 2021, the scale of China's online video (including short videos) users reached 975 million, an increase of 47.94 million compared with December 2020, accounting for 94.5% of the overall Internet users.

[0003] As a comprehensive media form integrating visual, auditory, and text information, video plays an important role in various fields. Referring to Figure 1 , a video structure hierarchy diagram in the prior art is shown. The hierarchical structure of a video can be divided into three logical units: frames, shots, and scenes from the upper layer to the lower layer.

[0004] Surveillance video, as an important type of video, plays an important role in public security protection in various industries. The shots of a video are the smallest semantic units of the video. There is a transitional state between two adjacent shots. The transitional state of a shot can be divided into abrupt transition and gradual transition according to the transition speed. The detection and segmentation of surveillance video shots are key upstream tasks for extracting key frames of a video and are of great significance for video content recognition and analysis.

[0005] Video shot detection is to use a video shot boundary detection algorithm to detect the boundaries of each shot in a video, and then segment the video into several individual shot units according to the detection results. At present, the commonly used video shot boundary detection algorithm is the double-threshold method. This method includes two thresholds, namely the abrupt threshold T1 and the gradual threshold T2. According to the size relationship between the similarity of video frames and T1 and T2, the abrupt transition and gradual transition of shots in the video are detected. The double-threshold method has the characteristics of low computational complexity and simple algorithm.

[0006] The specific steps of the video shot boundary detection method based on double thresholds are as follows:

[0007] Input: video file V, abrupt threshold T1, gradual threshold T2, maximum number of frames in the gradual transitional state N;

[0008] A1. Read video frames f n and video frame f n+1 ;

[0009] A2. Calculate the similarity between video frame f n and video frame f n+1 , denoted as S(f n , f n+1 );

[0010] A3. If S(f n , f n+1 ) ≥ T1, then the nth frame is a mutation transition, and store the index of this frame in the video into the array I; otherwise, execute A4;

[0011] A4. If T2 < S(f n , f n+1 ) < T1, then take the nth frame as a candidate frame for gradual transition and execute A5; otherwise, execute A6;

[0012] A5. If and S(f j , f j+1 ) ≤ T2, then take the nth frame as a gradual transition state, and store the index of this frame in the video into the array I, n = n + 1;

[0013] A6. If S(f n , f n+1 ) < T2, then continue to execute;

[0014] A7. If the detection of the shot transition state of all video frames is completed, execute A8; otherwise, iteratively execute A1 to A7;

[0015] A8. According to the mutation and gradual transition frame indices in the array I, split the video V into several shots and save the video shots;

[0016] A9. End

[0017] Output: Video shot file

[0018] The double - threshold algorithm is a commonly used technology in the field of video shot boundary detection. This algorithm has the characteristics of being simple and easy to implement. However, only through the method of threshold comparison, it cannot adapt to the rapid changes in video content, resulting in inaccurate shot boundary recognition. In addition, for the detection of the gradual transition state, the maximum number N of frames in the gradual transition state also depends on manual experience to determine, making it difficult to adapt to a variety of video scenarios. Summary of the Invention

[0019] In view of the defects in the prior art, embodiments of the present invention provide a method for detecting the transition state of a surveillance video shot, a method for segmenting a surveillance video shot, a device for detecting the transition state of a surveillance video shot, and a device for segmenting a surveillance video shot.

[0020] In a first aspect, an embodiment of the present invention provides a method for detecting the transitional state of a surveillance video shot, including:

[0021] Obtain a surveillance video file to be detected;

[0022] Calculate the similarity between adjacent video frames in the video file in sequence, and store the calculated similarity into a first array;

[0023] Normalize the similarity in the first array, and determine the inter-frame difference value according to the normalized similarity, and store the calculated inter-frame difference value into a second array;

[0024] Calculate the difference between adjacent inter-frame difference values in the second array in sequence, and store the calculated difference between adjacent inter-frame difference values into a third array;

[0025] Determine the rising segment and the falling segment according to the difference between adjacent inter-frame difference values in the third array, and use the determined rising segment and falling segment as candidate gradual change shot transition segments;

[0026] Judge in sequence whether the inter-frame difference value in the second array is greater than or equal to the mutation threshold Ta. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index into a first transition state array;

[0027] For each rising segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first rising segment is less than the gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index into a second transition state array;

[0028] For each falling segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index into the second transition state array;

[0029] Determine all transition frames of the surveillance video shot to be detected according to the indexes in the first transition state array and the second transition state array.

[0030] As the above method, optionally, normalizing the similarity in the first array and determining the inter-frame difference value according to the normalized similarity includes:

[0031] Normalize each similarity in the first array to the interval [0, 1] to obtain a standard similarity, and calculate the inter-frame difference value for each frame by subtracting the standard similarity from 1.

[0032] As in the above method, optionally, the determining the rising segment and the falling segment according to the difference between adjacent inter-frame difference values in the third array includes:

[0033] Determine the frames per second (FPS) of the video file.

[0034] If the number of consecutive differences between adjacent inter-frame difference values in the third array that are greater than 0 is greater than M, then determine the video frames corresponding to the consecutive differences between adjacent inter-frame difference values as the rising segment.

[0035] If the number of consecutive differences between adjacent inter-frame difference values in the third array that are all less than 0 is greater than M, then determine the video frames corresponding to the consecutive differences between adjacent inter-frame difference values as the falling segment; where M = FPS.

[0036] As in the above method, optionally, the gradual change threshold Tg is determined by the following method:

[0037] Calculate the average value of the inter-frame difference values in the second array.

[0038] Determine the gradual change threshold Tg according to the average value.

[0039] As in the above method, optionally, the mutation threshold Ta and the gradual change threshold Tg satisfy the following relationship:

[0040] Tg = k * Ta, where 0 < k < 1.

[0041] In a second aspect, an embodiment of the present invention provides a method for segmenting monitoring video shots, including:

[0042] Obtain the monitoring video file to be detected.

[0043] Calculate the similarities between adjacent video frames in the video file in sequence, and store the calculated similarities in the first array.

[0044] Normalize the similarities in the first array, and determine the inter-frame difference values according to the normalized similarities, and store the calculated inter-frame difference values in the second array.

[0045] Calculate the differences between adjacent inter-frame difference values in the second array in sequence, and store the calculated differences between adjacent inter-frame difference values in the third array.

[0046] Determine the rising segment and the falling segment according to the difference between the inter-frame difference values in the third array, and use the determined rising segment and falling segment as the candidate transition segments of the gradual change lens;

[0047] Determine whether the inter-frame difference value in the second array is greater than or equal to the mutation threshold Ta in sequence. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index in the first transition state array;

[0048] For each rising segment in the candidate transition segments of the gradual change lens, if the first inter-frame difference value in the first rising segment is less than the gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index in the second transition state array;

[0049] For each falling segment in the candidate transition segments of the gradual change lens, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index in the second transition state array;

[0050] Perform shot segmentation on the to-be-detected surveillance video according to the indexes in the first transition state array and the second transition state array.

[0051] In a third aspect, an embodiment of the present invention provides a surveillance video shot transition state detection device, including:

[0052] A first acquisition module, configured to acquire a to-be-detected surveillance video file;

[0053] A first similarity calculation module, configured to calculate the similarity between adjacent video frames in the video file in sequence, and store the calculated similarity in a first array;

[0054] A first inter-frame difference value calculation module, configured to standardize the similarity in the first array, and determine the inter-frame difference value according to the standardized similarity, and store the calculated inter-frame difference value in a second array;

[0055] A first difference calculation module, configured to calculate the difference between adjacent inter-frame difference values in the second array in sequence, and store the calculated difference between adjacent inter-frame difference values in a third array;

[0056] A first rising and falling judgment module, configured to determine the rising segment and the falling segment according to the difference between the adjacent inter-frame difference values in the third array, and use the determined rising segment and falling segment as the candidate transition segments of the gradual change lens;

[0057] The first mutation determination module is used to sequentially determine whether the inter-frame difference values in the second array are greater than or equal to the mutation threshold Ta. If so, it determines the index of the video frame corresponding to the inter-frame difference value in the video file and stores the index in the first transient state array;

[0058] The first gradual change determination module is used for each rising segment in the candidate gradual change shot transition segment. If the first inter-frame difference value in the first rising segment is less than the gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, it determines the index of the first frame in the first rising segment in the video file and stores the index in the second transient state array;

[0059] The second gradual change determination module is used for each falling segment in the candidate gradual change shot transition segment. If the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, it determines the index of the last frame in the first falling segment in the video file and stores the index in the second transient state array;

[0060] The transition frame determination module is used to determine all transition frames of the to-be-detected surveillance video shot according to the indexes in the first transient state array and the second transient state array.

[0061] In a fourth aspect, an embodiment of the present invention provides a surveillance video shot segmentation device, including:

[0062] The second acquisition module is used to acquire the to-be-detected surveillance video file;

[0063] The second similarity calculation module is used to sequentially calculate the similarity of adjacent video frames in the video file and store the calculated similarity in the first array;

[0064] The second inter-frame difference value calculation module is used to standardize the similarity in the first array and determine the inter-frame difference value according to the standardized similarity, and store the calculated inter-frame difference value in the second array;

[0065] The second difference calculation module is used to sequentially calculate the difference between adjacent inter-frame difference values in the second array and store the calculated difference between adjacent inter-frame difference values in the third array;

[0066] The second rising and falling determination module is used to determine the rising segment and the falling segment according to the difference between adjacent inter-frame difference values in the third array and use the determined rising segment and falling segment as the candidate gradual change shot transition segment;

[0067] A second mutation determination module, configured to sequentially determine whether the inter-frame difference values in the second array are greater than or equal to a mutation threshold Ta. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index in a first transient state array;

[0068] A third gradual change determination module, configured to, for each rising segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first rising segment is less than a gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index in a second transient state array;

[0069] A fourth gradual change determination module, configured to, for each falling segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index in the second transient state array;

[0070] A shot segmentation module, configured to perform shot segmentation on the to-be-detected surveillance video according to the indexes in the first transient state array and the second transient state array.

[0071] In a fifth aspect, an embodiment of the present invention provides an electronic device, including:

[0072] A memory and a processor, where the processor and the memory communicate with each other through a bus; the memory stores program instructions executable by the processor, and the processor can execute the following method by invoking the program instructions: obtaining a monitoring video file to be detected; calculating the similarity between adjacent video frames in the video file in sequence, and storing the calculated similarity into a first array; normalizing the similarity in the first array, and determining an inter-frame difference value according to the normalized similarity, and storing the calculated inter-frame difference value into a second array; calculating the difference between adjacent inter-frame difference values in the second array in sequence, and storing the calculated difference between adjacent inter-frame difference values into a third array; determining an ascending segment and a descending segment according to the difference between adjacent inter-frame difference values in the third array, and taking the determined ascending segment and descending segment as candidate gradual change shot transition segments; determining whether the inter-frame difference value in the second array is greater than or equal to a mutation threshold Ta in sequence, and if so, determining the index of the video frame corresponding to the inter-frame difference value in the video file, and storing the index into a first transition state array; for each ascending segment in the candidate gradual change shot transition segments, if the first inter-frame difference value in the first ascending segment is less than a gradual change threshold Tg and the last inter-frame difference value in the first ascending segment is greater than the gradual change threshold Tg, determining the index of the first frame in the first ascending segment in the video file, and storing the index into a second transition state array; for each descending segment in the candidate gradual change shot transition segments, if the first inter-frame difference value in the first descending segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first descending segment is less than the gradual change threshold Tg, determining the index of the last frame in the first descending segment in the video file, and storing the index into the second transition state array; determining all transition frames of the monitoring video shot to be detected according to the indexes in the first transition state array and the second transition state array.

[0073] Sixth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following method is implemented: obtaining a monitoring video file to be detected; sequentially calculating the similarity between adjacent video frames in the video file, and storing the calculated similarity into a first array; normalizing the similarity in the first array, and determining an inter-frame difference value according to the normalized similarity, and storing the calculated inter-frame difference value into a second array; sequentially calculating the difference between adjacent inter-frame difference values in the second array, and storing the calculated difference between adjacent inter-frame difference values into a third array; determining a rising segment and a falling segment according to the difference between adjacent inter-frame difference values in the third array, and using the determined rising segment and falling segment as candidate gradual change lens transition segments; sequentially determining whether the inter-frame difference value in the second array is greater than or equal to a mutation threshold Ta. If so, determining the index of the video frame corresponding to the inter-frame difference value in the video file, and storing the index into a first transition state array; for each rising segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first rising segment is less than a gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determining the index of the first frame in the first rising segment in the video file, and storing the index into a second transition state array; for each falling segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determining the index of the last frame in the first falling segment in the video file, and storing the index into the second transition state array; determining all transition frames of the monitoring video lens to be detected according to the indexes in the first transition state array and the second transition state array.

[0074] The monitoring video shot transition state detection and shot segmentation solution provided by the embodiments of the present invention. First, calculate the similarity of adjacent video frames in the video file in sequence. Then, after normalizing the similarity, determine the inter-frame difference value. By analyzing the change curve of the inter-frame difference value, determine the rising segment and the falling segment of the inter-frame difference value, and regard each rising segment or falling segment as a candidate gradual transition segment. Then, compare the inter-frame difference value with the mutation threshold to obtain the index of the mutation transition in the video. Finally, according to specific comparison rules, compare the first and last inter-frame difference values of the candidate gradual transition segments with the gradual transition threshold respectively. Among all the candidate gradual transition segments, screen out the real shot gradual transition state and obtain its index, completing the monitoring video shot transition state detection and shot segmentation process. This solution uses an edge detection method to process and analyze the similarity between video frames, determines the rising segment and the falling segment by specifying the inter-frame difference value, and compares the start frame and the end frame of the rising segment and the falling segment with the gradual transition threshold, replacing the original method for determining the gradual transition, solving the problem that the maximum number of frames in the gradual transition state in the double-threshold video shot detection algorithm needs to be determined manually, reducing the usage limitations of the algorithm and the requirements for prior experience, and improving the applicability and flexibility of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 is a video structure hierarchy diagram in the prior art;

[0076] Figure 2 is a flowchart of the steps of an embodiment of a method for detecting the transition state of a monitoring video shot according to the present invention;

[0077] Figure 3 is a flowchart of the steps of another embodiment of a method for detecting the transition state of a monitoring video shot according to the present invention;

[0078] Figure 4 is a flowchart of the steps of an embodiment of a method for segmenting a monitoring video shot according to the present invention;

[0079] Figure 5 is a block diagram of the structure of an embodiment of a device for detecting the transition state of a monitoring video shot according to the present invention;

[0080] Figure 6 is a block diagram of the structure of an embodiment of a device for segmenting a monitoring video shot according to the present invention;

[0081] Figure 7 is a block diagram of the structure of an embodiment of an electronic device according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0082] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0083] Reference Figure 1 , the step flowchart of an embodiment of the method for detecting the transition state of a monitoring video lens according to the present invention is shown. Specifically, it may include the following steps:

[0084] Step S210: Obtain the monitoring video file to be detected;

[0085] Specifically, when it is necessary to detect the transition frames in a certain monitoring video file, first obtain the monitoring video file V to be detected. The monitoring video file V to be detected contains multiple video frames. By detecting the transition state in the video frames, the video frames can be divided into multiple lenses, and then the key video frames can be extracted to identify and analyze the video content.

[0086] Step S220: Calculate the similarity of adjacent video frames in the video file in sequence, and store the calculated similarity into the first array;

[0087] Specifically, obtain the video frames f n and the adjacent video frames f n+1 from the monitoring video file V to be detected in sequence, where n = 1,... N, and N is the total number of video frames in the monitoring video file V to be detected. First, let n = 1, calculate the similarity between the video frame f n and the video frame f n+1 , denoted as S1(f n , f n+1 ), and store the obtained S1(f n , f n+1 ) into the array S1 in sequence, denoted as the first array S1, and then let n = n + 1. Continue to execute the above steps until the similarity calculation between all frames in the monitoring video file V to be detected is completed.

[0088] For example, use the structural similarity algorithm (SSIM) to calculate the similarity between the video frame f n and the video frame f n+1 , denoted as S1(f n , f n+1 ), and store the obtained S1(f n , f n+1 ) into the array S1 in sequence, denoted as the first array S1, and then let n = n + 1. Continue to execute the above steps until the similarity calculation between all frames in the monitoring video file V to be detected is completed, so as to obtain the first array S1. The first array S1 records the similarity between adjacent video frames in the monitoring video file V to be detected in sequence. In practical applications, the last similarity can be set as the similarity between the video frame f N and the video frame f1, denoted as S1(f N,f1), so the number of similarities in the first array S1 is the same as the number of video frames in the surveillance video file V to be detected, both of which are N.

[0089] Step S230, normalizing the similarity in the first array, determining the inter-frame difference value according to the normalized similarity, and storing the calculated inter-frame difference value in the second array;

[0090] Specifically, each similarity S1(f n ,f n+1 ) is standardized to the interval [0,1] to obtain the standard similarity S(f n ,f n+1 ), and according to the formula D(f n ,f n+1 )=1-S(f n ,f n+1 ) Calculate the difference value D(f) between each frame n ,f n+1 ), where D(f n ,f n+1 ) is the difference value between frames, and then the calculated D(f n ,f n+1 ) is stored in an inter-frame difference array, which is recorded as a second array D. The second array D records the inter-frame difference values ​​in sequence.

[0091] For example, the similarity S1(f n ,f n+1 ) The similarity value of the first array S1 is standardized to the interval [0,1] to obtain the standard similarity S(f n ,f n+1 ), that is, 0≤S(f n ,f n+1 )≤1, and then use the formula D(f n ,f n+1 )=1-S(f n ,f n+1 ) Calculate the difference value D(f) between each frame n ,f n+1 ), the calculated inter-frame difference value D(f n ,f n+1 ) is stored in the second array D.

[0092] Step S240, sequentially calculating the difference values ​​of the difference values ​​between adjacent frames in the second array, and storing the calculated difference values ​​of the difference values ​​between adjacent frames in a third array;

[0093] Specifically, according to the inter-frame difference value D(f n ,f n+1) in the order of the formula ΔD n = D n+1 - D n Calculate the difference ΔD of the difference values of the inter-frame differences between adjacent frames n , and store the difference ΔD n into the difference array, denoted as the third array X.

[0094] Step S250: Determine the rising segment and the falling segment according to the difference of the inter-frame difference values in the third array, and use the determined rising segment and falling segment as the candidate gradual change shot transition segment;

[0095] Specifically, determine the number of frames per second (Frames per Second, FPS) of the monitored video file V to be detected. If in the third array X, the number of differences ΔD of the consecutive inter-frame difference values n greater than 0 is greater than M, then determine the video frames corresponding to the differences of the consecutive inter-frame difference values as the rising segment;

[0096] If in the third array X, the number of differences ΔD of the consecutive inter-frame difference values n all less than 0 is greater than M, then determine the video frames corresponding to the differences of the consecutive inter-frame difference values as the falling segment; where M = FPS.

[0097] Specifically, the difference of the inter-frame difference value represents the change of the inter-frame difference value. According to the change curve of the inter-frame difference value, the rising segment and the falling segment of the inter-frame difference value can be determined. Specifically, if in the third array X, the number of consecutive ΔD n > 0 is greater than FPS, then the consecutive video frames corresponding to this ΔD n are the rising segment. On the contrary, if the number of consecutive ΔD n < 0 is greater than FPS, then the consecutive video frames corresponding to this ΔD n are the falling segment. Mark the rising segment and the falling segment in the second array D. Denote each rising segment and falling segment as a candidate gradual change shot transition segment, that is, there may be a gradual change shot transition segment in the rising segment and the falling segment, and further screening is required.

[0098] Step S260: Determine whether the inter-frame difference value in the second array is greater than or equal to the mutation threshold Ta in turn. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index into the first transition state array;

[0099] Specifically, when determining whether it is a lens transition frame, it is first necessary to determine the judgment thresholds, which are divided into a mutation threshold Ta and a gradual change threshold Tb. First, calculate the average value of the inter-frame difference values in the second array D, and then determine the gradual change threshold Tg according to the average value. For example, calculate the average value T of the inter-frame difference values in the second array D according to the formula T = ∑D / len(D), and then let Tg = T to determine the gradual change threshold Tg, where len(D) is the number of inter-frame difference values in the second array D.

[0100] Then, according to the formula Tg = k * Ta, where 0 < k < 1, determine the mutation threshold Ta. For example, set k = 1 / 3, and the mutation threshold Ta = 3 * Tg can be obtained.

[0101] After that, sequentially take out the inter-frame difference value D(f n , f n+1 ) from the second array D, and n = 1.

[0102] If the inter-frame difference value D(f n , f n+1 ) ≥ Ta, then the nth frame is a mutation transition, and store the index of this frame in the video file V to be detected into the first transition state array I1.

[0103] Let n = n + 1, and continue to execute the above steps until all the inter-frame difference values in the second array D have been compared. The finally obtained first transition state array I1 records the indexes of all the mutation transition frames in the video file V to be detected.

[0104] Step S270: For each rising segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first rising segment is less than the gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, then determine the index of the first frame in the first rising segment in the video file, and store the index into the second transition state array.

[0105] Specifically, sequentially take out a candidate gradual change lens transition segment xj from the second array D, where j = 1.

[0106] If xj is marked as a rising segment, determine whether the first inter-frame difference value of xj is less than the gradual change threshold Tg. If it is less, then continue to determine whether the last inter-frame difference value of xj is greater than the gradual change threshold Tg. If both are satisfied, then determine xj as a gradual change transition segment, and store the index of the first frame of xj in the video file V to be detected into the second transition state array I2. Let j = j + 1, and continue to execute the above steps until all the rising segments in all the candidate gradual change lens transition segments in the second array D have been detected.

[0107] Step S280: For each descending segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first descending segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first descending segment is less than the gradual change threshold Tg, determine the index of the last frame in the first descending segment in the video file, and store the index in the second transition state array.

[0108] Specifically, take out a candidate gradual change lens transition segment xj from the second array D in sequence, where j = 1.

[0109] If xj is marked as a descending segment, determine whether the first inter-frame difference value of xj is greater than the gradual change threshold Tg. If it is less, then continue to determine whether the last inter-frame difference value of xj is less than the gradual change threshold Tg. If both are satisfied, determine that xj is a gradual change transition segment, and store the index of the last frame of xj in the to-be-detected surveillance video file V in the second transition state array I2. Let j = j + 1, and continue to execute the above steps until all the descending segments in all the candidate gradual change lens transition segments in the second array D are detected. The second transition state array I2 stores the indices of all the gradual change transition frames.

[0110] Step S290: Determine all the transition frames of the to-be-detected surveillance video lens according to the indices in the first transition state array and the second transition state array.

[0111] Specifically, since the first transition state array I1 and the second transition state array I2 store the indices of the transition frames, store the indices in the first transition state array I1 and the second transition state array I2 in the lens transition index array I. After outputting all the indices in the lens transition index array I in sequence, the indices of all the transition frames in the to-be-detected surveillance video lens can be obtained, and thus all the transition frames in the to-be-detected surveillance video lens can be found according to these indices.

[0112] The monitoring video shot transition state detection method provided by the embodiment of the present invention first calculates the similarity between adjacent video frames in the video file in sequence. Then, after normalizing the similarity, the inter-frame difference value is determined. By analyzing the change curve of the inter-frame difference value, the rising segment and the falling segment of the inter-frame difference value are determined, and each rising segment or falling segment is used as a candidate gradual transition segment. Then, the inter-frame difference value is compared with the mutation threshold to obtain the index of the mutation transition in the video. Finally, according to specific comparison rules, the first and last inter-frame difference values of the candidate gradual transition segments are respectively compared with the gradual transition threshold. Among all the candidate gradual transition segments, the real shot gradual transition state is screened out and its index is obtained, completing the monitoring video shot transition state detection process. This method uses an edge detection method to process and analyze the similarity between video frames, determines the rising segment and the falling segment by specifying the inter-frame difference value, and compares the start frame and the end frame of the rising segment and the falling segment with the gradual transition threshold, replacing the original gradual transition determination method, solving the problem that the maximum number of frames in the gradual transition state in the double-threshold video shot detection algorithm needs to be determined manually, reducing the usage limitations of the algorithm and the requirements for prior experience, and improving the applicability and flexibility of the algorithm.

[0113] Furthermore, in the monitoring video shot transition state detection method provided by the embodiment of the present invention, in combination with the definition of the inter-frame difference value, the mutation threshold and the gradual transition threshold in the double threshold are determined according to the inter-frame difference value of the video frames in the video file. By comparing the inter-frame difference value with the mutation threshold and the gradual transition threshold, there is no need to manually set the threshold parameters, further reducing the usage limitations of the algorithm and the requirements for prior experience.

[0114] Refer to Figure 3 , which shows the step flowchart of another embodiment of the monitoring video shot transition state detection method of the present invention, and specifically may include:

[0115] Input the monitoring video file V to be detected;

[0116] Step B01: Read the monitoring video file V to be detected, and let n = 1;

[0117] Step B02: Sequentially obtain the video frame f n and the video frame f n+1 from the monitoring video file V to be detected;

[0118] Step B03: Use the structural similarity algorithm to calculate the similarity between the video frames f n and f n+1 , denoted as S1(f n , f n+1 );

[0119] Step B04: Normalize the calculated S1(f n , fn+1 ) Store it in the first array S1;

[0120] Step B05: Determine whether the calculation of the inter-frame similarity of all frames in the monitored video file V to be detected is completed. If so, execute Step B06; otherwise, let n = n + 1, and continue to execute Steps B02 to B05 until the calculation of the inter-frame similarity of all frames in the monitored video file V to be detected is completed;

[0121] Step B06: Normalize the similarity S1(f n , f n+1 ) in the first array S1 to the interval [0, 1] to obtain the standard similarity S(f n , f n+1 ), and define the video inter-frame difference value D(f n , f n+1 ) = 1 - S(f n , f n+1 ). Calculate each inter-frame difference value D(f n , f n+1 ), and store the calculated inter-frame difference value D(f n , f n+1 ) in the second array D;

[0122] Step B07: Sequentially calculate the difference ΔD of the adjacent inter-frame difference values according to the order of the inter-frame difference values D(f n , f n+1 ) in the second array D according to the formula ΔD n = D n+1 - D n , and store the difference ΔD n in the third array X; n Store it in the third array X;

[0123] Step B08: Define the number of consecutive ΔD n > 0 in the third array X greater than the FPS of the video as the rising segment;

[0124] Step B09: Define the number of consecutive ΔD n < 0 in the third array X greater than the FPS of the video as the falling segment;

[0125] Step B10: Denote each rising segment or falling segment in the third array X as a candidate gradual change shot transition segment and mark it in the second array D;

[0126] Step B11: Take out the value D(f n , f n+1 ) from the second array D in order, and n = 1.

[0127] Step B12: Calculate the gradual change threshold Tg through the formula Tg = ∑D / len(D), calculate the mutation threshold Ta through the formula Ta = 3*Tg, and determine whether D(f n ,f n+1 ) ≥ Ta. If so, execute Step B13; otherwise, execute Step B14.

[0128] Step B13: Determine that the nth frame is a mutation transition, and store the index of this frame in the video file V to be detected in the first transition state array I1.

[0129] Step B14: Determine whether all video frames have been compared. If so, execute Step B15; otherwise, set n = n + 1, and continue to execute Steps B12 to B14 until all inter-frame difference values in the array D have been compared.

[0130] Step B15: Take out a candidate gradual change lens transition segment xj from the third array X in order, where j = 1.

[0131] Step B16: If it is a rising segment, and the inter-frame difference value at the starting point of the rising segment is less than Tg, and the inter-frame difference value at the ending point is greater than Tg, then this rising segment is a gradual change transition segment, and store the index of the first frame of this rising segment in the video file V to be detected in the second transition state array I2.

[0132] Step B17: If it is a falling segment, and the inter-frame difference value at the starting point of the falling segment is greater than Tg, and the inter-frame difference value at the ending point is less than Tg, then this falling segment is a gradual change transition segment, and store the index of the last frame of this falling segment in the video file V to be detected in the second transition state array I2.

[0133] Step B18: Determine whether all candidate gradual change transition segments have been detected. If so, execute Step B19; otherwise, set j = j + 1, and continue to execute Steps B16 to B18 until all candidate gradual change transition segments have been detected.

[0134] Step B19: Store the indexes in the first transition state array I1 and the second transition state array I2 into the lens transition index array I.

[0135] Step B20: Sort the lens transition index array I in ascending order.

[0136] Output: The lens transition index array I.

[0137] The method for detecting the transitional state of a surveillance video lens provided by an embodiment of the present invention first calculates the similarity between adjacent video frames in a video file in sequence. Then, after normalizing the similarity, it determines the inter-frame difference value. By analyzing the change curve of the inter-frame difference value, it determines the rising segment and the falling segment of the inter-frame difference value, and takes each rising segment or falling segment as a candidate gradual transition segment. Then, it compares the inter-frame difference value with a mutation threshold to obtain the index of the mutation transition in the video. Finally, according to specific comparison rules, it compares the first and last inter-frame difference values of the candidate gradual transition segments with a gradual change threshold respectively, and screens out the true lens gradual transition state and obtains its index in all candidate gradual transition segments, thus completing the process of detecting the transitional state of the surveillance video lens. This method uses an edge detection method to process and analyze the similarity between video frames, determines the rising segment and the falling segment by specifying the inter-frame difference value, and compares the start frame and the end frame of the rising segment and the falling segment with the gradual change threshold, replacing the original method for determining the gradual transition, solving the problem that the maximum number of frames in the gradual transition state in the double-threshold video lens detection algorithm needs to be determined manually, reducing the usage limitations of the algorithm and the requirements for prior experience, and improving the applicability and flexibility of the algorithm.

[0138] Referring to Figure 4 , a flowchart of the steps of an embodiment of a method for segmenting a surveillance video lens according to the present invention is shown, which may specifically include:

[0139] Step S410: Obtain the surveillance video file to be detected;

[0140] Specifically, when it is necessary to segment a certain surveillance video file, first obtain the surveillance video file V to be detected. The surveillance video file V to be detected contains multiple video frames. By detecting the transitional state in the video frames, the video frames can be divided into multiple lenses, and then the key frames of the video can be extracted to identify and analyze the video content.

[0141] Step S420: Calculate the similarity between adjacent video frames in the video file in sequence, and store the calculated similarity in a first array;

[0142] Specifically, obtain video frames f n and adjacent video frames f n+1 from the surveillance video file V to be detected in sequence, where n = 1,... N, and N is the total number of video frames in the surveillance video file V to be detected. First, let n = 1, and calculate the similarity between video frame f n and video frame f n+1 , denoted as S1(f n , f n+1 ), and store the obtained S1(f n , f n+1) are sequentially stored in the array S1, recorded as the first array S1, and then n=n+1. The above steps are continued until the calculation of the similarity between all frames in the surveillance video file V to be detected is completed.

[0143] For example, the structural similarity algorithm (SSIM) is used to calculate the video frame f n and video frame f n+1 The similarity between them is denoted as S1(f n ,f n+1 ), and the calculated S1(f n ,f n+1 ) are sequentially stored in an array S1, recorded as the first array S1, and then n=n+1 is set, and the above steps are continued until all the inter-frame similarities in the surveillance video file V to be detected are calculated, thereby obtaining the first array S1, in which the similarities between adjacent video frames in the surveillance video file V to be detected are sequentially recorded. In practical applications, the last similarity can be set to the video frame f N and the similarity between the video frame f1, denoted as S1(f N ,f1), so the number of similarities in the first array S1 is the same as the number of video frames in the surveillance video file V to be detected, both of which are N.

[0144] Step S430, normalizing the similarity in the first array, determining the inter-frame difference value according to the normalized similarity, and storing the calculated inter-frame difference value in the second array;

[0145] Specifically, each similarity S1(f n ,f n+1 ) is standardized to the interval [0,1] to obtain the standard similarity S(f n ,f n+1 ), and according to the formula D(f n ,f n+1 )=1-S(f n ,f n+1 ) Calculate the difference value D(f) between each frame n ,f n+1 ), where D(f n ,f n+1 ) is the difference value between frames, and then the calculated D(f n ,f n+1 ) is stored in an inter-frame difference array, which is recorded as a second array D. The second array D records the inter-frame difference values ​​in sequence.

[0146] For example, the similarity S1(f n ,f n+1)Normalize the similarity value of the first array S1 to the interval [0, 1] to obtain the standard similarity S(f n , f n+1 ), that is, 0 ≤ S(f n , f n+1 ) ≤ 1. Then use the formula D(f n , f n+1 ) = 1 - S(f n , f n+1 ) to calculate the inter-frame difference value D(f n , f n+1 ). Store the calculated inter-frame difference value D(f n , f n+1 ) into the second array D.

[0147] Step S440: Calculate the differences between adjacent inter-frame difference values in the second array in sequence, and store the calculated differences between adjacent inter-frame difference values into the third array;

[0148] Specifically, according to the order of the inter-frame difference values D(f n , f n+1 ) in the second array D, calculate the differences ΔD n of adjacent inter-frame difference values in sequence according to the formula ΔD n+1 = D n - D n . Store the differences ΔD n into the difference array, denoted as the third array X.

[0149] Step S450: Determine the rising segment and the falling segment according to the differences between adjacent inter-frame difference values in the third array, and use the determined rising segment and falling segment as the candidate gradual change shot transition segment;

[0150] Specifically, determine the number of frames per second (Frames per Second, FPS) of the monitored video file V to be detected. If in the third array X, the number of consecutive differences ΔD n of adjacent inter-frame difference values greater than 0 is greater than M, then determine the video frames corresponding to the consecutive differences ΔD

[0151] of adjacent inter-frame difference values as the rising segment; n If in the third array X, the number of consecutive differences ΔD

[0152] of adjacent inter-frame difference values all less than 0 is greater than M, then determine the video frames corresponding to the consecutive differences ΔDn If the number of >0 is greater than the FPS, then the consecutive ΔD n The corresponding video frame is the rising segment. On the contrary, if ΔD is continuously satisfied n If the number of <0 is greater than the FPS, then the consecutive ΔD n The corresponding video frame is the falling segment. Mark the rising segment and the falling segment in the second array D. Denote each rising segment and falling segment as a candidate gradual change shot transition segment, that is, there may be a gradual change shot transition segment in the rising segment and the falling segment, and further screening is required.

[0153] Step S460: Sequentially determine whether the inter-frame difference value in the second array is greater than or equal to the mutation threshold Ta. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index in the first transition state array.

[0154] Specifically, when determining whether it is a shot transition frame, first, the judgment threshold needs to be determined, which is divided into the mutation threshold Ta and the gradual change threshold Tb. First, calculate the average value of each inter-frame difference value in the second array D, and then determine the gradual change threshold Tg according to the average value. For example, calculate the average value T of each inter-frame difference value in the second array D according to the formula T = ∑D / len(D), and then let Tg = T to determine the gradual change threshold Tg, where len(D) is the number of inter-frame difference values in the second array D.

[0155] Then, according to the formula Tg = k * Ta, where 0 < k < 1, determine the mutation threshold Ta. For example, set k = 1 / 3, and the mutation threshold Ta = 3 * Tg can be obtained.

[0156] After that, sequentially take out the inter-frame difference value D(f n , f n+1 ) from the second array D, where n = 1.

[0157] If the inter-frame difference value D(f n , f n+1 ) ≥ Ta, then the nth frame is a mutation transition, and store the index of this frame in the video file V to be detected in the first transition state array I1.

[0158] Let n = n + 1, and continue to execute the above steps until all inter-frame difference values in the second array D are compared. The finally obtained first transition state array I1 records the indexes of all mutation transition frames in the video file V to be detected.

[0159] Step S470: For each rising segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first rising segment is less than the gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index in the second transition state array;

[0160] Specifically, take out a candidate gradual change lens transition segment xj from the second array D in sequence, where j = 1.

[0161] If xj is marked as a rising segment, determine whether the first inter-frame difference value of xj is less than the gradual change threshold Tg. If it is less, then continue to determine whether the last inter-frame difference value of xj is greater than the gradual change threshold Tg. If both are satisfied, determine that xj is a gradual change transition segment, and store the index of the first frame of xj in the to-be-detected surveillance video file V in the second transition state array I2. Let j = j + 1, and continue to execute the above steps until all rising segments in all candidate gradual change lens transition segments in the second array D are detected.

[0162] Step S480: For each falling segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index in the second transition state array;

[0163] Specifically, take out a candidate gradual change lens transition segment xj from the second array D in sequence, where j = 1.

[0164] If xj is marked as a falling segment, determine whether the first inter-frame difference value of xj is greater than the gradual change threshold Tg. If it is less, then continue to determine whether the last inter-frame difference value of xj is less than the gradual change threshold Tg. If both are satisfied, determine that xj is a gradual change transition segment, and store the index of the last frame of xj in the to-be-detected surveillance video file V in the second transition state array I2. Let j = j + 1, and continue to execute the above steps until all falling segments in all candidate gradual change lens transition segments in the second array D are detected. The second transition state array I2 stores the indices of all gradual change transition frames.

[0165] An embodiment of the present invention provides a method for judging the gradual transition state of a monitoring video lens and a method for filtering lens noise. By filtering the rises / falls with the number of consecutive rising / falling segments less than the FPS, and further screening out the true lens transition frames from the rising / falling segments with the number of consecutive rising / falling segments greater than the FPS, the influence of video noise on lens detection is reduced, and the robustness, applicability, and flexibility of the algorithm are improved. Step S490: Segment the monitored video to be detected according to the indexes in the first transition state array and the second transition state array.

[0166] Specifically, since the indexes of the transition frames are stored in the first transition state array I1 and the second transition state array I2, the indexes in the first transition state array I1 and the second transition state array I2 are stored in the lens transition index array I. After all the indexes in the lens transition index array I are output in sequence, the indexes of all the transition frames in the monitored video lens to be detected can be obtained. According to these indexes, the monitored video to be detected is segmented, and the monitored video to be detected is divided into multiple lenses.

[0167] The method for segmenting the monitored video lens provided by the embodiment of the present invention first calculates the similarity of adjacent video frames in the video file in sequence, and then, after normalizing the similarity, determines the inter-frame difference value. By analyzing the change curve of the inter-frame difference value, the rising segment and the falling segment of the inter-frame difference value are determined, and each rising segment or falling segment is used as a candidate gradual transition segment; then, the inter-frame difference value is compared with the mutation threshold to obtain the indexes of the mutation transitions in the video; finally, according to specific comparison rules, the first and last inter-frame difference values of the candidate gradual transition segments are respectively compared with the gradual transition threshold, and among all the candidate gradual transition segments, the true lens gradual transition state is screened out and its indexes are obtained, so as to complete the process of segmenting the monitored video lens according to these indexes. This method uses the edge detection method to process and analyze the similarity between video frames, determines the rising segment and the falling segment by specifying the inter-frame difference value, and compares the start frame and the end frame of the rising segment and the falling segment with the gradual transition threshold, replacing the original method for determining the gradual transition, solving the problem that the maximum number of frames in the gradual transition state in the double-threshold video lens detection algorithm needs to be determined manually, reducing the usage limitations of the algorithm and the requirements for prior experience, and improving the applicability and flexibility of the algorithm.

[0168] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0169] Refer to Figure 5 As shown in Figure 5 , a structural block diagram of an embodiment of a monitoring video lens transition state detection device according to the present invention is shown, which may specifically include the following modules: a first acquisition module 510, a first similarity calculation module 520, a first inter-frame difference value calculation module 530, a first difference calculation module 540, a first rising and falling judgment module 550, a first mutation judgment module 560, a first gradual change judgment module 570, a second gradual change judgment module 580, and a transition frame determination module 590, where:

[0170] The first acquisition module 510 is configured to acquire a monitoring video file to be detected; the first similarity calculation module 520 is configured to sequentially calculate the similarity of adjacent video frames in the video file and store the calculated similarity in a first array; the first inter-frame difference value calculation module 530 is configured to normalize the similarity in the first array and determine an inter-frame difference value according to the normalized similarity, and store the calculated inter-frame difference value in a second array; the first difference calculation module 540 is configured to sequentially calculate the difference between adjacent inter-frame difference values in the second array and store the calculated difference between adjacent inter-frame difference values in a third array; the first rising and falling judgment module 550 is configured to determine a rising segment and a falling segment according to the difference between adjacent inter-frame difference values in the third array and use the determined rising segment and falling segment as candidate gradual change lens transition segments; the first mutation judgment module 560 is configured to sequentially determine whether the inter-frame difference value in the second array is greater than or equal to a mutation threshold Ta, and if so, determine the index of the video frame corresponding to the inter-frame difference value in the video file and store the index in a first transition state array; the first gradual change judgment module 570 is configured to, for each rising segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first rising segment is less than a gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determine the index of the first frame in the first rising segment in the video file and store the index in a second transition state array; the second gradual change judgment module 580 is configured to, for each falling segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determine the index of the last frame in the first falling segment in the video file and store the index in the second transition state array; the transition frame determination module 590 is configured to determine all transition frames of the monitoring video lens to be detected according to the indexes in the first transition state array and the second transition state array.

[0171] For the embodiment of the monitoring video shot transition state detection device, since it is basically similar to the embodiment of the monitoring video shot transition state detection method, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment, and details will not be repeated here.

[0172] Referring to Figure 6 , a structural block diagram of an embodiment of a monitoring video shot transition state detection device according to the present invention is shown, which may specifically include the following modules: a second acquisition module 610, a second similarity calculation module 620, a second inter-frame difference value calculation module 630, a second difference calculation module 640, a second rising and falling judgment module 650, a second mutation judgment module 660, a third gradual change judgment module 670, a fourth gradual change judgment module 680, and a shot segmentation module 690, where:

[0173] The second acquisition module 610 is configured to acquire a monitoring video file to be detected; the second similarity calculation module 620 is configured to calculate the similarity between adjacent video frames in the video file in sequence, and store the calculated similarity into a first array; the second inter-frame difference value calculation module 630 is configured to normalize the similarity in the first array, and determine an inter-frame difference value according to the normalized similarity, and store the calculated inter-frame difference value into a second array; the second difference calculation module 640 is configured to calculate the difference between adjacent inter-frame difference values in the second array in sequence, and store the calculated difference between adjacent inter-frame difference values into a third array; the second rising and falling judgment module 650 is configured to determine a rising segment and a falling segment according to the difference between adjacent inter-frame difference values in the third array, and use the determined rising segment and falling segment as candidate gradual change shot transition segments; the second mutation judgment module 660 is configured to judge in sequence whether the inter-frame difference value in the second array is greater than or equal to a mutation threshold Ta, if so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index into a first transition state array; the third gradual change judgment module 670 is configured to, for each rising segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first rising segment is less than a gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index into a second transition state array; the fourth gradual change judgment module 680 is configured to, for each falling segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index into the second transition state array; the shot segmentation module 690 is configured to perform shot segmentation on the monitoring video to be detected according to the indexes in the first transition state array and the second transition state array.

[0174] For the embodiment of the monitoring video shot segmentation device, since it is basically similar to the embodiment of the monitoring video shot segmentation method, the description is relatively simple. For related parts, refer to the partial description of the method embodiment, and details will not be repeated here.

[0175] Referring to Figure 7 , a structural block diagram of an embodiment of an electronic device according to the present invention is shown. The device includes: a processor 710, a memory 720, and a bus 730;

[0176] Wherein, the processor 710 and the memory 720 communicate with each other through the bus 730;

[0177] The processor 710 is used to call program instructions in the memory 720 to execute the methods provided in the above method embodiments. For example, it includes: obtaining a monitoring video file to be detected; sequentially calculating the similarity between adjacent video frames in the video file, and storing the calculated similarity into a first array; normalizing the similarity in the first array, and determining an inter-frame difference value according to the normalized similarity, and storing the calculated inter-frame difference value into a second array; sequentially calculating the difference between adjacent inter-frame difference values in the second array, and storing the calculated difference between adjacent inter-frame difference values into a third array; determining a rising segment and a falling segment according to the difference between adjacent inter-frame difference values in the third array, and using the determined rising segment and falling segment as candidate gradual change lens transition segments; sequentially determining whether the inter-frame difference value in the second array is greater than or equal to a mutation threshold Ta. If so, determining the index of the video frame corresponding to the inter-frame difference value in the video file, and storing the index into a first transition state array; for each rising segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first rising segment is less than a gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determining the index of the first frame in the first rising segment in the video file, and storing the index into a second transition state array; for each falling segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determining the index of the last frame in the first falling segment in the video file, and storing the index into the second transition state array; determining all transition frames of the monitoring video lens to be detected according to the indexes in the first transition state array and the second transition state array.

[0178] An embodiment of the present invention discloses a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided in the above-mentioned method embodiments. For example, it includes: obtaining a monitoring video file to be detected; sequentially calculating the similarity between adjacent video frames in the video file and storing the calculated similarity in a first array; normalizing the similarity in the first array and determining an inter-frame difference value according to the normalized similarity, and storing the calculated inter-frame difference value in a second array; sequentially calculating the difference between adjacent inter-frame difference values in the second array and storing the calculated difference between adjacent inter-frame difference values in a third array; determining an ascending segment and a descending segment according to the difference between adjacent inter-frame difference values in the third array and using the determined ascending segment and descending segment as candidate gradual change lens transition segments; sequentially determining whether the inter-frame difference value in the second array is greater than or equal to a mutation threshold Ta. If so, determining the index of the video frame corresponding to the inter-frame difference value in the video file and storing the index in a first transition state array; for each ascending segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first ascending segment is less than a gradual change threshold Tg and the last inter-frame difference value in the first ascending segment is greater than the gradual change threshold Tg, determining the index of the first frame in the first ascending segment in the video file and storing the index in a second transition state array; for each descending segment in the candidate gradual change lens transition segment, if the first inter-frame difference value in the first descending segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first descending segment is less than the gradual change threshold Tg, determining the index of the last frame in the first descending segment in the video file and storing the index in the second transition state array; determining all transition frames of the monitoring video lens to be detected according to the indexes in the first transition state array and the second transition state array.

[0179] An embodiment of the present invention provides a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the methods provided in the above method embodiments, for example, including: obtaining a monitoring video file to be detected; sequentially calculating the similarity between adjacent video frames in the video file, and storing the calculated similarity into a first array; normalizing the similarity in the first array, and determining an inter-frame difference value according to the normalized similarity, and storing the calculated inter-frame difference value into a second array; sequentially calculating the difference between adjacent inter-frame difference values in the second array, and storing the calculated difference between adjacent inter-frame difference values into a third array; determining an ascending segment and a descending segment according to the difference between adjacent inter-frame difference values in the third array, and using the determined ascending segment and descending segment as candidate transition segments for gradual change shots; sequentially determining whether the inter-frame difference value in the second array is greater than or equal to a mutation threshold Ta. If so, determining the index of the video frame corresponding to the inter-frame difference value in the video file, and storing the index into a first transition state array; for each ascending segment in the candidate transition segments for gradual change shots, if the first inter-frame difference value in the first ascending segment is less than a gradual change threshold Tg and the last inter-frame difference value in the first ascending segment is greater than the gradual change threshold Tg, determining the index of the first frame in the first ascending segment in the video file, and storing the index into a second transition state array; for each descending segment in the candidate transition segments for gradual change shots, if the first inter-frame difference value in the first descending segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first descending segment is less than the gradual change threshold Tg, determining the index of the last frame in the first descending segment in the video file, and storing the index into the second transition state array; determining all transition frames of the monitoring video shots to be detected according to the indexes in the first transition state array and the second transition state array.

[0180] The embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same and similar parts among the embodiments can be referred to each other.

[0181] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a device, or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] Embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal device generate means for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0183] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0185] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

[0186] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0187] The above has introduced in detail a method for detecting the transitional state of a surveillance video lens, a method for segmenting a surveillance video lens, a device for detecting the transitional state of a surveillance video lens and a device for segmenting a surveillance video lens provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for detecting the transitional state of a surveillance video shot, characterized in that, Including: Obtain the surveillance video file to be detected; Calculate the similarity of adjacent video frames in the video file in sequence, and store the calculated similarity into a first array; Normalize the similarity in the first array, and determine the inter-frame difference value according to the normalized similarity, and store the calculated inter-frame difference value into a second array; Calculate the difference between adjacent inter-frame difference values in the second array in sequence, and store the calculated difference between adjacent inter-frame difference values into a third array; Determine the rising segment and the falling segment according to the difference between adjacent inter-frame difference values in the third array, and use the determined rising segment and falling segment as candidate gradual change shot transition segments; Judge in sequence whether the inter-frame difference value in the second array is greater than or equal to the mutation threshold Ta. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index into a first transition state array; For each rising segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first rising segment is less than the gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index into a second transition state array; For each falling segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index into the second transition state array; Determine all transition frames of the surveillance video shot to be detected according to the indexes in the first transition state array and the second transition state array.

2. The method according to claim 1, characterized in that, Normalize the similarity in the first array, and determine the inter-frame difference value according to the normalized similarity, including: Normalize each similarity in the first array to the interval [0, 1] to obtain the standard similarity, and calculate each inter-frame difference value by using 1 minus the standard similarity.

3. The method according to claim 2, characterized in that, The determining the rising segment and the falling segment according to the difference between adjacent inter-frame difference values in the third array includes: Determine the number of frames displayed per second FPS of the video file; If in the third array, the number of consecutive adjacent inter-frame difference value differences greater than 0 is greater than M, determine the video frames corresponding to the consecutive adjacent inter-frame difference value differences as the rising segment; If in the third array, the number of consecutive adjacent inter-frame difference value differences all less than 0 is greater than M, determine the video frames corresponding to the consecutive adjacent inter-frame difference value differences as the falling segment; where M = FPS.

4. The method according to claim 3, characterized in that, The gradual change threshold Tg is determined by the following method: Calculate the average value of each inter-frame difference value in the second array; Determine the gradual change threshold Tg according to the average value.

5. The method according to claim 4, characterized in that, The mutation threshold Ta and the gradual change threshold Tg satisfy the following relationship: Tg = k * Ta, where 0 < k < 1.

6. A method for segmenting a surveillance video shot, characterized in that, Including: Obtain the surveillance video file to be detected; Calculate the similarity between adjacent video frames in the video file in sequence, and store the calculated similarity into a first array; Normalize the similarity in the first array, determine the inter-frame difference value according to the normalized similarity, and store the calculated inter-frame difference value into a second array; Calculate the difference between adjacent inter-frame difference values in the second array in sequence, and store the calculated difference between adjacent inter-frame difference values into a third array; Determine the rising segment and the falling segment according to the difference between adjacent inter-frame difference values in the third array, and use the determined rising segment and falling segment as candidate gradual change shot transition segments; Judge in sequence whether the inter-frame difference value in the second array is greater than or equal to the mutation threshold Ta. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index into a first transition state array; For each rising segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first rising segment is less than the gradual change threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradual change threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index into a second transition state array; For each falling segment in the candidate gradual change shot transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradual change threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradual change threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index into the second transition state array; Perform shot segmentation on the to-be-detected surveillance video according to the indexes in the first transition state array and the second transition state array.

7. A device for detecting the transitional state of a surveillance video shot, characterized in that, Include: A first acquisition module, configured to acquire a to-be-detected surveillance video file; A first similarity calculation module, configured to calculate the similarity between adjacent video frames in the video file in sequence, and store the calculated similarity into a first array; A first inter-frame difference value calculation module, configured to normalize the similarity in the first array, determine the inter-frame difference value according to the normalized similarity, and store the calculated inter-frame difference value into a second array; A first difference calculation module, configured to calculate the difference between adjacent inter-frame difference values in the second array in sequence, and store the calculated difference between adjacent inter-frame difference values into a third array; A first rising and falling judgment module, configured to determine the rising segment and the falling segment according to the difference between adjacent inter-frame difference values in the third array, and use the determined rising segment and falling segment as candidate gradual change shot transition segments; A first mutation judgment module, configured to judge in sequence whether the inter-frame difference value in the second array is greater than or equal to the mutation threshold Ta. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index into a first transition state array; The first gradient judgment module is used to, for each rising segment in the candidate gradient lens transition segment, if the first inter-frame difference value in the first rising segment is less than the gradient threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradient threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index in the second transition state array; The second gradient judgment module is used to, for each falling segment in the candidate gradient lens transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradient threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradient threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index in the second transition state array; The transition frame determination module is used to determine all transition frames of the to-be-detected surveillance video lens according to the indexes in the first transition state array and the second transition state array.

8. A monitoring video lens segmentation device, characterized in that, It includes: The second acquisition module is used to acquire the to-be-detected surveillance video file; The second similarity calculation module is used to calculate the similarity of adjacent video frames in the video file in sequence, and store the calculated similarity in the first array; The second inter-frame difference value calculation module is used to standardize the similarity in the first array, and determine the inter-frame difference value according to the standardized similarity, and store the calculated inter-frame difference value in the second array; The second difference calculation module is used to calculate the difference between adjacent inter-frame difference values in the second array in sequence, and store the calculated difference between adjacent inter-frame difference values in the third array; The second rising and falling judgment module is used to determine the rising segment and the falling segment according to the difference between adjacent inter-frame difference values in the third array, and use the determined rising segment and falling segment as the candidate gradient lens transition segment; The second mutation judgment module is used to judge in sequence whether the inter-frame difference value in the second array is greater than or equal to the mutation threshold Ta. If so, determine the index of the video frame corresponding to the inter-frame difference value in the video file, and store the index in the first transition state array; The third gradient judgment module is used to, for each rising segment in the candidate gradient lens transition segment, if the first inter-frame difference value in the first rising segment is less than the gradient threshold Tg and the last inter-frame difference value in the first rising segment is greater than the gradient threshold Tg, determine the index of the first frame in the first rising segment in the video file, and store the index in the second transition state array; The fourth gradient judgment module is used to, for each falling segment in the candidate gradient lens transition segment, if the first inter-frame difference value in the first falling segment is greater than the gradient threshold Tg and the last inter-frame difference value in the first falling segment is less than the gradient threshold Tg, determine the index of the last frame in the first falling segment in the video file, and store the index in the second transition state array; A shot segmentation module, configured to perform shot segmentation on the monitored video to be detected according to indexes in the first transition state array and the second transition state array.

9. An electronic device, characterized in that, It includes: A memory and a processor, and the processor and the memory complete communication with each other through a bus; The memory stores program instructions executable by the processor, and the processor can execute the method according to any one of claims 1 to 6 by invoking the program instructions.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images