Lens cutting method and device, electronic equipment and computer readable storage medium

Abstract

This invention provides a shot segmentation method, apparatus, electronic device, and computer-readable storage medium, relating to the field of security technology. It includes: improving the stability of the shot segmentation algorithm by calculating the similarity between video frames; and introducing video background information through background frames to fully consider the overall semantic changes of the video, assisting in guiding the dual-threshold algorithm for shot segmentation, thereby improving the information content and semantic integrity of the output shots. Furthermore, the deep integration of background information and video semantics effectively eliminates shots with low information content, increasing the value of shot information and facilitating the rapid location and identification of specific targets in the video.

Description

Technical Field

[0001] This invention relates to the field of security technology, and in particular to a lens splitting method, a lens splitting device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the development of communication technology and the improvement of hardware performance, the trend of information exchange and dissemination shifting towards video in my country has become increasingly significant. Currently, video surveillance technology is being used more and more widely, not only ubiquitous on streets and alleys, but also entering people's homes as security monitoring equipment.

[0003] As a comprehensive media form integrating visual, auditory, and textual information, video plays an important role in various fields. The hierarchical structure of video can be divided into three logical units from top to bottom: frame, shot, and scene.

[0004] As the smallest semantic unit of video, there is a transition state between two adjacent video shots. Depending on the semantic content of the transition and speed, this transition state can be divided into abrupt transitions and gradual transitions. The detection and segmentation of surveillance video shots is a crucial upstream task for video keyframe extraction, and it is of great significance for video content recognition and analysis.

[0005] Video shot segmentation utilizes video shot boundary detection algorithms to detect the boundaries of each shot in the video, and then divides the video into several individual shot units based on the detection results. Currently, the commonly used video shot boundary detection algorithm is the double threshold method, but this method has the following problems:

[0006] (1) The inability to consider stationary targets within the video results in a unified shot being divided into several parts;

[0007] (2) The semantic features of the video were not considered, resulting in incomplete semantics in the output shots;

[0008] (3) It is impossible to remove shots with low information content, which reduces the information content of the output shot. Summary of the Invention

[0009] In view of the above problems, embodiments of the present invention are proposed to provide a lens segmentation method, a lens segmentation device, an electronic device, and a computer-readable storage medium that overcome or at least partially solve the above problems.

[0010] To address the aforementioned problems, this invention discloses a lens segmentation method, comprising:

[0011] S1. Obtain an array of similarity between adjacent frames and an array of similarity between background frames; the array of similarity between adjacent frames contains multiple adjacent frames in sequence, each of which is calculated based on any two adjacent video frames in the video; the array of similarity between background frames contains multiple background frames in sequence, each of which is calculated based on each video frame in the video and the background frame of the video.

[0012] S2. Determine the inter-frame difference array based on the inter-frame similarity array, and determine the inter-frame difference array based on the background inter-frame similarity array;

[0013] S3. Determine the average inter-frame difference of the background frame in the video based on the background inter-frame difference array;

[0014] S4. Assign the start and end index to m, and obtain the nth inter-frame difference from the inter-frame difference array, where m and n are both positive integers.

[0015] S5. If the difference between the nth adjacent frames is not less than the mutation threshold, then the nth video frame in the video is a sudden transition, and S8 is executed; otherwise, S6 is executed.

[0016] S6. If the difference between the nth adjacent frames is within the range of the sudden change threshold and the gradual change threshold, and the candidate difference of the nth video frame is not less than the sudden change threshold, then the nth video frame is in a gradual transition state, and S8 is executed; otherwise, S7 is executed.

[0017] S7. If the difference between the nth adjacent frames is less than the gradient threshold, then execute S9.

[0018] S8. If the average value of the background inter-frame difference degree in the background inter-frame difference degree array that belongs to the index range [m, n] is greater than the average background inter-frame difference degree, then the index range is stored in the index array and m = n + 1 is executed.

[0019] S9. Execute n = n + 1 and S4 until all adjacent inter-frame differences in the adjacent inter-frame difference array are obtained, and then execute S10.

[0020] S10. The video is divided into multiple shot files based on the index array.

[0021] In one or more embodiments, the adjacent frame similarity array and the background frame similarity array are generated in the following manner:

[0022] Obtain the first k video frames of the video; k is a positive integer;

[0023] Calculate the average value of the first k video frames to obtain the background frame of the video;

[0024] The similarity between any two adjacent video frames in the video is calculated based on the order of the video frames, resulting in an adjacent frame similarity array containing multiple adjacent frame similarities arranged in order.

[0025] The similarity between each video frame and the background frame is calculated based on the order of the video frames, resulting in a background frame similarity array containing multiple background frame similarities arranged in order.

[0026] In one or more embodiments, determining an inter-frame difference array based on the inter-frame similarity array and determining a background inter-frame difference array based on the background inter-frame similarity array include:

[0027] The adjacent frame similarity array is normalized to a preset interval to obtain a normalized adjacent frame similarity array, and the background frame similarity array is normalized to the preset interval to obtain a normalized background frame similarity array.

[0028] Invert the normalized inter-frame similarity array to obtain an inter-frame difference array, and invert the normalized background inter-frame similarity array to obtain a background inter-frame difference array.

[0029] In one or more embodiments, determining the average inter-frame background difference of the video based on the background inter-frame difference array includes:

[0030] Calculate the sum of all background inter-frame differences in the background inter-frame difference array;

[0031] The average inter-frame difference of the background video is obtained by calculating the ratio of the sum to the length of the background inter-frame difference array.

[0032] In one or more embodiments, the step of determining that the nth video frame is in a gradual transition state if the difference between the nth adjacent frames is within the range of the abrupt change threshold and the gradual transition threshold, and the candidate difference of the nth video frame is not less than the abrupt change threshold, includes:

[0033] If the difference between the nth adjacent frames is within the range of the abrupt change threshold and the gradual change threshold, then the candidate difference of the nth video frame is calculated.

[0034] If the candidate difference degree of the nth video frame is not less than the mutation threshold, then the nth video frame is in a gradual transition state.

[0035] Accordingly, embodiments of the present invention disclose a lens splitting device, the device comprising:

[0036] The acquisition module is used to acquire an array of similarity between adjacent frames and an array of similarity between background frames; the array of similarity between adjacent frames contains multiple adjacent frame similarities arranged in order, each of which is calculated based on any two adjacent video frames in the video; the array of similarity between background frames includes multiple background frame similarities arranged in order, each of which is calculated based on each video frame in the video and the background frame of the video.

[0037] The determining module is configured to determine an inter-frame difference array based on the inter-frame similarity array, and to determine a background inter-frame difference array based on the background inter-frame similarity array.

[0038] The determining module is further configured to determine the average inter-frame difference of the background of the video based on the background inter-frame difference array;

[0039] The assignment module is used to assign the start and end indices to m.

[0040] The acquisition module is further configured to acquire the nth inter-frame difference from the inter-frame difference array, where m and n are both positive integers;

[0041] The execution module is used to call the storage module if the difference between the nth adjacent frames is not less than the abrupt change threshold, then the nth video frame in the video is an abrupt transition; otherwise, the execution module is called.

[0042] The execution module is further configured to call the storage module if the difference between the nth adjacent frames is within the range of the mutation threshold and the gradual transition threshold, and the candidate difference of the nth video frame is not less than the mutation threshold; otherwise, the execution module is called.

[0043] The execution module is further configured to invoke the execution module if the difference between the nth adjacent frames is less than the gradient threshold.

[0044] The storage module is configured to store the index range into the index array and execute m = n + 1 if the average value of the background inter-frame difference in the background inter-frame difference array that belongs to the index range [m, n] is greater than the average background inter-frame difference.

[0045] The execution module is also used to execute n = n + 1 and call the assignment module until all adjacent inter-frame differences in the adjacent inter-frame difference array are obtained, and then call the segmentation module.

[0046] The segmentation module is used to segment the video into multiple shot files based on the index array.

[0047] In one or more embodiments, the adjacent frame similarity array and the background frame similarity array are generated in the following manner:

[0048] Obtain the first k video frames of the video; k is a positive integer;

[0049] Calculate the average value of the first k video frames to obtain the background frame of the video;

[0050] The similarity between any two adjacent video frames in the video is calculated based on the order of the video frames, resulting in an adjacent frame similarity array containing multiple adjacent frame similarities arranged in order.

[0051] The similarity between each video frame and the background frame is calculated based on the order of the video frames, resulting in a background frame similarity array containing multiple background frame similarities arranged in order.

[0052] In one or more embodiments, the determining module is specifically used for:

[0053] The adjacent frame similarity array is normalized to a preset interval to obtain a normalized adjacent frame similarity array, and the background frame similarity array is normalized to the preset interval to obtain a normalized background frame similarity array.

[0054] Invert the normalized inter-frame similarity array to obtain an inter-frame difference array, and invert the normalized background inter-frame similarity array to obtain a background inter-frame difference array.

[0055] In one or more embodiments, the determining module is further configured to:

[0056] Calculate the sum of all background inter-frame differences in the background inter-frame difference array;

[0057] The average inter-frame difference of the background video is obtained by calculating the ratio of the sum to the length of the background inter-frame difference array.

[0058] In one or more embodiments, the execution module is specifically used for:

[0059] If the difference between the nth adjacent frames is within the range of the abrupt change threshold and the gradual change threshold, then the candidate difference of the nth video frame is calculated.

[0060] If the candidate difference degree of the nth video frame is not less than the mutation threshold, then the nth video frame is in a gradual transition state.

[0061] Accordingly, this invention discloses an electronic device, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various steps of the above-described lens segmentation method embodiments.

[0062] Accordingly, this invention discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the various steps of the above-described lens segmentation method embodiments.

[0063] The embodiments of the present invention have the following advantages:

[0064] By calculating the similarity between video frames, the stability of the shot segmentation algorithm is improved. Furthermore, by introducing background information through background frames, the algorithm fully considers the overall semantic changes in the video, assisting in guiding the dual-threshold algorithm to segment video shots and increasing the information content and semantic integrity of the output shots. In addition, the deep integration of background information with video semantics effectively eliminates shots with low information content, increasing the value of shot information and facilitating the rapid location and identification of specific targets in the video. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating the steps of an embodiment of the lens segmentation method of the present invention;

[0066] Figure 2 This is a structural block diagram of an embodiment of the lens splitting device of the present invention. Detailed Implementation

[0067] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] One of the core concepts of this invention is that by calculating the similarity between video frames, the stability of the shot segmentation algorithm is improved. Furthermore, by introducing video background information through background frames, the overall semantic changes of the video are fully considered, assisting in guiding the dual-threshold algorithm to segment video shots and improving the information content and semantic integrity of the output shots. In addition, the deep integration of background information with video semantics effectively eliminates shots with low information content, increasing the value of shot information and facilitating the rapid location and identification of specific targets in the video.

[0069] Reference Figure 1 The diagram illustrates a step flowchart of an embodiment of the lens segmentation method of the present invention, which may specifically include the following steps:

[0070] Step S1: Obtain the adjacent frame similarity array and the background frame similarity array; the adjacent frame similarity array contains multiple adjacent frame similarities arranged in order, each of which is calculated based on any two adjacent video frames in the video; the background frame similarity array includes multiple background frame similarities arranged in order, each of which is calculated based on each video frame in the video and the background frame of the video.

[0071] The embodiments of the present invention can be applied to a monitoring system. When performing shot segmentation on the video collected by the monitoring device, an array of similarity between adjacent frames (denoted as S1 for ease of description) and an array of similarity between background frames (denoted as S2) can be obtained.

[0072] S1 includes multiple adjacent frame similarities arranged in sequence, with each adjacent frame similarity calculated based on the corresponding two adjacent video frames. For example, the first adjacent frame similarity is calculated based on the first and second video frames, the second adjacent frame similarity is calculated based on the second and third video frames, and so on.

[0073] Similarly, S2 includes multiple background frame similarities arranged in sequence, each calculated based on the corresponding video frame and background frame. For example, the first background frame similarity is calculated based on the first video frame and background frame, the second is calculated based on the second video frame and background frame, and so on. Here, any background frame similarity is denoted as...

[0074] In this embodiment of the invention, the adjacent frame similarity array and the background frame similarity array are generated in the following manner:

[0075] Obtain the first k video frames of the video; k is a positive integer;

[0076] Calculate the average value of the first k video frames to obtain the background frame of the video;

[0077] The similarity between any two adjacent video frames in the video is calculated based on the order of the video frames, resulting in an adjacent frame similarity array containing multiple adjacent frame similarities arranged in order.

[0078] The similarity between each video frame and the background frame is calculated based on the order of the video frames, resulting in a background frame similarity array containing multiple background frame similarities arranged in order.

[0079] Specifically, after acquiring video, the monitoring equipment can obtain the first k video frames, where k is a positive integer, such as k = 10. Then, the average of these k video frames is calculated to obtain the background frame, denoted as f.b .

[0080] Calculate the similarity between all adjacent video frames in the order of the video frames, and store all similarities in an array according to the order of the video frames, to obtain the adjacent frame similarity array S1. The similarity between any two adjacent frames is denoted as S1(f n ,f n+1 ).

[0081] Similarly, following the order of all video frames in the video frame, the similarity between each video frame and the background frame is calculated, and all similarities are stored in an array according to the order of the video frames to obtain the background frame similarity array S2. Here, the similarity between any two background frames is denoted as S2(f b ,f n ).

[0082] It should be noted that in practical applications, the value of k can be set according to actual needs, and this embodiment of the invention does not impose any restrictions on this.

[0083] Furthermore, when calculating the similarity between video frames, the SSIM (structural similarity) algorithm can be used, or other algorithms can be used. In practical applications, the settings can be made according to actual needs, and the embodiments of the present invention do not impose any restrictions on this.

[0084] Step S2: Determine the inter-frame difference array based on the inter-frame similarity array, and determine the inter-frame difference array based on the background inter-frame similarity array.

[0085] Since the similarity between adjacent frames and the similarity between background frames are used to represent the degree of similarity between video frames, the difference between video frames (the degree of dissimilarity between video frames) can be determined based on the similarity between video frames. Therefore, the difference between adjacent frames can be determined based on the similarity between adjacent frames, and the difference between background frames can be determined based on the similarity between background frames.

[0086] In this embodiment of the invention, determining the inter-frame difference array based on the inter-frame similarity array and determining the background inter-frame difference array based on the background inter-frame similarity array include:

[0087] The adjacent frame similarity array is normalized to a preset interval to obtain a normalized adjacent frame similarity array, and the background frame similarity array is normalized to the preset interval to obtain a normalized background frame similarity array.

[0088] Invert the normalized inter-frame similarity array to obtain an inter-frame difference array, and invert the normalized background inter-frame similarity array to obtain a background inter-frame difference array.

[0089] Specifically, after obtaining S1 and S2, S1 and S2 can be normalized to a preset interval to obtain normalized S1 and S2. Then, the normalized S1 and S2 are inverted to obtain the inter-frame difference array (denoted as S1') and the background inter-frame difference array (denoted as S2').

[0090] It should be noted that the preset range can be [0,1] or other values. In practical applications, it can be set according to actual needs. This embodiment of the invention does not impose any restrictions on this.

[0091] Furthermore, when performing the inversion calculation, the formula S' = 1 - S (where S' is the difference array and S is the similarity array) can be used, or other methods can be used. In practical applications, the settings can be made according to actual needs, and the embodiments of the present invention do not impose any restrictions on this.

[0092] Step S3: Determine the average background inter-frame difference of the video based on the background inter-frame difference array.

[0093] After obtaining the background inter-frame difference array, the average background inter-frame difference can be further calculated, i.e., the average background inter-frame difference, denoted as S. diff .

[0094] In this embodiment of the invention, determining the average inter-frame background difference of the video based on the background inter-frame difference array includes:

[0095] Calculate the sum of all background inter-frame differences in the background inter-frame difference array;

[0096] The average inter-frame difference of the background video is obtained by calculating the ratio of the sum to the length of the background inter-frame difference array.

[0097] Specifically, first calculate the sum of all background frame differences in S2', denoted as sum(S2'). Then, calculate the ratio of sum(S2') to the length of the array len(S2'), thus obtaining the average background frame difference of the video, i.e., S diff = sum(S2') / len(S2').

[0098] Of course, in addition to using the above method to calculate the average difference between background frames, other methods can also be used. In practical applications, the settings can be made according to actual needs, and the embodiments of the present invention do not limit this.

[0099] Step S4: Assign the start and end index to m, and obtain the nth inter-frame difference from the inter-frame difference array, where m and n are both positive integers;

[0100] Step S5: If the difference between the nth adjacent frames is not less than the mutation threshold, then the nth video frame in the video is a mutation transition, and S8 is executed; otherwise, S6 is executed.

[0101] Specifically, the start and end indices (denoted as shot_start_index) are assigned the value m, for example, shot_start_index = 1, and the difference between the nth adjacent frames S1'(f) is obtained from S1'. n ,f n+1 ), where n is a positive integer.

[0102] If S1'(f n ,f n+1 Not less than the preset mutation threshold T a That is, S1'(f n ,f n+1 )>=T a If the nth video frame is a sudden transition, then step S8 is executed; otherwise, step S6 is executed.

[0103] Step S6: If the difference between the nth adjacent frames is within the range of the mutation threshold and the gradual transition threshold, and the candidate difference of the nth video frame is not less than the mutation threshold, then the nth video frame is in a gradual transition state, and proceed to S8; otherwise, proceed to S7.

[0104] If the difference between the nth adjacent frames is less than the abrupt change threshold and greater than the gradual change threshold, that is, T g <S1'(f n ,f n+1 ) <T a Furthermore, the candidate difference degree (denoted as A(j)) of the nth video frame is not less than the mutation threshold, that is, A(j)>=T a If the nth video frame is in a gradual transition state, then step S8 is executed.

[0105] If the difference between the nth adjacent frames is not within the range of the mutation threshold and the gradual threshold, or if the candidate difference of the nth video frame is less than the mutation threshold, then step S7 is executed.

[0106] In this embodiment of the invention, if the difference between the nth adjacent frames is within the range of the abrupt change threshold and the gradual transition threshold, and the candidate difference of the nth video frame is not less than the abrupt change threshold, then the nth video frame is in a gradual transition state, including:

[0107] If the difference between the nth adjacent frames is within the range of the abrupt change threshold and the gradual change threshold, then the candidate difference of the nth video frame is calculated.

[0108] If the candidate difference degree of the nth video frame is not less than the mutation threshold, then the nth video frame is in a gradual transition state.

[0109] Specifically, if T g <S1'(f n ,f n+1 ) <T a If the nth video frame is selected as a candidate frame for gradual transition, then the candidate difference degree of the nth video frame is calculated. If the candidate difference degree is not less than the mutation threshold, then the nth video frame is selected as the gradual transition state.

[0110] Among them, the candidate difference of the nth video frame can be obtained through... The calculation is performed, where N is the maximum number of transition frames. Of course, it can also be calculated in other ways. In practical applications, it can be set according to actual needs, and this embodiment of the invention does not impose any limitations on this.

[0111] Step S7: If the difference between the nth adjacent frames is less than the gradient threshold, then execute S9.

[0112] Specifically, if the difference between the nth adjacent frames is less than the gradient threshold, i.e., S1'(f n ,f n+1 ) <T g Then proceed to step S9.

[0113] Step S8: If the average value of the background inter-frame difference in the background inter-frame difference array that belongs to the index range [m, n] is greater than the average background inter-frame difference, then the index range is stored in the index array and m = n + 1 is executed.

[0114] Specifically, since S1' and S2' have the same length, after determining the type and index range [m, n] of the nth video frame based on S1', the inter-frame difference of the background frames in S2' that also belong to [m, n] can be obtained. Then, the average value of the inter-frame difference of the background frames belonging to [m, n] can be calculated and denoted as diff. If diff > S diff If so, the index range is stored in the preset index array, and then m = n + 1 and step S9 are executed.

[0115] Step S9: Execute n = n + 1 and S4 until all adjacent inter-frame differences in the adjacent inter-frame difference array are obtained, and then execute S10.

[0116] Step S10: Divide the video into multiple shot files based on the index array.

[0117] Specifically, after storing the current index range into the index array, step S4 can be executed from the (n+1)th video frame for iteration until all the differences between adjacent frames in S1' are iterated, thus obtaining an index array containing all index ranges. Then, the video is segmented into shots according to each index range to obtain each shot file.

[0118] For ease of understanding, the complete process of the embodiments of the present invention is illustrated below:

[0119] 1. Read the captured video file;

[0120] 2. Accumulate and average the first 10 frames of the video to obtain the background frame f. b ;

[0121] 3. Extract video frames f sequentially from the video. n and video frame f n+1 n = 1;

[0122] 4. Calculate adjacent frame f using the SSIM algorithm. n and f n+1 The similarity between them is denoted as S1(f n ,f n+1 ), and store them in array S1 in order;

[0123] 5. Using SSIM, calculate the background frame f b and video frame f n The similarity between them is denoted as S2(f b ,f n ), and store them in array S2 in order;

[0124] 6. n = n + 1;

[0125] 7. Iterate from 3 to 6 until the similarity between all adjacent frames in the video and the similarity between background frames have been calculated;

[0126] 8. Normalize S1 and S2 to the interval [0,1] respectively;

[0127] 9. Calculate the inter-frame difference array S1' and the inter-frame difference array S2' of the video background using the formula S' = 1 - S;

[0128] 10. Calculate the average inter-frame background difference S of the video. diff = sum(S2) / len(S2);

[0129] 11. Assign the value shot_start_index = 1;

[0130] 12. Extract the inter-frame similarity values ​​S1'(f) from array S1' in sequence. n ,f n+1 ), n=1;

[0131] 13. If S1'(f n ,f n+1 )>=T a If the nth frame is an abrupt transition, execute step 17; otherwise, execute step 14.

[0132] 14. If T g <S1'(f n ,f n+1 ) <T a If the result is positive, then frame n is selected as a candidate frame for the gradient transition and step 15 is executed; otherwise, step 16 is executed.

[0133] 15. If If frame n is used as the transition state, then execute 17; otherwise, execute 16.

[0134] 16. If S1'(f n ,f n+1 ) <T g Then execute 19;

[0135] 17. If the average inter-frame difference diff in S2' that belongs to the index range [shot_start_index,n] is greater than S... diff If [shot_start_index, n] is not found, then [shot_start_index, n] will be stored in the index array.

[0136] 18. Execute shot_start_index = n + 1, and then execute 19;

[0137] 19. n = n + 1, iterate from 12 to 18 until all differences in array S1' have been compared;

[0138] 20. Based on the index ranges in the index array, divide the video into multiple shot files and output them.

[0139] In this embodiment of the invention, the stability of the shot segmentation algorithm is improved by calculating the similarity between video frames. Furthermore, by introducing video background information through background frames, the overall semantic changes of the video are fully considered, assisting in guiding the dual-threshold algorithm to segment video shots and improving the information content and semantic integrity of the output shots. In addition, the deep integration of background information with video semantics effectively eliminates shots with low information content, increasing the value of shot information and facilitating the rapid location and identification of specific targets in the video.

[0140] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0141] Reference Figure 2 The diagram shows a structural block diagram of an embodiment of the lens splitting device of the present invention, which may specifically include the following modules:

[0142] The acquisition module 201 is used to acquire an array of similarity between adjacent frames and an array of similarity between background frames; the array of similarity between adjacent frames contains multiple adjacent frames in sequence, each of which is calculated based on any two adjacent video frames in the video; the array of similarity between background frames includes multiple background frames in sequence, each of which is calculated based on each video frame in the video and the background frame of the video.

[0143] The determining module 202 is used to determine the inter-frame difference array based on the inter-frame similarity array, and to determine the inter-frame difference array based on the background inter-frame similarity array;

[0144] The determining module is further configured to determine the average inter-frame difference of the background of the video based on the background inter-frame difference array;

[0145] Assignment module 203 is used to assign the start and end indices to m;

[0146] The acquisition module is further configured to acquire the nth inter-frame difference from the inter-frame difference array, where m and n are both positive integers;

[0147] The execution module 204 is used to call the storage module if the difference between the nth adjacent frames is not less than the abrupt change threshold, then the nth video frame in the video is an abrupt transition; otherwise, it calls the execution module.

[0148] The execution module is further configured to call the storage module if the difference between the nth adjacent frames is within the range of the mutation threshold and the gradual transition threshold, and the candidate difference of the nth video frame is not less than the mutation threshold; otherwise, the execution module is called.

[0149] The execution module is further configured to invoke the execution module if the difference between the nth adjacent frames is less than the gradient threshold.

[0150] The storage module 205 is used to store the index range into the index array and execute m = n + 1 if the average value of the background inter-frame difference degree in the background inter-frame difference degree array that belongs to the index range [m, n] is greater than the average background inter-frame difference degree.

[0151] The execution module is also used to execute n = n + 1 and call the assignment module until all adjacent inter-frame differences in the adjacent inter-frame difference array are obtained, and then call the segmentation module.

[0152] The segmentation module 206 is used to segment the video into multiple shot files based on the index array.

[0153] In this embodiment of the invention, the adjacent frame similarity array and the background frame similarity array are generated in the following manner:

[0154] Obtain the first k video frames of the video; k is a positive integer;

[0155] Calculate the average value of the first k video frames to obtain the background frame of the video;

[0156] The similarity between any two adjacent video frames in the video is calculated based on the order of the video frames, resulting in an adjacent frame similarity array containing multiple adjacent frame similarities arranged in order.

[0157] The similarity between each video frame and the background frame is calculated based on the order of the video frames, resulting in a background frame similarity array containing multiple background frame similarities arranged in order.

[0158] In this embodiment of the invention, the determining module is specifically used for:

[0159] The adjacent frame similarity array is normalized to a preset interval to obtain a normalized adjacent frame similarity array, and the background frame similarity array is normalized to the preset interval to obtain a normalized background frame similarity array.

[0160] Invert the normalized inter-frame similarity array to obtain an inter-frame difference array, and invert the normalized background inter-frame similarity array to obtain a background inter-frame difference array.

[0161] In this embodiment of the invention, the determining module is further configured to:

[0162] Calculate the sum of all background inter-frame differences in the background inter-frame difference array;

[0163] The average inter-frame difference of the background video is obtained by calculating the ratio of the sum to the length of the background inter-frame difference array.

[0164] In this embodiment of the invention, the execution module is specifically used for:

[0165] If the difference between the nth adjacent frames is within the range of the abrupt change threshold and the gradual change threshold, then the candidate difference of the nth video frame is calculated.

[0166] If the candidate difference degree of the nth video frame is not less than the mutation threshold, then the nth video frame is in a gradual transition state.

[0167] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0168] This invention also provides an electronic device, comprising:

[0169] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described shot segmentation method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0170] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described shot segmentation method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0171] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0172] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0174] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0176] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0177] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0178] The lens segmentation method and lens segmentation device provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A lens segmentation method, characterized in that, The method includes: S1. Obtain an array of similarity between adjacent frames and an array of similarity between background frames; the array of similarity between adjacent frames contains multiple adjacent frames in sequence, each of which is calculated based on any two adjacent video frames in the video; the array of similarity between background frames contains multiple background frames in sequence, each of which is calculated based on each video frame in the video and the background frame of the video. S2. Determine the inter-frame difference array based on the inter-frame similarity array, and determine the inter-frame difference array based on the background inter-frame similarity array; S3. Determine the average inter-frame difference of the background frame in the video based on the background inter-frame difference array; S4. Assign the start and end index to m, and obtain the nth inter-frame difference from the inter-frame difference array, where m and n are both positive integers. S5. If the difference between the nth adjacent frames is not less than the mutation threshold, then the nth video frame in the video is a sudden transition, and S8 is executed; otherwise, S6 is executed. S6. If the difference between the nth adjacent frames is within the range of the sudden change threshold and the gradual change threshold, and the candidate difference of the nth video frame is not less than the sudden change threshold, then the nth video frame is in a gradual transition state, and S8 is executed; otherwise, S7 is executed. S7. If the difference between the nth adjacent frames is less than the gradient threshold, then execute S9. S8. If the average value of the background inter-frame difference degree in the background inter-frame difference degree array that belongs to the index range [m, n] is greater than the average background inter-frame difference degree, then the index range is stored in the index array and m = n + 1 is executed. S9. Execute n = n + 1 and S4 until all adjacent inter-frame differences in the adjacent inter-frame difference array are obtained, and then execute S10. S10. Divide the video into multiple shot files based on the index array; The step of determining the inter-frame difference array based on the inter-frame similarity array and determining the background inter-frame difference array based on the background inter-frame similarity array includes: The adjacent frame similarity array is normalized to a preset interval to obtain a normalized adjacent frame similarity array, and the background frame similarity array is normalized to the preset interval to obtain a normalized background frame similarity array. Invert the normalized inter-frame similarity array to obtain an inter-frame difference array, and invert the normalized background inter-frame similarity array to obtain a background inter-frame difference array.

2. The lens segmentation method according to claim 1, characterized in that, The adjacent frame similarity array and the background frame similarity array are generated in the following manner: Obtain the first k video frames of the video; k is a positive integer; Calculate the average value of the first k video frames to obtain the background frame of the video; The similarity between any two adjacent video frames in the video is calculated based on the order of the video frames, resulting in an adjacent frame similarity array containing multiple adjacent frame similarities arranged in order. The similarity between each video frame and the background frame is calculated based on the order of the video frames, resulting in a background frame similarity array containing multiple background frame similarities arranged in order.

3. The lens segmentation method according to claim 1, characterized in that, Determining the average inter-frame background difference of the video based on the background inter-frame difference array includes: Calculate the sum of all background inter-frame differences in the background inter-frame difference array; The average inter-frame difference of the background video is obtained by calculating the ratio of the sum to the length of the background inter-frame difference array.

4. The lens segmentation method according to claim 1, characterized in that, If the difference between the nth adjacent frames is within the range of the abrupt change threshold and the gradual transition threshold, and the candidate difference of the nth video frame is not less than the abrupt change threshold, then the nth video frame is in a gradual transition state, including: If the difference between the nth adjacent frames is within the range of the abrupt change threshold and the gradual change threshold, then the candidate difference of the nth video frame is calculated. If the candidate difference degree of the nth video frame is not less than the mutation threshold, then the nth video frame is in a gradual transition state.

5. A lens cutting device, characterized in that, The device includes: The acquisition module is used to acquire an array of similarity between adjacent frames and an array of similarity between background frames; the array of similarity between adjacent frames contains multiple adjacent frames in sequence, each of which is calculated based on any two adjacent video frames in the video; the array of similarity between background frames includes multiple background frames in sequence, each of which is calculated based on each video frame in the video and the background frame of the video. The determining module is configured to determine an inter-frame difference array based on the inter-frame similarity array, and to determine a background inter-frame difference array based on the background inter-frame similarity array. The determining module is further configured to determine the average inter-frame difference of the background of the video based on the background inter-frame difference array; The assignment module is used to assign the start and end indices to m. The acquisition module is further configured to acquire the nth inter-frame difference from the inter-frame difference array, where m and n are both positive integers; The execution module is used to call the storage module if the difference between the nth adjacent frames is not less than the abrupt change threshold, then the nth video frame in the video is an abrupt transition; otherwise, the execution module is called. The execution module is further configured to call the storage module if the difference between the nth adjacent frames is within the range of the mutation threshold and the gradual transition threshold, and the candidate difference of the nth video frame is not less than the mutation threshold; otherwise, the execution module is called. The execution module is further configured to invoke the execution module if the difference between the nth adjacent frames is less than the gradient threshold. The storage module is configured to store the index range into the index array and execute m = n + 1 if the average value of the background inter-frame difference in the background inter-frame difference array that belongs to the index range [m, n] is greater than the average background inter-frame difference. The execution module is also used to execute n = n + 1 and call the assignment module until all adjacent inter-frame differences in the adjacent inter-frame difference array are obtained, and then call the segmentation module. The segmentation module is used to segment the video into multiple shot files based on the index array; The determining module is specifically used for: The adjacent frame similarity array is normalized to a preset interval to obtain a normalized adjacent frame similarity array, and the background frame similarity array is normalized to the preset interval to obtain a normalized background frame similarity array. Invert the normalized inter-frame similarity array to obtain an inter-frame difference array, and invert the normalized background inter-frame similarity array to obtain a background inter-frame difference array.

6. The lens cutting device according to claim 5, characterized in that, The adjacent frame similarity array and the background frame similarity array are generated in the following manner: Obtain the first k video frames of the video; k is a positive integer; Calculate the average value of the first k video frames to obtain the background frame of the video; The similarity between any two adjacent video frames in the video is calculated based on the order of the video frames, resulting in an adjacent frame similarity array containing multiple adjacent frame similarities arranged in order. The similarity between each video frame and the background frame is calculated based on the order of the video frames, resulting in a background frame similarity array containing multiple background frame similarities arranged in order.

7. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the lens segmentation method as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the lens segmentation method as described in any one of claims 1 to 4.

Images