Intelligent video strip splitting method and system, readable storage medium and computer

Abstract

The invention provides an intelligent video splitting method and system, a readable storage medium and a computer, and the method comprises the steps: distributing a plurality of non-overlapping windows for a video frame sequence after the introduction of position coding based on the image parameters of an input video, and calculating a self-attention mechanism in each window to obtain a key frame sequence; calculating the low-dimensional similarity between the template image in the image template library and the key frame sequence according to the index threshold value corresponding to each similarity evaluation index; performing feature extraction on the template image and the key frame sequence by using a deep neural network model to respectively obtain a feature map of the template image and a feature map of the key frame sequence; and calculating the high-dimensional similarity between the feature map of the template image and the feature map of the key frame sequence, and positioning the key frame sequence based on the low-dimensional similarity and the high-dimensional similarity to realize video splitting of the input video. According to the method, low-dimensional and high-dimensional similarities are combined, so that the key frame positioning precision is effectively improved, and video splitting is further realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, readable storage medium and computer for intelligent video stripping. Background Art

[0002] With the improvement of people's living standards and the rapid development of science and technology, video data has become one of the indispensable data in people's lives, and it has a huge amount of data.

[0003] However, the current video stripping method requires training of video generation models, which requires a large number of labeled training samples, making video editing and content splitting a large workload and prone to errors. In addition, the measurement indicators of this method are single, and the multi-dimensional matching mechanism is not taken into account, and there are many redundant key frames. In addition, it mainly processes the underlying detailed information of the image based on traditional image processing methods, lacking the matching of high-dimensional semantic information. Therefore, in order to improve training efficiency and data quality, an automated video processing algorithm is needed to automatically edit and split while organizing and classifying video data. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a video intelligent stripping method, system, readable storage medium and computer to at least solve the deficiencies in the above-mentioned technology.

[0005] The present invention provides a video intelligent stripping method, comprising: Obtaining a video frame sequence of an input video, and introducing a position code into the video frame sequence; Allocating a plurality of non-overlapping windows to the video frame sequence after the position encoding is introduced based on the image parameters of the input video, and calculating a self-attention mechanism in each of the windows to obtain a corresponding key frame sequence; Constructing a plurality of similarity evaluation indicators, and calculating the low-dimensional similarity between the template image in the preset image template library and the key frame sequence according to the indicator threshold corresponding to each of the similarity evaluation indicators; Performing feature extraction on the template image and the key frame sequence using a deep neural network model to obtain a feature map of the template image and a feature map of the key frame sequence respectively; The high-dimensional similarity between the feature map of the template image and the feature map of the key frame sequence is calculated, and the key frame sequence is positioned based on the low-dimensional similarity and the high-dimensional similarity to achieve video striping of the input video.

[0006] Furthermore, the step of obtaining a video frame sequence of an input video and introducing a position code into the video frame sequence includes: Acquire a video frame sequence of an input video, and perform frame-by-frame analysis according to the video frame sequence to obtain timing information corresponding to the video frame sequence; A corresponding position code is generated according to the timing information, and the position code is introduced into the video frame sequence.

[0007] Furthermore, the steps of allocating a plurality of non-overlapping windows to the video frame sequence after the position encoding is introduced based on the image parameters of the input video, and calculating the self-attention mechanism in each of the windows to obtain the corresponding key frame sequence include: Acquire the image size of the input video, and construct a window size of the video frame sequence after introducing the position coding according to the image size; Window allocation is performed for the video frame sequence after the introduction position encoding according to the window size so that the image area of ​​the input video is covered by at least one window, and a self-attention mechanism is calculated in each of the windows to obtain a corresponding key frame sequence.

[0008] Furthermore, the calculation formula of the self-attention is: ; In the formula, is the query matrix, is the key matrix, is the value matrix, for The transposed matrix of is the dimension of the key.

[0009] Furthermore, the similarity evaluation index includes a structural similarity index, a peak signal-to-noise ratio index and a mutual information index. The expression of the structural similarity index is: ; ; ; ; In the formula, Respectively represent the template image and key frame sequence in the image template library, are greater than 0, Indicates brightness comparison, Indicates contrast comparison, Indicates structural comparison. Respectively The average value of Respectively The standard deviation of express The covariance of , , are constants respectively.

[0010] The present invention also provides a video intelligent stripping system, comprising: A coding introduction module, used for acquiring a video frame sequence of an input video and introducing a position code into the video frame sequence; A window allocation module, used for allocating a plurality of non-overlapping windows to the video frame sequence after the position encoding is introduced based on the image parameters of the input video, and calculating the self-attention mechanism in each of the windows to obtain a corresponding key frame sequence; An indicator construction module is used to construct a plurality of similarity evaluation indicators, and calculate the low-dimensional similarity between the template image in the preset image template library and the key frame sequence according to the indicator threshold corresponding to each of the similarity evaluation indicators; A feature extraction module, used to extract features from the template image and the key frame sequence using a deep neural network model to obtain a feature map of the template image and a feature map of the key frame sequence respectively; The video stripping module is used to calculate the high-dimensional similarity between the feature map of the template image and the feature map of the key frame sequence, and locate the key frame sequence based on the low-dimensional similarity and the high-dimensional similarity to achieve video stripping of the input video.

[0011] Furthermore, the coding introduction module includes: A frame-by-frame analysis unit, used to obtain a video frame sequence of an input video, and perform frame-by-frame analysis according to the video frame sequence to obtain timing information corresponding to the video frame sequence; A coding introduction unit is used to generate a corresponding position coding according to the timing information, and introduce the position coding into the video frame sequence.

[0012] Furthermore, the window allocation module includes: A window construction unit, used for obtaining the image size of the input video, and constructing a window size of the video frame sequence after the position encoding according to the image size; A window allocation unit is used to allocate windows for the video frame sequence after the introduction position encoding according to the window size, so that the image area of ​​the input video is covered by at least one window, and calculate the self-attention mechanism in each of the windows to obtain a corresponding key frame sequence.

[0013] The present invention also provides a readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned video intelligent stripping method is implemented.

[0014] The present invention also proposes a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned video intelligent stripping method when executing the computer program.

[0015] The video intelligent stripping method, system, readable storage medium and computer in the present invention adopt a video scene classification method to determine the start and end frames of the video, and then obtain the corresponding moments, which greatly reduces the number of candidate key frames. Secondly, from the perspective of low-level detail information of the image, the three measurement indicators of SSIM, PSNR and mutual information are combined, and the low-dimensional similarity of structure, brightness, contrast and pixel value and information volume between the key frame and the template image is considered; from the perspective of high-level semantic information of the image, the feature map rich in image semantic information proposed by the deep pre-training model is compared to obtain high-dimensional similarity, and the low-dimensional and high-dimensional similarities are combined to effectively improve the key frame positioning accuracy, thereby realizing video stripping. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the video intelligent stripping method in the first embodiment of the present invention; Figure 2 for Figure 1 Detailed flow chart of step S101; Figure 3 for Figure 1 Detailed flow chart of step S102; Figure 4 It is a structural block diagram of the video intelligent stripping system in the second embodiment of the present invention; Figure 5 FIG. 4 is a structural block diagram of a computer in a third embodiment of the present invention.

[0017] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0018] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0020] Embodiment 1 See also Figure 1 , which is a video intelligent stripping method in the first embodiment of the present invention, and the method specifically includes steps S101 to S105: S101, obtaining a video frame sequence of an input video, and introducing a position code into the video frame sequence; For further information, see Figure 2 , the step S101 specifically includes steps S1011~S1012: S1011, obtaining a video frame sequence of an input video, and performing frame-by-frame analysis according to the video frame sequence to obtain timing information corresponding to the video frame sequence; S1012: Generate a corresponding position code according to the timing information, and introduce the position code into the video frame sequence.

[0021] In the specific implementation, obtain the video frame sequence of the input video ,in, Indicates The feature vector of the frame, Indicates the length of the video frame sequence, and analyzes the above video frame sequence frame by frame to obtain the timing information corresponding to the video frame sequence; Specifically, the timing information is converted into corresponding position codes, and the position codes are introduced into the above-mentioned video frame sequence. .

[0022] S102, allocating a plurality of non-overlapping windows to the video frame sequence after the position encoding is introduced based on the image parameters of the input video, and calculating a self-attention mechanism in each of the windows to obtain a corresponding key frame sequence; For further information, see Figure 3 , the step S102 specifically includes steps S1021~S1022: S1021, obtaining an image size of the input video, and constructing a window size of the video frame sequence after introducing the position coding according to the image size; S1022, performing window allocation for the video frame sequence after the introduction position encoding according to the window size, so that the image area of ​​the input video is covered by at least one window, and calculating the self-attention mechanism in each of the windows to obtain a corresponding key frame sequence.

[0023] In a specific implementation, the image size (H x W) of the input video is obtained, and the window size (M x M) of the video frame sequence after the position coding is introduced is constructed according to the image size, and the number of windows H / M x W / M (rounded up) into which the input video is split is calculated to (h / M + 1) x (w / M+1) (rounded up) by rounding up, ensuring that all image areas are covered by at least one window.

[0024] Furthermore, within each window, a multi-head self-attention mechanism is calculated. For each head, the self-attention is calculated as follows. Given the sequence in the window, the self-attention calculation can be expressed as: ; In the formula, is the query matrix, is the key matrix, is the value matrix, for The transposed matrix of is the dimension of the key.

[0025] Moving window. In order to establish connections between windows and capture a wider range of contextual information, a moving window mechanism is adopted. The position of the window will be offset between consecutive Transformer layers. For example, each window moves to the lower right by half the size of the window. The pixels originally inside a window will interact with the pixels of other windows at the next level.

[0026] Furthermore, we introduce text sequences ,in, Indicates The feature vector of a text unit, Indicates the length of the text sequence, and adds the above position code to the text sequence , using memory-enhanced self-attention mechanism and cross self-attention mechanism for the above text sequence and video frame sequence After processing, the corresponding key frame sequence is selected through an additional classification layer.

[0027] S103, constructing a plurality of similarity evaluation indicators, and calculating the low-dimensional similarity between the template image in a preset image template library and the key frame sequence according to the indicator threshold corresponding to each of the similarity evaluation indicators; In the specific implementation, an image template library is constructed as a benchmark, and three indicators are selected: structural similarity index (SSIM), peak signal-to-noise ratio index (PSNR) and mutual information index, and appropriate thresholds are selected for these three indicators to obtain the similarity scores corresponding to the three indicators. SSIM is defined as follows: ; ; ; ; In the formula, Respectively represent the template image and key frame sequence in the image template library, are greater than 0, Indicates brightness comparison, Indicates contrast comparison, Indicates structural comparison. Respectively The average value of Respectively The standard deviation of express The covariance of , , are constants respectively.

[0028] Among them, the peak signal-to-noise ratio index (PSNR) and the mutual information index are expressed in the same way as the structural similarity index (SSIM).

[0029] Specifically, the peak signal-to-noise ratio (PSNR) is defined as follows: 1. Calculate MSE (mean square error) Assume there are two m*n images, one is the template image I and the other is the key frame image K. MSE can be calculated by the following formula: ; in, is the template image, The pixel value of the position, is the key frame image in The pixel value at the location.

[0030] 2. Calculate PSNR Once the MSE is obtained, the PSNR can be calculated. The formula for PSNR is as follows: ; in,( ) is the maximum possible pixel value of the image. For an image with 8-bit depth, Usually this is 255 (because pixel values ​​range from 0 to 255).

[0031] Furthermore, the mutual information index is defined as follows: 1. Calculate the entropy of two images For an image X, its entropy H(X) is defined as follows: ; in, is the probability that X takes a particular value x.

[0032] 2. Calculate the joint entropy of two images For two template images X and key frame images Y, their joint entropy H(X, Y) is defined as follows: ; in, is the joint probability that X and Y take on specific values ​​x and y simultaneously.

[0033] 3. Calculate mutual information index Mutual Information It can be calculated from the entropy of X and Y and their joint entropy: ; in, is the quotient of the joint probability distribution of X and Y and their respective marginal probability distributions, indicating the degree of dependence between X and Y. If X and Y are independent, then , mutual information will be 0.

[0034] S104, performing feature extraction on the template image and the key frame sequence using a deep neural network model to obtain a feature map of the template image and a feature map of the key frame sequence respectively; In the specific implementation, the template image and the key frame are feature extracted based on the Swin Transformer to obtain the corresponding feature map, and then the similarity between the two is calculated, and the appropriate threshold is also set.

[0035] Specifically, the preprocessed template image is input into the Swin Transformer model, and the feature map of the template image is calculated by forward propagation. The feature map is usually a multi-channel tensor, where each channel represents a specific feature representation. The same preprocessing is performed on each frame image in the key frame sequence and sent to the Swin Transformer model in turn. For each key frame, the model outputs the corresponding feature map. Finally, the key frame sequence will be converted into a feature map sequence, and each feature map corresponds to a key frame.

[0036] S105, calculating the high-dimensional similarity between the feature map of the template image and the feature map of the key frame sequence, and positioning the key frame sequence based on the low-dimensional similarity and the high-dimensional similarity to achieve video striping of the input video.

[0037] In the specific implementation, the high-dimensional similarity between the feature map of the template image and the feature map of the key frame sequence is calculated, and the query vector, key vector and value vector are calculated: ; ; ; Further, calculate the attention weight: ; ; in, represents the query vector, Q represents the key vector; Specifically, calculate the output vector sequence: ; in, is the position of the output and input vector sequences, Indicates Input to The weight of the output attention; At the same time, the cosine similarity between the template image feature map and each feature map in the key frame sequence is calculated. Cosine similarity is a commonly used measure of the angular difference between two non-zero vectors. Its value is between -1 and 1. The larger the value, the more similar the two vectors are. The specific calculation formula is as follows: ; Among them, A and B represent the flattened vectors of the template image feature map and a key frame feature map, respectively. and is the eigenvalue of the corresponding position, and n is the feature dimension.

[0038] If all of the above four indicators are satisfied and exceed the set threshold, a key frame that meets the project requirements, i.e., the host frame, can be selected from the key frame list. Furthermore, the video can be split into strips according to the corresponding moments of the key frames obtained.

[0039] The feature of this embodiment is that it first uses the inter-frame difference algorithm to obtain a list of different scene change moments in the video, that is, a key frame list, and then calculates the similarity, SSIM, PSNR and mutual information between the template image and the key frame based on the traditional image processing method. Secondly, based on the visual depth pre-training model, the template image and the key frame are feature extracted, the corresponding feature map is obtained and the similarity between the two is calculated. Set a suitable threshold for each indicator. If the above indicators are all greater than the corresponding threshold, the key frame actually required can be obtained. At the same time, the traditional image processing method and the deep learning method are combined to effectively improve the video splitting performance.

[0040] In summary, the video intelligent stripping method in the above embodiment of the present invention adopts a video scene classification method to determine the start and end frames of the video, and then obtains the corresponding moments, which greatly reduces the number of candidate key frames. Secondly, from the perspective of low-level detail information of the image, combined with the three measurement indicators of SSIM, PSNR, and mutual information, the low-dimensional similarity of structure, brightness, contrast, pixel value and information volume between the key frame and the template image is considered; from the perspective of high-level semantic information of the image, the feature map rich in image semantic information proposed by the deep pre-training model is compared to obtain high-dimensional similarity. At the same time, the low-dimensional and high-dimensional similarities are combined to effectively improve the key frame positioning accuracy, thereby realizing video stripping.

[0041] Embodiment 2 Another aspect of the present invention is to provide a video intelligent stripping system, please refer to Figure 4 , shown is a video intelligent stripping system in a second embodiment of the present invention, the system comprising: The coding introduction module 11 is used to obtain a video frame sequence of an input video and introduce a position code into the video frame sequence; Furthermore, the code introduction module 11 includes: A frame-by-frame analysis unit, used to obtain a video frame sequence of an input video, and perform frame-by-frame analysis according to the video frame sequence to obtain timing information corresponding to the video frame sequence; A coding introduction unit is used to generate a corresponding position coding according to the timing information, and introduce the position coding into the video frame sequence.

[0042] A window allocation module 12 is used to allocate multiple non-overlapping windows to the video frame sequence after the position encoding is introduced based on the image parameters of the input video, and calculate the self-attention mechanism in each of the windows to obtain a corresponding key frame sequence; Furthermore, the window allocation module 12 includes: A window construction unit, used for obtaining the image size of the input video, and constructing a window size of the video frame sequence after the position encoding according to the image size; A window allocation unit is used to allocate windows for the video frame sequence after the introduction position encoding according to the window size, so that the image area of ​​the input video is covered by at least one window, and calculate the self-attention mechanism in each of the windows to obtain a corresponding key frame sequence.

[0043] An index construction module 13 is used to construct a plurality of similarity evaluation indexes, and calculate the low-dimensional similarity between the template image in the preset image template library and the key frame sequence according to the index threshold corresponding to each of the similarity evaluation indexes; A feature extraction module 14 is used to extract features from the template image and the key frame sequence using a deep neural network model to obtain a feature map of the template image and a feature map of the key frame sequence respectively; The video stripping module 15 is used to calculate the high-dimensional similarity between the feature map of the template image and the feature map of the key frame sequence, and locate the key frame sequence based on the low-dimensional similarity and the high-dimensional similarity to achieve video stripping of the input video.

[0044] The functions or operation steps implemented when the above modules and units are executed are generally the same as those in the above method embodiments, and will not be repeated here.

[0045] The video intelligent stripping system provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the system embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment.

[0046] Embodiment 3 The present invention also provides a computer, see Figure 5 , shown is a computer in the third embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored in the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, the above-mentioned video intelligent stripping method is implemented.

[0047] The memory 10 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 10 may be an internal storage unit of a computer, such as a hard disk of the computer. In other embodiments, the memory 10 may also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Further, the memory 10 may also include both an internal storage unit of the computer and an external storage device. The memory 10 may be used not only to store application software and various types of data installed in the computer, but also to temporarily store data that has been output or is to be output.

[0048] Among them, in some embodiments, the processor 20 can be an electronic control unit (Electronic Control Unit, abbreviated as ECU, also known as a vehicle computer), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run the program code stored in the memory 10 or process data, such as executing access restriction programs, etc.

[0049] It should be pointed out that Figure 5 The structure shown does not constitute a limitation on the computer. In other embodiments, the computer may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0050] The embodiment of the present invention further provides a readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned video intelligent stripping method is implemented.

[0051] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0052] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0053] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0054] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A video intelligent stripping method, characterized in that: include: Obtaining a video frame sequence of an input video, and introducing a position code into the video frame sequence; Allocating a plurality of non-overlapping windows to the video frame sequence after the position encoding is introduced based on the image parameters of the input video, and calculating a self-attention mechanism in each of the windows to obtain a corresponding key frame sequence; Constructing a plurality of similarity evaluation indicators, and calculating the low-dimensional similarity between the template image in the preset image template library and the key frame sequence according to the indicator threshold corresponding to each of the similarity evaluation indicators; Performing feature extraction on the template image and the key frame sequence using a deep neural network model to obtain a feature map of the template image and a feature map of the key frame sequence respectively; The high-dimensional similarity between the feature map of the template image and the feature map of the key frame sequence is calculated, and the key frame sequence is positioned based on the low-dimensional similarity and the high-dimensional similarity to achieve video striping of the input video.

2. The video intelligent stripping method according to claim 1, characterized in that: The steps of obtaining a video frame sequence of an input video and introducing a position code into the video frame sequence include: Acquire a video frame sequence of an input video, and perform frame-by-frame analysis according to the video frame sequence to obtain timing information corresponding to the video frame sequence; A corresponding position code is generated according to the timing information, and the position code is introduced into the video frame sequence.

3. The video intelligent stripping method according to claim 1, characterized in that: The steps of allocating a plurality of non-overlapping windows to the video frame sequence after the position encoding is introduced based on the image parameters of the input video, and calculating the self-attention mechanism in each of the windows to obtain the corresponding key frame sequence include: Acquire the image size of the input video, and construct a window size of the video frame sequence after introducing the position coding according to the image size; Window allocation is performed for the video frame sequence after the introduction position encoding according to the window size so that the image area of ​​the input video is covered by at least one window, and a self-attention mechanism is calculated in each of the windows to obtain a corresponding key frame sequence.

4. The video intelligent stripping method according to claim 1, characterized in that: The calculation formula of the self-attention is: ； In the formula, is the query matrix, is the key matrix, is the value matrix, for The transposed matrix of is the dimension of the key.

5. The video intelligent stripping method according to claim 1, characterized in that: The similarity evaluation index includes a structural similarity index, a peak signal-to-noise ratio index and a mutual information index. The expression of the structural similarity index is: ； ； ； ； In the formula, Respectively represent the template image and key frame sequence in the image template library, are greater than 0, Indicates brightness comparison, Indicates contrast comparison, Indicates structural comparison. Respectively The average value of Respectively The standard deviation of express The covariance of , , are constants respectively.

6. A video intelligent stripping system, characterized in that: include: A coding introduction module, used for acquiring a video frame sequence of an input video and introducing a position code into the video frame sequence; A window allocation module, configured to allocate a plurality of non-overlapping windows to the video frame sequence after the position encoding is introduced based on the image parameters of the input video, and calculate a self-attention mechanism in each of the windows to obtain a corresponding key frame sequence; An indicator construction module is used to construct a plurality of similarity evaluation indicators, and calculate the low-dimensional similarity between the template image in the preset image template library and the key frame sequence according to the indicator threshold corresponding to each of the similarity evaluation indicators; A feature extraction module, used to extract features from the template image and the key frame sequence using a deep neural network model to obtain a feature map of the template image and a feature map of the key frame sequence respectively; The video stripping module is used to calculate the high-dimensional similarity between the feature map of the template image and the feature map of the key frame sequence, and locate the key frame sequence based on the low-dimensional similarity and the high-dimensional similarity to achieve video stripping of the input video.

7. The video intelligent stripping system according to claim 6, characterized in that: The coding introduction module comprises: A frame-by-frame analysis unit, used to obtain a video frame sequence of an input video, and perform frame-by-frame analysis according to the video frame sequence to obtain timing information corresponding to the video frame sequence; A coding introduction unit is used to generate a corresponding position coding according to the timing information, and introduce the position coding into the video frame sequence.

8. The video intelligent stripping system according to claim 6, characterized in that: The window allocation module comprises: A window construction unit, used for obtaining the image size of the input video, and constructing the window size of the video frame sequence after the position coding according to the image size; A window allocation unit is used to allocate windows for the video frame sequence after the introduction position encoding according to the window size, so that the image area of ​​the input video is covered by at least one window, and calculate the self-attention mechanism in each of the windows to obtain a corresponding key frame sequence.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the video intelligent stripping method as described in any one of claims 1 to 5 is implemented.

10. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the video intelligent stripping method as described in any one of claims 1 to 5 is implemented.

Images