Video feature vector extraction method

Abstract

The invention provides a video feature vector extraction method, and belongs to the technical field of video processing. Comprising the following steps: splitting a video into video segments by using a segment detection mode; acquiring a key frame of each video clip; obtaining a feature vector of each key frame by using an image element analysis method based on an image HSV color space; and taking a set of the feature vectors of all the key frames as the feature vector of the video. According to the method, the feature vector of the video can be extracted, the video features are stored in the specific database, and when video retrieval needs to be carried out, similarity calculation is carried out by utilizing the features of the query image or the video and the features in the database, so that the required video clip can be queried; according to the image feature vector extracted by the method, color, texture, spatial distribution and shape information of the image can be considered, and the features of the image can be reflected more accurately.

Description

Technical Field

[0001] The invention belongs to the technical field of video processing, and in particular relates to a method for extracting feature vectors of a video. Background Art

[0002] Video retrieval is to analyze video files in order to find the required target or event. A large number of video files have been accumulated in the construction and operation of power grids, but currently most applications still use manual methods to perform video retrieval, which is time-consuming and labor-intensive, and has low video utilization.

[0003] With the development of computer technology, artificial intelligence technology is more and more widely used in the field of video retrieval, and video feature extraction is a basic task of video retrieval and even video processing. Therefore, an effective video feature extraction method is very necessary. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method for extracting feature vectors of a video.

[0005] In order to solve one, part or all of the above technical problems, the technical solution adopted by the present invention is: A method for extracting a feature vector of a video comprises: dividing a video into video segments by using a segment detection method; obtaining a key frame of each video segment; obtaining a feature vector of each key frame by using an image element analysis method based on an image HSV color space; and taking a set of feature vectors of all key frames as a feature vector of the video.

[0006] Furthermore, the method of splitting a video into video segments by using segmented detection includes: S11, taking the entire video as the segment to be detected; S12, performing a split judgment on the segment to be detected to determine whether the segment to be detected is a segment to be split; if so, continuing to step S13, if not, continuing to step S14; S13, if the number of frames of the segment to be split is greater than 2, then the segment to be split is divided into two segments to be tested, and returning to step S12 to perform a split judgment on the two segments to be tested; otherwise, the tail frame of the segment to be split is used as a boundary frame, and subsequent steps are continued; S14, when all the segments to be tested have completed the split judgment, if there is a boundary frame, the video is split into video segments according to the boundary frame, otherwise the entire video is used as a video segment.

[0007] Furthermore, the method for determining whether the fragment to be inspected is a fragment to be split includes: preprocessing the first and last frame images of the fragment to be inspected, calculating the frame difference according to the HSV color histogram of the first and last frame images after preprocessing, and if the frame difference is greater than a set first threshold, determining that the fragment to be inspected is a fragment to be split.

[0008] Furthermore, the method for obtaining the key frame of the video clip includes: traversing all frame images of the video clip; If it is the first frame of a video clip, a video cluster is constructed with the frame image as the cluster centroid; otherwise, the frame difference between the frame image and the centroid of an existing video cluster is obtained. If the frame difference with the centroid of a certain video cluster is not greater than the set second threshold, the frame image is added to the video cluster; if the frame difference with the centroids of all video clusters is greater than the set second threshold, a new video cluster is constructed with the frame image as the cluster centroid; after all frame images are traversed, the frame with the smallest difference with the video cluster centroid is obtained from all video clusters as the key frame of the video clip.

[0009] Furthermore, the value of the frame difference is ,in , and Respectively represent the hue value, saturation value and brightness value of one frame of the image. , and They respectively represent the hue value, saturation value and brightness value of another frame of image.

[0010] Furthermore, the method for obtaining the feature vector of the key frame includes: converting the key frame image from the RGB color space to the HSV color space, quantizing the HSV color into multiple color representations, and obtaining a color representation set of the image; splitting the image into different types of image elements, and obtaining an image element set and image element features of the image; performing connectivity analysis on the color representation set and the image element set, respectively, to obtain color connectivity features and image element connectivity features; performing feature fusion on the color connectivity features, image element connectivity features, and image element features, and obtaining a feature vector of the key frame image.

[0011] Furthermore, the color representation set is represented as , For point The color indicates the serial number. , m is the number of color representations after HSV color quantization of the image, X×Y is the size of the image; the image element size is 2×2, and the type of the image element is divided according to the change of the gradient on the diagonal line; the image element set is expressed as , , ; Image element The type number of the image element, x and y are the horizontal and vertical numbers of the image element, which are used to locate the image element; ; The image element feature is expressed as , , n is the number of image element types, and its value is 9; Used to represent image elements Whether it is the dth type of image element, the value is 1 or 0, .

[0012] Furthermore, the color connectivity feature , represents the connectivity feature represented by the ath color, ; The image element connectivity feature , represents the connectivity feature of the b-th image element, ; and Respectively represent the number of connected regions of the a-th color representation and the b-th image element; and They represent the average size of the connected areas of the a-th color representation and the b-th image element, respectively. , ; and Respectively represent the size of the maximum connected area of ​​the a-th color representation and the b-th image element; and They respectively represent the size of the second largest connected area of ​​the a-th color representation and the b-th image element.

[0013] Furthermore, the feature vector of the key frame image , where norm is the normalization function, is the weight function, , , f function is the square root of the 1-norm of the eigenvector x; r is the correlation coefficient, , mean is the mean value function.

[0014] Furthermore, the method further includes: storing the feature vector of the video as an index of the video.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention firstly divides the video into multiple video clips according to the changes of the pictures in the video, then obtains the key frame of each video clip, obtains the feature vector of each key frame by using the image element analysis method of the HSV color space, and finally uses the set of feature vectors of all key frames as the feature vector of the video. By extracting the feature vector of the video and storing it in a specific database, when video retrieval is required, the required video clip can be queried by calculating the similarity between the features of the query image or video and the features in the database.

[0016] The image element analysis method based on the HSV color space of the image can take into account the color, texture, spatial distribution and shape information of the image, can reflect the correlation between the target and the background in the image, and the extracted image features will be more sufficient and can more accurately reflect the characteristics of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0018] Figure 1 : Flowchart of the present invention; Figure 2 : Flow chart of step S1 of the present invention. DETAILED DESCRIPTION

[0019] In order to better understand the present invention, the content of the present invention is further clearly described below in conjunction with the embodiments and the accompanying drawings, but the protection content of the present invention is not limited to the following embodiments. In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without one or more of these details.

[0020] Example 1: See Figure 1-Figure 2 The purpose of this embodiment is to provide a method for extracting a feature vector of a video, comprising: S1, split the video into video segments using segment detection; S2, obtaining the key frame of each video clip; S3, obtaining the feature vector of each key frame; S4, taking the set of feature vectors of all key frames as the feature vector of the video; S5. The feature vector of the video is used as the index of the video for storage.

[0021] Further, such as Figure 2 As shown, step S1 includes: S11, taking the entire video as the segment to be inspected.

[0022] S12, split the fragment to be checked to determine whether the fragment to be checked is a fragment to be split; if so, continue to step S13, if not, continue to step S14.

[0023] The method for determining whether the to-be-checked segment is a to-be-split segment that needs to be split includes: The first and last frame images of the segment to be inspected are preprocessed, and the frame difference is calculated according to the HSV color histogram of the first and last frame images after preprocessing. If the frame difference is greater than the set first threshold, it means that a shot change occurs in the segment to be inspected, and the segment to be inspected is determined to be a segment to be split.

[0024] Frame Difference ,in , and Respectively represent the hue value, saturation value and brightness value of the first frame of the image of the segment to be inspected, , and They respectively represent the hue value, saturation value and brightness value of the last frame image of the segment to be inspected.

[0025] S13. If the number of frames of the to-be-split segment is greater than 2, the to-be-split segment is equally divided into two to-be-checked segments, and the process returns to step S12 to determine the splitting of the two to-be-checked segments; otherwise, the tail frame of the to-be-split segment is used as the boundary frame, and subsequent steps are continued.

[0026] The two fragments to be inspected after the fragment to be split is evenly divided have a repeated frame. For example, if the number of frames of the fragment to be split is 2n-1 (n is an integer), the number of frames of the two fragments to be inspected after the split is n. If the number of frames of the fragment to be split is 2n (n is an integer), the number of frames of the two fragments to be inspected after the split is n and n+1, or n+1 and n respectively.

[0027] S14, when all the segments to be checked have completed the splitting judgment, if there is a boundary frame, the video is split into video segments according to the boundary frame, otherwise the entire video is taken as a video segment.

[0028] Furthermore, the method for obtaining the key frame of the video clip in step S2 includes: S21, traverse all frame images of the video clip.

[0029] S22. If it is the first frame of the video clip, a video cluster is constructed with the frame image as the cluster centroid; otherwise, the frame difference between the frame image and the centroid of the existing video cluster is obtained. If the frame difference with the centroid of a certain video cluster is not greater than a set second threshold, the frame image is added to the video cluster; if the frame difference with the centroids of all video clusters is greater than the set second threshold, a new video cluster is constructed with the frame image as the cluster centroid.

[0030] When obtaining the frame difference between two frames of images, the images are first preprocessed, and then the frame difference is calculated based on the HSV color histogram of the first and last frame images after preprocessing.

[0031] Frame Difference ,in , and Respectively represent the hue value, saturation value and brightness value of one frame of the image. , and They respectively represent the hue value, saturation value and brightness value of another frame of image.

[0032] S23, after all frame images are traversed, the frame with the smallest centroid of the video cluster is obtained from all video clusters as the key frame of the video segment.

[0033] Furthermore, step S3 uses an image element analysis method based on the image HSV color space to extract features. Step S3 includes: S31. Convert the key frame image from the RGB color space to the HSV color space, quantize the HSV color into a plurality of color representations, and obtain a color representation set of the image.

[0034] After converting the image from RGB color space to HSV color space, the three color components H, S and V are quantized as class, Level and Level, generate m kinds of color representations, among which , , , Preferably, , .

[0035] The color representation set is a set constructed according to the color representation of each pixel in the image. The color representation set is expressed as , For point The color indicates the serial number. , m is the number of color representations after HSV color quantization of the image, and X×Y is the size of the image.

[0036] S32, splitting the image into different types of image elements, and obtaining an image element set and image element features of the image.

[0037] The image is split into multiple 2×2 image elements, and the image element set is represented as ,in , At the same time, the image elements in this step are also classified according to the gradient changes of the pixel values ​​on the diagonal line. There are 9 types in total: Image element The type number of the image element, x and y are the horizontal and vertical numbers of the image element, which are used to locate the image element; .

[0038] The image element feature is represented as , , n is the number of image element types, and its value is 9; i and j are the horizontal and vertical serial numbers of the image element, respectively, which are used to locate the image element; Used to represent image elements Whether it is the dth type of image element, the value is 1 or 0, .

[0039] S33, performing connectivity analysis on the color representation set and the image element set respectively to obtain color connectivity features and image element connectivity features.

[0040] When performing connectivity analysis on a color representation set, connected colors represent the same and adjacent pixels. Color connectivity features , represents the connectivity feature represented by the ath color, .

[0041] When performing connectivity analysis on a set of image elements, image elements with the same connectivity type and adjacent to each other are considered. Image element connectivity features , represents the connectivity feature of the b-th image element, .

[0042] and Respectively represent the number of connected regions of the a-th color representation and the b-th image element; and They represent the average size of the connected areas of the ath color representation and the bth image element respectively; The value is the total number of image pixels and of business, The value of is the total number of image elements and of business, , ; and Respectively represent the size of the maximum connected area of ​​the a-th color representation and the b-th image element; and They respectively represent the size of the second largest connected area of ​​the a-th color representation and the b-th image element.

[0043] S34, performing feature fusion on the color connectivity feature, the image element connectivity feature and the image element feature to obtain a feature vector of the key frame image.

[0044] use Obtain the weighted fusion result H of the three features of color connectivity feature A, image element connectivity feature B and image element feature D, where norm is the normalization function. is the weight function, , , f function is the square root of the 1-norm of the eigenvector x; r is the correlation coefficient, , mean is the mean value function.

[0045] The weighted fusion result H is the feature vector of the key frame image to be obtained in this step.

[0046] Furthermore, in step S5, the feature vector of the video is stored as an index of the video. When it is necessary to retrieve the video using the image, the feature vector of the image can be obtained using the method of step S3, and then the video can be retrieved using a vector similarity check.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in the field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for extracting a feature vector of a video, characterized in that: include: Split the video into video segments using segment detection; Get the key frames of each video clip; The feature vector of each key frame is obtained by using the image element analysis method based on the image HSV color space; The set of feature vectors of all key frames is taken as the feature vector of the video.

2. The method for extracting feature vectors of a video according to claim 1, characterized in that: Methods for splitting a video into video segments using segment detection include: S11, taking the entire video as the segment to be inspected; S12, splitting and judging the fragment to be inspected, and judging whether the fragment to be inspected is a fragment to be split; if so, continue to step S13, if not, continue to step S14; S13, if the number of frames of the to-be-split segment is greater than 2, the to-be-split segment is equally divided into two to-be-checked segments, and the process returns to step S12 to perform splitting judgment on the two to-be-checked segments; otherwise, the tail frame of the to-be-split segment is used as the boundary frame, and subsequent steps are continued; S14, when all the segments to be checked have completed the splitting judgment, if there is a boundary frame, the video is split into video segments according to the boundary frame, otherwise the entire video is taken as a video segment.

3. The method for extracting feature vectors of a video according to claim 2, characterized in that: The method for determining whether a fragment to be inspected is a fragment to be split includes: preprocessing the first and last frame images of the fragment to be inspected, calculating the frame difference according to the HSV color histogram of the first and last frame images after preprocessing, and if the frame difference is greater than a set first threshold, determining that the fragment to be inspected is a fragment to be split.

4. The method for extracting feature vectors of a video according to claim 1, characterized in that: Methods for obtaining key frames of video clips include: Traverse all frame images of the video clip; If it is the first frame of the video clip, a video cluster is constructed with the frame image as the cluster centroid; otherwise, the frame difference between the frame image and the centroid of the existing video cluster is obtained. If the frame difference with the centroid of a certain video cluster is not greater than the set second threshold, the frame image is added to the video cluster; if the frame difference with the centroids of all video clusters is greater than the set second threshold, a new video cluster is constructed with the frame image as the cluster centroid; After all frame images are traversed, the frame with the smallest centroid of the video cluster is obtained from all video clusters as the key frame of the video segment.

5. The method for extracting feature vectors of a video according to claim 3 or 4, characterized in that: The value of the frame difference is ,in , and Respectively represent the hue value, saturation value and brightness value of one frame of the image. , and They respectively represent the hue value, saturation value and brightness value of another frame of image.

6. The method for extracting feature vectors of a video according to claim 1, characterized in that: Methods for obtaining the feature vector of a key frame include: Convert the key frame image from RGB color space to HSV color space, quantize the HSV color into multiple color representations, and obtain a color representation set of the image; Split the image into different types of image elements, and obtain the image element set and image element features of the image; Performing connectivity analysis on the color representation set and the image element set respectively to obtain color connectivity features and image element connectivity features; The color connectivity features, image element connectivity features and image element features are fused to obtain the feature vector of the key frame image.

7. The method for extracting feature vectors of a video according to claim 6, characterized in that: The color representation set is represented as , For point The color indicates the serial number. , m is the number of color representations after HSV color quantization of the image, and X×Y is the size of the image; The size of the image element is 2×2, and the type of the image element is divided according to the change of the gradient on the diagonal line; The image element set is represented as , , ; Image element The type number of the image element, x and y are the horizontal and vertical numbers of the image element, which are used to locate the image element; ; The image element feature is expressed as , , n is the number of image element types, and its value is 9; Used to represent image elements Whether it is the dth type of image element, the value is 1 or 0, .

8. The method for extracting feature vectors of a video according to claim 7, characterized in that: The color connectivity feature , represents the connectivity feature represented by the ath color, ; The image element connectivity feature , represents the connectivity feature of the b-th image element, ; and Respectively represent the number of connected regions of the a-th color representation and the b-th image element; and They represent the average size of the connected areas of the a-th color representation and the b-th image element, respectively. , ; and Respectively represent the size of the maximum connected area of ​​the a-th color representation and the b-th image element; and They respectively represent the size of the second largest connected area of ​​the a-th color representation and the b-th image element.

9. The method for extracting feature vectors of a video according to claim 8, characterized in that: Feature vector of key frame image , where norm is the normalization function, is the weight function, , , f function is the square root of the 1-norm of the eigenvector x; r is the correlation coefficient, , mean is the mean value function.

10. The method for extracting feature vectors of a video according to claim 1, characterized in that: Also includes: The feature vector of the video is used as the index of the video for storage.

Images