Video duplicate checking method and system based on multi-modal feature fusion and dynamic strategy

Abstract

The application relates to the technical field of electric digital data processing, in particular to a video duplication checking method and system based on multi-modal feature fusion and dynamic strategy, which specifically comprises the following steps: performing regional division and numbering on a feature frame image; determining a structural color tendency degree of the feature frame image based on the distribution of edge contours in all regions in the feature frame image and the difference between the change directions of pixel gray values in different closed contours in the feature frame image; analyzing moving features in the current feature frame image through the difference between the current feature frame image and adjacent feature frame images, combining the serial numbers of the regions and the structural color tendency degree, and constructing a modal feature degree of the feature frame image; and performing video duplication checking based on the modal feature degrees of frame images in a video to be audited and user-stored videos. The influence of mosaics, watermarks and the like on video duplication checking is reduced, the similarity between videos can be better calculated, and the accuracy of video duplication checking is improved.

Description

Technical Field

[0001] This application relates to the field of electronic digital data processing technology, specifically to a video plagiarism detection method and system based on multimodal feature fusion and dynamic strategies. Background Technology

[0002] With the rapid development of video technology, video websites and software are growing at an unprecedented pace, with tens of millions of videos being uploaded daily. However, a large number of user-uploaded videos are either duplicates or infringing. To ensure the healthy development of video copyright, protect the legitimate rights and interests of creators, and prevent unauthorized users from stealing and distributing videos for illegal profit, it is necessary to perform deduplication checks on user-uploaded videos. This ensures the copyright of user videos, improves user experience, and provides strong protection for user video copyrights. During the deduplication process, dynamic feature extraction using multi-dimensional features is employed to extract image features from video frames.

[0003] Traditional techniques typically detect video duplication by comparing the similarity and repetition between video frames. To achieve efficient duplication detection, feature values ​​or feature vectors are extracted from each video frame for comparison, reducing computational load. However, because thieves may remove, blur, or add their own watermarks to stolen videos, discrepancies arise between image frames, causing feature values ​​to become dissimilar or identical. This mismatch leads to lower accuracy in video duplication detection. Summary of the Invention

[0004] To address the aforementioned technical problems, the purpose of this application is to provide a video plagiarism detection method and system based on multimodal feature fusion and dynamic strategies. The specific technical solution adopted is as follows:

[0005] In a first aspect, embodiments of this application provide a video deduplication method based on multimodal feature fusion and dynamic strategies, the method comprising the following steps:

[0006] Obtain the grayscale images of each frame of the video data, and denote them as feature frame images;

[0007] The feature frame image is uniformly divided into regions and numbered to obtain each region of the feature frame image and its sequence number; the structural synchronization of the feature frame image is constructed based on the distribution of edge contours in all regions of the feature frame image.

[0008] The structural color anisotropy of the feature frame image is constructed based on the difference in the direction of change of the gray value of pixels within different closed contours in the feature frame image. Combined with the structural synchronicity, the structural color tendency of the feature frame image is determined.

[0009] determine a moving frame image of the current feature frame image based on a difference between the current feature frame image and feature frame images at adjacent time points before and after the current feature frame image; construct a modality feature degree of the feature frame image based on a distribution of pixel values in each region in the moving frame image, in combination with the sequence number and the structure color trend degree;

[0010] construct a modality feature vector of each video through the modality feature degrees of the feature frame images of all frames of each video; and perform video duplication checking based on a similarity between the modality feature vectors of the video to be audited and the user-stored videos.

[0011] In one embodiment, the structure synchronicity acquisition process is as follows:

[0012] a sequence of the number of intra-regional edges of all regions in the feature frame image is denoted as a contour feature sequence, and a sequence of intra-regional edge pixel value sums of all regions in the feature frame image is denoted as a contour structure sequence; a similarity between the contour feature sequence and the contour structure sequence is obtained; a difference between a maximum value in the contour feature sequence and a maximum value in the contour structure sequence is obtained and denoted as a first difference;

[0013] The structure synchronicity of the feature frame image is positively correlated with the similarity and the first difference.

[0014] In one embodiment, the structure color anisotropy acquisition process is as follows:

[0015] a threshold segmentation is performed on the gray values of all pixel points in each closed contour in the feature frame image, and a straight line formed by connecting a coordinate point corresponding to a position average of all pixel points with a gray value greater than a corresponding segmentation threshold to a coordinate point corresponding to a position average of all pixel points with a gray value less than or equal to the corresponding segmentation threshold is denoted as a color change line;

[0016] a mean value of a difference between a maximum closed contour in the feature frame image and the color change line slopes of all closed contours is calculated as the structure color anisotropy of the feature frame image.

[0017] In one embodiment, the structure color trend degree acquisition process is as follows:

[0018] a sum value of the structure color anisotropy of the feature frame image and a preset non-zero adjustment factor is calculated as a first sum value; and a ratio of the structure synchronicity of the feature frame image to the first sum value is taken as the structure color trend degree of the feature frame image.

[0019] In one embodiment, the moving frame image acquisition process is as follows:

[0020] Subtract the current time characteristic frame image from the previous and next time characteristic frame images respectively to obtain two inter-frame difference images, and record the inter-frame difference image with the minimum mean pixel value in the image as the moving frame image of the current time characteristic frame image.

[0021] In one embodiment, the process of obtaining the modal feature degree is as follows:

[0022] In the moving frame image, the number of regions occupied by non-zero pixels is recorded as a first number, and the sum of the serial numbers of all non-zero pixel regions is recorded as a first feature sum value. The modal feature degree of the moving frame image is positively correlated with the first number and the first feature sum value respectively, and is negatively correlated with the structural color trend degree.

[0023] In one embodiment, the process of video duplication checking is as follows:

[0024] For any user-stored video, the length of the modal feature vector of the user-stored video and the to-be-audited video is compared to determine a matching modal feature vector and the length of a sliding window in another modal feature vector. The vectors in each sliding window of the another modal feature vector are recorded as modal feature sub-vectors. The similarity between the matching modal feature vector and each modal feature sub-vector is determined to determine a similar modal feature sub-vector of the matching modal feature vector.

[0025] The difference sequence between the similar modal feature sub-vector and the matching modal feature vector is obtained. If the number of elements less than a preset threshold in the difference sequence is more than the number of elements greater than the preset threshold, it is determined that the video is duplicated. Otherwise, it is determined that the video is not duplicated.

[0026] In one embodiment, the determination of the matching modal feature vector and the length of the sliding window in the another modal feature vector is as follows: the shortest value of the lengths of the modal feature vectors of the user-stored video and the to-be-audited video is obtained, the shortest value is taken as the length of the sliding window, and the modal feature vector corresponding to the shortest value is taken as the matching modal feature vector.

[0027] In one embodiment, the process of obtaining the similar modal feature sub-vector is as follows:

[0028] The maximum value of the similarity between the matching modal feature vector and all modal feature sub-vectors is obtained, and the modal feature sub-vector corresponding to the maximum value of the similarity is taken as the similar modal feature sub-vector.

[0029] In a second aspect, the embodiments of the present application also provide a video duplication checking system based on multi-modal feature fusion and dynamic strategy, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the method described in any of the above embodiments are implemented.

[0030] The embodiments of the present application have at least the following beneficial effects:

[0031] The present application can effectively eliminate the error between the image acquisition feature matching caused by the different resolutions of the stolen video by dividing the video frame image into regions, analyzing the edge attribute state in each region, and constructing the structure synchronization of each frame image, and can improve the accuracy of video duplication checking. The present application can effectively represent the change direction of the color of the video frame image by analyzing the convergence degree of the color change of the object in the video frame image, effectively representing the uniqueness of the frame image, improving the extraction of the uniqueness of the video frame image, distinguishing the different frame images, and reducing the influence of mosaic, watermark, etc. on video duplication checking. The present application can reduce the influence of the static scene on image analysis by combining the moving attribute of the object in the video frame image to analyze the multi-modal features of the video frame image, increase the uniqueness of the video frame image with the moving attribute, and better calculate the similarity between videos when performing video duplication checking. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed in the following embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0033] Figure 1 The step flow chart of the video duplication checking method based on multi-modal feature fusion and dynamic strategy provided by an embodiment of the present application is shown in the figure.

[0034] Figure 2 The schematic diagram for obtaining structural color anisotropy is shown in the figure.

[0035] Figure 3 The step block diagram of the video duplication checking method based on multi-modal feature fusion and dynamic strategy is shown in the figure. DETAILED DESCRIPTION

[0036] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following describes the video duplication checking method and system based on multi-modal feature fusion and dynamic strategy according to the present application in detail, including its specific implementation, structure, features and effects, with reference to the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0038] The specific scheme of the video duplicate detection method and system based on multi-modal feature fusion and dynamic strategy provided by the application will be specifically described below in combination with the drawings.

[0039] Please refer to Figure 1 which shows the step flowchart of the video duplicate detection method based on multi-modal feature fusion and dynamic strategy provided by an embodiment of the application, which comprises the following steps:

[0040] Step S1, obtaining the gray scale image of each frame image of the video data, denoted as the feature frame image.

[0041] Firstly, after the user uploads the video data, the video data will be stored in the review area. The video data is composed of multiple image data, and the frame image in the video data is read by the cap.read() function in OpenCV.

[0042] Then, the frame image of the video data is converted into a gray scale frame image by the weighted average method. Since there may be noise in the frame image, the noise will affect the extraction of data features, therefore, the gray scale frame image is taken as the input of the NLM algorithm (Non-Local Means), and the output is the gray scale frame image of the video data after noise reduction, denoted as the feature frame image. The cap.read() function, the weighted average method and the NLM algorithm are all known technologies, and the specific calculation steps are not described here.

[0043] Step S2, uniformly dividing and numbering the feature frame image to obtain each region of the feature frame image and its serial number; and constructing the structural synchronism of the feature frame image based on the distribution of the edge contour in all regions of the feature frame image.

[0044] The video data is composed of frame images, therefore, in video duplicate detection, only the similarity between images needs to be compared to detect duplicates of the video. Since the video has continuous action characteristics, at the same time, since the pirate will have a low resolution (the number of pixels of the frame image) problem when stealing the video, but the relative position of the image is different under different resolutions.

[0045] Thus, the feature frame image is evenly divided into N*N regions, N is 10 in the embodiment, each region is numbered, specifically, the serial number of the first region in the upper left corner of the image is set to 1, then the serial number is increased by 1 from left to right; the serial number of the first region in the second row is 11, and the serial number is increased by 1 from left to right; and so on, the serial number of the last region in the lower right corner is 100. The serial number of each region is used as the feature value of each region. Therefore, for images of different resolutions of the same scene, the part of an object in different resolution images is in the same region. Therefore, the region where the same feature object is located is the same.

[0046] In other embodiments of the application, the implementer can also number all regions according to actual conditions using other numbering methods.

[0047] Next, the feature frame image is input into the Canny edge detection algorithm, and the output is an edge image of the feature frame image. Further, the number of edges in each region of the feature frame image is obtained, denoted as a contour feature quantity; and the sum of the pixel values of all edge pixel points in each region is obtained, denoted as a contour structure quantity. The contour feature of the object in the image is represented by the contour feature quantity and the contour structure quantity. The contour feature quantity and the contour structure quantity of all regions are normalized respectively to eliminate the influence of the dimensions of the two quantities, and in the embodiment, maximum value normalization is used, and then the contour feature sequence and the contour structure sequence are obtained by arranging the region feature values from small to large. The calculation of the Canny edge detection algorithm and the maximum value normalization is a known technology, and the specific calculation steps are not described here.

[0048] Thus, the structure synchronicity of the feature frame image is calculated, which is used to represent the uniformity of the edge attribute distribution in the feature frame image, and the expression is:

[0049]

[0050] In the formula, the structure synchronicity of the feature frame image is represented , the contour feature sequence and the contour structure sequence of the feature frame image are represented respectively , the maximum values in the contour feature sequence and the contour structure sequence of the feature frame image are represented respectively the cosine similarity function is represented a preset zero exclusion adjustment factor is represented to prevent the denominator from being 0, the value range is [0.001, 1], and in the embodiment, the value of the zero exclusion adjustment factor is set to 0.01. In the formula, the first difference is represented

[0051] ​The similarity between the contour feature sequence and the contour structure sequence is calculated, which is used to represent the similarity between the edge distribution and the edge pixel point distribution in the feature frame image. The edge distribution and the edge pixel point distribution of different frame images are different. However, the normalized edge quantity of the images at different resolutions in the same position is the same. Therefore, the similarity between the edge distribution and the edge pixel point distribution of each frame image (i.e., the same scene image) is unique. The images at different resolutions of the same scene have the same structure, so the contour feature quantity of the same region is the same, and the contour structure quantity is different due to the different image resolutions. However, after normalization, the same contour structure quantity is normalized to a similar or same value. Therefore, the cosine similarity between the contour feature sequence and the contour structure sequence of the images at different resolutions of the same scene is the same.

[0052] Meanwhile, since the mosaic region is small and the detail and texture attribute is small, the maximum value in the contour feature sequence and the contour structure sequence of the feature frame image is used. The larger the ratio between the two values, the longer the length of the distributed edge, and the simpler the structure, so that the value of the structure synchronization of the feature frame image is larger, which means that the image can effectively represent the feature frame image. The structure synchronization of the feature frame images of the same scene is similar or the same, which can effectively match and efficiently check the video, and improve the efficiency of video checking.

[0053] In step S3, the structure color anisotropy of the feature frame image is constructed based on the difference between the change directions of the pixel point gray value in different closed contours in the feature frame image, and the structure color tendency degree of the feature frame image is determined in combination with the structure synchronization.

[0054] For the objects in the same scene, the area ratio in the images at different resolutions is the same, so the number of regions where the same object is located is the same. When the number of regions occupied by different objects at different resolutions is more similar, the modal feature has higher uniqueness in the video checking process, which can reduce the false judgment when matching different scene images.

[0055] The resolution difference results in different image clarity, but the relative positions of the texture distribution of the objects in the region are the same, so the development direction of the texture is the same. Thus, all edges in the edge image are obtained by the findContours function in OpenCV, and the contour area function is used to calculate the edge contour area. The edge contour with a closed contour relationship in the edge image of the feature frame image is taken as the edge contour with a closed contour relationship, which is denoted as a closed contour edge. A mask is formed by the closed contour edge and the positions of the pixels inside the closed contour edge. Thus, a plurality of mask regions in the edge image of the feature frame image are obtained. The distribution of the texture in the mask region represents the color change of the object. The calculation of the findContours function and the contour area function in OpenCV is a known technology, and the specific calculation steps are not described herein.

[0056] Thus, the gray value of each pixel point in each mask region is taken as the input of the Otsu method, and the output is the segmentation threshold value of each mask region. The position mean value of the pixel points with a gray value greater than the corresponding segmentation threshold value and the position mean value of the pixel points with a gray value less than or equal to the segmentation threshold value in each mask region are calculated. The two mean value points are connected to form a straight line, which is denoted as a color change line of the mask region and is used to represent the color change direction of the object color. The calculation of the Otsu method is a known technology, and the specific calculation steps are not described herein.

[0057] It should be noted that for the threshold segmentation of the gray value in the mask region, the present application only provides a threshold segmentation method. There are many existing threshold segmentation methods, and the implementer can also use other threshold segmentation algorithms to perform threshold segmentation on the gray value in the mask region. The present application does not make specific limitations.

[0058] Thus, the structural color anisotropy of the feature frame image is first calculated. The maximum mask region in all the mask regions of the feature frame image is obtained, the difference between the slope of the maximum mask region and the slope of the color change line of each mask region is calculated, which is denoted as a slope difference value. The mean value of all the slope difference values between the maximum mask region of the feature frame image and all the mask regions is calculated as the structural color anisotropy of the feature frame image, which is used to represent the difference of the color change direction in the feature frame image. The calculation of the slope is a known technology, and the specific calculation steps are not described herein.

[0059] Then, the structural color tendency degree of the feature frame image is calculated. The structural color tendency degree is used to represent the homotopy state of the color change of the object in the feature frame image, and the expression is as follows:

[0060]

[0061] In the formula, the structural color tendency degree of the feature frame image, representing structural synchrony of the feature frame image, representing structural color anisotropy of the feature frame image, representing a preset non-zero adjustment factor. In the embodiment, the value of the non-zero adjustment factor is set to 0.01. Wherein, is a first sum value. is a first sum value.

[0062] For the color change of the object in the feature frame image, the trend of the image color change is determined, and at the same time, the trend of the color of the largest object in the different resolution images of the same scene can determine the overall color change characteristics of the image, and the difference between the color change line slope of the largest mask region and other mask regions is used to represent the difference in the color change direction of the object in the feature frame image. The smaller the difference is, the higher the convergence of the color change is. At the same time, the higher the convergence is, the smaller the distribution difference between the objects is, which makes the object edge distribution more uniform, that is, the structural synchrony of the feature frame image is larger, the structural color trend degree of the feature frame image is higher, and it is easier to match when the frame image is matched, which can improve the video duplication detection efficiency.

[0063] Step S4, determining a moving frame image of the current feature frame image based on the difference between the current feature frame image and the feature frame images of the previous and next adjacent time; and constructing a modal feature degree of the feature frame image based on the pixel value distribution in each region of the moving frame image, in combination with the sequence number and the structural color trend degree.

[0064] Further, due to the existence of part of the moving object in the video, there is a certain difference between the images, so the current time feature frame image is subtracted from the feature frame images of the previous and next adjacent time respectively to obtain two frame difference images, wherein the subtraction of the two images is to subtract the same position elements in the two images. And calculate the mean value of all pixel values in each frame difference image, and the frame difference image with the smallest mean value is recorded as the moving frame image of the current time feature frame image. Wherein, the moving frame image of the first feature frame image in the video is the frame difference image between the first feature frame image and the next feature frame image, and the moving frame image of the last feature frame image is the frame difference image between the last feature frame image and the previous feature frame image.

[0065] When the number of regions occupied by non-zero pixels in the moving frame image is more, it means that the area proportion of the moving object is larger, and the feature frame image has obvious distinguishing features. Therefore, in the moving frame image, the number of regions occupied by non-zero pixels is recorded as a first number, the sum of feature values of all non-zero pixel regions is recorded as a first feature sum value, and the product of the first number and the first feature sum value is recorded as a moving recognition degree. For representing the feature degree of the feature frame image in the moving modal feature dimension.

[0066] Thus, the modal feature degree of the feature frame image is calculated, which is used to represent the structural change state of the feature frame image at adjacent time, and the expression is:

[0067]

[0068] In the formula, represents the modal feature degree of the feature frame image; represents the movement recognition degree of the feature frame image; represents a preset parameter adjustment factor, which prevents the movement recognition degree of the feature frame image from being zero due to the stillness of the video shooting area, and the default value range is [1, 10], and in the embodiment, the value of is set to 1; represents the structural color trend degree of the feature frame image.

[0069] For the feature frame image with a higher movement attribute, the greater the proportion of movement, the higher the unique feature of the feature frame image when compared, so when the movement attribute of the feature frame image is higher, the influence of the structural homotopy of the feature frame image is greater, so that the structural color trend degree of the feature frame image is smaller, and the value of the movement recognition degree is greater, so that the modal feature degree of the feature frame image is greater. In the video duplication checking process, the still scene is more likely to be matched, so when the modal feature degree of the feature frame image of the still scene is smaller, the same continuous time can be used for fast dynamic duplication checking.

[0070] Step S5, constructing the modal feature vector of each video through the modal feature degree of the feature frame image of each frame image of each video; and performing video duplication checking based on the similarity between the modal feature vectors of the video to be audited and the user stored videos.

[0071] Thus, the modal feature degree of each feature frame image in the video is calculated through the above steps, and the modal feature vector of the video is obtained by arranging the modal feature degree of the feature frame image in the video in the order of the feature frame image in the video, which is used to represent the feature state of the video in the duplication checking and matching.

[0072] The modal feature vector of the video to be audited is calculated through the above steps. The similarity between the modal feature vector of the video to be audited and the modal feature vector of the user stored video is calculated. The calculation method is: since the time length of the video is different, the length of the modal feature vector of the video is also different, so the sliding window algorithm is used to calculate the most similar part. The calculation method of the sliding window algorithm is: the shortest length of the two modal feature vectors is taken as the length of the sliding window in the sliding window algorithm, and the modal feature vector with the shortest length is recorded as the matching modal feature vector. The sliding window is slid in the modal feature vector with the longer length, and the vector in each sliding window is recorded as a modal feature sub-vector, and the cosine similarity between the modal feature sub-vector and the matching modal feature vector is calculated. The modal feature sub-vector with the maximum cosine similarity is recorded as the similar modal feature sub-vector. The calculation of the cosine similarity is a known technology, and the specific calculation steps are not described here.

[0073] It should be noted that for the calculation of the similarity between the matching modal feature vector and the modal feature sub-vector, only one similarity algorithm is provided in the embodiments of the present application. There are many existing similarity algorithms, and the implementer can also use other similarity algorithms to calculate the similarity between the matching modal feature vector and the modal feature sub-vector. The present application does not make specific limitations.

[0074] Next, the similar modal feature sub-vector and the corresponding elements in the matching modal feature vector are subtracted to obtain a difference sequence, and the difference sequence is taken as the input of the maximum inter-class variance method, and the output is a threshold value. The number of elements greater than the threshold value is recorded as P, and the number of elements less than or equal to the threshold value is recorded as Q. Since the difference between the modal features is small when the frame images are similar, when Q is greater than P, it is considered that the video is repeated; when Q is less than or equal to P, it is considered that the video is not repeated. Finally, the video of the user uploaded video and all stored videos is checked through the above steps. The calculation of the maximum inter-class variance method is a known technology, and the specific calculation steps are not described here.

[0075] The schematic diagram of the acquisition process of the structural color anisotropy is shown in Figure 2 The step block diagram of the video duplication checking method based on multi-modal feature fusion and dynamic strategy is shown in Figure 3

[0076] Based on the same inventive concept as the above method, the embodiments of the present application also provide a video duplication checking system based on multi-modal feature fusion and dynamic strategy, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above methods based on multi-modal feature fusion and dynamic strategy are implemented.

[0077] ​To sum up, the video duplication checking method based on multi-modal feature fusion and dynamic strategy is provided in the embodiments of the application, the video frame images are divided into regions, the state of the edge attribute in each region is analyzed, and the structural synchronization of each frame image is constructed, so that the error between the image acquisition feature matching caused by the different resolutions of the stolen videos can be effectively eliminated, and the accuracy of the video duplication checking can be improved; the structural color trend degree of the feature frame image is constructed by analyzing the convergence degree of the object color change in the video frame image, so that the change direction of the color of the video frame image can be effectively represented, the uniqueness of the frame image can be effectively represented, the extraction of the uniqueness of the video frame image can be improved, the different frame images can be distinguished, and the influence of the mosaic and the watermark on the video duplication checking can be reduced; the modal feature degree of the feature frame image is constructed by combining the moving attribute of the object in the video frame image, the multi-modal feature analysis of the video frame image is performed, the influence of the static scene on the image analysis can be reduced, the uniqueness of the video frame image with the moving attribute can be increased, and the similarity between the videos can be better calculated when the video duplication checking is performed, and the dynamic video duplication checking is performed.

[0078] It should be noted that the above-mentioned embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments. Moreover, the above-mentioned embodiments of the application are described. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0079] Each of the embodiments in the application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment mainly describes the difference from other embodiments.

[0080] The above only describes the preferred embodiments of the application, and does not limit the application, and any modification, equivalent replacement, improvement, etc. made within the principles of the application shall be included in the protection scope of the application.

Claims

1. A video duplicate detection method based on multi-modal feature fusion and dynamic strategy, characterized in that, The method comprises the following steps: Obtain a gray image of each frame image of video data, denoted as a feature frame image; Uniformly divide and number the feature frame image to obtain each region of the feature frame image and its serial number; and construct structure synchronism of the feature frame image based on the distribution of edge contours in all regions of the feature frame image; Construct structure color anisotropy of the feature frame image based on the difference between the change directions of the gray values of the pixels in different closed contours in the feature frame image, and determine structure color trend degree of the feature frame image in combination with the structure synchronism. Determine a moving frame image of the current feature frame image based on the difference between the current feature frame image and the feature frame images at the previous and next time points; and construct modal feature degree of the feature frame image based on the distribution of pixel values in each region of the moving frame image in combination with the serial number and the structure color trend degree. Construct a modal feature vector of each video through the modal feature degrees of the feature frame images of all the frame images of each video; and perform video duplication checking based on the similarity between the modal feature vectors of the video to be audited and the user-stored videos.

2. The video duplicate detection method based on multi-modal feature fusion and dynamic strategy according to claim 1, wherein, The structure synchronism is obtained by: A sequence of the number of in-region edges of all regions in the feature frame image is denoted as a contour feature sequence, and a sequence of the sum of the pixel values of the in-region edge pixels of all regions in the feature frame image is denoted as a contour structure sequence; the similarity between the contour feature sequence and the contour structure sequence is obtained; and the difference between the maximum value in the contour feature sequence and the maximum value in the contour structure sequence is obtained, denoted as a first difference. The structure synchronism of the feature frame image is positively correlated with the similarity and the first difference. 3.The video duplicate detection method based on multi-modal feature fusion and dynamic strategy according to claim 1, wherein, The structure color anisotropy is obtained by: The gray values of all the pixels in each closed contour in the feature frame image are threshold segmented, the average position of all the pixels with a gray value greater than the corresponding segmentation threshold in each closed contour is connected with the average position of all the pixels with a gray value less than or equal to the corresponding segmentation threshold to form a straight line, denoted as a color change line; The average of the difference between the slope of the color change line of the maximum closed contour and the slopes of the color change lines of all the closed contours in the feature frame image is calculated as the structure color anisotropy of the feature frame image. 4.The video duplicate detection method based on multi-modal feature fusion and dynamic strategy according to claim 1, wherein, The structure color trend degree is obtained by: The sum of the structure color anisotropy of the feature frame image and a preset non-zero adjustment factor is calculated, denoted as a first sum; The ratio of the structure synchronism of the feature frame image to the first sum is taken as the structure color trend degree of the feature frame image.

5. The video duplicate detection method based on multi-modal feature fusion and dynamic strategy according to claim 1, wherein, The moving frame image is obtained by: The current time feature frame image is subtracted from the feature frame images at the previous and next time points respectively to obtain two inter-frame difference images, and the inter-frame difference image with the smallest average pixel value is taken as the moving frame image of the current time feature frame image.

6. The video duplicate detection method based on multi-modal feature fusion and dynamic strategy according to claim 1, wherein, The modal feature degree is obtained by: In the moving frame image, the number of regions occupied by non-zero pixels is denoted as a first number, and the sum of the serial numbers of all the regions occupied by non-zero pixels is denoted as a first feature sum. The modal feature degree of the moving frame image is positively correlated with the first number and the first feature and value, and is negatively correlated with the structural color trend degree.

7. The video duplicate detection method based on multi-modal feature fusion and dynamic strategy according to claim 1, wherein, The video duplication checking process is as follows: For any user stored video, the length of the modal feature vector of the user stored video and the to-be-audited video is compared to determine a matching modal feature vector and the length of a sliding window in another modal feature vector; the vectors in each sliding window of the another modal feature vector are recorded as modal feature sub-vectors; the similarity between the matching modal feature vector and each modal feature sub-vector is determined to determine a similar modal feature sub-vector of the matching modal feature vector; A difference sequence between the similar modal feature sub-vector and the matching modal feature vector is obtained, and if the number of elements less than a preset threshold in the difference sequence is more than the number of elements greater than the preset threshold, it is determined that the video is duplicated. Otherwise, it is determined that the video is not duplicated.

8. The video duplicate detection method based on multi-modal feature fusion and dynamic strategy according to claim 7, characterized in that, The determination of the matching modal feature vector and the length of the sliding window in the another modal feature vector is as follows: the shortest value of the lengths of the modal feature vectors of the user stored video and the to-be-audited video is obtained, the shortest value is taken as the length of the sliding window, and the modal feature vector corresponding to the shortest value is taken as the matching modal feature vector.

9. The video duplicate detection method based on multi-modal feature fusion and dynamic strategy according to claim 7, wherein, The process of obtaining the similar modal feature sub-vector is as follows: The maximum value of the similarity between the matching modal feature vector and all modal feature sub-vectors is obtained, and the modal feature sub-vector corresponding to the maximum value of the similarity is taken as the similar modal feature sub-vector. 10.A video duplicate detection system based on multi-modal feature fusion and dynamic strategy, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1-9.

Images