Archive management system based on distributed storage

Abstract

The invention relates to the technical field of archive management, in particular to an archive management system based on distributed storage, which matches corresponding storage nodes through archive types of pre-stored video archives of a smart campus, and obtains archive features of the pre-stored video archives based on an image feature recognition algorithm. Segmenting the pre-stored video file by taking the file continuity similarity and the video file feature data length as a judgment basis of a distributed storage segmentation point so as to facilitate subsequent distributed storage; acquiring a first key frame video file feature and a second key frame video file feature by traversing the similarity of the segmented pre-stored video file, taking the features as a basis for data backup and data restoration, and performing distributed storage based on file types; through the method, the security of the smart campus video file can be improved, and meanwhile, based on image recognition algorithm feature selection and distributed storage, the completeness during video backup and restoration can be ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of archive management, and in particular to an archive management system based on distributed storage. Background Art

[0002] Distributed storage is a data storage method that divides single data into multiple data blocks and sends them to multiple nodes for storage. Distributed storage is mainly used to solve the storage of large amounts of data that are growing explosively today. For large amounts of data such as TB, PB, and ZB, the transmission speed of hard disks is limited. Therefore, by dividing these large amounts of data into smaller amounts of data, storage efficiency can be effectively improved. At the same time, distributed storage also solves the problem of data loss for single data. It backs up data while dividing it. Even if data on a certain node is lost, the data can be restored through backup, which significantly improves the reliability and fault tolerance of data storage.

[0003] A smart campus uses advanced computer technologies such as the Internet of Things and big data to achieve intelligent and digital transformation of teaching, services and management on campus; and the digital security management of campus archives is one of them. In addition to traditional archives such as student academic records, faculty and staff personnel files and employee files, educational video archives such as teaching videos and lecture videos, campus security surveillance video archives and campus meeting activity record archives also need to be preserved and managed as new types of archival data. Compared with traditional archives, the characteristics of video archives are that single archive data is larger than traditional archive data and has a coherent feature. Therefore, the security and integrity detection of video archives are more difficult during the storage process.

[0004] The existing technology has certain limitations and low applicability for the security and integrity issues of video archive storage in smart campuses. Therefore, an archive management system based on distributed storage is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide an archive management system based on distributed storage, which matches the archive type of the pre-stored video archive of the smart campus with the corresponding storage node, obtains the archive features of the pre-stored video archive based on the image feature recognition algorithm, and uses the archive continuity similarity and the video archive feature data length as the basis for judging the distributed storage segmentation point, so as to segment the pre-stored video archive for subsequent distributed storage; obtains the first key frame video archive feature and the second key frame video archive feature by traversing the similarity of the pre-stored video archive after segmentation, as the basis for data backup and data restoration, and performs distributed storage based on the archive type; the method can improve the security of the video archive of the smart campus, and at the same time, based on the image recognition algorithm feature selection and distributed storage, can ensure the integrity of the video during backup and restoration.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A file management system based on distributed storage includes a smart campus video file acquisition module, a smart campus video file segmentation and judgment module, a smart campus video file first segmentation module, a smart campus video file second segmentation module, a smart campus video file storage module, a smart campus primary feature backup module and a smart campus secondary feature backup module, specifically:

[0008] The smart campus video archive acquisition module obtains the archive type of the pre-stored video archive and matches multiple corresponding storage nodes, screens the storage nodes, and generates a target storage node;

[0009] The storage node corresponds to a file type, and the storage nodes of the same file type include multiple groups, each group includes multiple storage nodes; the file types include personal video files, laboratory video files, open class video files, monitoring video files, activity video files, conference video files, temporary video files and unclassified video files;

[0010] Furthermore, the number of the multiple groups of storage nodes of the same file type is at least three groups, each group of storage nodes is divided according to functional type, and the multiple groups of storage nodes include a video original file storage group, a video feature file backup group and a video feature file restoration group;

[0011] The smart campus video archive segmentation judgment module extracts multiple picture archive data of the pre-stored video archive to obtain video archive feature data, and calculates the archive continuity similarity of the video archive feature data;

[0012] Furthermore, the video archive feature data and the archive continuity similarity are judged by a smart campus video archive feature model; the smart campus video archive feature model includes an archive video data input unit, an archive video data feature extraction unit, an archive video data similarity calculation unit, an archive video data similarity marking unit, an archive video feature output unit, an archive video similarity output unit and an archive video feature restoration unit;

[0013] The archive video data input unit classifies and marks the archive types of the pre-stored video archives, and pre-processes the pre-stored video archives to generate archive video input data and archive video type marks;

[0014] Specifically, the archive video similarity data marks the pre-stored video archive by marking the pre-stored video archive of each timestamp with the archive video similarity data;

[0015] If the archive video similarity data of the t-th timestamp is less than the archive continuity similarity, a first mark is performed;

[0016] Calculate the difference θ between the similarity data of the archive video at the tth timestamp and the similarity data of the archive video at the t+1th timestamp t1 , calculate the difference θ between the similarity data of the archive video at the t-th timestamp and the similarity data of the archive video at the t-1th timestamp t2 ;

[0017] If θ t1 or θ t2 If it is greater than the difference in file similarity, a second mark is performed;

[0018] The archive video data feature extraction unit calls the corresponding video feature extraction module according to the archive video type tag, extracts the archive video input data features through the video feature extraction module, and generates archive video feature data;

[0019] The video feature extraction module is composed of a coding layer and a decoding layer, the coding layer is composed of an improved yolov5 acquisition and a Transformer encoder, the video feature extraction module only uses the backbone layer and neck layer of yolov5, and the video feature extraction module includes position feature extraction and color feature extraction; the position feature extraction uses the backbone layer and neck layer of yolov5; the color feature extraction uses the bankbone layer and the neck layer improved based on the dilated convolution; the color feature extraction is composed of:

[0020] output TCN=neck(backbone TCN (input));

[0021]

[0022] Among them, output TCN is the output feature, neck is the neck layer, backbone TCN is the backbone layer improved based on the dilated convolution, input is the input data; Conv6*6 is the convolution layer with a convolution kernel size of 6*6, TCNV1, TCNV2, TCNV3, TCNV4, TCNV m is the feature extraction layer based on the dilated convolution, SPPF is the SPPF layer used for pooling in yolov5, TCNS m is the mth dilated convolution group, TCN11 m The mth CSP structure based on dilated convolution;

[0023] The TCNS m and the TCN11 m The structure is as follows:

[0024]

[0025] Among them, RELU is the activation layer, BN is the BN layer, TCN is the dilated convolution layer, and CNN1*1 is the convolution layer with a convolution kernel of 1*1;

[0026] The formula for the convolution kernel size of the dilated convolution layer is as follows:

[0027] KT λ =λ×(KT0-1)+1;

[0028] λ j =max(λ j+1 -2λ j ,λ j+1 -2(λ j+1 -λ j ),λ j );

[0029] Among them, KT λ is the size of the expanded convolution kernel, λ is the expansion factor, KT0 is the set convolution kernel size, max is the maximum value, λ j is the dilation factor of the jth layer, λ j+1 is the dilation factor of the j+1th layer;

[0030] After feature extraction, the feature is input into the Transformer encoder to generate position features and color features;

[0031] The decoding layer is restored by Transformer decoder and reverse dilation convolution;

[0032] The convolution kernel size and image size of the inverse dilation convolution are calculated as follows:

[0033]

[0034] λ k =max(λ k+1 -2λ k ,λ k+1 -2(λ k+1 -λ k ),λ k );

[0035]

[0036] Among them, KT A is the size of the reverse dilation convolution kernel, λ A is the reverse expansion factor, KT C is the reverse convolution kernel size, λ k is the dilation factor of the kth layer, λ k+1 is the dilation factor of the k+1th layer, outa is the output image size, ina is the input image size, padding is the padding size, and d is the moving step size;

[0037] The archive video data similarity calculation unit sorts all the archive video feature data in chronological order, and sequentially calculates the similarities of adjacent archive video feature data to generate archive video similarity data;

[0038] Specifically, the archive video similarity data is calculated as:

[0039]

[0040] Among them, γ sim is the similarity data of the archive video, N is the number of features, P i t is the i-th color feature of the archive video feature data at the t-th timestamp, P i t-1 is the i-th color feature of the archive video feature data at the t-1-th timestamp, is the i-th position feature of the archive video feature data at the t-th timestamp, is the i-th position feature of the archive video feature data at the t-1th timestamp, α c and α l is the calculation weight of the archive video similarity data;

[0041] The archive video data similarity marking unit marks the pre-stored video archive according to the archive video similarity data to generate marked pre-stored video archive data;

[0042] The archive video feature output unit outputs and stores the archive video feature data sorted in the time sequence;

[0043] The archive video similarity output unit outputs and stores the marked pre-stored video archive data;

[0044] The archive video feature restoration unit restores the archive video feature data;

[0045] Generate segmentation points based on the file continuity similarity and the length of the video file feature data as segmentation judgment;

[0046] Furthermore, the timestamp of the first frame of the video is used as the start timestamp; the segmentation point is determined by the file continuity similarity and the length of the video file feature data;

[0047] If the file continuity similarity is less than the continuity threshold, the timestamp corresponding to the file continuity similarity being less than the continuity threshold is used as the segmentation point; if the length of the video file feature data exceeds the segmentation continuity threshold, the timestamp corresponding to the segmentation continuity threshold length is used as the segmentation point;

[0048] The first segmentation module of the smart campus video archive segments the pre-stored video archive by the segmentation point, obtains the video archive feature data corresponding to the multiple segmented video archives, and obtains multiple first key frame video archive features by combining the archive continuity similarity calculation;

[0049] Furthermore, the first key frame video file feature is the frame number of the pre-stored video file and the video file feature data after segmentation;

[0050] Calculate based on the similarity of the video file feature data and the number of frames, and obtain the positions of the plurality of first key frame video file features from the video file feature data:

[0051]

[0052]

[0053] Among them, η F is the position of the first key frame video file feature, φ F and φ V is the preselected position, φ videois the frame number; To calculate and The position closest to the value, k n the number of said similarities calculated for the preselected positions, is the similarity of the feature data of the i-th video archive, k n The similarity of the pre-selected positions, min is the minimum value and max is the maximum value;

[0054] The second segmentation module of the smart campus video archive obtains the video archive feature data corresponding to the segmentation point, combines the archive continuity similarity mean of the video archive feature data after adjacent segmentation, and generates multiple second key frame video archive features; the second key frame video archive feature is obtained to obtain the video archive feature data of the segmentation point A for storage; the archive continuity similarity mean of the similarity range (M-1, M) and (M, M+1) of the segmentation point is obtained for storage; wherein M is the segmentation point, M-1 is the previous segmentation point with the smallest number of M frames, and M+1 is the next segmentation point with the smallest number of M frames.

[0055] A smart campus video archive storage module stores the pre-stored video archive after segmentation into the target storage node;

[0056] A smart campus primary feature backup module stores the segmented video archive feature data into the target storage node;

[0057] A smart campus secondary feature backup module stores the first key frame video file feature and the second key frame video file feature into the target storage node;

[0058] The smart campus archive storage module, the smart campus primary feature backup module and the smart campus secondary feature backup module also include:

[0059] The smart campus archive storage module stores similarity data of video archive feature data corresponding to the pre-stored video archive; the smart campus primary feature backup module and the smart campus secondary feature backup module store similarity data of the corresponding video archive feature data;

[0060] The smart campus primary feature backup module and the smart campus secondary feature backup module perform feature data restoration through the archive video feature restoration unit of the smart campus video archive feature model.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. The present invention obtains feature data of pre-stored video archives through an image feature algorithm, and divides the data based on the archive continuity similarity between feature data and the length of video archive feature data as a judgment basis, so as to retain the integrity of the video archive data to the greatest extent, and also reduce the data size, reduce the required storage capacity space, and facilitate subsequent distributed storage; at the same time, the data feature data is also retained to provide a basis for subsequent data backup and restoration work, so as to ensure the implementation of subsequent video archive data security processing.

[0063] 2. In order to ensure the accuracy of feature similarity analysis, the present invention extracts all feature data of pre-stored video files, calculates the similarity between adjacent feature data, and judges the color features and position features of related content to improve the accuracy of the similarity content of the video file, thereby ensuring the integrity of the video file content during segmentation. Even if distributed storage is performed, the video content will not be destroyed, thereby ensuring the continuity and local integrity of the video content after segmentation. If a small amount of data is missing in the worst case, the continuity and integrity of other content can be guaranteed as much as possible. At the same time, the characteristics of distributed storage can also be combined to ensure the security of video file data.

[0064] 3. The present invention is based on the feature data after segmentation and the similarity data of the calculated adjacent feature data. The feature position of the first key frame video file of the pre-stored video after segmentation is obtained through the similarity data and the frame number data, and the corresponding feature data is obtained to generate the first key frame video file as the basis for backup and restoration for storage. While ensuring that the data can be restored in the event of loss, it can also ensure that the content of the restored video file is coherent, thereby ensuring the integrity of the video file. At the same time, the feature data can also ensure that when the backup data is leaked, the content of the video file cannot be understood through the feature data, thereby improving the security of the backup data. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic diagram of the system flow of the present invention;

[0066] Figure 2 It is a schematic diagram of the flow chart of the smart campus video archive feature model of the present invention;

[0067] Figure 3 A schematic diagram of the process flow of the video feature extraction module of the smart campus video archive feature model of the present invention;

[0068] Figure 4 It is a structural schematic diagram of the color feature extraction part in the video feature extraction module of the present invention;

[0069] Figure 5 TCNS in the video feature extraction module of the present invention m and TCN11 mSchematic diagram of the structure. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] With the gradual maturity of new computer technologies such as big data, the smart campus realized by these related technologies can realize efficient digital management of traditional teaching and service businesses on campus; among them, security management has always been one of the focuses of digital management. In addition to the security of traditional archive data such as personnel or faculty and staff, the security issues of video archive data as a new type of archive data cannot be ignored; compared with traditional archive data, since the capacity of a single video archive data is larger than that of traditional archive data, and the information therein has the characteristics of continuity, if the traditional distributed algorithm is used for storage, it is easy to destroy the integrity of the video archive data, and the video archive data is more complex than the traditional archive data, and it is impossible to ensure the same security as the traditional archive data. For this reason, the present invention provides an archive management system based on distributed storage, and the technical solution is as follows:

[0072] Reference Figure 1 As shown, the smart campus video archive acquisition module obtains the archive type of the pre-stored video archive and matches multiple corresponding storage nodes, screens the storage nodes, and generates a target storage node;

[0073] The smart campus video archive segmentation judgment module extracts multiple image archive data of the pre-stored video archive to obtain video archive feature data, calculates the archive continuity similarity of the video archive feature data; generates segmentation points based on the archive continuity similarity and the length of the video archive feature data as segmentation judgment;

[0074] The first segmentation module of the smart campus video archive segments the pre-stored video archive by the segmentation point, obtains the video archive feature data corresponding to the multiple segmented video archives, and obtains multiple first key frame video archive features by combining the archive continuity similarity calculation;

[0075] The second segmentation module of the smart campus video archive obtains the video archive feature data corresponding to the segmentation point, combines the archive continuity similarity mean of the video archive feature data after adjacent segmentation, and generates a plurality of second key frame video archive features;

[0076] A smart campus video archive storage module stores the pre-stored video archive after segmentation into the target storage node;

[0077] A smart campus primary feature backup module stores the segmented video archive feature data into the target storage node;

[0078] The smart campus secondary feature backup module stores the first key frame video file feature and the second key frame video file feature into the target storage node.

[0079] In order to ensure the integrity and security of each video archive data of the smart campus;

[0080] First, in order to facilitate storage and management, the file type of the storage node is matched with the file type of the pre-stored video file, and the storage node is screened to generate the pre-stored file target storage node, so that the pre-stored video can be placed in the corresponding storage node for distributed storage and management;

[0081] Secondly, in order to ensure the continuity of distributed storage of video archive data, the image data of each timestamp in the video archive is extracted, and the image data is obtained through the image feature recognition algorithm for similarity analysis. Based on the video archive continuity similarity and the length of the video archive feature data as the judgment basis, the video archive data is distributedly divided, which can ensure the continuity of the video archive to the greatest extent on the basis of the original data, prevent the destruction of the video archive information, and improve the local integrity after distributed storage based on the security of distributed storage;

[0082] Finally, for the corresponding video archive feature data of the segmented video archive data, multiple first key frame video archive features are obtained through continuity similarity calculation, and the continuity similarity between the video archive feature data of the segmentation point and the adjacent segmented archive data is used as the second key frame video archive feature. By storing and backing up the video feature data, on the one hand, it can improve the ability to restore data when data is missing and ensure data integrity. On the other hand, these feature data can also ensure that video archive information cannot be obtained after data leakage, thereby improving data security.

[0083] Embodiment 1

[0084] For the purpose of detailed description, the present invention is described in conjunction with a case where there is only a single file type. For the purpose of specific description, this embodiment takes the file type of campus conference video as an example:

[0085] A file management system based on distributed storage includes a smart campus video file acquisition module, a smart campus video file segmentation and judgment module, a smart campus video file first segmentation module, a smart campus video file second segmentation module, a smart campus video file storage module, a smart campus primary feature backup module and a smart campus secondary feature backup module, specifically:

[0086] The smart campus video archive collection module obtains the archive type of the pre-stored video archive and matches multiple corresponding storage nodes, screens the storage nodes, and generates the target storage node;

[0087] A storage node corresponds to a file type. Storage nodes of the same file type include multiple groups, and each group contains multiple storage nodes. The file types include personal video files, laboratory video files, open class video files, monitoring video files, activity video files, conference video files, temporary video files, and unclassified video files.

[0088] In the specific implementation process, since the type of the stored video archive is campus conference video, all storage nodes of the type campus conference video will be matched during the traversal and screening process; the implementation method can be in three forms;

[0089] In a first implementation mode, a storage node storing a campus conference video is determined as a corresponding storage node;

[0090] In a second implementation mode, a storage node storing campus conference videos and accounting for 50% or more of the total storage capacity is determined as a corresponding storage node;

[0091] A third implementation method is to divide the storage nodes into storage nodes dedicated to storing campus conference videos;

[0092] Distributed storage is used to reduce the storage pressure of each storage server and ensure data security. Therefore, the data will be divided into multiple copies, each of which is stored on a different storage server. In order to facilitate data search and management, different file types will be divided for each storage server. The stored campus conference videos are matched with the corresponding storage server file types as candidate storage nodes to prevent the situation where data is scattered and inconvenient to manage.

[0093] Furthermore, the number of multiple groups of storage nodes of the same file type is at least three groups, each group of storage nodes is divided according to the functional type, and the multiple groups of storage nodes include a video original file storage group, a video feature file backup group and a video feature file restoration group; in the specific implementation process, all corresponding storage nodes after the campus conference video is matched will be divided into at least three groups according to function, namely, a video original file storage group, a video feature file backup group and a video feature file restoration group;

[0094] In the specific implementation process, there may be further divisions. For example, according to the type of meeting, the video original archive storage group may be divided into the school important meeting video original archive storage group, the school routine meeting video original archive storage group, etc.; or according to the date, it may be further divided into the first quarter meeting video original archive storage group, the second quarter meeting video original archive storage group, etc.;

[0095] In order to accurately divide, in addition to dividing according to data types, it also divides the video original archive storage group, video feature archive backup group and video feature archive restoration group. By dividing the functional data, it can ensure that when performing original data storage, data backup and data restoration, the corresponding data can be accurately located, the data processing speed can be improved, and at the same time, data mixing can be avoided, resulting in data restoration failure or data loss during restoration;

[0096] The smart campus video archive segmentation and judgment module extracts multiple image archive data of pre-stored video archives to obtain video archive feature data, and calculates the archive continuity similarity of the video archive feature data;

[0097] Furthermore, the video archive feature data and the archive continuity similarity are judged by the smart campus video archive feature model; the smart campus video archive feature model includes an archive video data input unit, an archive video data feature extraction unit, an archive video data similarity calculation unit, an archive video data similarity marking unit, an archive video feature output unit, an archive video similarity output unit and an archive video feature restoration unit, referring to Figure 2 As shown;

[0098] The archive video data input unit classifies and marks the archive types of the pre-stored video archives, and pre-processes the pre-stored video archives to generate archive video input data and archive video type marks;

[0099] Specifically, the archive video similarity data marks the pre-stored video archives by marking the archive video similarity data for each time stamped pre-stored video archive;

[0100] If the archive video similarity data of the t-th timestamp is less than the archive continuity similarity, a first mark is performed;

[0101] Calculate the difference θ between the similarity data of the archive video at the tth timestamp and the similarity data of the archive video at the t+1th timestamp t1 , calculate the difference θ between the similarity data of the archive video at the t-th timestamp and the similarity data of the archive video at the t-1th timestamp t2 ;

[0102] If θ t1 or θ t2If it is greater than the file similarity difference, a second mark is performed; the similarity difference defaults to 5%, which can be modified according to the specific implementation situation combined with the file type of the actual video data or expert experience;

[0103] After the video archive continuity similarity is calculated, in order to improve the data processing speed and facilitate the subsequent processing of related video archive data, the corresponding data can be accurately obtained by marking the similarity. At the same time, additional marking is performed on the data with a large similarity gap, which is also convenient for the confirmation of subsequent segmentation points and improves the data processing speed. At the same time, recording related data can also assist in data query and restoration, improve the integrity of data during query and restoration, and similarity data can also be used as one of the signs of data security verification;

[0104] The archive video data feature extraction unit calls the corresponding video feature extraction module according to the archive video type tag, extracts the archive video input data features through the video feature extraction module, and generates archive video feature data; in this embodiment, the video feature extraction module trained based on campus video conference archive data is called;

[0105] The video feature extraction module is composed of an encoding layer and a decoding layer. Figure 3 As shown, Figure 3 Output1 is the feature output, and output2 is the feature restoration output; the encoding layer is composed of the improved yolov5 acquisition and Transformer encoder. The video feature extraction module only uses the backbone layer and neck layer of yolov5. The video feature extraction module includes position feature extraction and color feature extraction; the position feature extraction uses the backbone layer and neck layer of yolov5; the color feature extraction uses the bankbone layer and neck layer improved based on the dilated convolution, refer to Figure 4 As shown; the composition of color feature extraction is:

[0106] output TCN =neck(backbone TCN (input));

[0107]

[0108] Among them, output TCN is the output feature, neck is the neck layer, backbone TCN It is the backbone layer improved based on dilated convolution, input is the input data; Conv6*6 is the convolution layer with a convolution kernel size of 6*6, TCNV1, TCNV2, TCNV3, TCNV4, TCNV mis a feature extraction layer based on dilated convolution, SPPF is the SPPF layer used for pooling in yolov5, and TCNS m is the mth dilated convolution group, TCN11 m The mth CSP structure based on dilated convolution;

[0109] TCNS m and TCN11 m The structure of Figure 5 As shown below:

[0110]

[0111] Among them, RELU is the activation layer, BN is the BN layer, TCN is the dilated convolution layer, and CNN1*1 is the convolution layer with a convolution kernel of 1*1;

[0112] The formula for the convolution kernel size of the dilated convolution layer is as follows:

[0113] KT λ =λ×(KT0-1)+1;

[0114] λ j =max(λ j+1 -2λ j ,λ j+1 -2(λ j+1 -λ j ),λ j );

[0115] Among them, KT λ is the size of the expanded convolution kernel, λ is the expansion factor, KT0 is the set convolution kernel size, max is the maximum value, λ j is the dilation factor of the jth layer, λ j+1 is the dilation factor of the j+1th layer; the initial convolution kernel size of the dilated convolution is adjusted and set according to the training data of the campus conference video;

[0116] After feature extraction, it is input into the Transformer encoder to generate position features and color features;

[0117] The decoding layer is restored by Transformer decoder and reverse dilation convolution;

[0118] The convolution kernel size and image size of the inverse dilated convolution are calculated as follows:

[0119]

[0120] λ k =max(λ k+1 -2λ k ,λ k+1 -2(λk+1 -λ k ),λ k );

[0121]

[0122] Among them, KT A is the inverse expansion convolution kernel size, λ A is the reverse expansion factor, KT C is the reverse convolution kernel size, λ k is the expansion factor of the kth layer, λ k+1 is the expansion factor of the k+1th layer, outa is the output image size, ina is the input image size, padding is the padding size, and d is the moving step size;

[0123] The archive video data similarity calculation unit sorts all archive video feature data in chronological order, and sequentially calculates the similarities of adjacent archive video feature data to generate archive video similarity data;

[0124] The smart campus video archive feature model can realize the similarity calculation, labeling and related data storage. When data of different lengths and sizes are input, the model can be used to segment the data according to the prescribed process to improve the data processing speed. At the same time, combined with the image feature recognition algorithm, it can quickly and accurately extract the corresponding features in the video archive data, calculate the corresponding similarity, accurately grasp the continuity relationship between the video archive data, ensure the coherence characteristics of the video archive data during segmentation, and ensure the local and global integrity of the video.

[0125] Specifically, the archive video similarity data is calculated as:

[0126]

[0127] Among them, γ sim is the archive video similarity data, N is the number of features, P i t is the i-th color feature of the archive video feature data at the t-th timestamp, P i t-1 is the i-th color feature of the archive video feature data at the t-1th timestamp, is the i-th position feature of the archive video feature data at the t-th timestamp, is the i-th position feature of the archive video feature data at the t-1th timestamp, α c and α l The calculation weight for the archive video similarity data;

[0128] α c and αl The default values ​​are 0.4 and 0.6. The values ​​are analyzed in combination with the simulated campus conference data to obtain the results. Each weight is calculated 30 times, and the average is taken after removing the highest and lowest values ​​and compared with the actual data. The actual data with different weights are the same, as shown in Table 1:

[0129] Table 1α c and α l Comparison of similarity values ​​calculated with different weights and actual similarity values

[0130] <![CDATA[α c ]]> <![CDATA[α l ]]> Calculated similarity Actual similarity Difference 0.7 0.3 92.3％ 94.6％ 2.3％ 0.6 0.4 94.1％ 94.6％ 0.5％ 0.5 0.5 93.7％ 94.6％ 0.9％ 0.4 0.6 93.2％ 94.6％ 1.4％ 0.3 0.7 92.5％ 94.6％ 2.1％

[0131] In the specific implementation process, α c and α l It can be modified based on the data type and size of historical video archives combined with expert experience;

[0132] Considering the content of the video itself, in order to ensure that the similarity calculation can achieve a high accuracy, both color features and position features are calculated. The color feature simulates the wide-area perspective to obtain the characteristics of the entire image, and the position feature simulates the local perspective to determine the position characteristics of different key objects in the current picture. Through the above calculation, the similarity between adjacent pictures can be accurately calculated, and the continuity and integrity of similar data during segmentation can be guaranteed to the greatest extent;

[0133] The archive video data similarity marking unit marks the pre-stored video archive according to the archive video similarity data to generate marked pre-stored video archive data;

[0134] The archive video feature output unit outputs and stores the archive video feature data sorted in chronological order;

[0135] The archive video similarity output unit outputs the marked pre-stored video archive data and stores it;

[0136] The segmentation points are generated based on the file continuity similarity and the length of the video file feature data as segmentation judgment;

[0137] Furthermore, the timestamp of the first frame of the video is used as the start timestamp; the segmentation point is determined by the file continuity similarity and the length of the video file feature data;

[0138] If the file continuity similarity is less than the continuity threshold, the timestamp corresponding to the file continuity similarity being less than the continuity threshold is used as the segmentation point; if the length of the video file feature data exceeds the segmentation continuity threshold, the timestamp corresponding to the segmentation continuity threshold length is used as the segmentation point;

[0139] The continuity threshold is obtained through a large amount of data and combined with expert experience. The default setting is 90%. It can be modified according to the actual situation on campus. The continuity similarity ratio of the continuity similarity of the adjacent archive feature data is compared with the continuity threshold; the continuity similarity ratio is calculated as follows:

[0140]

[0141] Among them, ω γ is the continuity similarity ratio, is the continuity similarity of the t-th timestamp, is the continuity similarity of the t+1th timestamp;

[0142] The segmentation continuity threshold uses the frame rate of the current video as the segmentation continuity threshold. For example, for a 120-frame campus conference video, the segmentation continuity threshold is 120, which means 120 images after the campus conference video is split by frame; for a 60-frame campus conference video, the segmentation continuity threshold is 60, which means 60 images after the campus conference video is split by frame.

[0143] In the process of distributed storage, it is possible that the content in the picture is similar for a long time. For example, at a teacher's meeting on campus, when a teacher or relevant leader is speaking, the picture may be similar for a long time. If it is divided only by similarity, a certain period of time may be too long, resulting in too much data after segmentation, which violates the original purpose of distributed storage. For this reason, a segmentation continuity threshold is added. If the current data length exceeds the limit, it is also segmented. On the one hand, it ensures the continuity of the segmented data and the local integrity of the segmented data. At the same time, it also avoids the situation where the data is too long and the data volume is too large, ensuring that distributed storage can be implemented to ensure data security.

[0144] The first segmentation module of the smart campus video archive segments the pre-stored video archive by segmentation points, obtains the video archive feature data corresponding to the multiple segmented video archives, and obtains multiple first key frame video archive features by combining the archive continuity similarity calculation;

[0145] Furthermore, the first key frame video file feature is to obtain the frame number of the pre-stored video file and the segmented video file feature data;

[0146] Calculate based on the similarity and frame number of the video file feature data, and obtain the positions of multiple first key frame video file features from the video file feature data:

[0147]

[0148] Among them, η Fis the position of the first key frame video file feature, φ F and φ V is the preselected position, φ video is the number of frames; To calculate and The position closest to the value, k n The number of similarities calculated for the preselected positions, is the similarity of the feature data of the i-th video file, k n The similarity of the pre-selected positions, min is the minimum value and max is the maximum value;

[0149] 20 is the minimum frame number of continuous video that can be observed by the human eye. If it is less than this value, there will be a freeze. In actual video acquisition, 30 frames, 60 frames, 120 frames and above are mainly used. Since the existing video data generally uses 60 frames and above, the present invention takes 60 frames as the reference object when considering it, so 3 is used as the division node. The 30-frame data is retained because the difference between it and the 20-frame data is small. Taking the campus conference video of the specific implementation as an example, if the frame number φ of the campus conference video collected is video ≤60; for example, the campus conference video has 30 frames, even if calculated according to the formula, it is 1.5 frames, but this number cannot be obtained in actual data acquisition, and the difference between 30 frames of data and 20 frames of data is small, so each frame is rounded down as the first key frame video file feature, and no position calculation is performed. All 30 frames of data are used as the first key frame video file feature;

[0150] In order to ensure the security of data after backup storage and the integrity of data when restoring the backup, the location of the first key frame video file feature is obtained to obtain the more critical data among all feature data. At the same time, in order to ensure continuity, 20 frames are used as the minimum integrity flow level for calculation; through the selection of the first key frame video file feature, the security of the data can be guaranteed in the event of leakage, and even if a large amount of data is lost, the minimum integrity and fluency can be guaranteed after the data is restored; at the same time, a small amount of feature data, both in terms of the data itself and the amount of data, can save the capacity space of the distributed storage server and improve the space utilization efficiency;

[0151] The second segmentation module of the smart campus video archive obtains the video archive feature data corresponding to the segmentation point, and generates multiple second key frame video archive features by combining the archive continuity similarity mean of the video archive feature data after adjacent segmentation; the second key frame video archive feature is obtained by obtaining the video archive feature data of the segmentation point A for storage; the archive continuity similarity mean of the similarity range (M-1, M) and (M, M+1) of the segmentation point is obtained for storage; wherein M is the segmentation point, M-1 is the previous segmentation point with the smallest number of M frames, and M+1 is the next segmentation point with the smallest number of M frames. In the restoration process, in order to facilitate the query of the segmented data, the similarity data can be quickly associated with the original data, and by backing up the segmentation points, the integrity and fluency of the data during connection can be guaranteed. At the same time, storage based on feature data can also reduce capacity and improve data security to prevent key information from being obtained when data is leaked;

[0152] The smart campus video archive storage module stores the pre-stored video archives after segmentation into the target storage node;

[0153] The smart campus primary feature backup module stores the segmented video archive feature data into the target storage node;

[0154] The smart campus secondary feature backup module stores the first key frame video file feature and the second key frame video file feature into the target storage node;

[0155] The smart campus archive storage module, the smart campus primary feature backup module and the smart campus secondary feature backup module also include:

[0156] The smart campus archive storage module stores similarity data of video archive feature data corresponding to the pre-stored video archive; the smart campus primary feature backup module and the smart campus secondary feature backup module store similarity data of the corresponding video archive feature data;

[0157] The smart campus primary feature backup module and the smart campus secondary feature backup module perform feature data restoration through the archive video feature restoration unit of the smart campus video archive feature model;

[0158] By storing these similarity data, when data is called, the key information of data search can be improved through these data, which speeds up the matching speed; at the same time, when restoring data, the corresponding backup data can be accurately searched for restoration, which can also reduce the number of data traversals and improve efficiency.

[0159] Embodiment 2

[0160] For the purpose of detailed description, this embodiment is described in conjunction with the case where there are only multiple file types. For the purpose of specific description, this embodiment takes the file types of open classes, campus conference videos, and surveillance videos as examples, and the selected campus conference video is the same as the campus conference video in the first embodiment:

[0161] A file management system based on distributed storage includes a smart campus video file acquisition module, a smart campus video file segmentation and judgment module, a smart campus video file first segmentation module, a smart campus video file second segmentation module, a smart campus video file storage module, a smart campus primary feature backup module and a smart campus secondary feature backup module, specifically:

[0162] The smart campus video archive collection module obtains the archive type of the pre-stored video archive and matches multiple corresponding storage nodes, screens the storage nodes, and generates the target storage node;

[0163] A storage node corresponds to a file type, and the storage nodes of the same file type include multiple groups, each group including multiple storage nodes; the file types include personal video files, laboratory video files, open class video files, monitoring video files, activity video files, conference video files, temporary video files and unclassified video files; in this embodiment, since open classes, campus conference videos and monitoring videos are included, the storage nodes corresponding to the open class video files, monitoring video files and conference video files are searched for storage;

[0164] Furthermore, the number of multiple groups of storage nodes of the same file type is at least three groups, each group of storage nodes is divided according to the functional type, and the multiple groups of storage nodes include a video original file storage group, a video feature file backup group and a video feature file restoration group; each of the open class video files, the monitoring video files and the conference video files corresponds to the three functions of the original file storage group, the video feature file backup group and the video feature file restoration group;

[0165] The smart campus video archive segmentation and judgment module extracts multiple image archive data of pre-stored video archives to obtain video archive feature data, and calculates the archive continuity similarity of the video archive feature data;

[0166] Further, the video archive feature data and the archive continuity similarity are judged by a smart campus video archive feature model; the smart campus video archive feature model includes an archive video data input unit, an archive video data feature extraction unit, an archive video data similarity calculation unit, an archive video data similarity marking unit, an archive video feature output unit, an archive video similarity output unit and an archive video feature restoration unit;

[0167] The archive video data input unit classifies and marks the archive types of the pre-stored video archives, and pre-processes the pre-stored video archives to generate archive video input data and archive video type marks;

[0168] Specifically, the archive video similarity data marks the pre-stored video archives by marking the archive video similarity data for each time stamped pre-stored video archive;

[0169] If the archive video similarity data of the t-th timestamp is less than the archive continuity similarity, a first mark is performed;

[0170] Calculate the difference θ between the similarity data of the archive video at the tth timestamp and the similarity data of the archive video at the t+1th timestamp t1 , calculate the difference θ between the similarity data of the archive video at the t-th timestamp and the similarity data of the archive video at the t-1th timestamp t2 ;

[0171] If θ t1 or θ t2 If it is greater than the difference in file similarity, a second mark is performed;

[0172] The archive video data feature extraction unit calls the corresponding video feature extraction module according to the archive video type tag, extracts the archive video input data features through the video feature extraction module, and generates archive video feature data; in this embodiment, the video feature extraction module trained based on campus video conference archive data, the video feature extraction module trained based on monitoring video archive data, and the video feature extraction module trained based on open class video archive data are called;

[0173] The video feature extraction module is composed of an encoding layer and a decoding layer. The encoding layer is composed of an improved yolov5 acquisition and a Transformer encoder. The encoding layer is composed of an improved yolov5 acquisition and a Transformer encoder. The video feature extraction module only uses the backbone layer and neck layer of yolov5. The video feature extraction module includes position feature extraction and color feature extraction; the position feature extraction uses the backbone layer and neck layer of yolov5; the color feature extraction uses the bankbone layer and neck layer improved based on the dilated convolution; the color feature extraction is composed of:

[0174] output TCN =neck(backbone TCN (input));

[0175]

[0176]

[0177] Among them, output TCN is the output feature, neck is the neck layer, backbone TCN It is the backbone layer improved based on dilated convolution, input is the input data; Conv6*6 is the convolution layer with a convolution kernel size of 6*6, TCNV1, TCNV2, TCNV3, TCNV4, TCNV m is a feature extraction layer based on dilated convolution, SPPF is the SPPF layer used for pooling in yolov5, and TCNS m is the mth dilated convolution group, TCN11 m The mth CSP structure based on dilated convolution;

[0178] TCNS m and TCN11 m The structure is as follows:

[0179]

[0180] Among them, RELU is the activation layer, BN is the BN layer, TCN is the dilated convolution layer, and CNN1*1 is the convolution layer with a convolution kernel of 1*1; the initial convolution kernel size of the dilated convolution is adjusted and set according to the training data of the campus conference video, open class and surveillance video;

[0181] The formula for the convolution kernel size of the dilated convolution layer is as follows:

[0182] KT λ =λ×(KT0-1)+1;

[0183] λ j =max(λ j+1 -2λ j ,λ j+1 -2(λ j+1 -λ j ),λ j );

[0184] Among them, KT λ is the size of the expanded convolution kernel, λ is the expansion factor, KT0 is the set convolution kernel size, max is the maximum value, λ j is the dilation factor of the jth layer, λ j+1 is the expansion factor of the j+1th layer;

[0185] After feature extraction, it is input into the Transformer encoder to generate position features and color features;

[0186] The decoding layer is restored by Transformer decoder and reverse dilation convolution;

[0187] The convolution kernel size and image size of the inverse dilated convolution are calculated as follows:

[0188]

[0189] λ k =max(λ k+1 -2λ k ,λ k+1 -2(λ k+1 -λ k ),λ k );

[0190]

[0191] Among them, KT A is the inverse expansion convolution kernel size, λ A is the reverse expansion factor, KT C is the reverse convolution kernel size, λ k is the expansion factor of the kth layer, λ k+1 is the expansion factor of the k+1th layer, outa is the output image size, ina is the input image size, padding is the padding size, and d is the moving step size;

[0192] The archive video data similarity calculation unit sorts all archive video feature data in chronological order, and sequentially calculates the similarities of adjacent archive video feature data to generate archive video similarity data;

[0193] Specifically, the archive video similarity data is calculated as:

[0194]

[0195] Among them, γ sim is the archive video similarity data, N is the number of features, P i t is the i-th color feature of the archive video feature data at the t-th timestamp, P i t-1 is the i-th color feature of the archive video feature data at the t-1th timestamp, is the i-th position feature of the archive video feature data at the t-th timestamp, is the i-th position feature of the archive video feature data at the t-1th timestamp, α c and α l The calculation weight for the archive video similarity data;

[0196] α of open classes and surveillance videos c and α lDifferent values ​​were used, 0.3 and 0.7 for open classes, and 0.5 and 0.5 for surveillance videos. The corresponding simulation data were calculated 30 times, and the average was taken after removing the highest and lowest values ​​for comparison with the actual data. The actual data with different weights all used the same data; see Table 2 and Table 3:

[0197] Table 2α c and α l Comparison of similarity values ​​calculated with different weights and actual similarity values ​​in open courses

[0198] <![CDATA[α c ]]> <![CDATA[α l ]]> Calculated similarity Actual similarity Difference 0.7 0.3 90.2％ 93.2％ 3.0％ 0.6 0.4 91.8％ 93.2％ 1.5％ 0.5 0.5 92.2％ 93.2％ 1.0％ 0.4 0.6 92.6％ 93.2％ 0.6％ 0.3 0.7 92.8％ 93.2％ 0.4％ 0.2 0.85 92.3％ 93.2％ 0.9％

[0199] Table 3α c and α l Comparison of similarity values ​​calculated with different weights of surveillance videos and actual similarity values

[0200] <![CDATA[α c ]]> <![CDATA[α l ]]> Calculated similarity Actual similarity Difference 0.7 0.3 94.7％ 96.9％ 2.2％ 0.6 0.4 95.0％ 96.9％ 1.9％ 0.5 0.5 96.1％ 96.9％ 0.8％ 0.4 0.6 95.9％ 96.9％ 1.0％ 0.3 0.7 94.3％ 96.9％ 2.6％

[0201] The archive video data similarity marking unit marks the pre-stored video archive according to the archive video similarity data to generate marked pre-stored video archive data;

[0202] The archive video feature output unit outputs and stores the archive video feature data sorted in chronological order;

[0203] The archive video similarity output unit outputs the marked pre-stored video archive data and stores it;

[0204] The segmentation points are generated based on the file continuity similarity and the length of the video file feature data as segmentation judgment;

[0205] Furthermore, the timestamp of the first frame of the video is used as the start timestamp; the segmentation point is determined by the file continuity similarity and the length of the video file feature data;

[0206] If the file continuity similarity is less than the continuity threshold, the timestamp corresponding to the file continuity similarity being less than the continuity threshold is used as the segmentation point; if the length of the video file feature data exceeds the segmentation continuity threshold, the timestamp corresponding to the segmentation continuity threshold length is used as the segmentation point;

[0207] Since it contains a variety of data, the continuity threshold is set by default to 90% for campus conference videos, 85% for open classes, and 95% for surveillance videos after combining a large amount of data with expert experience. The specific situation can be modified according to the actual needs and conditions of each campus.

[0208] The continuity threshold is set according to the frame rate standard of each video file. For example, if the open class has 30 frames, the continuity threshold of the open class is 30, and if the monitoring video has 60 frames, the continuity threshold of the monitoring video is 60.

[0209] The first segmentation module of the smart campus video archive segments the pre-stored video archive by segmentation points, obtains the video archive feature data corresponding to the multiple segmented video archives, and obtains multiple first key frame video archive features by combining the archive continuity similarity calculation;

[0210] Furthermore, the first key frame video file feature is to obtain the frame number of the pre-stored video file and the segmented video file feature data;

[0211] Calculate based on the similarity and frame number of the video file feature data, and obtain the positions of multiple first key frame video file features from the video file feature data:

[0212]

[0213] Among them, η F is the position of the first key frame video file feature, φ F and φ V is the preselected position, φ video is the number of frames; To calculate and The position closest to the value, k n The number of similarities calculated for the preselected positions, is the similarity of the feature data of the i-th video file, k n The similarity of the pre-selected positions, min is the minimum value and max is the maximum value;

[0214] In the specific implementation process, regardless of the video type, 20 is the minimum number of frames that the human eye can observe in a continuous video. Therefore, if the number of frames of open classes, surveillance videos, and campus conference videos is less than 3 after calculation, each frame is the first key frame video file;

[0215] The second segmentation module of the smart campus video archive obtains the video archive feature data corresponding to the segmentation point, combines the archive continuity similarity mean of the video archive feature data after adjacent segmentation, and generates multiple second key frame video archive features; the second key frame video archive feature is obtained to obtain the video archive feature data of the segmentation point A for storage; the archive continuity similarity mean of the similarity range (M-1, M) and (M, M+1) of the segmentation point is obtained for storage; wherein M is the segmentation point, M-1 is the previous segmentation point with the smallest number of frames from M, and M+1 is the next segmentation point with the smallest number of frames from M.

[0216] The smart campus video archive storage module stores the pre-stored video archives after segmentation into the target storage node;

[0217] The smart campus primary feature backup module stores the segmented video archive feature data into the target storage node;

[0218] The smart campus secondary feature backup module stores the first key frame video file feature and the second key frame video file feature into the target storage node;

[0219] The smart campus archive storage module, the smart campus primary feature backup module and the smart campus secondary feature backup module also include:

[0220] The smart campus archive storage module stores similarity data of video archive feature data corresponding to the pre-stored video archive; the smart campus primary feature backup module and the smart campus secondary feature backup module store similarity data of the corresponding video archive feature data;

[0221] The smart campus primary feature backup module and the smart campus secondary feature backup module restore feature data through the archive video feature restoration unit of the smart campus video archive feature model.

[0222] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A file management system based on distributed storage, characterized in that: include: The smart campus video archive acquisition module obtains the archive type of the pre-stored video archive and matches multiple corresponding storage nodes, screens the storage nodes, and generates a target storage node; The smart campus video archive segmentation judgment module extracts multiple image archive data of the pre-stored video archive to obtain video archive feature data, calculates the archive continuity similarity of the video archive feature data; uses the archive continuity similarity and the length of the video archive feature data as segmentation judgment to generate segmentation points; The first segmentation module of the smart campus video archive segments the pre-stored video archive by the segmentation point, obtains the video archive feature data corresponding to the multiple segmented video archives, and obtains multiple first key frame video archive features by combining the archive continuity similarity calculation; The second segmentation module of the smart campus video archive obtains the video archive feature data corresponding to the segmentation point, combines the archive continuity similarity mean of the video archive feature data after adjacent segmentation, and generates a plurality of second key frame video archive features; A smart campus video archive storage module stores the pre-stored video archive after segmentation into the target storage node; A smart campus primary feature backup module stores the segmented video archive feature data into the target storage node; The smart campus secondary feature backup module stores the first key frame video file feature and the second key frame video file feature into the target storage node.

2. The archive management system based on distributed storage according to claim 1, characterized in that: The storage node corresponds to an archive type, and the storage nodes of the same archive type include multiple groups, each group contains multiple storage nodes; the archive types include personal video archives, laboratory video archives, open class video archives, monitoring video archives, activity video archives, conference video archives, temporary video archives and unclassified video archives.

3. The archive management system based on distributed storage according to claim 2 is characterized in that: The number of the multiple groups of storage nodes of the same archive type is at least three, each group of storage nodes is divided according to functional type, and the multiple groups of storage nodes include a video original archive storage group, a video feature archive backup group and a video feature archive restoration group.

4. The archive management system based on distributed storage according to claim 1, characterized in that: The segmentation judgment based on the file continuity similarity and the length of the video file feature data includes: The timestamp of the first frame of the video is used as the start timestamp; the segmentation point is determined by the continuity similarity of the file and the length of the characteristic data of the video file; If the file continuity similarity is less than the continuity threshold, the timestamp corresponding to the file continuity similarity being less than the continuity threshold is used as the segmentation point; if the length of the video file feature data exceeds the segmentation continuity threshold, the timestamp corresponding to the segmentation continuity threshold length is used as the segmentation point; The next start timestamp is determined with the segmentation point as the start timestamp until the entire video is traversed.

5. The archive management system based on distributed storage according to claim 1 is characterized in that: The video archive feature data and the archive continuity similarity are judged by a smart campus video archive feature model, including: The smart campus video archive feature model includes an archive video data input unit, an archive video data feature extraction unit, an archive video data similarity calculation unit, an archive video data similarity marking unit, an archive video feature output unit, an archive video similarity output unit and an archive video feature restoration unit; The archive video data input unit classifies and marks the archive types of the pre-stored video archives, and pre-processes the pre-stored video archives to generate archive video input data and archive video type marks; The archive video data feature extraction unit calls the corresponding video feature extraction module according to the archive video type tag, extracts the archive video input data features through the video feature extraction module, and generates archive video feature data; The archive video data similarity calculation unit sorts all the archive video feature data in chronological order, and sequentially calculates the similarities of adjacent archive video feature data to generate archive video similarity data; The archive video data similarity marking unit marks the pre-stored video archive according to the archive video similarity data to generate marked pre-stored video archive data; The archive video feature output unit outputs and stores the archive video feature data sorted in the time sequence; The archive video similarity output unit outputs and stores the marked pre-stored video archive data; The archive video feature restoration unit restores the archive video feature data.

6. The archive management system based on distributed storage according to claim 5, characterized in that: Marking the pre-stored video archive according to the archive video similarity data includes: Marking the archive video similarity data for each timestamp of the pre-stored video archive; If the archive video similarity data of the t-th timestamp is less than the archive continuity similarity, a first mark is performed; Calculate the difference θ between the similarity data of the archive video at the tth timestamp and the similarity data of the archive video at the t+1th timestamp t1 , calculate the difference θ between the similarity data of the archive video at the t-th timestamp and the similarity data of the archive video at the t-1th timestamp t2 ; If θ t1 and / or θ t2 If it is greater than the file similarity difference, a second mark is performed.

7. The archive management system based on distributed storage according to claim 5, characterized in that: The archive video similarity data calculation includes: Among them, γ sim is the similarity data of the archive video, N is the number of features, P i t is the i-th color feature of the archive video feature data at the t-th timestamp, P i t-1 is the i-th color feature of the archive video feature data at the t-1-th timestamp, is the i-th position feature of the archive video feature data at the t-th timestamp, is the i-th position feature of the archive video feature data at the t-1th timestamp, α c and α l is the calculation weight of the archive video similarity data.

8. The archive management system based on distributed storage according to claim 1, characterized in that: The first key frame video file feature acquisition includes: Obtaining the number of frames of the pre-stored video file and feature data of the segmented video file; Calculate based on the similarity of the video file feature data and the number of frames, and obtain the positions of the plurality of first key frame video file features from the video file feature data: Among them, η F is the position of the first key frame video file feature, φ F and φ V is the preselected position, φ video is the frame number; To calculate and The position closest to the value, k n the number of said similarities calculated for the preselected positions, is the similarity of the feature data of the i-th video archive, k n The similarity of the pre-selected positions, min is the minimum value and max is the maximum value.

9. The archive management system based on distributed storage according to claim 1, characterized in that: The second key frame video file feature is obtained as follows: Acquire the video file feature data of the segmentation point for storage; Obtain the mean of the file continuity similarity of the similarity range (M-1, M) and (M, M+1) of the segmentation point for storage; wherein M is the segmentation point, M-1 is the previous segmentation point with the smallest number of M frames away, and M+1 is the next segmentation point with the smallest number of M frames away.

10. The archive management system based on distributed storage according to claim 5, characterized in that: The smart campus archive storage module, the smart campus primary feature backup module and the smart campus secondary feature backup module also include: The smart campus archive storage module stores similarity data of video archive feature data corresponding to the pre-stored video archive; the smart campus primary feature backup module and the smart campus secondary feature backup module store similarity data of the corresponding video archive feature data; The smart campus primary feature backup module and the smart campus secondary feature backup module perform feature data restoration through the archive video feature restoration unit of the smart campus video archive feature model.

Images