A video network asset identity credible identification method and system
Through the combination of graph neural network and deep belief network, the complementarity of multimodal data is used to generate a comprehensive feature matrix, which solves the problem of insufficient accuracy and robustness of video network asset identity recognition in the existing technology, and realizes the complex environment Efficient and reliable authentication under this method enhances the security and adaptability of the system.
Patent Information
- Application Number
- CN202411298135.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-18
AI Technical Summary
The existing video network asset identity recognition technology relies on single modal data, ignores the complementarity and correlation of multimodal data, resulting in a decrease in recognition accuracy, and it is difficult to provide robust identification results under complex environments and noise interference, and lacks effective Optimization mechanism leads to insufficient model adaptability and generalization capabilities.
Graph neural network is used to aggregate multi-dimensional feature of video, audio, sensor and text data to generate a comprehensive feature matrix, generate identity authentication feature codes through hashing operations and encryption processing, build a deep belief network model and optimize historical data to realize online Identify and verify.
Maintain efficient identification accuracy and robustness in complex environments, ensure accurate and reliable verification of the identity of the video network asset, enhance the security and adaptability of the system, and can trigger alarms immediately and record abnormal information to prevent unauthorized content transmission or storage .
Smart Images

Figure CN118821029B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of asset identification, and in particular to a video network asset identity credible identification method and system. Background Art
[0002] With the rapid development of video surveillance systems and multimedia technologies, video network assets have been widely used in various fields. These video assets not only contain a large amount of visual information, but may also be accompanied by audio, sensor data and text information, forming a collection of multimodal data. Especially in the fields of security, traffic management and smart cities, the trusted identity of video assets has become particularly important, which is directly related to the security of information and the reliability of the system. Therefore, how to effectively identify and verify the identity of video network assets to ensure their authenticity and security is an important research direction in the current technology field.
[0003] However, the existing video network asset identification technology has certain shortcomings. Traditional methods often rely on single modal data (such as video images or audio), while ignoring the complementarity and correlation between multimodal data, which leads to a decrease in recognition accuracy. In addition, when dealing with complex environments and dynamic scenes, existing methods are susceptible to interference from noise and data redundancy, and it is difficult to provide robust identification results. At the same time, the existing recognition model lacks an effective optimization mechanism when processing data, and cannot make full use of historical data for continuous optimization and improvement of the model, resulting in insufficient adaptability and generalization of the model.
[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and to propose a video network asset identity trusted identification method and system, which can maintain high recognition accuracy and robustness in a complex and changing environment, and ultimately achieve accurate and reliable verification and monitoring of the identity of video network assets. Summary of the invention
[0005] Based on the above objectives, the present invention provides a method and system for credible identification of video network asset identities.
[0006] A method for credible identification of video network asset identities comprises the following steps:
[0007] S1, video asset acquisition and preprocessing: the video data of the target video asset is acquired through the acquisition device of the video network, and the acquired video data is preprocessed, including denoising, format conversion and frame sequence arrangement;
[0008] S2, multimodal data acquisition: acquiring multimodal data related to video assets, including audio data, sensor data, and text data;
[0009] S3, multi-dimensional feature aggregation: Use graph neural network (GNN) to perform multi-dimensional feature aggregation on video data and multimodal data to generate a comprehensive feature matrix as the identity feature of video assets;
[0010] S4, identity authentication feature generation: performing hash operation and encryption processing on the comprehensive feature matrix to generate an identity authentication feature code for the video asset;
[0011] S5, model construction and optimization: Based on the generated identity authentication feature code, a video network asset identity trust recognition model is constructed, and the video network asset identity trust recognition model is optimized through historical video data, including:
[0012] S51, model construction: based on the generated identity authentication feature code, a video network asset identity trustworthy identification model is constructed to identify and distinguish different video asset identity features;
[0013] S52, historical data integration: collecting and integrating stored historical video data and its corresponding identity authentication feature codes;
[0014] S53, continuous optimization: Based on historical video data, the parameters of the video network asset identity trust recognition model are optimized and adjusted through optimization algorithms;
[0015] S6, online identification and verification: During the transmission or storage of video assets, the comprehensive feature matrix of the target video is extracted in real time and compared with the identity authentication feature code in the video network asset identity trustworthy identification model. When the identification result matches the pre-stored video network asset identity information, the identity of the video asset is confirmed to be trustworthy. Otherwise, the alarm mechanism is triggered and the abnormal information is recorded.
[0016] Optionally, the video asset acquisition and preprocessing in S1 includes:
[0017] S11, video data acquisition: acquiring video data of target video assets in real time through acquisition devices of the video network, including cameras and sensors;
[0018] S12, video data denoising: performing denoising processing on video frames in the video data by using a Gaussian filter;
[0019] S13, format conversion: converting the format of the denoised video data;
[0020] S14, frame sequence arrangement: arrange the video frames in sequence by interpolation method and perform frame rate standardization.
[0021] Optionally, the multimodal data collection in S2 includes:
[0022] S21, audio data acquisition: audio data related to the target video asset is collected in real time through a microphone or an audio sensor synchronized with a video acquisition device;
[0023] S22, sensor data acquisition: acquiring environmental data and device status data through various sensor devices deployed at the video acquisition site, including temperature sensors, humidity sensors, accelerometers, and light sensors;
[0024] S23, text data acquisition: acquiring text information related to video acquisition, including real-time subtitles, metadata tags, and operation logs.
[0025] Optionally, the multi-dimensional feature aggregation in S3 includes:
[0026] S31, multimodal feature representation and standardization: extract features from video data, audio data, sensor data, and text data, and standardize the extracted features, including:
[0027] S311, video feature representation: Use convolutional neural network (CNN) to perform convolution operation on each frame of video data to extract spatial features;
[0028] S312, Audio Feature Representation: Use Mel Frequency Cepstral Coefficients (MFCC) to represent audio signals Convert to Mel frequency cepstral coefficients;
[0029] S313, Sensor feature representation: Use Fourier transform (FFT) to analyze sensor data Perform fast Fourier transform to extract frequency domain features;
[0030] S314, Text Feature Representation: Using word vector embedding to embed text data Convert to word vector representation through pre-trained word vector model;
[0031] S315, feature standardization: standardize all extracted features;
[0032] S32, Graph Structure Construction and Feature Association Modeling: Mapping Multimodal Features to Graph Structure In which the node represents the standardized eigenvector of each mode, Represents the association between features and the weight of the edge It is calculated by the cosine similarity between feature vectors;
[0033] S33, graph neural network aggregation: a multi-layer graph neural network (GNN) is used to propagate and aggregate the features in the graph structure. Through the feature propagation mechanism of each layer, the information from neighboring nodes is gradually integrated to update the feature representation of the node, which is expressed as:
[0034] ;
[0035] in, It is Node in layer The characteristic vector of Is a node The set of neighbor nodes of It is the edge The weight of is the normalization constant, and It is The learnable weight matrix of the layer, is the ReLU activation function;
[0036] S34, multi-layer feature fusion: In the last layer of the graph neural network (GNN), the feature representations of all nodes are aggregated to generate a comprehensive feature matrix , each row corresponds to the identity feature of a video asset, expressed as:
[0037] ;
[0038] in, is the number of layers of the graph neural network (GNN), To concatenate the final feature vectors of all nodes to form a comprehensive feature matrix.
[0039] Optionally, the identity authentication feature generation in S4 includes:
[0040] S41, hash operation: Use the SHA-256 algorithm to aggregate the comprehensive feature matrix generated by multi-dimensional features. Perform hash operations;
[0041] S42, encryption processing: the feature summary generated by hash operation Perform encryption processing;
[0042] S43, identity authentication feature code generation: the final generated video asset identity authentication feature code It is the encrypted hash value.
[0043] Optionally, the video network asset identity trustworthy identification model in S51 adopts a deep belief network (DBN) model, and the deep belief network (DBN) model includes:
[0044] S511, Multimodal Data Preprocessing and Representation: Standardize and extract features from video, audio, sensor, and text data respectively;
[0045] S512, restricted Boltzmann machine layer modeling: introduce multimodal data fusion mechanism, take each modal data as the input of the visible layer, and adaptively adjust the weight of the hidden layer to reflect the correlation between multimodal data, expressed as:
[0046] ;
[0047] in, and Represent the states of the visible layer and hidden layer units respectively, and are the biases of the visible layer and the hidden layer respectively, is the weight matrix in the standard restricted Boltzmann machine (RBM), is the newly added multimodal correlation coefficient;
[0048] S513, layer-by-layer pre-training and parameter optimization: A greedy algorithm is used to pre-train each layer of the restricted Boltzmann machine (RBM) layer by layer. After each layer is trained, the output of the layer is used as the input of the next layer of the restricted Boltzmann machine (RBM), and the weight is updated by the contrast divergence algorithm, which is expressed as:
[0049] ;
[0050] in, is the learning rate, is the expected value based on actual data, is the expected value based on the reconstructed data;
[0051] S514, multimodal feature fusion and deep feature representation: After completing the pre-training of all restricted Boltzmann machine (RBM) layers, the outputs of each layer are fused to form a unified deep feature representation, expressed as:
[0052] ;
[0053] in, is the weight matrix of the fully connected layer, Respectively The first RBM The hidden layer output, is the ReLU activation function, is the bias of the fully connected layer;
[0054] S515, identity authentication feature generation and classification: Use the generated deep feature representation for classification to generate an identity authentication feature code for the video asset, which is represented as:
[0055] ;
[0056] in, Is the feature belonging to the category The probability of and are the weights and biases of the softmax layer, and For the Class weights and biases, Represented as transposed;
[0057] S516, model fine-tuning and overall optimization: use the back propagation algorithm to optimize the overall network parameters, expressed as:
[0058] ;
[0059] in, is the actual label, is the predicted probability.
[0060] Optionally, the historical data integration in S52 includes:
[0061] S521, historical data collection: extracting stored historical video data from the video network asset database, including original video files and their corresponding multimodal data;
[0062] S522, identity authentication feature code acquisition: extracting the identity authentication feature code corresponding to each historical video data from the video network asset database;
[0063] S523, data integration: Integrate the collected historical video data and its corresponding identity authentication feature code, and perform correlation matching through timestamps.
[0064] Optionally, the continuous optimization in S53 includes:
[0065] S531, optimizing based on historical video data: collecting and using stored historical video data and its corresponding identity authentication feature codes as training sets and verification sets to optimize the video network asset identity trust recognition model;
[0066] S532, optimization algorithm application: Adam optimizer is used to adjust the model parameters, expressed as:
[0067] ;
[0068] ;
[0069] ;
[0070] ;
[0071] in, For the The model parameters at the iteration, is the current parameter The gradient of the loss function under and are the moving averages of the first and second moments of the gradient, and is the momentum parameter, is the learning rate, is a small constant.
[0072] Optionally, the online identification and verification in S6 includes:
[0073] S61, real-time feature extraction: extracting a comprehensive feature matrix of a target video in real time during the transmission or storage of the video asset;
[0074] S62, feature comparison: using Euclidean distance to compare the comprehensive feature matrix extracted in real time with the identity authentication feature code pre-stored in the video network asset identity trustworthy identification model;
[0075] S63, identity authentication and exception handling: When the result of feature comparison shows that the feature matrix extracted in real time matches the pre-stored identity authentication feature code, the identity of the video asset is confirmed to be credible and normal transmission or storage is allowed. If the comparison result does not match, the alarm mechanism is triggered and the exception information is recorded, including time, location, and device ID.
[0076] A video network asset identity trustworthy identification system, used to implement the above-mentioned video network asset identity trustworthy identification method, includes the following modules:
[0077] Video asset acquisition and preprocessing module: acquires the video data of the target video asset through the acquisition device of the video network, and preprocesses the acquired video data, including denoising, format conversion and frame sequence arrangement;
[0078] Multimodal data acquisition module: acquires multimodal data related to video assets, including audio data, sensor data, and text data;
[0079] Multi-dimensional feature aggregation module: Use graph neural network to aggregate multi-dimensional features of video data and multimodal data to generate a comprehensive feature matrix as the identity feature of video assets;
[0080] Identity authentication feature generation module: performs hash operations and encryption on the comprehensive feature matrix to generate identity authentication feature codes for video assets;
[0081] Model building and optimization module: Based on the generated identity authentication feature code, a video network asset identity trust recognition model is built, and the model is continuously optimized using historical video data;
[0082] Online identification and verification module: During the transmission or storage of video assets, the comprehensive feature matrix of the target video is extracted in real time and compared with the pre-stored video network asset identity authentication feature code to verify the identity credibility of the video asset. If the identification result does not match, the alarm mechanism is triggered and the abnormal information is recorded.
[0083] Beneficial effects of the present invention:
[0084] The present invention, through video asset acquisition and preprocessing, multimodal data collection and multi-dimensional feature aggregation, fully utilizes the complementarity of video, audio, sensor and text data, generates a comprehensive feature matrix as the identity feature of the video asset, and models and integrates the deep correlation of multimodal data through graph neural network, which effectively improves the expressiveness and robustness of the identity authentication feature, ensuring that the identity of the video asset can be accurately identified in complex and changing environments.
[0085] The present invention, by adopting a deep belief network model, combined with layer-by-layer pre-training, multimodal data fusion and deep feature representation, realizes efficient recognition of the identity of video network assets. The continuous optimization based on historical video data further improves the recognition accuracy and stability of the model, so that the model can maintain high recognition accuracy and adaptability in the face of diversified application scenarios and complex data environments.
[0086] The present invention, through an online identification and verification mechanism, extracts a comprehensive feature matrix of video assets in real time, and compares it with a pre-stored identity authentication feature code, thereby ensuring the identity credibility of video assets during transmission or storage. When an abnormality is found, the system can immediately trigger an alarm and record abnormal information, thereby enhancing the security and reliability of video network assets, preventing unauthorized content transmission or storage, and helping to maintain the overall security and stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0087] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0088] Figure 1 A schematic diagram of a recognition method flow chart of an embodiment of the present invention;
[0089] Figure 2Schematic diagram of system function modules according to an embodiment of the present invention. DETAILED DESCRIPTION
[0090] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.
[0091] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).
[0092] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.
[0093] like Figure 1 As shown, a method for reliable identification of video network asset identities includes the following steps:
[0094] S1, video asset acquisition and preprocessing: The video data of the target video asset is acquired through the acquisition equipment of the video network, and the acquired video data is preprocessed, including denoising, format conversion and frame sequence arrangement, to ensure the integrity and standardization of the video data;
[0095] S2, multimodal data acquisition: Acquire multimodal data related to video assets, including audio data, sensor data, and text data, to provide more information dimensions for subsequent feature extraction;
[0096] S3, multi-dimensional feature aggregation: Use graph neural network (GNN) to perform multi-dimensional feature aggregation on video data and multimodal data to generate a comprehensive feature matrix as the identity feature of video assets;
[0097] S4, identity authentication feature generation: Hash operation and encryption processing are performed on the comprehensive feature matrix to generate the identity authentication feature code of the video asset to ensure its uniqueness, tamper resistance and robustness in different application scenarios;
[0098] S5, model construction and optimization: Based on the generated identity authentication feature code, a video network asset identity trust recognition model is constructed, and the video network asset identity trust recognition model is optimized through historical video data to improve the accuracy and reliability of the model, including:
[0099] S51, model construction: based on the generated identity authentication feature code, a video network asset identity trustworthy identification model is constructed to identify and distinguish different video asset identity features;
[0100] S52, historical data integration: collect and integrate the stored historical video data and its corresponding identity authentication feature codes for model training to enhance the model's recognition capabilities for different scenes and diverse video assets;
[0101] S53, continuous optimization: Based on historical video data, the optimization algorithm is used to optimize and adjust the parameters of the video network asset identity trust recognition model, continuously improving the recognition accuracy and robustness of the model, and ensuring that the model can maintain efficiency and accuracy in complex and dynamic application environments;
[0102] S6, online identification and verification: During the transmission or storage of video assets, the comprehensive feature matrix of the target video is extracted in real time, and compared with the identity authentication feature code in the video network asset identity trustworthy identification model. When the identification result matches the pre-stored video network asset identity information, the identity of the video asset is confirmed to be trustworthy. Otherwise, the alarm mechanism is triggered and the abnormal information is recorded;
[0103] Through the above steps, effective fusion of multimodal data and efficient identity feature generation and verification are achieved, ensuring the accuracy and security of video asset recognition in complex and dynamic environments. At the same time, through the integration and continuous optimization of historical data, the robustness and adaptability of the model are improved, which can meet the needs of video asset recognition in different scenarios.
[0104] Video asset acquisition and preprocessing in S1 include:
[0105] S11, video data acquisition: acquiring video data of target video assets in real time through acquisition devices of the video network, including cameras and sensors;
[0106] S12, video data denoising: denoising the video frames in the video data through a Gaussian filter to reduce sensor noise and environmental interference, expressed as:
[0107] ;
[0108] in, represents the weight of the filter, and is the image coordinate, is the standard deviation of the filter;
[0109] S13, format conversion: converting the format of the denoised video data to facilitate subsequent feature extraction and processing;
[0110] S14, frame sequence arrangement: the video frames are arranged in sequence by interpolation method to ensure that the video frames are correctly arranged in time sequence and the frame rate is standardized, which is expressed as:
[0111] ;
[0112] in, is the interpolated video frame, is the original frame rate of the video frame, is the target frame rate, is the original video frame;
[0113] Through the above content, the temporal order and frame rate consistency of the video frames are ensured, which improves the availability and processing efficiency of the video data. These steps ensure the standardization and consistency of the video data, and provide a solid foundation for the subsequent identity authentication feature generation and model construction.
[0114] Multimodal data acquisition in S2 includes:
[0115] S21, audio data acquisition: audio data related to the target video asset is collected in real time through a microphone or an audio sensor synchronized with a video acquisition device;
[0116] S22, sensor data acquisition: acquiring environmental data and device status data through various sensor devices deployed at the video acquisition site, including temperature sensors, humidity sensors, accelerometers, and light sensors;
[0117] S23, text data acquisition: acquiring text information related to video acquisition, including real-time subtitles, metadata tags, and operation logs;
[0118] Through the above content, the efficient integration of various types of data is ensured, providing more comprehensive and reliable input for subsequent feature extraction and identity recognition, improving the recognition accuracy and robustness of the system in complex scenarios, and being able to better adapt to the challenges in diverse application environments.
[0119] Multidimensional feature aggregation in S3 includes:
[0120] S31, Multimodal Feature Representation and Standardization: Extract features from video data, audio data, sensor data, and text data respectively, and standardize the extracted features to ensure that the features of different modalities are within the same scale range, so that they are processed more consistently in the graph neural network. Specifically, it includes:
[0121] S311, video feature representation: Use a convolutional neural network (CNN) to perform a convolution operation on each frame of the video data to extract spatial features, which are expressed as:
[0122] ;
[0123] in, is the convolution kernel weight matrix, is the bias term, is the activation function, is a video frame;
[0124] S312, Audio Feature Representation: Use Mel Frequency Cepstral Coefficients (MFCC) to represent audio signals Converted to Mel frequency cepstral coefficients, expressed as:
[0125] Short-time Fourier transform (STFT): ;
[0126] in, is the audio signal, is the window function, is the frame shift amount, is the frequency index, is the number of FFT points;
[0127] Mel filter bank transform: ;
[0128] in, is a Mel filter bank;
[0129] Logarithmic transformation: ;
[0130] Discrete Cosine Transform (DCT): ;
[0131] S313, Sensor feature representation: Use Fourier transform (FFT) to analyze sensor data Perform fast Fourier transform and extract frequency domain features, expressed as:
[0132] ;
[0133] in, is the time domain data of the sensor, is the frequency domain feature, is the frequency index, is the signal length;
[0134] S314, Text Feature Representation: Using word vector embedding to embed text data The pre-trained word vector model is converted into a word vector representation, expressed as:
[0135] ;
[0136] in, The function converts each word Mapped into a vector space of fixed dimension;
[0137] S315, feature standardization: All extracted features are standardized to ensure that different modal features are within the same scale range, expressed as:
[0138] ;
[0139] in, is the feature vector, is the mean of the feature, is the standard deviation of the feature;
[0140] S32, Graph Structure Construction and Feature Association Modeling: Mapping Multimodal Features to Graph Structure In which the node represents the standardized eigenvector of each mode, Represents the association between features and the weight of the edge It is calculated by the cosine similarity between feature vectors and is expressed as:
[0141] ;
[0142] S33, graph neural network aggregation: a multi-layer graph neural network (GNN) is used to propagate and aggregate the features in the graph structure. Through the feature propagation mechanism of each layer, the information from neighboring nodes is gradually integrated to update the feature representation of the node, which is expressed as:
[0143] ;
[0144] in, It is Node in layer The characteristic vector of Is a node The set of neighbor nodes of It is the edge The weight of is the normalization constant, and It is The learnable weight matrix of the layer, is the ReLU activation function;
[0145] S34, multi-layer feature fusion: In the last layer of the graph neural network (GNN), the feature representations of all nodes are aggregated to generate a comprehensive feature matrix , each row corresponds to the identity feature of a video asset, expressed as:
[0146] ;
[0147] in, is the number of layers of the graph neural network (GNN), To concatenate the final feature vectors of all nodes to form a comprehensive feature matrix;
[0148] Through the above content, we can not only fully utilize the complementarity of video, audio, sensor and text data, but also capture the deep connection between each modality, improve the expressiveness and recognition accuracy of comprehensive features, enhance the robustness and accuracy of the system in complex and changing environments, and provide more comprehensive and accurate feature support for the identity recognition of video network assets.
[0149] The authentication feature generation in S4 includes:
[0150] S41, hash operation: Use the SHA-256 algorithm to aggregate the comprehensive feature matrix generated by multi-dimensional features. A hash operation is performed to generate a fixed-length feature summary, expressed as:
[0151] ;
[0152] in, It is a feature summary after hash operation, has a fixed binary length, and can uniquely identify the comprehensive feature matrix of the input ;
[0153] S42, encryption processing: the feature summary generated by hash operation Encryption processing is performed to ensure the security of the identity authentication feature code, which is expressed as:
[0154] ;
[0155] in, It is the encrypted feature code. Represents an encryption operation using a public key;
[0156] S43, identity authentication feature code generation: the final generated video asset identity authentication feature code It is an encrypted hash value that is unique, tamper-resistant, and highly secure, and can be used for subsequent video asset authentication and identification.
[0157] Through the above content, not only can the identity authentication feature code be generated quickly, but also the identity information of video assets can be protected in a complex network environment, thereby improving the security and reliability of video network asset identification.
[0158] The video network asset identity trust recognition model in S51 adopts the deep belief network (DBN) model, which includes:
[0159] S511, Multimodal Data Preprocessing and Representation: Standardize and extract features of video, audio, sensor, and text data to ensure that data of different modalities are within the same scale range, represented as:
[0160] ;
[0161] in, Represents each modal data, and are the mean and standard deviation of the modal data respectively;
[0162] S512, restricted Boltzmann machine layer modeling: introduce multimodal data fusion mechanism, take each modal data as the input of the visible layer, and adaptively adjust the weight of the hidden layer to reflect the correlation between multimodal data, expressed as:
[0163] ;
[0164] in, and Represent the states of the visible layer and hidden layer units respectively, and are the biases of the visible layer and the hidden layer respectively, is the weight matrix in the standard restricted Boltzmann machine (RBM), It is a newly added multimodal correlation coefficient, which is used to reflect the dependency relationship between multimodal data;
[0165] S513, layer-by-layer pre-training and parameter optimization: A greedy algorithm is used to pre-train each layer of the restricted Boltzmann machine (RBM) layer by layer. After each layer is trained, the output of the layer is used as the input of the next layer of the restricted Boltzmann machine (RBM), and the weight is updated by the contrast divergence algorithm, which is expressed as:
[0166] ;
[0167] in, is the learning rate, is the expected value based on actual data, is the expected value based on the reconstructed data;
[0168] S514, multimodal feature fusion and deep feature representation: After completing the pre-training of all restricted Boltzmann machine (RBM) layers, the outputs of each layer are fused to form a unified deep feature representation, expressed as:
[0169] ;
[0170] in, is the weight matrix of the fully connected layer, Respectively The first RBM The hidden layer output, is the ReLU activation function, is the bias of the fully connected layer;
[0171] S515, identity authentication feature generation and classification: Use the generated deep feature representation for classification to generate an identity authentication feature code for the video asset, which is represented as:
[0172] ;
[0173] in, Is the feature belonging to the category The probability of and are the weights and biases of the softmax layer, and For the Class weights and biases, Represented as transposed;
[0174] S516, model fine-tuning and overall optimization: Use the back propagation algorithm to optimize the overall network parameters to ensure that the parameters of each layer are coordinated and consistent, expressed as:
[0175] ;
[0176] in, is the actual label, is the predicted probability;
[0177] Through the above content, the recognition accuracy and generalization ability are improved, so that it can still maintain high-precision identity recognition capabilities in complex dynamic environments, making the DBN model particularly suitable for video network asset identity trusted recognition tasks that require processing complex data structures and multimodal inputs.
[0178] Historical data integration in S52 includes:
[0179] S521, historical data collection: extract the stored historical video data from the video network asset database, including the original video files and their corresponding multimodal data (such as audio, sensor data, text data, etc.). This data set covers video assets in different time periods and scenarios to ensure data diversity and representativeness;
[0180] S522, identity authentication feature code acquisition: extracting the identity authentication feature code corresponding to each historical video data from the video network asset database. These feature codes are generated when the video assets are first collected and processed, and reflect the unique identity characteristics of each video asset;
[0181] S523, data integration: integrating the collected historical video data and its corresponding identity authentication feature code, and performing correlation matching through timestamps;
[0182] Through the above content, the recognition accuracy and robustness of the model in practical applications are significantly enhanced, enabling the system to more effectively cope with complex and changing application environments.
[0183] Ongoing improvements in S53 include:
[0184] S531, optimizing based on historical video data: collecting and using stored historical video data and its corresponding identity authentication feature codes as training sets and verification sets to optimize the video network asset identity trust recognition model;
[0185] S532, optimization algorithm application: Adam optimizer is used to adjust the model parameters, expressed as:
[0186] ;
[0187] ;
[0188] ;
[0189] ;
[0190] in, For the The model parameters at the iteration, is the current parameter The gradient of the loss function under and are the moving averages of the first and second moments of the gradient, and is the momentum parameter, with values of 0.9 and 0.999, is the learning rate, is a small constant used to prevent the denominator from being zero;
[0191] Through the above content, not only the convergence of parameters is accelerated, but also the stability and robustness of the model are improved, ensuring high-precision identity recognition in complex multimodal data environments, so that the model can continuously improve its recognition ability and adapt to new data changes, thereby maintaining good performance in practical applications.
[0192] Online identification and verification in S6 includes:
[0193] S61, real-time feature extraction: extracting a comprehensive feature matrix of a target video in real time during the transmission or storage of the video asset;
[0194] S62, feature comparison: the comprehensive feature matrix extracted in real time is compared with the identity authentication feature code pre-stored in the video network asset identity trustworthy identification model using the Euclidean distance. The smaller the Euclidean distance, the more similar the two are, which is expressed as:
[0195] ;
[0196] in, It is the first part of the comprehensive feature matrix extracted in real time during the transmission or storage of video assets. eigenvalues, It is the first one of the pre-stored identity authentication feature codes. eigenvalues, is the number of features in the feature matrix or feature code;
[0197] S63, identity authentication and exception handling: When the result of feature comparison shows that the feature matrix extracted in real time matches the pre-stored identity authentication feature code, the identity of the video asset is confirmed to be credible and normal transmission or storage is allowed. If the comparison result does not match, the alarm mechanism is triggered and the exception information, including time, location, and device ID, is recorded for subsequent review and processing;
[0198] Through the above content, it is possible to quickly identify trusted video assets, and immediately trigger alarms and record abnormal information when anomalies are found, which enhances the security of video network assets, prevents unauthorized content transmission or storage, and helps maintain the overall security and stability of the system.
[0199] like Figure 2 As shown, a video network asset identity trusted identification system is used to implement the above-mentioned video network asset identity trusted identification method, including the following modules:
[0200] Video asset acquisition and preprocessing module: acquires the video data of the target video asset through the acquisition device of the video network, and preprocesses the acquired video data, including denoising, format conversion and frame sequence arrangement;
[0201] Multimodal data acquisition module: acquires multimodal data related to video assets, including audio data, sensor data, and text data;
[0202] Multi-dimensional feature aggregation module: Use graph neural network to aggregate multi-dimensional features of video data and multimodal data to generate a comprehensive feature matrix as the identity feature of video assets;
[0203] Identity authentication feature generation module: performs hash operations and encryption on the comprehensive feature matrix to generate identity authentication feature codes for video assets;
[0204] Model building and optimization module: Based on the generated identity authentication feature code, a video network asset identity trust recognition model is built, and the model is continuously optimized using historical video data;
[0205] Online identification and verification module: During the transmission or storage of video assets, the comprehensive feature matrix of the target video is extracted in real time and compared with the pre-stored video network asset identity authentication feature code to verify the identity credibility of the video asset. If the identification result does not match, the alarm mechanism is triggered and the abnormal information is recorded.
[0206] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.
[0207] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A video network asset identity trustworthy identification method, characterized in that: The following steps are involved: S1, video asset acquisition and preprocessing: the video data of the target video asset is acquired through the acquisition device of the video network, and the acquired video data is preprocessed, including denoising, format conversion and frame sequence arrangement; S2, multimodal data acquisition: acquiring multimodal data related to video assets, including audio data, sensor data, and text data; S3, multi-dimensional feature aggregation: Use graph neural network to perform multi-dimensional feature aggregation on video data and multimodal data to generate a comprehensive feature matrix as the identity feature of the video asset. The multi-dimensional feature aggregation includes: S31, multimodal feature representation and standardization: extract features from video data, audio data, sensor data, and text data, and standardize the extracted features, including: S311, video feature representation: use a convolutional neural network to perform a convolution operation on each frame of the video data to extract spatial features; S312, audio feature representation: use Mel frequency cepstral coefficients to transform the audio signal Convert to Mel frequency cepstral coefficients; S313, Sensor feature representation: Using Fourier transform to represent sensor data Perform fast Fourier transform to extract frequency domain features; S314, Text Feature Representation: Using word vector embedding to embed text data Convert to word vector representation through pre-trained word vector model; S315, feature standardization: standardize all extracted features; S32, Graph Structure Construction and Feature Association Modeling: Mapping Multimodal Features to Graph Structure In which the node represents the standardized eigenvector of each mode, Represents the association between features and the weight of the edge It is calculated by the cosine similarity between feature vectors; S33, graph neural network aggregation: A multi-layer graph neural network is used to propagate and aggregate the features in the graph structure. Through the feature propagation mechanism of each layer, the information from neighboring nodes is gradually integrated to update the feature representation of the node, which is expressed as: ; in, It is Node in layer The characteristic vector of Is a node The set of neighbor nodes of It is the edge The weight of is the normalization constant, and It is The learnable weight matrix of the layer, is the ReLU activation function; S34, multi-layer feature fusion: In the last layer of the graph neural network, the feature representations of all nodes are aggregated to generate a comprehensive feature matrix , each row corresponds to the identity feature of a video asset, expressed as: ; in, is the number of layers of the graph neural network, To concatenate the final feature vectors of all nodes to form a comprehensive feature matrix; S4, identity authentication feature generation: performing hash operation and encryption processing on the comprehensive feature matrix to generate an identity authentication feature code for the video asset; S5, model construction and optimization: Based on the generated identity authentication feature code, a video network asset identity trust recognition model is constructed, and the video network asset identity trust recognition model is optimized through historical video data, including: S51, model construction: based on the generated identity authentication feature code, a video network asset identity trustworthy identification model is constructed to identify and distinguish different video asset identity features; S52, historical data integration: collecting and integrating stored historical video data and its corresponding identity authentication feature codes; S53, continuous optimization: Based on historical video data, the parameters of the video network asset identity trust recognition model are optimized and adjusted through optimization algorithms; S6, online identification and verification: During the transmission or storage of video assets, the comprehensive feature matrix of the target video is extracted in real time and compared with the identity authentication feature code in the video network asset identity trustworthy identification model. When the identification result matches the pre-stored video network asset identity information, the identity of the video asset is confirmed to be trustworthy. Otherwise, the alarm mechanism is triggered and the abnormal information is recorded.
2. A video network asset identity trustworthy identification method according to claim 1, characterized in that: The video asset acquisition and preprocessing in S1 include: S11, video data acquisition: acquiring video data of target video assets in real time through acquisition devices of the video network, including cameras and sensors; S12, video data denoising: performing denoising processing on video frames in the video data by using a Gaussian filter; S13, format conversion: converting the format of the denoised video data; S14, frame sequence arrangement: arrange the video frames in sequence by interpolation method and perform frame rate standardization.
3. A video network asset identity trustworthy identification method according to claim 2, characterized in that: The multimodal data collection in S2 includes: S21, audio data acquisition: audio data related to the target video asset is collected in real time through a microphone or an audio sensor synchronized with a video acquisition device; S22, sensor data acquisition: acquiring environmental data and device status data through various sensor devices deployed at the video acquisition site, including temperature sensors, humidity sensors, accelerometers, and light sensors; S23, text data acquisition: acquiring text information related to video acquisition, including real-time subtitles, metadata tags, and operation logs.
4. A video network asset identity trustworthy identification method according to claim 1, characterized in that: The identity authentication feature generation in S4 includes: S41, hash operation: Use the SHA-256 algorithm to aggregate the comprehensive feature matrix generated by multi-dimensional features. Perform hash operations; S42, encryption processing: the feature summary generated by hash operation Perform encryption processing; S43, identity authentication feature code generation: the final generated video asset identity authentication feature code It is the encrypted hash value.
5. A video network asset identity trustworthy identification method according to claim 1, characterized in that: The video network asset identity trustworthy identification model in S51 adopts a deep belief network model, and the deep belief network model includes: S511, Multimodal Data Preprocessing and Representation: Standardize and extract features from video, audio, sensor, and text data respectively; S512, restricted Boltzmann machine layer modeling: introduce multimodal data fusion mechanism, take each modal data as the input of the visible layer, and adaptively adjust the weight of the hidden layer to reflect the correlation between multimodal data, expressed as: ; in, and Represent the states of the visible layer and hidden layer units respectively, and are the biases of the visible layer and the hidden layer respectively, is the weight matrix in the standard restricted Boltzmann machine, is the newly added multimodal correlation coefficient; S513, layer-by-layer pre-training and parameter optimization: A greedy algorithm is used to pre-train each layer of restricted Boltzmann machine layer by layer. After each layer is trained, the output of the layer is used as the input of the next layer of restricted Boltzmann machine, and the weight is updated by the contrast divergence algorithm, which is expressed as: ; in, is the learning rate, is the expected value based on actual data, is the expected value based on the reconstructed data; S514, multimodal feature fusion and deep feature representation: After completing the pre-training of all restricted Boltzmann machine layers, the outputs of each layer are fused to form a unified deep feature representation, which is expressed as: ; in, is the weight matrix of the fully connected layer, Respectively The first RBM The hidden layer output, is the ReLU activation function, is the bias of the fully connected layer; S515, identity authentication feature generation and classification: Use the generated deep feature representation for classification to generate an identity authentication feature code for the video asset, which is represented as: ; in, Is the feature belonging to the category The probability of and are the weights and biases of the softmax layer, and For the Class weights and biases, Represented as transposed; S516, model fine-tuning and overall optimization: use the back propagation algorithm to optimize the overall network parameters, expressed as: ; in, is the actual label, is the predicted probability.
6. A video network asset identity trustworthy identification method according to claim 5, characterized in that: The historical data integration in S52 includes: S521, historical data collection: extracting stored historical video data from the video network asset database, including original video files and their corresponding multimodal data; S522, identity authentication feature code acquisition: extracting the identity authentication feature code corresponding to each historical video data from the video network asset database; S523, data integration: Integrate the collected historical video data and its corresponding identity authentication feature code, and perform correlation matching through timestamps.
7. A video network asset identity trustworthy identification method according to claim 6, characterized in that: The continuous optimization in S53 includes: S531, optimizing based on historical video data: collecting and using stored historical video data and its corresponding identity authentication feature codes as training sets and verification sets to optimize the video network asset identity trust recognition model; S532, optimization algorithm application: Adam optimizer is used to adjust the model parameters, expressed as: ; ; ; ; in, For the The model parameters at the iteration, is the current parameter The gradient of the loss function under and are the moving averages of the first and second moments of the gradient, and is the momentum parameter, is the learning rate, is a small constant.
8. A video network asset identity trustworthy identification method according to claim 1, characterized in that: The online identification and verification in S6 includes: S61, real-time feature extraction: extracting a comprehensive feature matrix of a target video in real time during the transmission or storage of the video asset; S62, feature comparison: using Euclidean distance to compare the comprehensive feature matrix extracted in real time with the identity authentication feature code pre-stored in the video network asset identity trustworthy identification model; S63, identity authentication and exception handling: When the result of feature comparison shows that the feature matrix extracted in real time matches the pre-stored identity authentication feature code, the identity of the video asset is confirmed to be credible and normal transmission or storage is allowed. If the comparison result does not match, the alarm mechanism is triggered and the exception information is recorded, including time, location, and device ID.
9. A video network asset identity trustworthy identification system, used to implement a video network asset identity trustworthy identification method as claimed in any one of claims 1 to 8, characterized in that: Includes the following modules: Video asset acquisition and preprocessing module: acquires the video data of the target video asset through the acquisition device of the video network, and preprocesses the acquired video data, including denoising, format conversion and frame sequence arrangement; Multimodal data acquisition module: acquires multimodal data related to video assets, including audio data, sensor data, and text data; Multi-dimensional feature aggregation module: Use graph neural network to aggregate multi-dimensional features of video data and multimodal data to generate a comprehensive feature matrix as the identity feature of video assets; Identity authentication feature generation module: performs hash operations and encryption on the comprehensive feature matrix to generate identity authentication feature codes for video assets; Model building and optimization module: Based on the generated identity authentication feature code, a video network asset identity trust recognition model is built, and the model is continuously optimized using historical video data; Online identification and verification module: During the transmission or storage of video assets, the comprehensive feature matrix of the target video is extracted in real time and compared with the pre-stored video network asset identity authentication feature code to verify the identity credibility of the video asset. If the identification result does not match, the alarm mechanism is triggered and the abnormal information is recorded.
Citation Information
Cited By
Multi-dimensional credible authentication system based on data processing
CN120746804A
A multi-dimensional trusted authentication system based on data processing
CN120746804B
