Video stream online feature extraction method and device based on Gaussian Copula

Abstract

The present invention is a method and device for extracting online features of video streams based on Gaussina Copula, and belongs to the technical field of computer big data processing. The device comprises a receiving module, a storage module, a data filling module, a feature extraction module, and a parameter initialization module; the method comprises the following steps: S1: receiving collected video data, S2: receiving specific target instructions in online extraction of video, S3: parameter initialization, S4: target detection, S5: completing feature values ​​using the Gaussina Copula method, S6: correlation analysis, S7: redundant feature analysis, and S8: outputting features. The present invention utilizes the Gaussina Copula method in combination with a missing value optimization model to reduce the search space, quickly and accurately complete the missing feature data, and at the same time reduces the influence of uncertain relationships based on a three-branch decision method, thereby realizing accurate and rapid extraction of online video features.

Description

Technical Field

[0001] The present invention relates to the field of computer big data image recognition, and is particularly applicable to a video stream online feature extraction method and device based on Gaussian Copula. Background Art

[0002] In video data, image recognition is mainly about extracting features from image data; and a collection of multi-frame image data can be regarded as a high-dimensional data scene. In the era of big data, feature selection is the main method of dimensionality reduction when facing high-dimensional scenes. With the update of technology and the development of the Internet, the requirement of real-time dimensionality reduction has been proposed. Online streaming feature selection (OSFS) is a method for online real-time dimensionality reduction processing of high-dimensional streaming data. For OSFS, we believe that the number of instances or features is not fixed, but new instances or features continue to flow in over time. Existing OSFS algorithms, such as Fast-OSFS, SAOLA, OSFASW, SFS_FI, etc., are designed to solve the problem of feature selection on complete streaming data without considering scenarios with sparse features.

[0003] At the same time, the existing OSFS model is designed for complete streaming feature scenarios, that is, the target streaming features are complete and there is no missing data [1]. Especially in the scene of video image recognition, the detected target is often blocked due to relative motion, resulting in the loss of part of the data information of the detected target. On the other hand, due to equipment limitations, it is impossible to collect complete video data. Therefore, when the feature stream is full of missing data, how to accurately select streaming features is the difficulty and focus that needs to be studied at present.

[0004] In order to deal with incomplete flow features, based on the literature [2], Wu Di et al. [3] innovated an online sparse flow feature selection framework, the principle of which is to accurately estimate the missing flow features before selection. However, the error between the estimated data and the true data of the sparse flow features may cause uncertainty in the interaction between labels and sparse features. Direct feature selection with traditional ad hoc strategies may result in decreased accuracy and efficiency. Therefore, it is crucial to minimize the impact of this uncertain relationship during the selection process.

[0005] [1] Y.He, J.Dong, B.-J.Hou, Y.Wang and F.Wang, "Online Learning inVariable Feature Spaces with Mixed Data," 2021 IEEE International Conference onData Mining (ICDM), Auckland, New Zealand, 2021, pp.181-190, doi:10.1109 / ICDM51629.2021.00028.

[0006] [2] Zhao, Yuxuan and Madeleine Udell. "Missing Value Imputation for Mixed Data via Gaussian Copula." Proceedings of the 26th ACMSIGKDDInternational Conference on Knowledge Discovery&Data Mining(2019).

[0007] [3] D. Wu, Y. He, X. Luo, M. Shang, and X. Wu, "Online feature selection with capricious streaming features: A general framework," in 2019IEEE International Conference on Big Data (Big Data), 2019, pp. 683–688. Summary of the invention

[0008] In view of this, the present invention provides a method and device for online feature extraction of video streams based on Gaussian Copula, which aims to quickly and accurately supplement the missing feature data when the detected target is occluded during the video image recognition process, and realize online accurate extraction of features.

[0009] To achieve the above object, the present invention provides a video stream online feature extraction method based on Gaussian Copula, combined with Figure 1 , including the following steps:

[0010] S1: The receiving module receives the video data collected by the video acquisition device and stores it frame by frame into the video cache unit of the storage module;

[0011] S2: The feature extraction module receives an instruction for online extraction of specific target features in a video sent by the server;

[0012] S3: The parameter initialization module initializes the parameters involved in the online feature extraction process of the video stream based on Gaussian Copula;

[0013] S4: The video cache unit segments the target to be detected frame by frame using the target detection method, and stores the image of the target area to be detected in the form of a matrix to obtain a matrix to be detected. At the same time, the video cache unit makes a difference to the matrix to be detected frame by frame, and uses the difference of each element position of the matrix to be detected to establish an interpolation function for calculating the initial value of the iteration of the Gaussian copula method; wherein the element value of the matrix to be detected of the blocked part of the target to be detected is set as a missing value;

[0014] S5: The data filling module uses the Gaussian copula method to fill the missing values ​​in the test matrix obtained in step S4;

[0015] S6: The correlation analysis unit in the feature extraction module performs correlation analysis on the filled matrix row by row to calculate the matrix features of the current video data;

[0016] S7: The redundancy analysis unit in the feature extraction module analyzes the matrix features and the historical features in the feature storage unit, screens out the redundant features, and updates the historical features;

[0017] S8: the iterative output unit in the feature extraction module repeatedly executes steps S4 to S7 until all video streams have completed the feature extraction of the target, and sends all the target features in the feature storage unit to the server;

[0018] The parameters in step S3 include Gaussian Copula method parameters and feature extraction parameters; wherein the Gaussian Copula method parameters include: the matrix size H×W corresponding to the target to be detected, the covariance matrix ∑ of the normal distribution; the feature extraction parameters include: the classification attribute C corresponding to the feature, the significance levels α, β, and the cost cost; wherein 0≤β≤α≤1;

[0019] In particular, if the element value in the matrix is ​​missing, the difference calculation of the current frame is not performed.

[0020] The Gaussian Copula method described in step S5 is specifically:

[0021] S501: For the known element x in the matrix to be tested O Processing is performed to construct a mapping relationship and obtain a normal matrix Z; specifically, it includes:

[0022] S5011: Establishing a mapping function ψ(·)=[ψ j (·)] 1≤j≤W , use the known elements of the jth column in the matrix to be tested Transform, traverse j = 1, ..., W, and obtain the normalized matrix E; where ψ j (·) is to normalize the known elements of the jth column in the matrix to be tested to the interval [0,1] according to each column;

[0023] S5012: Establishing a mapping function Φ(·)=[Φ j (·)] 1≤j≤W , transform each column in the normalized matrix E using the normal distribution N(0,∑) to obtain the normal matrix Z;

[0024] S502: Using the interpolation function of step S4 to obtain the missing value x of the matrix to be tested in the current frame M The predicted value x′ is used to establish a missing value optimization model to obtain the missing value; the optimization model is:

[0025] obj min||x M -x′||

[0026]

[0027] in, is the coefficient to be determined; Σ M,O ,Σ O,O are the submatrices consisting of all rows corresponding to missing values ​​in Σ and the submatrix consisting of all rows corresponding to non-missing values; -1 is the inverse matrix operation; E(·) is the expected function; Φ -1 (·) is the inverse mapping of Φ(·); ψ -1 (·) is the inverse mapping of ψ(·);

[0028] S503: Using likelihood estimation, combined with the missing value x calculated in step S502 M , update the covariance matrix Σ; specifically:

[0029] S5031: Update the covariance matrix Σ using likelihood estimation: = G(Σ, x) = E(z·z T |x,Σ); where x=[x O ,x M ],z=[z O ,z M ];

[0030] S5032: Calculate the approximate covariance matrix

[0031] S5033: Update the covariance matrix again

[0032] Where P = D -1 / 2 ·Σ·D -1 / 2 , D = diag(Σ), diag(·) is the diagonal matrix;

[0033] S504: Repeat S501 to S503, update the covariance matrix Σ, until it converges to the required accuracy range, and output the normal matrix at this time

[0034] S505: Using the inverse mapping Φ(·) -1 (·), the normal matrix Mapping to obtain the normalized matrix

[0035] S506: Using the inverse mapping ψ of ψ(·) -1 (·) Normalize the matrix Mapping to get the filled matrix

[0036] The redundant feature analysis described in step S7 is specifically as follows: all features in the matrix features of the current video data are traversed one by one, the independence of the features is calculated using conditional probability, and redundant features are deleted; wherein, features that are not independent are redundant features.

[0037] Furthermore, the target detection method described in step S4 is one of the deep learning networks of the RCNN series and the YOLO series; the interpolation function is a Lagrange interpolation function or a spline interpolation function.

[0038] Further, the correlation analysis in step S6 is performed using a three-branch decision method; specifically, the three-branch decision method is:

[0039] S601: Initialize the significance level α, β, and cost according to the parameter initialization module; at the same time, set the feature set Intermediate Collection

[0040] S602: learning the labeled historical data to make the cost smaller, and updating the significance levels α and β using the deep learning network;

[0041] S603: Calculate the filled matrix F corresponding to the current frame t The corresponding conditional correlation coefficient Dep(C,F t );

[0042] S604: If the conditional correlation coefficient Dep(C,F t )<β, the feature corresponding to the current frame does not belong to the classification attribute C, and the current frame matrix F is abandoned. t ; If the conditional correlation coefficient Dep(C,F t )>α, then the feature corresponding to the current frame belongs to the classification attribute C, and the current frame matrix F t Put it into the feature set pos; otherwise, the current frame matrix F t Put it into the intermediate set bnd, and execute step S605;

[0043] S605: When the feature set When any X F ∈pos, traverse F t ′∈bnd, will satisfy P(C|F t ′,X F )≠P(C|X F ) t ′ is put into the feature set pos; where P(·) is the probability function;

[0044] S606: Output the matrix features of the current video data, that is, the feature set pos.

[0045] Furthermore, the calculation method for updating the significance levels α and β in step S602 can also be to solve the following optimization model; the target equation is: The constraint equation is: r PP ≤r BP ≤r EP , r EE ≤r BE ≤r PE , At the same time: (1) If the features in the historical data belongs to the classification attribute C, then R P ≤R B , R P ≤R E ; (2) If the characteristics in the historical data does not belong to the classification attribute C, then R B ≤R P , R B ≤R E ; (3) Other R E ≤R P , R E ≤R B ;in, For non-operation; r EP , rPE , r BP , r BE , r PP , r EE , is the undetermined hyperparameter, which is calculated by solving the optimization model.

[0046] The invention is applied to a method for extracting online features of a video stream based on Gaussian Copula. The invention also provides a device for extracting online features of a video stream based on Gaussian Copula. Figure 2 , characterized in that the device comprises a receiving module, a storage module, a data filling module, a feature extraction module, and a parameter initialization module; the receiving module is used to receive the real-time image of the video data acquisition device and instruct the storage module to store the acquired video data; the storage module comprises a video cache unit and a feature storage unit, the video cache unit is used to store the video data transmitted by the receiving module in a matrix form, and the feature storage unit is used to store the features of the extracted video data; the data filling module is used to fill the missing data in the video data stored in the matrix form based on the Gaussian Copula method; the feature extraction module is used to perform correlation analysis on the filled video data matrix, extract features and store them in the feature storage unit, and delete redundant features in the feature storage unit; the parameter initialization module is used to initialize the parameters of the video stream online feature extraction method based on Gaussian Copula.

[0047] Furthermore, the feature extraction module includes a correlation analysis unit, a redundancy analysis unit and an iterative output unit; the correlation analysis unit is used to realize feature correlation calculation online and judge the feature category of video data; the redundancy analysis unit is used to screen out redundant features among all features in the feature storage unit online; the iterative output unit is used to iteratively update the video data features and output them to the feature storage unit.

[0048] The beneficial effects of the present invention are as follows: the present invention provides a method and device for online feature extraction of video streams based on Gaussian Copula. In view of the situation where the detected target is occluded during the video image recognition process, the Gaussian Copula method is firstly combined with a missing value optimization model to reduce the search space and quickly and accurately complete the missing feature data. Then, the influence of uncertain relationships is reduced based on a three-branch decision method, and redundant features are further screened out, thereby realizing accurate and fast extraction of online video features. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to make the purpose and technical solution of the present invention, the present invention provides the following drawings for explanation:

[0050] Figure 1 It is a flow chart of the online feature extraction method of video stream based on Gaussian Copula of the present invention;

[0051] Figure 2 It is a framework diagram of a video stream online feature extraction device based on Gaussian Copula of the present invention;

[0052] Figure 3 This is an example diagram of Gaussian Copula in Example 1 of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose and technical solution of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0054] Embodiment: In the scenario of performing feature analysis on a moving target using a video captured by a camera, if the target is partially blocked during movement. The present invention provides a "video stream online feature extraction device based on Gaussian Copula", combined with Figure 2 , specifically comprising: a receiving module 110, a storage module 120, a data filling module 130, a feature extraction module 140, and a parameter initialization module 150; the receiving module 110 is used to receive the real-time image of the video data acquisition device and instruct the storage module 120 to store the acquired video data; the storage module 120 comprises a video cache unit 121 and a feature storage unit 122, the video cache unit 121 is used to store the video data transmitted by the receiving module 110 in a matrix form, and the feature storage unit 122 is used to store the features of the extracted video data; the data filling module 130 is used to fill the missing data in the video data stored in the matrix form based on the Gaussian Copula method; the feature extraction module 140 is used to perform correlation analysis on the filled video data matrix, extract features and store them in the feature storage unit, and delete the redundant features in the feature storage unit; the parameter initialization module 150 is used to initialize the parameters of the video stream online feature extraction method based on Gaussian Copula.

[0055] The feature extraction module 140 includes a correlation analysis unit 141, a redundancy analysis unit 142 and an iterative output unit 143; the correlation analysis unit 141 is used to implement feature correlation calculation online and determine the feature category of video data; the redundancy analysis unit 142 is used to screen out redundant features from all features in the feature storage unit online; the iterative output unit 143 is used to iteratively update the video data features and output them to the feature storage unit.

[0056] The present invention also provides a method for extracting online features of video streams based on Gaussian Copula. Figure 1 , which specifically includes the following steps:

[0057] Step 1: The receiving module 110 receives the video data collected by the video acquisition device and stores the data frame by frame into the video buffer unit 121 of the storage module.

[0058] Step 2: The feature extraction module 140 receives an instruction for online extraction of specific target features in a video sent by the server.

[0059] Step 3: The parameter initialization module 150 initializes the parameters involved in the online feature extraction process of the video stream based on Gaussian Copula.

[0060] The parameters include Gaussian Copula method parameters and feature extraction parameters; wherein, the Gaussian Copula method parameters include: the matrix size H×W corresponding to the target to be detected, the covariance matrix Σ of the normal distribution; the feature extraction parameters include: the classification attribute C corresponding to the feature, the significance levels α, β, and the cost cost; wherein, 0≤β≤α≤1, cost=∞.

[0061] Step 4, the video cache unit 121 uses the target detection method to segment the target to be detected frame by frame, and re-stores the image of the target area to be detected in matrix form to obtain the matrix to be tested. At the same time, the video cache unit 121 subtracts the matrix to be tested frame by frame, and uses the difference of each element position of the matrix to be tested to establish an interpolation function; wherein the element value of the matrix to be tested of the obscured part of the target to be detected is set to the missing value.

[0062] The target detection method is YOLOv3; the interpolation function is a Lagrange interpolation function, the dimension is two-dimensional, and the grid points correspond to the rows and columns of the matrix, that is, binary linear interpolation.

[0063] Step 5: The data filling module 130 uses the Gaussian copula method to fill the missing values ​​in the matrix to be tested obtained in step S4.

[0064] Combination Figure 3, for the matrix to be tested in one frame, the Gaussian Copula method is specifically as follows:

[0065] S501: For the known element x in the matrix to be tested O Processing is performed to construct a mapping relationship and obtain a normal matrix Z; specifically, it includes:

[0066] S5011: Establishing a mapping function ψ(·)=[ψ j (·)] 1≤j≤W , use the known elements of the jth column in the matrix to be tested Transform, traverse j = 1, ..., W, and obtain the normalized matrix E; where ψ j (·) is to normalize the known elements of the jth column in the matrix to be tested to the interval [0,1] according to each column;

[0067] S5012: Establishing a mapping function Φ(·)=[Φ j (·)] 1≤j≤W , transform each column in the normalized matrix E using the normal distribution N(0,∑) to obtain the normal matrix Z;

[0068] S502: Using the interpolation function obtained in step 4, obtain the missing value x of the matrix to be tested in the current frame. M The predicted value x′ is used to establish a missing value optimization model to obtain the missing value; the optimization model is:

[0069] obj min||x M -x′||

[0070]

[0071] in, is the coefficient to be determined; Σ M,O ,Σ O,O The submatrices are the submatrices consisting of all rows corresponding to missing values ​​in Σ and the submatrices consisting of all rows corresponding to non-missing values; ·- 1 is the inverse matrix operation; E(·) is the expected function; Φ -1 (·) is the inverse mapping of Φ(·); ψ -1 (·) is the inverse mapping of ψ(·);

[0072] S503: Using likelihood estimation, combined with the missing value x calculated in step S502 M , update the covariance matrix ∑; specifically:

[0073] S5031: Update the covariance matrix Σ using likelihood estimation: = G(Σ, x) = E(z·z T |x,Σ); where x=[xO ,x M ],z=[z O ,z M ];

[0074] S5032: Calculate the approximate covariance matrix

[0075] S5033: Update the covariance matrix again

[0076] Where P = D -1 / 2 ·Σ·D -1 / 2 , D = diag(Σ), diag(·) is the diagonal matrix;

[0077] S504: Repeat S501 to S503, update the covariance matrix ∑, until it converges to the required accuracy range, and output the normal matrix at this time

[0078] S505: Using the inverse mapping Φ(·) -1 (·), the normal matrix Mapping to obtain the normalized matrix

[0079] S506: Using the inverse mapping ψ of ψ(·) -1 (·) Normalize the matrix Mapping to get the filled matrix

[0080] Step 6: The correlation analysis unit 141 in the feature extraction module 140 performs correlation analysis on the filled matrix row by row to calculate the matrix features of the current video data.

[0081] The correlation analysis is performed using a three-branch decision method; specifically, the three-branch decision method is:

[0082] S601: Initialize the significance level α, β, and cost according to the parameter initialization module; at the same time, set the feature set Intermediate Collection

[0083] S602: learning the labeled historical data to make the cost smaller, and updating the significance levels α and β using the optimization model;

[0084] The objective equation in the optimization model is: The constraint equation is: r PP ≤r BP ≤r EP , r EE≤r BE ≤r PE , At the same time: (1) If the characteristics in the historical data belongs to the classification attribute C, then R P ≤R B , R P ≤R E ; (2) If the characteristics in the historical data does not belong to the classification attribute C, then R B ≤R P , R B ≤R E ; (3) Other R E ≤R P , R E ≤R B ;in, For non-operation; r EP , r PE , r BP , r BE , r PP , r EE , is the undetermined hyperparameter, which is calculated by solving the optimization model.

[0085] S603: Calculate the filled matrix F corresponding to the current frame t The corresponding conditional correlation coefficient Dep(C,F t );

[0086] S604: If the conditional correlation coefficient Dep(C,F t )<β, the feature corresponding to the current frame does not belong to the classification attribute C, and the current frame matrix F is abandoned. t ; If the conditional correlation coefficient Dep(C,F t )>α, then the feature corresponding to the current frame belongs to the classification attribute C, and the current frame matrix F t Put it into the feature set pos; otherwise, the current frame matrix F t Put it into the intermediate set bnd, and execute step S605;

[0087] S605: When the feature set When any X F ∈pos, traverse F t ′∈bnd, will satisfy P(C|F t ′,X F )≠P(C|X F )t ′ is put into the feature set pos; where P(·) is the probability function;

[0088] S606: Output the matrix features of the current video data, that is, the feature set pos.

[0089] Step 7: The redundancy analysis unit 142 in the feature extraction module 140 analyzes the matrix features and the historical features in the feature storage unit, screens out the redundant features therein, and updates the historical features.

[0090] Step 8: The iterative output unit 143 in the feature extraction module 140 repeats steps 4 to 7 until all the video streams have completed the target feature extraction, and sends all the target features in the feature storage unit 122 to the server.

[0091] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A video stream online feature extraction method based on Gaussian Copula, characterized in that: The method comprises the following steps: S1: The receiving module receives the video data collected by the video acquisition device and stores it frame by frame into the video cache unit of the storage module; S2: The feature extraction module receives an instruction for online extraction of specific target features in a video sent by the server; S3: The parameter initialization module initializes the parameters involved in the online feature extraction process of the video stream based on Gaussian Copula; S4: The video cache unit segments the target to be detected frame by frame using the target detection method, and stores the image of the target area to be detected in the form of a matrix to obtain a matrix to be detected. At the same time, the video cache unit makes a difference to the matrix to be detected frame by frame, and uses the difference of each element position of the matrix to be detected to establish an interpolation function for calculating the initial value of the iteration of the Gaussian copula method; wherein the element value of the matrix to be detected of the blocked part of the target to be detected is set as a missing value; S5: The data filling module uses the Gaussian copula method to fill the missing values ​​in the test matrix obtained in step S4; S6: The correlation analysis unit in the feature extraction module performs correlation analysis on the filled matrix row by row to calculate the matrix features of the current video data; S7: The redundancy analysis unit in the feature extraction module analyzes the matrix features and the historical features in the feature storage unit, screens out the redundant features, and updates the historical features; S8: the iterative output unit in the feature extraction module repeatedly executes steps S4 to S7 until all video streams have completed the feature extraction of the target, and sends all the target features in the feature storage unit to the server; The parameters in step S3 include Gaussian Copula method parameters and feature extraction parameters; wherein the Gaussian Copula method parameters include: the matrix size H×W corresponding to the target to be detected, the covariance matrix ∑ of the normal distribution; the feature extraction parameters include: the classification attribute C corresponding to the feature, the significance levels α, β, and the cost cost; wherein 0≤β≤α≤1; The Gaussian Copula method described in step S5 is specifically: S501: For the known element x in the matrix to be tested O Processing is performed to construct a mapping relationship and obtain a normal matrix Z; specifically, it includes: S5011: Establishing a mapping function ψ(·)=[ψ j (·)] 1≤j≤W , use the known elements of the jth column in the matrix to be tested Transform, traverse j = 1, ..., W, and obtain the normalized matrix E; where ψ j (·) is to normalize the known elements of the jth column in the matrix to be tested to the interval [0,1] according to each column; S5012: Establishing a mapping function Φ(·)=[Φ j (·)] 1≤j≤W , transform each column in the normalized matrix E using the normal distribution N(0,Σ) to obtain the normal matrix Z; S502: Using the interpolation function of step S4 to obtain the missing value x of the matrix to be tested in the current frame M The predicted value x′ is used to establish a missing value optimization model to obtain the missing value; the optimization model is: obj min||x M -x′|| in, is the coefficient to be determined; Σ M,O ,∑ O,O are the submatrices consisting of all rows corresponding to missing values ​​in ∑ and the submatrix consisting of all rows corresponding to non-missing values; -1 is the inverse matrix operation; E(·) is the expected function; Φ -1 (·) is the inverse mapping of Φ(·); ψ -1 (·) is the inverse mapping of ψ(·); S503: Using likelihood estimation, combined with the missing value x calculated in step S502 M , update the covariance matrix ∑; specifically: S5031: Update the covariance matrix Σ using likelihood estimation: = G(Σ, x) = E(z·z T |x,Σ); where x=[x O ,x M ],z=[z O ,z M ]; S5032: Calculate the approximate covariance matrix S5033: Update the covariance matrix again Where P = D -1 / 2 ·Σ·D -1 / 2 , D = diag(Σ), diag(·) is the diagonal matrix; S504: Repeat S501 to S503, update the covariance matrix ∑, until it converges to the required accuracy range, and output the normal matrix Z at this time; S505: Using the inverse mapping Φ(·) -1 (·), mapping the normal matrix Z to obtain the normalized matrix S506: Using the inverse mapping ψ of ψ(·) -1 (·) Map the normalized matrix E to get the filled matrix The redundant feature analysis described in step S7 is specifically as follows: all features in the matrix features of the current video data are traversed one by one, the independence of the features is calculated using conditional probability, and redundant features are deleted; wherein, features that are not independent are redundant features.

2. The method for online feature extraction of video stream based on Gaussian Copula according to claim 1, characterized in that: The target detection method described in step S4 is one of the deep learning networks of the RCNN series and the YOLO series; the interpolation function is a Lagrange interpolation function or a spline interpolation function.

3. The method for online feature extraction of video stream based on Gaussian Copula according to claim 1, characterized in that: The correlation analysis described in step S6 is performed using a three-branch decision method; specifically, the three-branch decision method is: S601: Initialize the significance level α, β, and cost according to the parameter initialization module; at the same time, set the feature set Intermediate Collection S602: learning the labeled historical data to make the cost smaller, and updating the significance levels α and β using the deep learning network; S603: Calculate the filled matrix F corresponding to the current frame t The corresponding conditional correlation coefficient Dep(C,F t ); S604: If the conditional correlation coefficient Dep(C,F t )<β, the feature corresponding to the current frame does not belong to the classification attribute C, and the current frame matrix F is abandoned. t ; If the conditional correlation coefficient Dep(C,F t )>α, then the feature corresponding to the current frame belongs to the classification attribute C, and the current frame matrix F t Put it into the feature set pos; otherwise, the current frame matrix F t Put it into the intermediate set bnd, and execute step S605; S605: When the feature set When any X F ∈pos, traverse F t ′∈bnd, will satisfy P(C|F t ′,X F )≠P(C|X F ) t ′ is put into the feature set pos; where P(·) is the probability function; S606: Output the matrix features of the current video data, that is, the feature set pos.

4. The method for online feature extraction of video stream based on Gaussian Copula according to claim 3, characterized in that: The calculation method for updating the significance levels α and β in step S602 is to solve the following optimization model; the objective equation is: The constraint equation is: r PP ≤r BP ≤r EP , r EE ≤r BE ≤r PE , At the same time: (1) If the characteristics in the historical data belongs to the classification attribute C, then R P ≤R B , R P ≤R E ; (2) If the characteristics in the historical data does not belong to the classification attribute C, then R B ≤R P , R B ≤R E ; (3) Other R E ≤R P , R E ≤R B ;in, For non-operation; r EP , r PE , r BP , r BE , r PP , r EE , is the undetermined hyperparameter, which is calculated by solving the optimization model.

5. A video stream online feature extraction device based on Gaussian Copula based on the method according to any one of claims 1 to 4, characterized in that: The device comprises a receiving module, a storage module, a data filling module, a feature extraction module and a parameter initialization module; the receiving module is used for receiving the real-time image of the video data acquisition device and instructing the storage module to store the acquired video data; the storage module comprises a video cache unit and a feature storage unit, the video cache unit is used for storing the video data transmitted by the receiving module in the form of a matrix, and the feature storage unit is used for storing the features of the extracted video data; the data filling module is used for filling the missing data in the video data stored in the form of a matrix based on the Gaussian Copula method; the feature extraction module is used for performing correlation analysis on the filled video data matrix, extracting features and storing them in the feature storage unit, and deleting redundant features in the feature storage unit; the parameter initialization module is used for initializing the parameters of the video stream online feature extraction method based on Gaussian Copula.

6. The video stream online feature extraction device based on Gaussian Copula according to claim 5, characterized in that: The feature extraction module includes a correlation analysis unit, a redundancy analysis unit and an iterative output unit; the correlation analysis unit is used to realize feature correlation calculation online and judge the feature category of video data; the redundancy analysis unit is used to screen out redundant features from all features in the feature storage unit online; The iterative output unit is used to iteratively update the video data features and output them to the feature storage unit.

Images