Unstructured data feature extraction method and system based on deep learning

Abstract

The invention discloses an unstructured data feature extraction method and system based on deep learning, and relates to the technical field of data processing, and the method comprises the steps: constructing the probability distribution of a trunk mode and an auxiliary mode, calculating a bidirectional distribution difference through Kullback-Leibler divergence, defining a relative information gain, and converting the bidirectional distribution difference into an auxiliary mode fusion weight, and carrying out weighted fusion on the trunk modal features and the auxiliary modal features to generate trunk features after preliminary fusion. According to the method, independent coding of modal features is combined with a modal fusion strategy based on relative information gain, the capacity of mining the interaction relation between different modal data is improved, the distribution difference between the modals is calculated by adopting Kullback-Leibler divergence, the relative information gain is defined, the fusion weight of the auxiliary modal is dynamically adjusted according to the information gain, and the fusion accuracy of the modal data is improved. And the semantic relevance among the multi-modal data is enhanced.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for feature extraction of unstructured data based on deep learning. Background Technology

[0002] With the rapid development of artificial intelligence and deep learning technologies, the processing and feature extraction of unstructured data has become an important topic in the field of computer science and engineering. Unstructured data usually includes forms such as video, voice, and text. Due to their complexity and diversity, these data are difficult to effectively parse and apply using traditional data processing methods.

[0003] Current methods for extracting features from unstructured data still have shortcomings. The challenge lies in how to effectively integrate data features from different modalities. Traditional methods often process data from each modality independently and obtain the final features through simple concatenation or weighted fusion, ignoring the complex interaction relationships between different modalities. Most existing multimodal learning methods rely on relatively simple feature fusion strategies and cannot fully explore the interrelationships and semantic gains between different modalities. Summary of the Invention

[0004] In view of the aforementioned existing problems, the inventors have proposed this invention.

[0005] Therefore, this invention provides a method and system for extracting unstructured data features based on deep learning to solve the problem of how to effectively fuse data features from different modalities. Traditional methods often process data from each modality independently and obtain the final features through simple concatenation or weighted fusion, ignoring the complex interaction relationships between different modalities. Most existing multimodal learning methods rely on relatively simple feature fusion strategies and cannot fully explore the interrelationships and semantic gains between different modalities.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for feature extraction of unstructured data based on deep learning, comprising: Collect unstructured data and preprocess it. Segment the unstructured data according to the same time window to generate an aligned set of segments and perform normalization processing. Encode the normalization results independently to generate feature sequences. Define the backbone and auxiliary sequences, construct the probability distributions of the backbone mode and auxiliary mode, calculate the bidirectional distribution difference through Kullback-Leibler divergence, define the relative information gain and convert it into auxiliary mode fusion weights, and perform weighted fusion of backbone mode features and auxiliary mode features to generate the backbone features after preliminary fusion. Layer normalization and GeLU activation function are applied to the backbone features after initial fusion, and dimensionality reduction is performed to generate dimensionality-reduced fused features. Channel attention weights are generated through a multilayer perceptron, and corresponding attention weights are applied to each feature channel to generate the final feature vector. Build a visual interface to display the final feature vector.

[0007] As a preferred embodiment of the deep learning-based unstructured data feature extraction method of the present invention, the following steps are included: defining the backbone and auxiliary sequences, constructing the probability distributions of the backbone mode and auxiliary modes, calculating the bidirectional distribution difference through Kullback-Leibler divergence, defining the relative information gain and converting it into auxiliary mode fusion weights, and performing weighted fusion of the backbone mode features and auxiliary mode features to generate the preliminary fused backbone features, including: The video feature sequence is used as the backbone sequence, and the speech and text feature sequences are used as auxiliary sequences. A linear projection transformation is applied to the modal features of the backbone sequence to generate a query vector; Global average pooling is performed along the time dimension on the auxiliary modal features in the auxiliary sequence to generate a global feature vector of the auxiliary modality. A linear projection transformation is then performed on the global feature vector of the auxiliary modality to generate a key vector. Based on the query vector and key vector, the probability distributions of the main mode and the auxiliary mode are constructed respectively. The difference between the two-way distributions is calculated by Kullback-Leibler divergence, the relative information gain is defined, and the relative information gain is mapped to the (0,1) interval by the sigmoid activation function to generate the auxiliary mode fusion weight. Linear interpolation is performed on the auxiliary modal features to generate time-aligned auxiliary modal features; The backbone modal features are weighted and fused with the time-aligned auxiliary modal features to generate the preliminary fused backbone features.

[0008] As a preferred embodiment of the deep learning-based unstructured data feature extraction method of the present invention, the step of applying layer normalization and GeLU activation function to the initially fused backbone features and performing dimensionality reduction processing to generate dimensionality-reduced fused features includes: Layer normalization and GeLU activation function are applied to the preliminary fusion backbone features to generate refined backbone fusion features; The feature dimensions of the refined backbone fusion features are reduced by one-dimensional convolution operation to generate dimensionality-reduced fusion features.

[0009] As a preferred embodiment of the deep learning-based unstructured data feature extraction method of the present invention, the step of generating channel attention weights through a multilayer perceptron, applying a corresponding attention weight to each feature channel, and generating a final feature vector includes: Channel attention weights are generated using a multilayer perceptron, and corresponding attention weights are applied to each feature channel to obtain weighted fused features. The fusion feature weighted at level 5 is used as the final distillation feature; Global average pooling is applied to the final distilled features, followed by L2 norm normalization to generate the final feature vector.

[0010] As a preferred embodiment of the deep learning-based unstructured data feature extraction method of the present invention, wherein: the construction of a visual interface to display the final feature vector includes: A visualization interface is built using the visualization tool Matplotlib to display the final feature vector in real time.

[0011] As a preferred embodiment of the deep learning-based unstructured data feature extraction method of the present invention, the step of collecting and preprocessing unstructured data includes: Use smart devices to collect unstructured data and perform timestamp synchronization, noise reduction, and standardization. The smart devices include webcams, microphones, and smartphones; The unstructured data includes video, audio, and text data.

[0012] As a preferred embodiment of the deep learning-based unstructured data feature extraction method of the present invention, the step of segmenting the unstructured data according to the same time window, generating an aligned set of segments and performing normalization processing, and independently encoding the normalization results to generate feature sequences includes: Set a time window, divide the unstructured data into segments according to the same time window, and generate an aligned set of segments; The aligned set of segments is normalized to generate normalized results, including video tensors, Mel spectrograms, and token ID sequences; Load the pre-trained ImageBind large model to independently encode the normalization results and generate modal features, including video, speech and text features; Modal features are stacked along the time dimension to obtain feature sequences, including visual, speech, and text feature sequences.

[0013] Secondly, this invention provides a deep learning-based unstructured data feature extraction system, comprising: The preprocessing module is used to collect unstructured data and perform timestamp synchronization, noise reduction, and standardization. The segmentation and encoding module is used to segment the data. It loads a pre-trained model to encode the segmented data independently and generate feature sequences. The preliminary fusion module is used to construct the probability distributions of the backbone mode and the auxiliary mode. It calculates the difference between the two-way distributions through Kullback-Leibler divergence, defines the relative information gain, and performs weighted fusion of the modal features to generate the backbone features after preliminary fusion. The weighted feature module is used to generate channel attention weights and apply corresponding weights to each feature channel to generate the final feature vector. The presentation module is used to build a visual interface to display the final feature vector.

[0014] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the deep learning-based unstructured data feature extraction method described in the first aspect of the present invention.

[0015] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the deep learning-based unstructured data feature extraction method as described in the first aspect of the present invention.

[0016] The beneficial effects of this invention are as follows: By combining modal feature independent encoding with a modal fusion strategy based on relative information gain, this invention improves the ability to mine the interaction relationship between different modal data, thereby significantly improving the accuracy and robustness of feature extraction. Kullback-Leibler divergence is used to calculate the distribution difference between modalities, a relative information gain is defined, and the fusion weight of the auxiliary modality is dynamically adjusted according to the information gain, thereby enhancing the semantic correlation between multimodal data. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the deep learning-based unstructured data feature extraction method in Example 1.

[0019] Figure 2This is a schematic diagram of the deep learning-based unstructured data feature extraction system in Example 1. Detailed Implementation

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a method for extracting features from unstructured data based on deep learning, including the following steps: S1. Collect unstructured data and preprocess it. Segment the unstructured data according to the same time window to generate an aligned set of segments and perform normalization processing. Encode the normalization results independently to generate feature sequences. Specifically, unstructured data is collected and preprocessed, including: Use smart devices to collect unstructured data and perform timestamp synchronization, noise reduction, and standardization. The smart devices include webcams, microphones, and smartphones; The unstructured data includes video, audio, and text data.

[0024] Denoising, timestamp synchronization, and standardization processes improve data quality by removing sensor noise, system bias, and other factors that may affect the accuracy of the analysis.

[0025] Furthermore, the unstructured data is segmented according to the same time window to generate an aligned set of segments, which are then normalized. The normalization results are independently encoded to generate feature sequences, including: The sliding window algorithm is used to set time windows, and unstructured data is divided into segments according to the same time window to generate an aligned set of segments; The aligned set of segments is normalized to generate normalized results, including video tensors, Mel spectrograms, and token ID sequences; Load the pre-trained ImageBind large model to independently encode the normalization results and generate modal features, including video, speech and text features; Modal features are stacked along the time dimension to obtain feature sequences, including visual, speech, and text feature sequences; The normalization process for the aligned fragment set includes: Extract video from the segment set, perform linear normalization on the RGB three-channel pixel values ​​of the video frames, and generate a video tensor; Speech segments are extracted from the collection of segments and subjected to short-time Fourier transform to obtain a complex spectrum. The amplitude of the complex spectrum is squared to generate a power spectrum. The power spectrum is weighted and summed using a Mel filter bank to obtain the Mel energy. The natural logarithm of the Mel energy is then taken to form a Mel spectrogram. Extract the text from the fragment set and perform WordPiece word segmentation. Then, use WordPiece Tokenizer to convert the segmented text into a token ID sequence.

[0026] By segmenting different modal data into small segments according to time windows and aligning these segments, the consistency and comparability of the data are enhanced, ensuring the synchronization of multimodal data over time. Independent encoding steps enhance the expressive power of each data modality, ensuring that the features of each modality are presented in the optimal way. By stacking feature sequences of different modalities, not only is the important information of each modality preserved, but also the information transmission and synergy between different modalities are enhanced by integrating time-series data. Normalization processing ensures that video, audio, and text data can be compared under the same standard, eliminating scale and unit differences between different modalities and improving the comparability and effectiveness of features from different data sources.

[0027] S2. Define the backbone and auxiliary sequences, construct the probability distributions of the backbone mode and auxiliary mode, calculate the bidirectional distribution difference through Kullback-Leibler divergence, define the relative information gain and convert it into auxiliary mode fusion weights, perform weighted fusion of backbone mode features and auxiliary mode features, and generate the backbone features after preliminary fusion. Specifically, the core and auxiliary sequences are defined, the probability distributions of the core and auxiliary modes are constructed, the bidirectional distribution difference is calculated using Kullback-Leibler divergence, the relative information gain is defined and converted into auxiliary mode fusion weights, and the core and auxiliary mode features are weighted and fused to generate the preliminary fused core features, including: The video feature sequence is used as the backbone sequence, and the speech and text feature sequences are used as auxiliary sequences. A linear projection transformation is applied to the modal features of the backbone sequence to generate a query vector, as shown in the formula: , Where Q is the query vector. , representing the backbone modal features of the i-th segment. The learnable query projection matrix of the backbone mode. For the real number field, Let be the symbol, d be the feature dimension, and the embedding dimension of the large ImageBind model is 1024. The length of the visual modality sequence; Global average pooling is performed along the time dimension on the auxiliary modality features in the auxiliary sequence to generate a global feature vector for the auxiliary modality, as shown in the formula: , in To assist the global feature vector of the modality, For auxiliary mode sequence length, Let be the auxiliary modal feature at the j-th time step, and let represent the auxiliary modal feature of the i-th data segment at the j-th time step. A linear projection transformation is applied to the global eigenvectors of the auxiliary modes to generate the key vectors, as shown in the formula: , in For auxiliary modal key vectors, The learnable key projection matrix for auxiliary modes; Based on the query vector and key vector, the probability distributions of the main mode and auxiliary modes are constructed using the softmax function, respectively, as follows: , , in The main mode probability distribution For auxiliary modal probability distribution, This is the normalization function; The difference between the two-way distributions is calculated using the Kullback-Leibler divergence, and the relative information gain is defined by the following formula: , , , in Relative information gain measures the semantic gain of the auxiliary modality relative to the main modality. The Kullback–Leibler divergence operator measures the information difference between two probability distributions, where r is the dimension of the projected distribution. and These are the probability distributions of the main mode and the auxiliary mode at the j-th time step, respectively. The relative information gain is mapped to the (0,1) interval using the sigmoid activation function to generate auxiliary modality fusion weights, as shown in the formula: , in To assist in modal fusion weights, It is the sigmoid activation function; Linear interpolation is performed on the auxiliary modal features to make the time axis of the auxiliary modes consistent with that of the main modes, thus generating time-aligned auxiliary modal features; The backbone modal features are weighted and fused with the time-aligned auxiliary modal features to generate the preliminary fused backbone features. The formula is as follows: , in These are the main features after initial fusion. These are the auxiliary modal features after interpolation.

[0028] By defining and constructing probability distributions for the backbone and auxiliary modalities, we can ensure that information between different data modalities is reasonably expressed and provide a solid foundation for subsequent modality fusion. The construction of probability distributions provides a measure of the relative importance between modalities, providing a necessary basis for weighted fusion. Through linear projection transformation, the query vector can accurately capture the potential information of the backbone modal features and play a crucial role in subsequent modality fusion. Global pooling and linear projection can effectively reduce data complexity while retaining key information of auxiliary modalities. The calculation of relative information gain provides dynamic adjustment of modality fusion, enabling the fusion effect of auxiliary modalities to adaptively adjust according to their semantic gain on the backbone modality. Time-aligned weighted fusion ensures that data from different modalities can be effectively fused within the same time step, thereby greatly improving the efficiency and accuracy of multimodal learning.

[0029] S3. Apply layer normalization and GeLU activation function to the backbone features after preliminary fusion, and perform dimensionality reduction to generate dimensionality-reduced fused features. Generate channel attention weights through a multilayer perceptron, apply the corresponding attention weights to each feature channel, and generate the final feature vector. Specifically, layer normalization and GeLU activation function are applied to the backbone features after initial fusion, and dimensionality reduction is performed to generate dimensionality-reduced fused features, including: Apply layer normalization and GeLU activation function to the initially fused backbone features to generate refined backbone fused features, as shown in the formula: , in The refined core fusion characteristics Let Gaussian error be the activation function of the linear unit. For layer normalization operation; The refined backbone fusion features are reduced in dimensionality using a one-dimensional convolution operation to generate the dimensionality-reduced fusion features, as shown in the formula: , , in The fused features are after dimensionality reduction to level a, where a is the distillation level index. , For a one-dimensional convolution operation with a kernel size of 1, The length of the visual modality sequence. For the a-th level target feature dimension Using a proportional decrease setting, The initial feature dimension is set to 1024. The final feature dimension is 64, and A is the total number of distillation stages, with a value of 5.

[0030] The combination of layer normalization and GeLU activation function not only improves the convergence speed of training, but also enhances the expressive power of the model. Dimensionality reduction not only reduces the computational complexity, but also effectively removes noise and redundant features from the data, making subsequent processing more efficient and accurate.

[0031] Furthermore, channel attention weights are generated using a multilayer perceptron, and corresponding attention weights are applied to each feature channel to generate the final feature vector, including: Channel attention weights are generated using a multilayer perceptron, with the following formula: , in Let c be the attention weight of the a-th channel. and These are the weight matrices for the first and second layers of level a, respectively. and denoted as the mean and variance of the c-th channel at level a, respectively, are calculated based on the dimensionality-reduced fusion features. To determine the global information gain feature, the mean value of the auxiliary modality fusion weights is calculated. and These are the offsets of the first and second layers of level a, respectively; Applying corresponding attention weights to each feature channel yields the weighted fused features, as shown in the formula: , in The fusion feature is the weighted feature of the a-th channel. Let be the attention weight vector for the a-th channel. For the Hadamard accumulation of each channel; The fusion feature weighted at level 5 is used as the final distillation feature; Global average pooling is applied to the final distilled features, followed by L2 norm normalization to generate the final feature vector, as shown in the formula: , in Let i be the feature vector after average pooling of the i-th segment. For the final distillation feature, in level 5, the weighted fusion feature at time step j.

[0032] The channel attention mechanism can automatically adjust the influence of each channel in the final decision, effectively suppressing the influence of redundant or noisy features and strengthening the expression of key features. The biggest advantage of the distillation feature generation process is that it compresses redundant information in multimodal data and retains the most representative and discriminative features. The combination of global pooling and L2 norm normalization effectively reduces the differences between different feature dimensions, making the feature vector more stable and avoiding unnecessary interference to model training caused by different scales of certain features.

[0033] S4. Construct a visual interface to display the final feature vector; Specifically, a visual interface is constructed to display the final feature vector, including: A visualization interface is built using the visualization tool Matplotlib to display the final feature vector in real time.

[0034] This embodiment also provides a deep learning-based unstructured data feature extraction system, including: The preprocessing module is used to collect unstructured data and perform timestamp synchronization, noise reduction, and standardization. The segmentation and encoding module is used to segment the data. It loads a pre-trained model to encode the segmented data independently and generate feature sequences. The preliminary fusion module is used to construct the probability distributions of the backbone mode and the auxiliary mode. It calculates the difference between the two-way distributions through Kullback-Leibler divergence, defines the relative information gain, and performs weighted fusion of the modal features to generate the backbone features after preliminary fusion. The weighted feature module is used to generate channel attention weights and apply corresponding weights to each feature channel to generate the final feature vector. The presentation module is used to build a visual interface to display the final feature vector.

[0035] This embodiment also provides a computer device applicable to the case of unstructured data feature extraction method based on deep learning, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the unstructured data feature extraction method based on deep learning as proposed in the above embodiment.

[0036] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0037] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the deep learning-based unstructured data feature extraction method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0038] In summary, this invention improves the ability to mine interaction relationships between different modal data by combining modal feature independent encoding with a modal fusion strategy based on relative information gain, thereby significantly improving the accuracy and robustness of feature extraction. It uses Kullback-Leibler divergence to calculate the distribution differences between modalities, defines relative information gain, and dynamically adjusts the fusion weights of auxiliary modalities based on this information gain, thereby enhancing the semantic correlation between multimodal data.

[0039] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep learning-based unstructured data feature extraction method, characterized in that: The method comprises the following steps: Collecting and preprocessing unstructured data, dividing the unstructured data according to the same time window, generating an aligned segment set and performing normalization processing, independently encoding the normalized results, and generating a feature sequence; Defining the main sequence and the auxiliary sequence, constructing the probability distribution of the main mode and the auxiliary mode, calculating the bidirectional distribution difference through Kullback-Leibler divergence, defining the relative information gain, and converting it into the auxiliary mode fusion weight, weighting and fusing the main mode feature and the auxiliary mode feature to generate the preliminary fused main feature; Applying layer normalization and GeLU activation function to the preliminary fused main feature, and performing dimension reduction processing to generate the reduced fusion feature, generating the channel attention weight through the multilayer perception, and applying the corresponding attention weight to each feature channel to generate the final feature vector; Constructing a visual interface to display the final feature vector. 2.The deep learning based unstructured data feature extraction method of claim 1, wherein: The definition of the main sequence and the auxiliary sequence, the construction of the probability distribution of the main mode and the auxiliary mode, the calculation of the bidirectional distribution difference through Kullback-Leibler divergence, the definition of the relative information gain, and the conversion into the auxiliary mode fusion weight, the weighting and fusion of the main mode feature and the auxiliary mode feature to generate the preliminary fused main feature, comprising: Taking the video feature sequence as the main sequence, and taking the speech and text feature sequence as the auxiliary sequence; Performing linear projection transformation on the main mode feature in the main sequence to generate a query vector; Performing global average pooling on the auxiliary mode feature in the auxiliary sequence along the time dimension to generate an auxiliary mode global feature vector, and performing linear projection transformation on the auxiliary mode global feature vector to generate a key vector; Based on the query vector and the key vector, the probability distribution of the main mode and the auxiliary mode is constructed respectively, the bidirectional distribution difference is calculated through Kullback-Leibler divergence, the relative information gain is defined, the relative information gain is mapped to the interval (0, 1) through the sigmoid activation function, and the auxiliary mode fusion weight is generated; Performing linear interpolation on the auxiliary mode feature to generate the time-aligned auxiliary mode feature; Weighting and fusing the main mode feature and the time-aligned auxiliary mode feature to generate the preliminary fused main feature. 3.The deep learning based unstructured data feature extraction method of claim 2, wherein: The layer normalization and GeLU activation function are applied to the preliminary fused main feature, and the dimension reduction processing is performed to generate the reduced fusion feature, comprising: Applying layer normalization and GeLU activation function to the preliminary fused main feature to generate a refined main fusion feature; The feature dimension of the refined main fusion feature is reduced through one-dimensional convolution operation to generate the reduced fusion feature. 4.The deep learning based unstructured data feature extraction method of claim 3, wherein: The channel attention weight is generated through the multilayer perception, and the corresponding attention weight is applied to each feature channel to generate the final feature vector, comprising: The channel attention weight is generated through the multilayer perception, and the corresponding attention weight is applied to each feature channel to generate the weighted fusion feature; The fifth-level weighted fusion feature is taken as the final distillation feature; The global average pooling is performed on the final distillation feature, and the L2 norm normalization is performed to generate the final feature vector. 5.The deep learning based unstructured data feature extraction method of claim 4, wherein: The constructed visualization interface displays the final feature vector, including: The visualization interface is constructed using the visualization tool Matplotlib to display the final feature vector in real time. 6.The deep learning based unstructured data feature extraction method of claim 1, wherein: The collection and preprocessing of unstructured data includes: Unstructured data is collected using intelligent devices and is time-stamped, denoised, and standardized; The intelligent devices include webcams, microphones, and smartphones; The unstructured data includes video, audio, and text data. 7.The deep learning based unstructured data feature extraction method of claim 6, wherein: The unstructured data is divided into aligned segment sets according to the same time window, and the normalized results are independently encoded to generate feature sequences, including: Set the time window and divide the unstructured data into aligned segment sets according to the same time window to generate aligned segment sets; Normalize the aligned segment sets to generate normalized results, including video tensors, mel spectrograms, and token ID sequences; Load the pre-trained ImageBind large model to independently encode the normalized results to generate modal features, including video, audio, and text features; Stack the modal features along the time dimension to obtain the feature sequence, including visual, audio, and text feature sequences.

8. The system for feature extraction of unstructured data based on deep learning according to any one of claims 1 to 7, characterized in that: It includes, The preprocessing module is used to collect unstructured data and perform time stamp synchronization, denoising, and standardization; The segmentation and encoding module is used to segment the data and independently encode the segmented data by loading a pre-trained model to generate feature sequences; The preliminary fusion module is used to construct the probability distribution of the main modal and the auxiliary modal, calculate the bidirectional distribution difference through Kullback-Leibler divergence, define the relative information gain, and perform weighted fusion on the modal features to generate the preliminary fused main features; The weight feature module is used to generate channel attention weights and apply corresponding weights to each feature channel to generate the final feature vector; The display module is used to construct a visualization interface to display the final feature vector. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the deep learning-based unstructured data feature extraction method of any one of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the deep learning-based unstructured data feature extraction method of any one of claims 1-7.

Images