Multi-granularity extraction and enhancement method, device and equipment for multi-source heterogeneous data

Abstract

The invention relates to the technical field of data processing, and discloses a multi-granularity extraction and enhancement method, device and equipment for multi-source heterogeneous data. The method comprises the following steps: performing multi-level feature extraction on a multi-modal data set and original feature parameters to obtain a multi-modal feature set; performing correlation analysis on the multi-modal feature set to obtain a feature correlation matrix and optimized feature representation; performing cross-modal knowledge migration processing on the feature incidence matrix and the optimized feature representation to obtain migration feature parameters; performing dynamic fusion processing on the migration feature parameters to obtain target fusion features; and performing multi-task segmentation processing according to the target fusion features to obtain a plurality of sub-task sequences, performing time sequence enhancement and multi-task fusion analysis on each sub-task sequence to obtain target extraction data, and performing visual display on the target extraction data. According to the method and the device, the efficiency and the accuracy of multi-granularity extraction and enhancement for multi-source heterogeneous data are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method, device and equipment for multi-granularity extraction and enhancement of multi-source heterogeneous data. Background Art

[0002] With the rapid development of information technology, multi-source heterogeneous data has been widely used in various fields. These data come from various sources, including different modalities such as text, images, videos and audio, and the data formats and structures are also different. Existing data processing technologies mainly focus on the processing of single-modal data. For example, text processing technology focuses on semantic analysis, image processing technology focuses on visual feature extraction, video processing technology focuses on dynamic information capture, and audio processing technology focuses on acoustic feature analysis. At the same time, some cross-modal data processing methods have begun to emerge, attempting to establish associations and mapping relationships between different modal data.

[0003] However, the existing technology still has the following shortcomings: first, feature extraction for multi-source heterogeneous data is mostly carried out independently, lacking in-depth exploration of the correlation between features; second, the knowledge transfer mechanism between different modal data is imperfect, making it difficult to fully utilize the complementary information between modalities; third, the feature fusion process often adopts a simple splicing or weighting method, failing to fully consider the hierarchy and dynamics of features; finally, the feature sharing and task association mechanisms in the multi-task processing process are not sound enough, making it difficult to achieve efficient use of resources. Summary of the invention

[0004] The present application provides a method, apparatus and device for multi-granularity extraction and enhancement of multi-source heterogeneous data, which are used to improve the efficiency and accuracy of multi-granularity extraction and enhancement of multi-source heterogeneous data.

[0005] In a first aspect, the present application provides a multi-granularity extraction and enhancement method for multi-source heterogeneous data, the multi-granularity extraction and enhancement method for multi-source heterogeneous data comprising: preprocessing the multi-source heterogeneous data to obtain a multimodal data set and original feature parameters, the original feature parameters comprising: content feature values, distribution feature values ​​and quality feature values; performing multi-level feature extraction on the multimodal data set and the original feature parameters to obtain a multimodal feature set; performing correlation analysis on the multimodal feature set to obtain a feature association matrix and an optimized feature representation; performing cross-modal knowledge transfer processing on the feature association matrix and the optimized feature representation to obtain transfer feature parameters; performing dynamic fusion processing on the transfer feature parameters to obtain target fusion features, the target fusion features comprising: bottom-level fusion features, middle-level fusion features and high-level fusion features; performing multi-task segmentation processing according to the target fusion features to obtain multiple sub-task sequences, and performing timing enhancement and multi-task fusion analysis on each sub-task sequence to obtain target extracted data, and visually displaying the target extracted data.

[0006] In a second aspect, the present application provides a multi-granularity extraction and enhancement device for multi-source heterogeneous data, the multi-granularity extraction and enhancement device for multi-source heterogeneous data comprising: A processing module is used to pre-process the multi-source heterogeneous data to obtain a multimodal data set and original feature parameters, wherein the original feature parameters include: content feature values, distribution feature values, and quality feature values; An extraction module, used for performing multi-level feature extraction on the multimodal data set and the original feature parameters to obtain a multimodal feature set; An analysis module, used to perform correlation analysis on the multimodal feature set to obtain a feature correlation matrix and an optimized feature representation; A migration module, used for performing cross-modal knowledge migration processing on the feature association matrix and the optimized feature representation to obtain migration feature parameters; A fusion module is used to dynamically fuse the migration feature parameters to obtain target fusion features, where the target fusion features include: bottom-level fusion features, middle-level fusion features, and high-level fusion features; The segmentation module is used to perform multi-task segmentation processing according to the target fusion features to obtain multiple sub-task sequences, and perform timing enhancement and multi-task fusion analysis on each sub-task sequence to obtain target extraction data, and visualize the target extraction data.

[0007] The third aspect of the present application provides a computer device, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate through a bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned multi-granularity extraction and enhancement method for multi-source heterogeneous data are performed.

[0008] In the technical solution provided by the present application, by preprocessing multi-source heterogeneous data, a multimodal data set and original feature parameters are obtained, which effectively solves the problems of inconsistent data formats and irregular feature expressions, and lays the foundation for subsequent feature extraction. Multi-level feature extraction is performed on the multimodal data set and the original feature parameters, which realizes the mining of data features from different granularities and levels, and improves the integrity and accuracy of feature expression. In the correlation analysis stage, by generating feature association matrices and optimizing feature representations, the intrinsic connections between different features are fully explored, and the expressive power of features is enhanced. The cross-modal knowledge transfer processing link realizes knowledge sharing and migration between different modalities by obtaining migration feature parameters, and improves the utilization efficiency of information between modalities. The target fusion features generated in the dynamic fusion processing stage include bottom-level fusion features, middle-level fusion features, and high-level fusion features, constructing a complete feature hierarchy structure and enhancing the richness of feature expression. Finally, multiple subtask sequences are obtained through multi-task segmentation processing, and time series enhancement and multi-task fusion analysis are performed to realize efficient decomposition and collaborative processing of tasks, and at the same time, the target extraction data is visualized, which improves the intuitiveness and interpretability of data analysis. The entire solution forms a complete multi-source heterogeneous data processing framework through the organic combination of multiple links, which not only improves the accuracy and efficiency of feature extraction, but also enhances the knowledge sharing ability between modalities, realizes the collaborative processing of multiple tasks, and ultimately achieves the unity of efficiency, accuracy and reliability of data analysis and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0010] Figure 1 A schematic diagram of an embodiment of a multi-granularity extraction and enhancement method for multi-source heterogeneous data in an embodiment of the present application; Figure 2 A schematic diagram of an embodiment of a multi-granularity extraction and enhancement device for multi-source heterogeneous data in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0011] The embodiments of the present application provide a multi-granularity extraction and enhancement method, device and equipment for multi-source heterogeneous data. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0012] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In one embodiment of the present application, a multi-granularity extraction and enhancement method for multi-source heterogeneous data includes: Step S101, preprocessing multi-source heterogeneous data to obtain a multimodal data set and original feature parameters, the original feature parameters including: content feature values, distribution feature values, and quality feature values; Step S102: performing multi-level feature extraction on the multimodal data set and the original feature parameters to obtain a multimodal feature set; Step S103: performing correlation analysis on the multimodal feature set to obtain a feature correlation matrix and an optimized feature representation; Step S104: performing cross-modal knowledge transfer processing on the feature association matrix and the optimized feature representation to obtain transfer feature parameters; Step S105: dynamically fuse the migration feature parameters to obtain target fusion features, which include bottom-layer fusion features, middle-layer fusion features, and high-layer fusion features; Step S106: perform multi-task segmentation processing according to the target fusion features to obtain multiple sub-task sequences, and perform time series enhancement and multi-task fusion analysis on each sub-task sequence to obtain target extraction data, and visualize the target extraction data.

[0013] It is understandable that the execution subject of the present application may be a multi-granularity extraction and enhancement device for multi-source heterogeneous data, or a terminal or a server, which is not limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0014] Specifically, multi-source heterogeneous data is preprocessed. Among them, multi-source heterogeneous data includes text, image, video and audio data. Text data contains a large number of invalid characters and noisy text; image data contains blur, uneven lighting and other problems; video data has problems such as different frame rates and jitter; audio data has problems such as noise interference and different sampling rates. In the preprocessing stage, adaptive data cleaning is performed, wavelet transform is used to remove noise, and text is segmented and standardized; histogram equalization is used to adjust image contrast, and median filtering is used to eliminate noise; video stability correction and frame rate standardization are performed; audio is denoised and sampling rate is unified. In feature parameter extraction, content feature values ​​reflect the core information volume of the data, which are obtained through semantic analysis, image analysis, time series analysis and other means; distribution feature values ​​describe the statistical laws of the data, including statistical quantities such as mean and variance; quality feature values ​​characterize the integrity and reliability of the data.

[0015] In the multi-level feature extraction stage, a hierarchical extraction strategy is adopted for different types of data. For text data, morphological features are extracted from the character level, semantic features are extracted from the word level, grammatical features are extracted from the sentence level, and thematic features are extracted from the document level. For image data, color and texture features are extracted from the pixel level, shape and edge features are extracted from the region level, semantic features are extracted from the object level, and context features are extracted from the scene level. Spatial features and temporal features are extracted from video data, including action features and scene change features. Time domain features, frequency domain features, and time-frequency joint features are extracted from audio data. In the correlation analysis, by constructing a feature correlation map, the correlation between different features is calculated to generate a feature correlation matrix. Feature selection and optimization algorithms are used to screen out the most representative feature combination to form an optimized feature representation. In the knowledge transfer process, the knowledge of the information-rich modality is transferred to the information-sparse modality to enhance the feature expression capability.

[0016] Taking the quality control of intelligent manufacturing production lines as an example, the input data includes production parameter records, product appearance inspection images, production process videos, and equipment operation audio. In the parameter records, each production record contains 50 process parameters such as temperature, pressure, and speed. After removing outliers through data cleaning, the distribution characteristic value of each parameter is calculated. For product appearance inspection images, the contrast is increased by 35% and the clarity is increased by 40% through image enhancement, and the color, texture and other features of the product surface are extracted. The production process video records the complete assembly process. Through the key frame extraction algorithm, representative frames are extracted from the original video, and the timing characteristics of the assembly process are calculated. The equipment operation audio retains the effective signal of 100Hz-5kHz through bandpass filtering to extract the working status characteristics of the equipment.

[0017] After feature extraction and association analysis, it was found that the correlation coefficient between process parameters and product appearance features reached 0.85, and the temporal correlation between assembly action features and equipment operation audio reached 0.78. Through knowledge transfer, the defect feature knowledge of appearance inspection was transferred to parameter analysis, which improved the recognition accuracy of parameter anomalies. In the dynamic fusion stage, the bottom-level fusion features include basic data features, such as temperature curves, pressure changes, etc.; the middle-level fusion features include state-related features, such as the correlation between parameter anomalies and appearance defects; the high-level fusion features include comprehensive evaluation features for the overall evaluation of product quality. Finally, the quality control task is decomposed into subtasks such as parameter monitoring, appearance inspection, and assembly verification to achieve quality control of the entire process. Through multi-dimensional data analysis and feature fusion, product quality problems can be accurately identified and traced back to specific process links to provide data support for production optimization.

[0018] In the embodiment of the present application, by preprocessing multi-source heterogeneous data, a multimodal data set and original feature parameters are obtained, which effectively solves the problems of inconsistent data format and non-standard feature expression, and lays the foundation for subsequent feature extraction. Multi-level feature extraction is performed on the multimodal data set and the original feature parameters, which realizes the mining of data features from different granularities and levels, and improves the integrity and accuracy of feature expression. In the correlation analysis stage, by generating a feature association matrix and optimizing feature representation, the intrinsic connection between different features is fully explored, and the expressive ability of features is enhanced. The cross-modal knowledge transfer processing link realizes knowledge sharing and migration between different modalities by obtaining migration feature parameters, and improves the utilization efficiency of information between modalities. The target fusion features generated in the dynamic fusion processing stage include bottom-level fusion features, middle-level fusion features, and high-level fusion features, constructing a complete feature hierarchy structure and enhancing the richness of feature expression. Finally, multiple subtask sequences are obtained through multi-task segmentation processing, and time series enhancement and multi-task fusion analysis are performed to realize efficient decomposition and collaborative processing of tasks, and at the same time, the target extraction data is visualized, which improves the intuitiveness and interpretability of data analysis. The entire solution forms a complete multi-source heterogeneous data processing framework through the organic combination of multiple links, which not only improves the accuracy and efficiency of feature extraction, but also enhances the knowledge sharing ability between modalities, realizes the collaborative processing of multiple tasks, and ultimately achieves the unity of efficiency, accuracy and reliability of data analysis and processing.

[0019] In a specific embodiment, the process of executing step S101 may specifically include the following steps: (1) Perform data cleaning on multi-source heterogeneous data to obtain initial data labeling results, and perform type recognition on the initial data labeling results to obtain data type identifiers; (2) Perform data analysis on the data type identifier to obtain basic data sequence and basic feature data; (3) Perform modal division processing on the basic data sequence to obtain a multimodal data set, and perform content analysis processing on the basic feature data to obtain content feature values; (4) Performing time-frequency decomposition processing on the basic feature data to obtain a multi-scale feature sequence, and performing distribution calculation processing on the multi-scale feature sequence to obtain a distribution feature value; (5) Perform quality assessment on the multi-scale feature sequence to obtain quality feature values, and combine the content feature values, distribution feature values, and quality feature values ​​to obtain original feature parameters.

[0020] Specifically, multi-source heterogeneous data refers to a collection of data from different sources and in different formats, including text, images, videos, and audio data. Data cleaning processing refers to the process of preprocessing the original data to remove outliers, noise, and redundant data. Specifically: for text data, remove special characters, repeated content, and meaningless symbols; for image data, remove blurred pixels, noise, and distorted areas; for video data, remove jittery frames, blurred frames, and intermittent frames; for audio data, remove background noise and distorted fragments. The initial data labeling results obtained after data cleaning contain basic attribute labels of the data, such as data generation time, data format, data size, and other information. These labeling results are processed by type recognition to identify the specific type and characteristics of the data and generate data type identifiers. The data type identifier contains information such as the modal type, format type, and structure type of the data.

[0021] Data analysis is performed on the data type identifier to analyze the intrinsic characteristics and structure of the data. Basic data sequences refer to the time series of raw data that have been preliminarily processed, and basic feature data refers to the basic features extracted from these sequences, such as word frequency features of text, color features of images, motion features of videos, and frequency features of audio.

[0022] Modal division processing is to classify and organize the basic data sequence according to different data modes (text, image, video, audio) to form a multimodal data set. Content analysis is performed on the basic feature data to extract the core features of the data content and obtain the content feature value. The content feature value reflects the main content information of the data, such as the theme features of the text, the target features of the image, the scene features of the video, and the voice features of the audio.

[0023] Time-frequency decomposition is to decompose the basic feature data in two dimensions, time and frequency, to obtain a multi-scale feature sequence. The multi-scale feature sequence contains feature information at different time scales and frequency scales. These sequences are processed by distribution calculation to calculate the statistical distribution characteristics of the features, such as mean, variance, skewness, kurtosis, etc., to obtain the distribution feature value.

[0024] The quality assessment process measures the quality of multi-scale feature sequences, evaluates the integrity, accuracy and reliability of the data, and obtains quality feature values. Finally, the content feature values ​​(reflecting data content), distribution feature values ​​(reflecting data distribution) and quality feature values ​​(reflecting data quality) are combined and processed to form a complete set of original feature parameters.

[0025] Taking intelligent manufacturing environment monitoring as an example, the input temperature sensor data (10 values ​​per second) is first cleaned to remove abnormal values ​​outside the range of 20-80℃, and the cleaned data is obtained and the sampling time and validity are marked. Type recognition determines that this is numerical time series data, and data analysis generates basic data sequences (temperature change sequence over time) and basic feature data (temperature change rate, temperature fluctuation range, etc.). Modal division classifies temperature data as numerical sensor data, and content analysis extracts temperature change feature values. Time-frequency decomposition obtains temperature change characteristics at different time scales (hours, days), and distribution calculation obtains statistical characteristics of temperature. Quality assessment calculates the completeness and accuracy of the data, and finally combines them to form original feature parameters containing temperature characteristics, distribution characteristics and quality indicators.

[0026] In a specific embodiment, the process of executing step S102 may specifically include the following steps: (1) Performing hierarchical parsing processing on the multimodal data set to obtain initial text data, initial image data, initial video data, and initial audio data, and performing feature matching processing on the initial text data, initial image data, initial video data, and initial audio data with the original feature parameters to obtain a basic feature sequence; (2) Perform deep semantic processing on the text part of the basic feature sequence to obtain text-level features, and perform spatial structure processing on the image part of the basic feature sequence to obtain image-level features; (3) Performing spatiotemporal change processing on the video part of the basic feature sequence to obtain video-level features, and performing spectrum analysis processing on the audio part of the basic feature sequence to obtain audio-level features; (4) Performing feature integration processing on text-level features, image-level features, video-level features, and audio-level features to obtain a multi-level feature matrix, and performing feature screening processing on the multi-level feature matrix to obtain a target feature vector; (5) Perform feature recombination processing on the target feature vector to obtain a multimodal feature set.

[0027] Specifically, hierarchical parsing is the process of hierarchical splitting of multimodal data sets. Structural parsing of text data, including sentence segmentation, word segmentation, and part-of-speech tagging, generates initial text data; pixel-level parsing of image data, including color space conversion, edge detection, and region segmentation, generates initial image data; frame-level parsing of video data, including frame extraction, scene segmentation, and motion detection, generates initial video data; sampling-level parsing of audio data, including time domain analysis, frequency domain conversion, and acoustic feature extraction, generates initial audio data. Feature matching is to correspond and map these initial data with original feature parameters, which include content feature values, distribution feature values, and quality feature values. Feature matching is performed by calculating the similarity matrix to obtain a basic feature sequence. The text part in the basic feature sequence is deeply semantically processed, and natural language processing technology is used to extract semantic information. First, a word vector space is constructed to calculate the semantic similarity between words, then sentence structure features are extracted through syntactic analysis, and finally document-level semantic features are extracted through topic modeling to form text hierarchical features including character level, word level, sentence level, and document level. The spatial structure of the image part is processed by first performing multi-scale decomposition, extracting color and texture features from the pixel level, shape and edge features from the region level, semantic features from the object level, and finally extracting global features from the scene level to obtain image hierarchical features.

[0028] The spatiotemporal change processing of the video part includes the extraction of spatial features and temporal features. The visual features of each frame are extracted in the spatial dimension, and the changes between frames are analyzed in the temporal dimension to capture motion information and scene transitions to obtain video hierarchical features. The spectrum analysis processing of the audio part includes short-time Fourier transform, Mel frequency cepstral coefficient extraction, pitch analysis, etc., and acoustic features are extracted from different frequency dimensions to obtain audio hierarchical features. Feature integration processing is to unify the representation and organization of hierarchical features of different modes. Through feature mapping, the features of different modes are projected into the same feature space to construct a multi-level feature matrix. The matrix is ​​subjected to feature screening, and the most representative feature combination is selected by principal component analysis and feature importance evaluation to obtain the target feature vector. Finally, through feature reorganization processing, the target feature vector is reorganized into a multimodal feature set to achieve structured representation of features.

[0029] Taking intelligent manufacturing quality control as a specific example, the input data includes production process parameter text (50 parameters are recorded per minute), product appearance inspection images (2048×1536 resolution), production process video (1080p, 30fps) and equipment operation audio (48kHz sampling rate). In the hierarchical parsing stage, the process parameter text is segmented to identify the process parameter name and value; the appearance inspection image is decomposed at multiple scales to construct a 4-layer image pyramid; the key frames of the production video are extracted at 2-second intervals; and the equipment audio is framed in units of 50ms.

[0030] In the feature matching stage, process parameters are compared with the set range, image features are matched with standard templates, video sequences are aligned with standard process flows, and audio features are compared with acoustic features of normal equipment operation. Deep semantic processing extracts the changing patterns and abnormal patterns of process parameters, spatial structure processing identifies surface defect features of products, spatiotemporal change processing captures abnormal movements in the production process, and spectrum analysis processing identifies abnormal vibration features of equipment. Through feature integration, these features are organized into a multidimensional feature matrix, where each row of the matrix represents a time point and each column represents a feature dimension. Feature screening retains the feature combination with the highest contribution, and ultimately generates a feature set that can fully characterize the production quality status.

[0031] In a specific embodiment, the process of executing step S103 may specifically include the following steps: (1) Perform semantic association processing on text-level features to obtain text association vectors, and perform spatial association processing on image-level features to obtain image association vectors; (2) Perform temporal correlation processing on the video level features to obtain a video correlation vector, and perform spectral correlation processing on the audio level features to obtain an audio correlation vector; (3) Perform cross-modal matching on the text association vector and the image association vector to obtain a text-image association matrix, and perform cross-modal matching on the video association vector and the audio association vector to obtain a video-audio association matrix; (4) Perform matrix fusion processing on the text-image correlation matrix and the audio-visual correlation matrix to obtain a feature correlation matrix, and perform feature extraction processing on the feature correlation matrix to obtain an optimized feature representation.

[0032] Specifically, semantic association processing is to calculate the association of semantic information in text-level features. For each text unit, the semantic relevance between words is first calculated, and the text is converted into a vector representation in a high-dimensional space through word vectorization. Semantic association processing includes word-level association, sentence-level association, and document-level association, and finally generates a text association vector. For spatial association processing of image-level features, the spatial dependency between different regions of the image is calculated, including the feature similarity of adjacent regions, the positional relationship and structural relationship between regions, and the spatial association graph of image features is constructed and converted into an image association vector. When performing temporal association processing on video-level features, it is necessary to analyze the change law of the video frame sequence in the time dimension. Calculate the feature changes between adjacent frames, extract motion trajectory information, identify scene transition points, construct temporal dependencies, and form a video association vector. The spectral association processing of audio-level features focuses on the association between different frequency components, and generates an audio association vector by analyzing the spectral energy distribution, frequency band correlation, and harmonic structure.

[0033] Cross-modal matching processing is a key step in establishing the correspondence between different modal data. The text association vector and the image association vector are matched, the correspondence between the semantic content and the visual content is calculated, and a text-image association matrix that reflects the strength of the association between the two is constructed. Similarly, the video association vector and the audio association vector are matched, and the temporal correspondence between visual changes and acoustic features is analyzed to obtain the audio-visual association matrix. The matrix fusion processing integrates the text-image association matrix and the audio-visual association matrix, and constructs a complete feature association matrix by calculating the mutual information between different modal features. The feature extraction processing extracts the most representative feature combination from the matrix to form an optimized feature representation.

[0034] Taking equipment monitoring in an intelligent manufacturing environment as an example, the input includes text records of equipment operation parameters (20 parameters are recorded per second), equipment surface temperature images (resolution 1024×768), equipment operation videos (720p, 25fps), and equipment vibration audio (96kHz sampling rate). Through semantic association processing, key parameters such as "speed", "temperature", and "vibration" in the parameter records are associated and analyzed, and the correlation coefficients between the parameters are calculated. When the correlation coefficient between speed and vibration exceeds 0.8, it is marked as a strongly correlated parameter pair. Spatial association processing analyzes the hotspot area of ​​the temperature image, calculates the spatial autocorrelation coefficient of the hotspot distribution, and identifies the spatial distribution pattern of the temperature abnormality area. Time series association processing analyzes the motion features in the equipment operation video, calculates the motion vector every 25 frames (1 second), and constructs a motion feature sequence. Spectral association processing performs short-time Fourier transform on the vibration audio according to a 100ms window, analyzes the spectral correlation in the range of 0-20kHz, and identifies the characteristic frequency combination. Text-image matching establishes a corresponding relationship between the temperature data in the parameter record and the temperature distribution of the thermal image, and constructs a 16×16 association matrix. Audio-visual matching generates a 32×32 correlation matrix by aligning the motion features of the video and the vibration features of the audio. Finally, through matrix fusion, the correlation information of all modalities is integrated into a 64×64 feature correlation matrix, and each element reflects the correlation strength between different features. Feature extraction retains feature combinations with correlation strengths exceeding 0.7, and ultimately obtains an optimized feature representation that can fully characterize the operating status of the device. It can not only discover feature associations within a single modality, but also identify feature correspondences across modalities.

[0035] In a specific embodiment, the process of executing step S104 may specifically include the following steps: (1) Perform matrix decomposition on the feature correlation matrix to obtain the correlation feature components, and perform probability distribution mapping on the correlation feature components to obtain the feature distribution sequence; (2) Calculate the entropy value of the feature distribution sequence to obtain the information entropy matrix, and perform feature probability processing on the optimized feature representation to obtain the conditional probability vector; (3) Perform mutual information calculation on the information entropy matrix and the conditional probability vector to obtain the knowledge transfer matrix, and perform feature weight allocation on the knowledge transfer matrix to obtain a weight coefficient sequence; (4) Perform marginal probability calculation on the weight coefficient sequence to obtain a probability distribution graph, and perform distribution consistency processing on the probability distribution graph to obtain a migration probability sequence; (5) Perform feature correlation processing on the migration probability sequence to obtain a correlation matrix, and perform threshold screening processing on the correlation matrix to obtain a screening feature sequence; (6) Performing migration loss calculation on the screening feature sequence to obtain a loss function vector, and performing parameter optimization on the loss function vector and the weight coefficient sequence to obtain an optimized parameter group; (7) Performing feature enhancement processing on the optimized parameter group to obtain an enhanced feature sequence, and performing parameter mapping processing on the enhanced feature sequence to obtain migration feature parameters, wherein the migration feature parameters include: feature importance parameters, structural similarity parameters, and modal conversion parameters.

[0036] Specifically, the feature association matrix is ​​subjected to matrix decomposition, and the singular value decomposition method is used to decompose the feature association matrix into eigenvectors and eigenvalues. The associated feature components represent the basic components of different feature dimensions, and each component reflects a main direction in the feature space. The probability distribution mapping process maps these associated feature components to the probability space, and generates a feature distribution sequence by calculating the distribution density function of each feature component. The entropy value of the feature distribution sequence is calculated, and the information entropy reflects the degree of uncertainty of the feature. For each feature dimension, the information entropy of its probability distribution is calculated to form an information entropy matrix. At the same time, the optimized feature representation is subjected to feature probability processing, and the probability distribution of each feature under given conditions is calculated to form a conditional probability vector. The mutual information calculation process measures the degree of information dependence between different features, and the knowledge transfer matrix is ​​constructed by calculating the mutual information between the information entropy matrix and the conditional probability vector. The feature weight allocation assigns importance weights to different features according to the size of the mutual information, and generates a weight coefficient sequence.

[0037] The marginal probability calculation process calculates the marginal distribution of each feature in the weight coefficient sequence to obtain a probability distribution graph. The distribution consistency process compares the probability distribution similarity of different features and adjusts the distribution parameters so that the feature distribution of the source domain and the target domain tends to be consistent, forming a migration probability sequence. The feature correlation process calculates the correlation coefficient between each feature in the migration probability sequence and constructs a correlation matrix. The threshold screening process sets the correlation threshold, retains the feature pairs with high correlation, and obtains the screening feature sequence. The migration loss calculation process evaluates the information loss in the knowledge transfer process, calculates the distance measure between the source domain features and the target domain features, and forms a loss function vector. The parameter optimization process minimizes the loss function while considering the weight coefficient sequence, adjusts the migration parameters, and obtains the optimized parameter group. The feature enhancement process uses the optimized parameter group to enhance the original features, improve the expressiveness of the features, and generate an enhanced feature sequence. The parameter mapping process maps the enhanced feature sequence to the standard feature space to obtain the migration feature parameters, including feature importance parameters, structural similarity parameters, and modal conversion parameters.

[0038] Taking the equipment status monitoring of smart factories as an example, the input data includes vibration sensor data (sampling rate 10kHz), temperature sensor data (sampling rate 1Hz), current sensor data (sampling rate 1kHz) and acoustic sensor data (sampling rate 48kHz). First, the 64×64 feature association matrix is ​​subjected to singular value decomposition, and the eigenvectors corresponding to the first 10 largest eigenvalues ​​are retained as the associated feature components. These feature components are mapped to probability distributions through kernel density estimation to obtain a 10-dimensional feature distribution sequence. The information entropy is calculated for each feature distribution to construct a 10×10 information entropy matrix. The conditional probability is calculated through Bayesian estimation to obtain a 64-dimensional conditional probability vector. The mutual information calculation shows that the mutual information value between the vibration feature and the acoustic feature is 0.85, and the mutual information value between the temperature feature and the current feature is 0.72. Based on this, the knowledge transfer matrix is ​​constructed and the feature weights are assigned.

[0039] Through calculation, it is found that the probability distribution of vibration signals in the 20-100Hz frequency band and acoustic signals in the 1-5kHz frequency band are highly similar (similarity 0.92), and the migration parameters are adjusted accordingly. Feature correlation analysis shows that the correlation coefficient between temperature change and current fluctuation is 0.78, and a threshold value of 0.7 is set for feature screening. Migration loss calculation shows that the feature dimension can be reduced from 64 dimensions to 32 dimensions while retaining 90% of the information. The migration feature parameters finally obtained through parameter optimization include: feature importance parameters (reflecting the weight of vibration-acoustic feature pairs 1.5, and the weight of temperature-current feature pairs 1.2), structural similarity parameters (spectral structure similarity 0.92, temporal structure similarity 0.85) and modal conversion parameters (vibration to acoustic mapping coefficient 0.8, temperature to current mapping coefficient 0.75). These parameters effectively guide the knowledge transfer process between different modes and achieve optimization and enhancement of data representation.

[0040] In a specific embodiment, the process of executing step S105 may specifically include the following steps: (1) Perform feature grouping processing on the feature importance parameters to obtain the underlying feature group, and perform edge feature extraction processing on the underlying feature group to obtain the underlying fusion feature; (2) Perform semantic association processing on the structural similarity parameters to obtain a mid-level feature group, and perform content feature association processing on the mid-level feature group to obtain a mid-level fusion feature; (3) Perform high-dimensional mapping processing on the modal conversion parameters to obtain a high-level feature group, and perform context feature association processing on the high-level feature group to obtain a high-level fusion feature; (4) Perform feature hierarchy division processing on the bottom-level fusion features, the middle-level fusion features, and the high-level fusion features to obtain a multi-level feature sequence, and perform inter-layer relationship analysis on the multi-level feature sequence to obtain an inter-layer feature vector; (5) Adaptively adjust the inter-layer feature vectors to obtain an adjustment parameter matrix, and perform feature combination processing on the adjustment parameter matrix and multi-level feature sequences to obtain the target fusion features.

[0041] Specifically, the feature importance parameters are grouped. The feature importance parameters are numerical indicators that reflect the contribution of each feature to the target task. Through cluster analysis, features with similar importance levels are grouped to form a bottom-level feature group. The bottom-level feature group contains basic data features, such as word frequency features of text, pixel features of images, frame features of videos, and waveform features of audio. The edge feature extraction process is to extract local significant features from the bottom-level feature group, including keyword features of text, edge contour features of images, motion boundary features of videos, and spectral edge features of audio. These features together constitute the bottom-level fusion features. The structural similarity parameters describe the structural correspondence between different features. Through semantic association processing, features with similar semantic structures are combined to form a middle-level feature group. The middle-level feature group contains higher-level semantic information, such as the syntactic structure of text, the target structure of images, the scene structure of videos, and the harmonic structure of audio. The content feature association process analyzes the semantic connection between these structural features, extracts co-occurrence patterns and dependencies, and generates middle-level fusion features.

[0042] The modal conversion parameters describe the conversion rules between different modal data. These parameters are projected into the high-dimensional feature space through high-dimensional mapping processing to form a high-level feature group. The high-level feature group contains semantic concepts at the abstract level, such as the theme features of text, the scene semantics of images, the event features of videos, and the semantic features of audio. The context feature association processing analyzes the semantic associations of these high-level features in different scenarios, constructs the context dependency of features, and obtains high-level fusion features. The feature hierarchy division processing organizes the bottom-level fusion features, the middle-level fusion features, and the high-level fusion features according to the semantic hierarchy to construct a multi-level feature sequence. The inter-layer relationship analysis processing studies the dependency between features at different levels, including the feature abstraction relationship from bottom to top and the feature refinement relationship from top to bottom, and generates inter-layer feature vectors. The adaptive adjustment processing dynamically adjusts the weights of features at different levels according to the importance of the features and the task requirements to form an adjustment parameter matrix. The feature combination processing performs weighted fusion of the adjustment parameter matrix and the multi-level feature sequence to obtain the final target fusion feature.

[0043] Taking intelligent video analysis as a specific example, the input includes video subtitle text, visual image sequence, action video clip and scene audio. The feature importance parameter shows that the key action words in the text (importance weight 0.8), the human target in the image (importance weight 0.9), the action trajectory in the video (importance weight 0.85) and the speech part in the audio (importance weight 0.75) have high importance, and these features are grouped into the bottom feature group. Through edge feature extraction, the action description words in the text, the human outline in the image (edge ​​points with confidence greater than 0.9), the motion boundary in the video (area with optical flow gradient greater than 0.5) and the speech paragraph in the audio (segment with signal-to-noise ratio greater than 15dB) are identified. In the middle-level feature processing, the structural similarity parameter shows that the structural similarity between the action description words and the motion trajectory is 0.82, and the structural similarity between the human outline and the speech feature is 0.78. Through semantic association analysis, these features are organized into an action-speech correspondence graph, and each node contains a feature descriptor and an association strength. Content feature correlation processing found that when the movement amplitude increased, the speech intensity also increased accordingly, and the correlation coefficient reached 0.85.

[0044] In high-level feature processing, modality conversion parameters guide feature mapping between different modalities, such as mapping visual action features to speech feature space (mapping accuracy 0.88), and mapping text semantics to visual scene space (mapping accuracy 0.85). Context feature association analysis shows that the combination pattern of action-speech features is significantly different in different scenarios, and the scene correlation reaches 0.92. Feature hierarchy division organizes these features into a three-layer structure: the bottom layer contains basic features (64-dimensional feature vectors), the middle layer contains semantic features (32-dimensional feature vectors), and the high layer contains scene features (16-dimensional feature vectors). Inter-layer relationship analysis shows that the feature abstraction accuracy from the bottom layer to the middle layer is 0.85, and the feature abstraction accuracy from the middle layer to the high layer is 0.82. Through adaptive adjustment, the weight of the bottom layer feature is set to 0.3, the weight of the middle layer feature is set to 0.4, and the weight of the high layer feature is set to 0.3, generating a 112-dimensional target fusion feature, realizing the effective fusion of multimodal data.

[0045] In a specific embodiment, the process of executing step S106 may specifically include the following steps: (1) Perform density clustering on the target fusion features to obtain a feature density map, and perform task boundary detection on the feature density map to obtain the initial task sequence; (2) Calculate the task relevance of the initial task sequence to obtain a relevance vector, and perform hierarchical clustering on the relevance vector to obtain multiple subtask sequences; (3) Extract the time series features of multiple subtask sequences to obtain a time series feature matrix, and perform autocorrelation analysis on the time series feature matrix to obtain a periodic feature vector; (4) Performing characteristic spectrum analysis on the periodic characteristic vector to obtain a spectrum distribution diagram, and performing principal component extraction on the spectrum distribution diagram to obtain a principal component sequence; (5) Calculate the temporal correlation of the principal component sequence to obtain a correlation feature group, and perform subtask feature mapping on the correlation feature group to obtain a mapping matrix set; (6) performing multi-task feature fusion processing on the mapping matrix set to obtain a fused feature vector, and performing feature reconstruction processing on the fused feature vector to obtain target extraction data, wherein the target extraction data includes: time series feature data, correlation feature data, and fused feature data; (7) Perform dimensionality reduction projection processing on the time series feature data to obtain a two-dimensional feature map, and perform hierarchical layout processing on the associated feature data to obtain a layout structure map; (8) Dynamically render the fused feature data to obtain an interactive view, and perform scene synthesis processing on the two-dimensional feature map, layout structure map, and interactive view to obtain a visualization interface.

[0046] Specifically, the target fusion features are subjected to density clustering. Density clustering is a method of clustering based on the density distribution of data points in the feature space. The core is to calculate the local density of each data point and the distance to the high-density point. By setting the density threshold (local density is greater than 1.5 times the average density) and the distance threshold (distance is less than 0.5 times the average distance in the feature space), the density center point is identified and a feature density map is generated. The task boundary detection process determines the boundaries of different task categories by analyzing the density jump area in the density map to form an initial task sequence. Task relevance calculation is to quantitatively analyze the degree of association between each task in the initial task sequence. The cosine similarity is used to calculate the similarity between task feature vectors and generate a relevance vector. Hierarchical clustering clusters the relevance vector in a top-down manner, sets the clustering threshold (relevance is greater than 0.7), and combines related tasks into subtask sequences. Each subtask sequence represents a set of subtasks with similar features and goals.

[0047] The time series feature extraction process analyzes the changing rules of the subtask sequence in the time dimension. The sequence is segmented through a sliding window (the window size is 30 time points), and the statistical features, trend features, and fluctuation features of each segment are extracted to construct a time series feature matrix. The autocorrelation analysis process calculates the correlation coefficient of the feature sequence under different time delays, identifies the periodic pattern of the sequence, and obtains the periodic feature vector. The feature spectrum analysis process converts the periodic feature vector to the frequency domain, and calculates the spectrum distribution map through fast Fourier transform. The principal component extraction process performs principal component analysis on the spectrum distribution map, retains the principal components with a cumulative contribution rate of more than 90%, and forms a principal component sequence. The time series correlation calculation process analyzes the correlation of the principal component sequence in the time dimension, calculates the correlation coefficient at different time scales, and generates a correlation feature group. The subtask feature mapping process maps the correlation feature group to the feature space of each subtask and constructs a mapping matrix set.

[0048] The multi-task feature fusion process performs a weighted combination of the features in the mapping matrix set, with the weights determined based on the task importance and feature significance, to generate a fused feature vector. The feature reconstruction process converts the fused feature vector into an interpretable data form to obtain target extraction data containing time series feature data, associated feature data, and fused feature data.

[0049] Taking the production line monitoring of smart factories as an example, the input data includes continuous monitoring records of equipment operation parameters (100 parameters per second), quality inspection data (10 indicators per minute), and environmental monitoring data (1 group every 10 seconds). Density cluster analysis found three main task density centers, corresponding to equipment maintenance, quality control, and environmental management tasks. Task correlation analysis showed that the correlation between equipment maintenance and quality control was 0.85, and the correlation between quality control and environmental management was 0.72. Based on this, the tasks were decomposed into 8 subtask sequences. Time series feature analysis showed that equipment parameters fluctuated periodically every 8 hours (corresponding to production shifts), quality indicators fluctuated every 2 hours (corresponding to product batches), and environmental parameters showed a diurnal variation every 24 hours. Spectral analysis extracted the frequency components corresponding to these three main cycles, and principal component analysis retained 5 main components, explaining 92% of the data variance.

[0050] Correlation analysis found that there is a significant correlation between equipment temperature and product quality, and between environmental humidity and product performance (correlation coefficients are 0.82 and 0.75, respectively). Feature mapping converts these correlations into a 64×64 mapping matrix, and generates a 128-dimensional feature vector through feature fusion. t-SNE dimension reduction is performed on the time series feature data to generate a 2D feature scatter plot; a hierarchical force-directed graph is constructed for the associated feature data to display the correlation between features; and the fused feature data is dynamically rendered to generate an interactive heat map. The three views are integrated into a unified visualization interface to achieve intuitive display and interactive analysis of multi-dimensional data.

[0051] The above describes the multi-granularity extraction and enhancement method for multi-source heterogeneous data in the embodiment of the present application. The following describes the multi-granularity extraction and enhancement device for multi-source heterogeneous data in the embodiment of the present application. Figure 2 In one embodiment of the present application, a multi-granularity extraction and enhancement device for multi-source heterogeneous data includes: The processing module 201 is used to pre-process the multi-source heterogeneous data to obtain a multi-modal data set and original feature parameters, wherein the original feature parameters include: content feature values, distribution feature values, and quality feature values; An extraction module 202, configured to perform multi-level feature extraction on the multimodal data set and the original feature parameters to obtain a multimodal feature set; An analysis module 203 is used to perform correlation analysis on the multimodal feature set to obtain a feature correlation matrix and an optimized feature representation; A migration module 204, configured to perform cross-modal knowledge migration processing on the feature association matrix and the optimized feature representation to obtain migration feature parameters; A fusion module 205 is used to dynamically fuse the migration feature parameters to obtain target fusion features, where the target fusion features include: bottom-level fusion features, middle-level fusion features, and high-level fusion features; The segmentation module 206 is used to perform multi-task segmentation processing according to the target fusion features to obtain multiple sub-task sequences, and perform time enhancement and multi-task fusion analysis on each sub-task sequence to obtain target extraction data, and visualize the target extraction data.

[0052] Through the collaborative cooperation of the above components, multi-source heterogeneous data are preprocessed to obtain multimodal data sets and original feature parameters, which effectively solves the problems of inconsistent data formats and irregular feature expressions, and lays the foundation for subsequent feature extraction. Multi-level feature extraction is performed on multimodal data sets and original feature parameters, which realizes the mining of data features from different granularities and levels, and improves the completeness and accuracy of feature expression. In the correlation analysis stage, by generating feature association matrices and optimizing feature representations, the intrinsic connections between different features are fully mined, and the feature expression ability is enhanced. The cross-modal knowledge transfer processing link realizes knowledge sharing and transfer between different modalities by obtaining transfer feature parameters, and improves the utilization efficiency of information between modalities. The target fusion features generated in the dynamic fusion processing stage include bottom-level fusion features, middle-level fusion features, and high-level fusion features, constructing a complete feature hierarchy structure and enhancing the richness of feature expression. Finally, multiple subtask sequences are obtained through multi-task segmentation processing, and time series enhancement and multi-task fusion analysis are performed to achieve efficient decomposition and collaborative processing of tasks. At the same time, the target extraction data is visualized, which improves the intuitiveness and interpretability of data analysis. The entire solution forms a complete multi-source heterogeneous data processing framework through the organic combination of multiple links, which not only improves the accuracy and efficiency of feature extraction, but also enhances the knowledge sharing ability between modalities, realizes the collaborative processing of multiple tasks, and ultimately achieves the unity of efficiency, accuracy and reliability of data analysis and processing.

[0053] Based on the same technical concept, the embodiment of the present application also provides an electronic device. Figure 3 As shown, it is a schematic diagram of the structure of the electronic device 300 provided in the embodiment of the present application, including a processor 301, a memory 302, and a bus 303. Among them, the memory 302 is used to store execution instructions, including a memory 3021 and an external memory 3022; the memory 3021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 301, and the data exchanged with the external memory 3022 such as a hard disk. The processor 301 exchanges data with the external memory 3022 through the memory 3021. When the electronic device 300 is running, the processor 301 communicates with the memory 302 through the bus 303.

[0054] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0055] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0056] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-granularity extraction and enhancement method for multi-source heterogeneous data, characterized in that: The multi-granularity extraction and enhancement method for multi-source heterogeneous data includes: Preprocessing multi-source heterogeneous data to obtain a multimodal data set and original feature parameters, wherein the original feature parameters include: content feature values, distribution feature values, and quality feature values; Performing multi-level feature extraction on the multimodal data set and the original feature parameters to obtain a multimodal feature set; Performing correlation analysis on the multimodal feature set to obtain a feature correlation matrix and an optimized feature representation; Performing cross-modal knowledge transfer processing on the feature association matrix and the optimized feature representation to obtain transfer feature parameters; Dynamically fusing the migration feature parameters to obtain target fusion features, where the target fusion features include: bottom-level fusion features, middle-level fusion features, and high-level fusion features; Multi-task segmentation processing is performed according to the target fusion features to obtain multiple sub-task sequences, and time series enhancement and multi-task fusion analysis are performed on each sub-task sequence to obtain target extraction data, and the target extraction data is visualized.

2. The multi-granularity extraction and enhancement method for multi-source heterogeneous data according to claim 1, characterized in that: The multi-source heterogeneous data is preprocessed to obtain a multimodal data set and original feature parameters, wherein the original feature parameters include: content feature values, distribution feature values, and quality feature values, including: Performing data cleaning processing on multi-source heterogeneous data to obtain initial data labeling results, and performing type recognition processing on the initial data labeling results to obtain data type identifiers; Performing data analysis on the data type identifier to obtain a basic data sequence and basic feature data; Performing modal division processing on the basic data sequence to obtain a multimodal data set, and performing content analysis processing on the basic feature data to obtain a content feature value; Performing time-frequency decomposition processing on the basic feature data to obtain a multi-scale feature sequence, and performing distribution calculation processing on the multi-scale feature sequence to obtain a distribution feature value; The multi-scale feature sequence is subjected to quality assessment processing to obtain a quality feature value, and the content feature value, the distribution feature value and the quality feature value are subjected to combination processing to obtain an original feature parameter.

3. The multi-granularity extraction and enhancement method for multi-source heterogeneous data according to claim 1, characterized in that: The multi-level feature extraction is performed on the multi-modal data set and the original feature parameters to obtain a multi-modal feature set, including: Performing hierarchical parsing processing on the multimodal data set to obtain initial text data, initial image data, initial video data and initial audio data, and performing feature matching processing on the initial text data, the initial image data, the initial video data and the initial audio data with the original feature parameters to obtain a basic feature sequence; Performing deep semantic processing on the text part in the basic feature sequence to obtain text-level features, and performing spatial structure processing on the image part in the basic feature sequence to obtain image-level features; Performing spatiotemporal change processing on the video part of the basic feature sequence to obtain video level features, and performing spectrum analysis processing on the audio part of the basic feature sequence to obtain audio level features; Performing feature integration processing on the text level features, the image level features, the video level features and the audio level features to obtain a multi-level feature matrix, and performing feature screening processing on the multi-level feature matrix to obtain a target feature vector; The target feature vector is subjected to feature recombination processing to obtain a multimodal feature set.

4. The multi-granularity extraction and enhancement method for multi-source heterogeneous data according to claim 3 is characterized in that: The performing correlation analysis on the multimodal feature set to obtain a feature correlation matrix and an optimized feature representation includes: Performing semantic association processing on the text-level features to obtain a text association vector, and performing spatial association processing on the image-level features to obtain an image association vector; Performing temporal correlation processing on the video level features to obtain a video correlation vector, and performing spectrum correlation processing on the audio level features to obtain an audio correlation vector; Performing cross-modal matching processing on the text association vector and the image association vector to obtain a text-image association matrix, and performing cross-modal matching processing on the video association vector and the audio association vector to obtain a video-audio association matrix; The text-image association matrix and the audio-visual association matrix are subjected to matrix fusion processing to obtain a feature association matrix, and feature extraction processing is performed on the feature association matrix to obtain an optimized feature representation.

5. The multi-granularity extraction and enhancement method for multi-source heterogeneous data according to claim 1, characterized in that: The cross-modal knowledge transfer processing is performed on the feature association matrix and the optimized feature representation to obtain the transfer feature parameters, including: Performing matrix decomposition processing on the feature association matrix to obtain associated feature components, and performing probability distribution mapping processing on the associated feature components to obtain a feature distribution sequence; Performing entropy calculation processing on the feature distribution sequence to obtain an information entropy matrix, and performing feature probability processing on the optimized feature representation to obtain a conditional probability vector; Performing mutual information calculation processing on the information entropy matrix and the conditional probability vector to obtain a knowledge transfer matrix, and performing feature weight allocation processing on the knowledge transfer matrix to obtain a weight coefficient sequence; Performing edge probability calculation processing on the weight coefficient sequence to obtain a probability distribution graph, and performing distribution consistency processing on the probability distribution graph to obtain a migration probability sequence; Performing feature correlation processing on the migration probability sequence to obtain a correlation matrix, and performing threshold screening processing on the correlation matrix to obtain a screening feature sequence; Performing migration loss calculation processing on the screening feature sequence to obtain a loss function vector, and performing parameter optimization processing on the loss function vector and the weight coefficient sequence to obtain an optimization parameter group; The optimized parameter group is subjected to feature enhancement processing to obtain an enhanced feature sequence, and the enhanced feature sequence is subjected to parameter mapping processing to obtain migration feature parameters, wherein the migration feature parameters include: feature importance parameters, structural similarity parameters and modal conversion parameters.

6. The multi-granularity extraction and enhancement method for multi-source heterogeneous data according to claim 5, characterized in that: The dynamic fusion processing is performed on the migration feature parameters to obtain target fusion features, and the target fusion features include: bottom-level fusion features, middle-level fusion features and high-level fusion features, including: Performing feature grouping processing on the feature importance parameters to obtain bottom-level feature groups, and performing edge feature extraction processing on the bottom-level feature groups to obtain bottom-level fusion features; Performing semantic association processing on the structural similarity parameters to obtain a middle-level feature group, and performing content feature association processing on the middle-level feature group to obtain a middle-level fusion feature; Performing high-dimensional mapping processing on the modal conversion parameters to obtain a high-level feature group, and performing context feature association processing on the high-level feature group to obtain a high-level fusion feature; Performing feature hierarchy division processing on the bottom-layer fusion features, the middle-layer fusion features, and the high-layer fusion features to obtain a multi-level feature sequence, and performing inter-layer relationship analysis processing on the multi-level feature sequence to obtain an inter-layer feature vector; The inter-layer feature vectors are adaptively adjusted to obtain an adjustment parameter matrix, and the adjustment parameter matrix and the multi-level feature sequence are subjected to feature combination processing to obtain target fusion features.

7. The multi-granularity extraction and enhancement method for multi-source heterogeneous data according to claim 1, characterized in that: The multi-task segmentation processing is performed according to the target fusion feature to obtain multiple sub-task sequences, and each sub-task sequence is respectively subjected to time series enhancement and multi-task fusion analysis to obtain target extraction data, and the target extraction data is visualized, including: Performing density clustering processing on the target fusion features to obtain a feature density map, and performing task boundary detection processing on the feature density map to obtain an initial task sequence; Performing task relevance calculation processing on the initial task sequence to obtain a relevance vector, and performing hierarchical clustering processing on the relevance vector to obtain multiple subtask sequences; Performing time series feature extraction processing on the plurality of subtask sequences to obtain a time series feature matrix, and performing autocorrelation analysis processing on the time series feature matrix to obtain a period feature vector; Performing characteristic spectrum analysis processing on the periodic characteristic vector to obtain a spectrum distribution diagram, and performing principal component extraction processing on the spectrum distribution diagram to obtain a principal component sequence; Performing time series correlation calculation processing on the principal component sequence to obtain a correlation feature group, and performing subtask feature mapping processing on the correlation feature group to obtain a mapping matrix set; Performing multi-task feature fusion processing on the mapping matrix set to obtain a fused feature vector, and performing feature reconstruction processing on the fused feature vector to obtain target extraction data, wherein the target extraction data includes: time series feature data, correlation feature data and fused feature data; Performing dimensionality reduction projection processing on the time series feature data to obtain a two-dimensional feature graph, and performing hierarchical layout processing on the associated feature data to obtain a layout structure graph; The fused feature data is dynamically rendered to obtain an interactive view, and the two-dimensional feature map, the layout structure map and the interactive view are subjected to scene synthesis to obtain a visualization interface.

8. A multi-granularity extraction and enhancement device for multi-source heterogeneous data, used to implement the multi-granularity extraction and enhancement method for multi-source heterogeneous data as described in any one of claims 1 to 7, characterized in that: The multi-granularity extraction and enhancement device for multi-source heterogeneous data includes: A processing module is used to pre-process the multi-source heterogeneous data to obtain a multimodal data set and original feature parameters, wherein the original feature parameters include: content feature values, distribution feature values, and quality feature values; An extraction module, used for performing multi-level feature extraction on the multimodal data set and the original feature parameters to obtain a multimodal feature set; An analysis module, used for performing correlation analysis on the multimodal feature set to obtain a feature correlation matrix and an optimized feature representation; A migration module, used for performing cross-modal knowledge migration processing on the feature association matrix and the optimized feature representation to obtain migration feature parameters; A fusion module is used to dynamically fuse the migration feature parameters to obtain target fusion features, where the target fusion features include: bottom-level fusion features, middle-level fusion features, and high-level fusion features; The segmentation module is used to perform multi-task segmentation processing according to the target fusion features to obtain multiple sub-task sequences, and perform timing enhancement and multi-task fusion analysis on each sub-task sequence to obtain target extraction data, and visualize the target extraction data.

9. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the multi-granularity extraction and enhancement method for multi-source heterogeneous data as described in any one of claims 1 to 7 are performed.

Images