Multi-modal data enhancement method and system based on feature space alignment

Abstract

The invention discloses a multi-modal data enhancement method and system based on feature space alignment. The method comprises the following steps: collecting modal original data streams of multiple unmanned aerial vehicles; reversely deducing an internal generation rule from a data final state, and constructing a sparse coding feature representation library through causal pilot signal extraction and singular trajectory analysis; constructing an adaptive weight map according to the feature representation library, and identifying a space alignment path through dynamic calibration processing to generate an alignment imbalance index; performing cross information entropy analysis to establish a collaborative enhancement chain, and identifying a collaborative enhancement mode based on the information coupling degree to generate a complementary enhancement vector; performing multi-dimensional projection reconstruction on the complementary enhancement vector, and determining feature mapping probability distribution through diffusion time inversion to obtain a multi-dimensional projection feature map; carrying out nonlinear enhancement processing to generate an enhancement expansion mode, and identifying an enhancement key node to construct an enhancement regulation and control sequence; and outputting hierarchical enhancement data based on the regulation and control sequence and the projection feature map, and realizing accurate space alignment and collaborative enhancement of multi-sensor data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-modal data enhancement method and system based on feature space alignment. BACKGROUND

[0002] Modern unmanned aerial vehicle systems usually carry multiple sensor devices, including optical cameras, infrared sensors, radar systems, inertial navigation devices, etc., for obtaining multi-dimensional perception data of target areas. These heterogeneous sensors can describe the same target or scene from different physical dimensions, providing rich information sources for intelligent perception and autonomous decision-making of unmanned aerial vehicles.

[0003] However, existing unmanned aerial vehicle multi-sensor data processing technologies have many shortcomings. First, the data obtained by different sensors has large differences in sampling frequency, data format, coordinate system, etc., and simple data splicing or superposition processing cannot fully utilize the complementary advantages of each modal data. Second, traditional data fusion methods lack deep mining of the internal correlation of sensor data, and cannot effectively identify and utilize the collaborative relationship between modalities. In addition, existing technologies usually adopt a unified processing strategy, lack the ability to adaptively adjust according to specific task requirements and system resource constraints, resulting in uneven processing effects in different application scenarios. These technical limitations seriously restrict the application value of unmanned aerial vehicle multi-sensor data and affect the perception accuracy and decision-making ability of unmanned aerial vehicles in complex environments. SUMMARY

[0004] The present application provides a multi-modal data enhancement method and system based on feature space alignment, which deeply analyzes the internal generation mechanism of unmanned aerial vehicle multi-sensor data through reverse feature mining technology, constructs an adaptive weight atlas to realize accurate spatial alignment of heterogeneous data, uses cross information entropy analysis to mine inter-modal collaborative enhancement patterns, and combines multi-dimensional projection reconstruction and nonlinear enhancement processing technology to realize hierarchical adaptive data enhancement for different unmanned aerial vehicle task scenarios, improving the feature expression ability, spatial consistency and task adaptability of unmanned aerial vehicle multi-modal perception data.

[0005] The present application provides a multi-modal data enhancement method and system based on feature space alignment, which deeply analyzes the internal generation mechanism of unmanned aerial vehicle multi-sensor data through reverse feature mining technology, constructs an adaptive weight atlas to realize accurate spatial alignment of heterogeneous data, uses cross information entropy analysis to mine inter-modal collaborative enhancement patterns, and combines multi-dimensional projection reconstruction and nonlinear enhancement processing technology to realize hierarchical adaptive data enhancement for different unmanned aerial vehicle task scenarios, improving the feature expression ability, spatial consistency and task adaptability of unmanned aerial vehicle multi-modal perception data. Collecting multi-modal raw data streams, applying reverse feature mining to the multi-modal raw data streams to generate a feature representation library; Constructing an adaptive weight atlas according to the feature representation library, performing dynamic calibration processing on the adaptive weight atlas to identify a spatial alignment path, deriving an alignment coefficient from the spatial alignment path to generate an alignment coefficient library, and performing bias detection using the alignment coefficient library to generate an alignment imbalance index; Cross information entropy analysis is performed on the feature representation library to establish a synergistic enhancement chain, the information coupling degree of different modal data is determined by using the synergistic enhancement chain, the synergistic enhancement mode is identified by using the information coupling degree, and the complementary enhancement vector is generated by amplifying the alignment imbalance index according to the synergistic enhancement mode; Multi-dimensional projection reconstruction is performed on the complementary enhancement vector to generate a projection coding feature, diffusion time inversion is performed on the projection coding feature to determine the probability distribution of feature mapping, a mapping path is reconstructed by sampling from the probability distribution, and a multi-dimensional projection feature map is obtained by deriving a structure enhancement feature according to the mapping path; The alignment imbalance index and the complementary enhancement vector are subjected to nonlinear enhancement processing to generate an enhanced expansion mode, an enhanced key node is identified from the enhanced expansion mode, an enhanced control coefficient is determined according to the enhanced key node, and an enhanced control sequence is constructed by using the enhanced control coefficient; Based on the enhanced control sequence and the multi-dimensional projection feature map, hierarchical enhancement data is output, and multi-modal data enhancement based on feature space alignment is completed.

[0006] The second aspect of the application proposes a multi-modal data enhancement system based on feature space alignment, comprising: A data mining module is used to collect multi-modal original data streams, and reverse feature mining is performed on the multi-modal original data streams to generate a feature representation library; A space alignment module is used to construct an adaptive weight atlas according to the feature representation library, perform dynamic calibration processing on the adaptive weight atlas to identify a space alignment path, derive an alignment coefficient from the space alignment path to generate an alignment coefficient library, and perform deviation detection to generate an alignment imbalance index by using the alignment coefficient library; A synergistic enhancement module is used to perform cross information entropy analysis on the feature representation library to establish a synergistic enhancement chain, determine the information coupling degree of different modal data by using the synergistic enhancement chain, identify a synergistic enhancement mode by using the information coupling degree, and generate a complementary enhancement vector by amplifying the alignment imbalance index according to the synergistic enhancement mode; A projection reconstruction module is used to perform multi-dimensional projection reconstruction on the complementary enhancement vector to generate a projection coding feature, perform diffusion time inversion on the projection coding feature to determine the probability distribution of feature mapping, sample a reconstruction mapping path from the probability distribution, and derive a structure enhancement feature to obtain a multi-dimensional projection feature map according to the mapping path; An enhancement control module is used to perform nonlinear enhancement processing on the alignment imbalance index and the complementary enhancement vector to generate an enhanced expansion mode, identify an enhanced key node from the enhanced expansion mode, determine an enhanced control coefficient according to the enhanced key node, and construct an enhanced control sequence by using the enhanced control coefficient; The data output module is configured to output hierarchical enhancement data based on the enhanced regulation sequence and the multi-dimensional projection feature map, and complete multi-modal data enhancement based on feature space alignment.

[0007] The beneficial effects of the present application are reflected in the following points: first, by reverse feature mining, the evolution terminal state of the unmanned aerial vehicle multi-sensor data is reversely traced to generate the feature generation law, compared with the traditional forward extraction method, the key feature components can be more accurately identified, the adaptive weight atlas construction and dynamic calibration processing realize the accurate alignment of heterogeneous sensor data in a unified feature space, effectively eliminate the feature drift and semantic conflict between modalities, and improve the feature extraction accuracy and spatial registration accuracy. Second, the cross information entropy analysis is used to correlate and mine different modal data, accurately quantify the information coupling strength between modalities, and the synergistic enhancement chain can fully mine the complementary value of each sensor data, the multi-dimensional projection reconstruction combines the diffusion time inversion technology to optimize the feature mapping through probability distribution sampling, optimizes the data dimension structure under the premise of maintaining the integrity of the key information, and improves the overall representation quality and information utilization efficiency of the multi-modal data. Third, the nonlinear enhancement processing is adopted to realize the differential enhancement control based on the feature importance, the enhancement key node identification and regulation sequence construction technology are combined for accurate enhancement parameter adjustment, and the hierarchical enhancement output mechanism provides corresponding level of data support according to the unmanned aerial vehicle task demand, so that the system can not only guarantee the data quality requirement of high-precision tasks, but also effectively control the calculation resource consumption of general tasks, and realize the optimization balance of performance and resource efficiency.

[0008] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a flow diagram of a multi-modal data enhancement method based on feature space alignment.

[0010] Figure 2 is a structural block diagram of a multi-modal data enhancement system based on feature space alignment. DETAILED DESCRIPTION

[0011] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0012] It should be noted that all directional indications, such as upper, lower, left, right, front, back, and the like, used in the embodiments described herein are only used to explain the relative position relationship, movement condition and the like between components, and if the specific posture (as shown in the drawings) changes, the directional indications will also change accordingly.

[0013] It should also be noted that when an element is referred to as being "fixed" or "set" on another element, it can be directly on the other element or there can be a middle element. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or there can be a middle element.

[0014] In addition, the description involving "first", "second" and the like in the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection claimed by the present application.

[0015] The technical solutions of the embodiments of the present application are introduced as follows.

[0016] As shown in Figure 1 The embodiment of the present application provides a multi-modal data enhancement method based on feature space alignment, which includes the following steps S110-S160: Step S110, collecting a multi-modal original data stream, and applying reverse feature mining to the multi-modal original data stream to generate a feature representation library.

[0017] Specifically, multi-modal raw data streams are collected. By distributed multi-source perception devices mounted on the UAV platform, multi-dimensional heterogeneous data streams of the target area are synchronously acquired during the flight task execution. The UAV multi-modal sensor system includes high-resolution visible light cameras, multi-spectral imaging devices, laser radar scanners, infrared thermal imaging sensors, and inertial measurement units, etc. Various sensors adopt a unified time synchronization protocol to ensure that the time alignment accuracy of different modal data reaches milliseconds. The data acquisition frequency is set according to the sensor type and task requirements, the visible light camera acquisition frequency is 30 Hz, the laser radar scanning frequency is 10 Hz, and the inertial measurement unit acquisition frequency is 100 Hz. All raw data are added with accurate time stamps and spatial coordinate markers to form time and space related data records. Real-time data compression and preprocessing techniques are used to reduce data storage space and transmission bandwidth requirements. The continuity of the collected data is checked to detect and mark the time period of data loss or abnormality. A circular buffer is used to manage the data stream, supporting historical data access and real-time data update within a sliding window.

[0018] In some embodiments, the applying reverse feature mining to the multi-modal raw data stream to generate a feature representation library comprises: acquiring a causal precursor signal by reverse time tracing the multi-modal raw data stream; extracting a singular trajectory by applying feature attractor projection to the causal precursor signal; generating sparse coding by taking the singular trajectory as a feature activation pattern; and constructing a feature representation library according to the sparse coding.

[0019] The multi-modal raw data stream is reverse time traced to acquire a causal precursor signal. Reverse feature mining starts from the current state of the data stream, reversely analyzes the historical trajectory of data changes, and identifies a precursor signal pattern that can predict the current state. The collected multi-modal raw data stream is accurately sorted and time-aligned according to the time stamp. Reverse time tracing starts from the latest time of the data stream and reversely analyzes the historical evolution process of data changes along the time axis. A fixed-length time sliding window is used, with a window size of 60 seconds and a sliding step of 5 seconds. The first-order difference and the second-order difference of the data sequence are calculated during reverse tracing to identify the key turning points of data changes. The causal precursor signal is defined as a signal feature that appears before data changes in the time series and has a strong correlation with the changes. The Pearson correlation coefficient under different time delays is calculated to identify the optimal delay time, and the correlation coefficient threshold is set to 0.7. A moving average filter is used to smooth the precursor signal, and the filter length is 10 sampling points. The identified causal precursor signals are screened according to signal amplitude and duration, and signals with an amplitude greater than 2 times the standard deviation of the mean value and a duration of more than 3 seconds are retained.

[0020] The characteristic attractor projection is applied to the causal precursor signal to extract the singular trajectory. The obtained causal precursor signal is converted into a phase space representation, and the one-dimensional time signal is expanded into a multi-dimensional phase space trajectory by the delay coordinate embedding method. The delay coordinate embedding uses a fixed delay time and embedding dimension, and the delay time is set to 3 times the signal sampling interval, and the embedding dimension is set to 5 dimensions. The Euclidean distance between the trajectory points in the reconstructed phase space is calculated, and the local density distribution of the trajectory is identified. The singular trajectory refers to the abnormal trajectory segment deviating from the normal trajectory distribution, which reflects the atypical behavior pattern of the system, which may correspond to sudden environmental changes or equipment state abnormalities in the unmanned aerial vehicle application. The high-density and low-density regions of the trajectory are identified by the density clustering method, and the trajectory in the low-density region is marked as a singular trajectory. The density threshold is set to 10% of the number of trajectory points in the neighborhood, and the neighborhood radius is 2 times the average distance between trajectory points. The geometric feature extraction is performed on the singular trajectory, and the length, curvature and bifurcation point number of the trajectory are calculated. The number of trajectory points is reduced using the trajectory simplification algorithm, and the main geometric features of the trajectory are preserved. The start time, duration and geometric feature parameters of each singular trajectory are recorded.

[0021] The singular trajectory is used as a feature activation pattern to generate sparse coding. Based on the extracted singular trajectory data, the geometric and timing information of the trajectory is converted into a high-dimensional feature vector representation. The feature activation pattern is defined as a combination of numerical vectors that can effectively represent the key characteristics of the singular trajectory. The geometric features and timing features of the singular trajectory are combined into a unified feature description vector, and the feature dimension is set to 128 dimensions. Sparse coding method is used to compress the feature vector representation, and the sparse coding optimization formula is min||X-DW||2²+λ||W||1, where X is the input feature matrix, D is the learned dictionary matrix, W is the sparse coefficient matrix, and λ is the sparsity control parameter. The sparsity control parameter λ is set to 0.1, and the number of dictionary atoms is set to 256. The coordinate descent method is used to solve the sparse coding optimization problem, and the iteration number is limited to 100 times. The reconstruction error is used to control the coding quality, and the reconstruction error threshold is set to 5% of the original signal energy. The generated sparse coding is sorted according to the activation intensity, and the top 50 coefficients with the highest activation intensity are selected as the effective coding. The sparse coefficient is converted to an integer representation using the quantization method, reducing the storage space requirement.

[0022] According to sparse coding, a feature representation library is constructed. Based on the generated sparse coding data, a systematic feature representation library storage and retrieval system is constructed. The feature representation library adopts a key-value pair storage structure, with the hash value of the sparse coding as the key and the complete coding vector as the value. The MD5 hash algorithm is used to generate a unique identifier for the sparse coding, ensuring fast retrieval and de-duplication of the coding. The feature representation library supports similarity-based neighbor search, and feature matching is achieved by calculating the cosine similarity between coding vectors. The similarity threshold is set to 0.8, and the coding with a similarity higher than the threshold is classified into the same feature category. An inverted index structure is used to speed up the feature retrieval process, and the index maintains the mapping relationship between the active position and the coding identifier. The feature representation library supports dynamic expansion, and new sparse coding can be added to the library in real time. LRU cache strategy is used to manage memory access of frequently used features, and the cache size is set to 1000 feature entries. The feature representation library is regularly sorted and optimized, and redundant features with a usage frequency lower than the threshold are deleted. A feature usage statistics mechanism is established to track the access frequency and matching success rate of each feature.

[0023] In step S120, an adaptive weight graph is constructed according to the feature representation library, dynamic calibration is performed on the adaptive weight graph to identify a spatial alignment path, an alignment coefficient is derived from the spatial alignment path to generate an alignment coefficient library, and deviation detection is performed using the alignment coefficient library to generate an alignment imbalance index.

[0024] Specifically, an adaptive weight graph is constructed according to the feature representation library. The sparse coding data in the feature representation library is taken as input, and the correlation between each feature code is extracted to construct a network topology. The adaptive weight graph describes the collaborative relationship between different sensor features in the multi-sensor system of the unmanned aerial vehicle. When the unmanned aerial vehicle performs forest fire monitoring tasks, the high-temperature area features detected by the infrared thermal imaging sensor have strong correlation with the smoke features captured by the visible light camera, and this correlation is represented as a high-weight connection in the graph. Similarity calculation is performed on each feature coding vector in the feature representation library, and cosine similarity is used as the main measurement index. The cosine similarity between any two feature codes is calculated, and the formula is cosine value equals two vector dot products divided by vector modulus product. Feature code pairs with a similarity greater than the set threshold of 0.7 are connected to form an initial feature correlation network. The connectivity of each node in the network is calculated, which reflects the importance and influence range of the feature in the entire system. The node connectivity is used as the basic index of node weight, and nodes with high connectivity obtain larger weight values. The weighted graph structure is constructed according to the node weight and edge connection relationship, and the weight distribution in the graph can adaptively reflect the relative importance between features. Normalization method is used to standardize the weight, ensuring that the weight value is within a reasonable range and has comparability. The weight distribution of the graph is updated regularly, and the node weight and connection strength are dynamically adjusted according to new feature data.

[0025] A dynamic calibration process is performed on the adaptive weight graph to identify spatial alignment paths. Using the constructed adaptive weight graph, a graph search algorithm is used to find high-quality propagation paths connecting different feature regions. Spatial alignment paths are information propagation channels connecting key sensor features. When a UAV inspects a power transmission line, alignment paths are formed between the 3D structural features scanned by the laser radar, the defect image features of the visible light camera, and the heat point features of the infrared sensor, ensuring the spatial consistency of multi-modal data. Starting from the core node with the highest weight value, an improved breadth-first search strategy is used to expand layer by layer to find the connection path. During the search process, paths that pass through high-weight nodes are preferred to ensure that the path has sufficient propagation ability and stability. The path length limit is set to avoid information attenuation and propagation delay caused by excessively long paths. The upper limit of the path length is set to 6 nodes, which can control the computational complexity while ensuring the propagation effect. The total weight value of each candidate path is calculated, and the total weight is equal to the cumulative sum of all edge weights on the path. The top K paths with the highest total weight are selected as candidate spatial alignment paths, and K is determined according to the size of the graph, usually set to 10% of the total number of nodes. The connectivity of the candidate paths is verified to ensure the integrity and effectiveness of the paths. Invalid paths containing broken or low-weight edges are filtered out. The verified paths are used as the final spatial alignment paths, and the node sequence and total weight information of the paths are recorded.

[0026] Alignment coefficients are derived from the spatial alignment paths to generate an alignment coefficient library. Based on the identified spatial alignment path structure, the alignment relationship parameters between nodes on the path are extracted. For any two adjacent nodes on the path, the similarity of their corresponding feature vectors is calculated as the basic alignment coefficient. Using the cosine similarity calculation method, the similarity value obtained is directly used as the alignment coefficient of the node pair. The alignment coefficient has a value range of [-1, 1], with positive values indicating consistent feature vector directions and negative values indicating opposite directions. All spatial alignment paths are traversed to extract the alignment coefficients of each pair of adjacent nodes on the path, forming a complete alignment relationship dataset. The alignment coefficients are indexed and stored according to the identifiers of the node pairs, and a hash table data structure is used to achieve fast lookup. Statistical information of the alignment coefficients is established, including basic statistics such as mean, standard deviation, and distribution range. The alignment coefficient library supports real-time query and update operations, and can dynamically update related alignment coefficients when new feature data is added. A caching mechanism is used to optimize frequently accessed alignment coefficients, improving query efficiency and response speed. The alignment coefficient library is periodically organized, and expired data that has not been used for a long time is deleted.

[0027] In some embodiments, the bias detection is performed by using the alignment coefficient library to generate an alignment imbalance index, including: activating a dynamic monitoring mode of the alignment coefficient library, obtaining a coefficient change trajectory; identifying abnormal fluctuations in the coefficient change trajectory, establishing a fluctuation type library, the fluctuation type library including sudden fluctuation, gradual fluctuation and periodic fluctuation; performing influence quantization processing on the fluctuation type library to generate an alignment imbalance index.

[0028] The dynamic monitoring mode of the alignment coefficient library is activated to obtain a coefficient change trajectory. The alignment coefficient library is switched to a real-time monitoring running state to start continuous data collection and change tracking functions. The change process of the alignment coefficient value is continuously recorded at a fixed time interval, and the sampling frequency is set to 10 times per second. The coefficient values obtained by sampling are stored in chronological order to form a complete coefficient change time sequence. The time sequence data is subjected to basic preprocessing, including missing value interpolation and abnormal value detection. The original data is smoothed using a simple moving average method, and the moving average window length is set to 5 sampling points. The instantaneous change rate of the coefficient change is calculated, and the change speed is obtained by dividing the value difference between adjacent time points by the time interval. The original coefficient value and the change speed are combined to form a two-dimensional coefficient change trajectory, which completely describes the dynamic change characteristics of the coefficient. A storage and management mechanism for trajectory data is established to support query and statistical analysis of historical trajectories. The retention period of the trajectory data is set, and historical data exceeding the period is automatically cleaned to save storage space.

[0029] The coefficient change trajectory is subjected to abnormal fluctuation identification to establish a fluctuation type library. The abnormal fluctuation patterns in the coefficient change trajectory are automatically identified and classified. The sudden fluctuation identification is based on the sudden jump of the change rate, and when the change rate exceeds the historical mean plus 3 times the standard deviation, it is determined to be a sudden fluctuation. The sudden fluctuation usually has a short duration but a large amplitude, reflecting the sharp change of the system state. The gradual fluctuation identification is based on the continuous unidirectional change of the coefficient, which requires more than 5 consecutive time steps to maintain the same change direction and the cumulative change amplitude to exceed 0.2. The gradual fluctuation has a long duration but a gentle change, reflecting the slow drift of the system. The periodic fluctuation identification detects the repeating pattern of the change trajectory, and uses the autocorrelation analysis method to identify the periodic characteristics of the trajectory. When the autocorrelation function has a significant peak at a certain delay time, it is determined that there is a periodic fluctuation. The parameters of each type of fluctuation are recorded, the peak amplitude and occurrence time of the sudden fluctuation are recorded, the change trend and duration of the gradual fluctuation are recorded, and the main period and amplitude of the periodic fluctuation are recorded. The fluctuation type library uses a classified storage structure, and the data is organized and managed according to the fluctuation type. Flexible data retrieval and statistics are supported according to the time range, fluctuation intensity and type.

[0030] The impact of the fluctuation type library is quantified to generate an alignment imbalance index. The alignment imbalance index is used to quantify the degree of deviation in data coordination between sensors in a drone multi-sensor system. An elevated index indicates an abnormality in the spatial alignment relationship between sensors, which may be caused by factors such as sensor vibration, temperature drift, electromagnetic interference, or mechanical looseness. Based on the abnormal fluctuation data recorded in the fluctuation type library, the impact of different types of fluctuations on system alignment stability is quantified. Differentiated impact calculation methods are designed for the three types of fluctuations. The impact of sudden fluctuations primarily considers their peak intensity, and the impact is equal to the square root of the peak amplitude. The impact of gradual fluctuations primarily considers their duration, and the impact is equal to the product of the average rate of change and the duration. The impact of periodic fluctuations primarily considers their amplitude and frequency, and the impact is equal to the product of the amplitude and frequency. Impact weights are set based on the threat level of different fluctuation types to system stability: sudden fluctuations are weighted at 0.5, gradual fluctuations at 0.3, and periodic fluctuations at 0.2. Calculate the weighted combined impact I_weighted = 0.5 × I_burst + 0.3 × I_gradient + 0.2 × I_period, where I_weighted represents the weighted combined impact, I_burst, I_gradient, and I_period represent the normalized impact values ​​of the three types of fluctuations. Map the weighted impact to the [0, 1] interval as the final alignment imbalance index. Values ​​closer to 1 indicate more severe imbalance between sensors, requiring system calibration or equipment maintenance. Establish a grading standard for the imbalance index: 0-0.3 represents mild imbalance, 0.3-0.7 represents moderate imbalance, and 0.7-1.0 represents severe imbalance.

[0031] In step S130, a cross-information entropy analysis is performed on the feature representation library to establish a collaborative enhancement chain. The collaborative enhancement chain is used to determine the information coupling degree of different modal data. The information coupling degree is used to identify the collaborative enhancement mode. The alignment imbalance index is amplified according to the collaborative enhancement mode to generate a complementary enhancement vector.

[0032] Specifically, cross-information entropy analysis of the feature representation library is performed to establish a synergistic enhancement chain. Different modal feature data in the feature representation library are taken as input to analyze the correlation between modalities and construct a synergistic network structure. The synergistic enhancement chain describes the data synergy relationship between multiple sensors of the unmanned aerial vehicle. When the unmanned aerial vehicle performs a search and rescue task, there is a synergistic enhancement effect between the human body heat signal detected by the infrared thermal imaging sensor and the moving target captured by the visible light camera and the terrain height data measured by the laser radar. These correlation relationships constitute the synergistic enhancement chain. The multi-modal sparse coding data stored in the feature representation library are classified by modalities, and the feature codes belonging to the same sensor type are grouped together. The cross-information amount between different modalities is calculated, and the information sharing degree between modalities is evaluated by using a simplified information overlap measurement method. For the feature data of any two modalities, the information overlap degree is calculated by counting their common activation patterns. The modal pairs with an information overlap degree greater than a set threshold of 0.6 are connected synergistically to form the basic topology structure of the synergistic enhancement chain. The synergistic enhancement chain is represented by a graph structure, with modalities as nodes and synergistic connections as edges, and the edge weight is equal to the information overlap degree of the corresponding modality pair. The top K connections with the highest information overlap degree are selected as the core connections of the synergistic enhancement chain, and the value of K is determined according to the total number of modalities, usually set to 30% of the number of modalities.

[0033] In some embodiments, the information coupling degree of different modal data is determined by using the synergistic enhancement chain, including: constructing a modality correlation network by performing modality correlation analysis through the synergistic enhancement chain; obtaining a modality similarity matrix by performing similarity measurement on the modality correlation network; obtaining a mutual information coefficient by performing mutual information calculation based on the modality similarity matrix; and taking the standardized value of the mutual information coefficient as the information coupling degree.

[0034] A modality correlation network is constructed by performing modality correlation analysis through the synergistic enhancement chain. The node connection relationship in the synergistic enhancement chain is used to expand and construct a complete modality correlation network structure. The neighborhood of each modality node on the synergistic enhancement chain is analyzed to identify the indirect correlation relationship between the node and other modalities. The correlation between any two modalities is calculated, and a simplified correlation measurement method is used. The modality pairs with a correlation value greater than a set threshold of 0.4 are identified as potential correlations, and a connection edge is added in the modality correlation network. The connection structure of the modality correlation network is gradually improved by iterative expansion to ensure that the network can reflect the multi-level correlation between modalities. The constructed modality correlation network contains all relevant modality nodes and their correlation connection relationships. The connection density of the correlation network is calculated to evaluate the tightness of the network by the ratio of the number of connection edges to the total number of possible connections. The connectivity of the modality correlation network is checked to ensure that all important modalities can be connected to each other through network paths. The basic statistical information of the network is recorded, including the number of nodes, the number of edges, and the average connection degree, etc.

[0035] The similarity of the modal correlation network is measured to obtain a modal similarity matrix. Based on the constructed complete modal correlation network, the overall similarity between each modal node in the network is calculated using a comprehensive similarity measurement method. The local topological features of each modal node in the network are extracted, including degree, clustering coefficient, centrality index, and adjacent node features, and other multi-dimensional topological description information. The structural similarity between modes is calculated using a structural similarity measurement method, which reflects the similarity of the structural role and functional position of different modes in the network. The comprehensive similarity index between modes is calculated by combining the intrinsic feature similarity and network topology similarity of the modes through a weighted fusion method. The comprehensive similarity calculation formula is S_total=α×S_feature+β×S_topology, where S_feature is the feature similarity, S_topology is the topology similarity, and α and β are weight coefficients, satisfying α+β=1. The comprehensive similarity values of all mode pairs are organized into a symmetric similarity matrix, and the matrix element Sij represents the overall similarity between modal i and modal j. The diagonal line of the similarity matrix is standardized to ensure that the diagonal element Sii of the matrix is equal to 1, indicating that the similarity of each mode to itself is completely similar. The similarity matrix comprehensively reflects the similarity distribution pattern and correlation strength change of modes in multiple dimensions.

[0036] For example, the mutual information coefficient is calculated based on the modal similarity matrix, including: based on the modal similarity matrix, joint distribution estimation is performed to generate a joint probability table and an edge probability table; the joint probability table is independently tested using the edge probability table to form a dependency index; the dependency index is logarithmically transformed to generate an information gain value; and the information gain value is normalized to obtain a mutual information coefficient.

[0037] The joint probability table and the marginal probability table are generated based on the modal similarity matrix. The joint probability distribution and the marginal probability distribution between the modes are estimated by statistical methods using the distribution characteristics of the similarity values in the modal similarity matrix. The continuous value space of the similarity values is discretely divided according to equal-width intervals, and K equal-width intervals are set to cover the complete value range of [0, 1]. The sample frequency in each value interval in the similarity matrix is counted, and the probability density of each interval is estimated by frequency statistics. The joint probability p(x, y) = N(x, y) / N_total is calculated, where N(x, y) is the number of samples falling into the interval (x, y) at the same time, N_total is the total number of samples, and p(x, y) is the joint probability. A complete joint probability table is constructed, and the rows and columns of the table correspond to different similarity value intervals, and the table elements are the joint probability values of the corresponding interval combinations. The marginal probability of each interval is calculated by marginalization, and the marginal probability formula is p(x) = Σyp(x, y) and p(y) = Σxp(x, y), where the summation is performed on all related intervals. The row marginal probability table and the column marginal probability table are constructed respectively to record the probability distribution characteristics of each interval in the row direction and the column direction. The joint probability table and the marginal probability table together constitute the complete probability distribution description of the modal similarity.

[0038] The dependence index is formed by performing independence test on the joint probability table using the marginal probability table. Based on the data of the joint probability table and the marginal probability table, the statistical dependence relationship strength between the modes is analyzed by statistical independence test method. Under the condition of statistical independence assumption, the expected joint probability distribution is calculated, and the expected joint probability under the independence assumption is the product of the two marginal probabilities. The expected joint probability calculation formula is p_expected(x, y) = p(x) x p(y), where p(x) and p(y) are the marginal probabilities. The deviation degree between the actual joint probability and the expected joint probability is calculated, and the deviation size reflects the degree of deviation from statistical independence between the modes. Absolute deviation and relative deviation are used as two indexes to quantify the deviation degree, and the absolute deviation is the direct difference between the two probabilities, and the relative deviation is the ratio of the absolute deviation to the expected probability. The deviation values of all interval combinations are statistically summarized, and the total deviation index D_total = ΣΣ|p(x, y) - p_expected(x, y)| is calculated, where the summation is performed on all interval combinations. The larger the D_total value is, the stronger the statistical dependence between the modes is, and the smaller the D_total value is, the closer to the statistical independence state. The total deviation index D_total is used as the quantification index of the dependence between the modes, which comprehensively reflects the mutual influence and constraint relationship between the modes at the statistical level.

[0039] The dependency index is logarithmically transformed to generate the information gain value. The calculated dependency index D_total is logarithmically transformed to convert the dependency strength into information gain. The logarithmic transformation compresses and standardizes the range of the dependency index while highlighting the importance and sensitivity of small changes. Using the natural logarithm for the transformation, the information gain formula is G=ln(1+D_total), where G is the information gain value and ln is the natural logarithm function. The mathematical properties of the logarithmic transformation ensure that when the dependency index is 0 (corresponding to complete statistical independence), the information gain value is also 0, which conforms to the theoretical requirement in information theory that independent events do not generate information gain. As the dependency index increases, the information gain increases, but the rate of increase gradually slows. This nonlinear transformation property helps balance the influence of strong and weak dependencies. From an information theory perspective, the information gain value quantifies the amount of additional information generated by deviations from the independence assumption between modalities. A larger value indicates greater mutual information between modalities. In drone-based forest fire monitoring, knowing that a thermal infrared sensor detects a high-temperature area can significantly increase the probability of a visible light camera detecting smoke in that area. A larger value indicates greater mutual information between modalities.

[0040] The information enhancement values ​​are normalized to obtain the mutual information coefficient. The calculated information gain values ​​are mapped to a standard numerical range through a normalization transformation to generate a mutual information coefficient that is easily comparable. A maximum-minimum normalization method is used, with the normalization formula being: the mutual information coefficient equals the information gain minus the minimum value, divided by the difference between the maximum and minimum values. The purpose of normalization is to standardize the correlation strengths of different sensor pairs to a common scale. In UAV systems, the correlation strengths between thermal infrared and visible light, and between lidar and GPS, can vary significantly. Normalization allows for direct comparison of which sensor pairs offer the best synergy. The normalized mutual information coefficient is constrained to the interval [0, 1], where 0 indicates complete statistical independence between the modalities and 1 indicates complete statistical dependence. The mutual information coefficient directly reflects the strength of information coupling between the modalities and can serve as a standardized quantitative indicator of modal correlation.

[0041] The normalized value of the mutual information coefficient is taken as the information coupling degree. The calculated mutual information coefficient is finally standardized to generate a unified scale information coupling degree index. The information coupling degree reflects the tightness of data association between multiple sensors of the unmanned aerial vehicle. When performing precision agriculture monitoring, the vegetation index features of the multispectral camera have high coupling degrees with the soil temperature features of the thermal infrared sensor and the canopy height features of the laser radar, indicating that these sensor data are interdependent in crop growth state assessment. The mutual information coefficients of all modal pairs are globally standardized to ensure that the coupling degree value range is controlled within the [0, 1] interval. The standardized information coupling degree is taken as a quantitative index of modal correlation strength, which comprehensively reflects the correlation degree between modes in terms of statistical dependence and information sharing. A complete information coupling degree matrix is constructed, and the matrix elements represent the standardized coupling strength between the corresponding modes. The information coupling degree matrix is checked for symmetry to ensure that the matrix meets the symmetry requirement.

[0042] Identify the collaborative enhancement mode using the information coupling degree. Based on the calculated information coupling degree matrix, the collaborative enhancement mode type between modes is identified by clustering analysis method. The collaborative enhancement mode describes the combination of sensors with similar collaborative characteristics. The combination of atmospheric sensors (temperature and humidity, air pressure, wind speed sensors) forms one collaborative mode, and the combination of image sensors (visible light, multispectral, thermal infrared cameras) forms another collaborative mode. According to the coupling degree value, the modal pairs are divided into different collaborative strength levels. Coupling degrees greater than 0.7 are marked as strong collaboration, 0.4-0.7 are marked as moderate collaboration, and less than 0.4 are marked as weak collaboration. Cluster analysis is performed on the strongly collaborative modal pairs to identify the modal set that can form a collaborative group. A simplified clustering method is used to automatically group the modes, and the modes with high coupling degrees are grouped into the same collaborative group. Each group in the clustering result is defined as a collaborative enhancement mode, and each mode represents a combination of modes with similar collaborative characteristics. Record the basic attributes of each collaborative enhancement mode, including the list of included modes and the internal average coupling strength. Analyze the coverage of the collaborative enhancement mode to identify the relationship between the modes. The identified collaborative enhancement mode is checked for stability to ensure the consistency of the mode.

[0043] According to the synergistic enhancement mode amplification, the alignment imbalance index is generated to generate a complementary enhancement vector. For each synergistic enhancement mode, the average coupling strength of its internal mode is calculated as the synergistic ability index of the mode. The stronger the synergistic ability of the mode, the more significant the amplification effect on the alignment imbalance index. Multiply the alignment imbalance index by the mode synergistic ability to obtain the amplified imbalance index, and the amplification formula is I_amplified=I×S_pattern, wherein I is the original alignment imbalance index, S_pattern is the mode synergistic ability, and I_amplified is the amplified imbalance index. Decompose the amplified imbalance index according to the mode dimension to form a multi-dimensional vector representation. The complementary enhancement vector is represented as a numerical array containing the enhancement weights of all modes, and each component corresponds to the enhancement weight of a mode. The numerical value of each component of the vector reflects the importance of the corresponding mode in the overall enhancement process, and the larger the component value, the more enhancement processing is required for the mode. The generated complementary enhancement vector is standardized to ensure that the numerical values of the components of the vector are consistent. The standardized complementary enhancement vector maintains the relative importance relationship between the modes and has a unified numerical scale. The complementary enhancement vector provides weight guidance for subsequent sensor calibration and data fusion, indicating the priority and resource allocation ratio of each mode data in system optimization.

[0044] In step S140, a multi-dimensional projection reconstruction is performed on the complementary enhancement vector to generate a projection coding feature, a diffusion time reversal is performed on the projection coding feature to determine a probability distribution of the feature mapping, a mapping path is reconstructed by sampling from the probability distribution, and a multi-dimensional projection feature map is obtained according to the structure enhancement feature derived from the mapping path.

[0045] Specifically, a multi-dimensional projection reconstruction is performed on the complementary enhancement vector to generate a projection coding feature. The multi-dimensional projection reconstruction is a process of mapping high-dimensional sensor coordination to a low-dimensional representation space. In the cooperative operation of a UAV formation, the sensor data of multiple UAVs forms a high-dimensional feature vector, and the key feature mode reflecting the cooperative effect of the formation can be extracted through projection reconstruction. A principal component analysis method is used to extract the main variation direction of the vector data, and the most important projection basis vector is determined through feature decomposition. The feature vectors corresponding to the top k largest eigenvalues are selected according to the size of the eigenvalues, and these feature vectors constitute the basis space of the projection transformation. The selected feature vectors are combined into a projection matrix, and each column of the matrix corresponds to a main variation direction. The original high-dimensional vector is mapped to a low-dimensional space through the projection transformation, and the projection calculation formula is Y = V^TP, where Y is the projection coding feature matrix, V is the original complementary enhancement vector matrix, and P is the projection transformation matrix. The projection coding feature retains the core features of the coordination, compresses the complex UAV group flight mode into a few key coordination parameters, simplifies the data processing complexity, and maintains the essential features of the group coordination. The projection coding feature is standardized to map the feature values to the same numerical scale range, providing standardized input data for diffusion analysis.

[0046] In some embodiments, the performing diffusion time reversal on the projection coding feature to determine the probability distribution of the feature mapping comprises: inputting the projection coding feature into diffusion analysis to obtain a forward propagation trajectory; performing time reversal on the forward propagation trajectory to generate a reverse path; sampling the intermediate state distribution on the reverse path; and aggregating the intermediate state distribution into the probability distribution of the feature mapping.

[0047] The projection coding feature is input into diffusion analysis to obtain a forward propagation trajectory. The projection coding feature data is used as the initial condition of the diffusion system, and the propagation and evolution process of the feature in the spatial dimension is calculated through diffusion simulation. Diffusion time reversal is a method of finding the optimal propagation path by simulating the diffusion of features from a concentrated state to a dispersed state, which can analyze the optimal coverage mode of the multi-sensor detection range in the UAV search and rescue mission. The spatial range and parameter configuration of the diffusion calculation are set according to the distribution characteristics of the projection coding feature. A simplified diffusion model is used to calculate the numerical value of the feature propagation, starting from the initial concentrated state and gradually calculating the feature distribution change at each time step. A suitable diffusion coefficient and time step are set to ensure the stability and accuracy of the calculation. The motion trajectory of the feature propagation during the diffusion process is recorded, including the position change of the propagation front and the evolution of the diffusion range. The forward propagation trajectory is recorded as a sequence of space-time coordinates, which completely describes the whole process of the gradual diffusion of the feature from the initial concentrated state.

[0048] Time reversal is performed on the forward propagation trajectory to generate the reverse path. The time coordinate of the forward propagation trajectory is reversed, and the trajectory data is reorganized to generate a reverse propagation path that traces back from the diffusion terminal state to the initial state. Time reversal transformation is performed on all spatiotemporal coordinate points recorded in the forward propagation trajectory, and the time coordinate is reversed by subtracting the original time from the total time. The trajectory points are rearranged in the order of the reversed time coordinate to generate the spatiotemporal sequence of the reverse propagation path. The starting point of the reverse path corresponds to the termination state of the forward diffusion, and the termination point corresponds to the initial state of the forward diffusion, and the path direction is completely opposite to the forward propagation. The direction of the feature concentration change at each point on the reverse path is calculated, and the feature concentration gradually concentrates on the reverse path. The reverse path completely retains the spatial information of the forward propagation process, and at the same time realizes the reverse tracking analysis from dispersion to concentration through time reversal.

[0049] Intermediate state distribution is sampled on the reverse path. State sampling is performed along the reverse propagation path at a predetermined time interval to extract feature distribution snapshots at multiple key moments in the inversion process. The total time span of the reverse path is determined to determine the sampling interval, and the feature distribution state is extracted at equal time intervals on the reverse path. For each sampling time, the concentration distribution of the feature in the spatial domain at that time is recorded to form intermediate state distribution data. The intermediate state distribution is similar to the sensor coverage range snapshot of the unmanned aerial vehicle at different altitude layers, and each snapshot records the distribution of the sensor influence in space at a specific moment. Discrete concentration data points are converted to continuous distribution functions through interpolation methods to ensure the continuity of the distribution function. The basic statistical features of each intermediate state distribution are calculated, including the mean and variance of the distribution. The intermediate state distribution sequence completely reveals the concentration change rule and spatial redistribution process of the feature in the reverse evolution process.

[0050] The intermediate state distributions are aggregated into a feature map probability distribution. All extracted intermediate state distributions are weighted and aggregated to generate a unified probability distribution function that can comprehensively reflect the characteristics of the entire reverse evolution process. A time weight function is designed to balance the relative importance of intermediate state distributions at different times, ensuring that intermediate states close to the end of the reverse path obtain greater weights. The weighted aggregated distribution p agg(x) =∑wi×ui(x) / ∑wi is calculated, where p agg(x) is the aggregated probability distribution function, wi is the weight coefficient of the i-th intermediate state, and ui(x) is the distribution function value of the i-th intermediate state at position x. Normalization of the aggregated distribution ensures that it meets the mathematical requirements of a probability distribution, ensuring that the integral of the distribution function over the entire domain equals 1. The aggregated probability distribution integrates the distribution information of all stages in the reverse evolution process, containing both the dispersion characteristics of the diffusion final state and the concentration characteristics of the diffusion initial state. The feature map probability distribution reflects the spatial position and intensity distribution of the most likely occurrence of the synergistic enhancement effect in the UAV sensor system, providing probabilistic guidance for determining the key enhancement region.

[0051] Sampling from the probability distribution to reconstruct the mapping path. Based on the determined feature map probability distribution, random sampling techniques are used to extract representative sample points from the distribution, and the main path of feature propagation is reconstructed by connecting the sample points. The importance sampling method is used to improve the sampling efficiency. First, the distribution space is divided into high-density and low-density regions according to the probability density. Dense sampling is performed in the high-density region to capture the main distribution characteristics, and sparse sampling is performed in the low-density region to cover the boundary characteristics of the distribution. The number of samples is allocated according to the importance of the region to ensure that the samples can fully represent the characteristics of the entire probability distribution. Standard random sampling methods are used to generate sample points that conform to the probability distribution, and inverse transform techniques are used to convert uniform random numbers into samples of the target distribution. The generated sample points are sorted according to spatial position and probability density, and the top M sample points with the highest probability density are selected as the key nodes of path reconstruction. The Euclidean distance between sample points is calculated, and a simplified connection method is used to construct the connection network between sample points. The initial connection network is optimized to remove unnecessary connections and fill in key missing connections, ensuring the continuity of the path. The final mapping path is represented as an ordered node sequence, each node containing spatial coordinates and corresponding probability weight information.

[0052] The structural enhanced features are obtained according to the mapping path derivation structure to obtain a multi-dimensional projection feature map. Based on the reconstructed mapping path structure, the geometric features and connection properties of the path are extracted as structural enhanced features, and these features are organized as a representation of a multi-dimensional projection feature map. The geometric parameters of the mapping path are calculated, including the total length of the path, the node density distribution, and the connection strength, and other key description indicators. The total length of the path is obtained by accumulating the distances between all adjacent nodes, reflecting the overall scale of the path. The spatial distribution density of the path nodes is calculated to analyze the aggregation degree of the nodes in different spatial regions. The connection characteristics of the path are counted, including the number of connections and connectivity, and other basic parameters. The extracted geometric features and connection features are numerically encoded to form a multi-dimensional data description of the structural enhanced features. A multi-dimensional projection feature map is constructed, which includes three basic components: a node set, an edge set, and a feature set. The node set includes all key position points on the mapping path, and each node carries spatial coordinates and local feature information. The edge set describes the connection relationship between nodes and the continuity of the path, and each edge contains connection weights and basic attribute parameters. The feature set records the overall properties of the graph structure, including the connectivity, density, and complexity of the graph, and other global features. The multi-dimensional projection feature map is the topological map of the UAV sensor network, which shows the importance of each sensor node, the mutual connection relationship, and the cooperative working mode, supporting hierarchical feature analysis and enhanced processing operations.

[0053] In step S150, the alignment imbalance index and the complementary enhancement vector are subjected to nonlinear enhancement processing to generate an enhanced expansion mode, an enhanced key node is identified from the enhanced expansion mode, an enhancement control coefficient is determined according to the enhanced key node, and an enhancement control sequence is constructed using the enhancement control coefficient.

[0054] Specifically, the alignment imbalance index and the complementary enhancement vector are subjected to nonlinear enhancement processing to generate an enhanced expansion mode. The alignment imbalance index and the complementary enhancement vector are taken as inputs, and through nonlinear transformation technology, the two groups of data are fused and the enhancement effect is amplified. Nonlinear enhancement processing is a technology that amplifies the imbalance between sensors through special mathematical transformation. When an infrared sensor detects an abnormal heating area, but a visible light camera fails to identify the defect synchronously, nonlinear enhancement can amplify the inconsistency between sensors and highlight the abnormal area that needs attention. A nonlinear mapping function is used to transform the input data into an enhanced space, and a hyperbolic tangent function is used for nonlinear transformation. The mapping result is subjected to amplitude amplification processing, and a power-law enhancement function is used to adaptively amplify the signal. The enhanced values are reorganized according to the spatial coordinates and modal dimensions to form a matrix representation of the enhanced expansion mode. The enhanced expansion mode matrix reflects the combined effect of alignment imbalance and complementary enhancement, and the high-value area in the matrix represents the key position that needs to be enhanced.

[0055] In some embodiments, the identifying an enhanced key node from the enhanced expansion mode comprises: performing information entropy distribution analysis on the enhanced expansion mode to identify an entropy concentrated domain; performing node information capacity measurement in the entropy concentrated domain to obtain a capacity peak value; and performing entropy value correlation matching screening on the capacity peak value to determine the enhanced key node.

[0056] The information entropy distribution analysis on the enhanced expansion mode identifies an entropy concentrated domain. The numerical distribution of the enhanced expansion mode matrix is converted into a probability distribution form, and the information-intensive spatial region is identified through information entropy calculation. The enhanced key node is the position point with the highest information density and the most significant enhancement effect in the enhanced expansion mode. When the multispectral sensor detects vegetation anomalies and the thermal infrared sensor simultaneously finds temperature rise, the position where these sensor data intersect may become the enhanced key node. The matrix elements are normalized to calculate the probability distribution, ensuring that all elements constitute an effective probability distribution. The sliding window technique is used to calculate the information entropy distribution of the local region. The window slides on the matrix to calculate the local information entropy of the elements within the window. A suitable window size is set, usually a 3x3 or 5x5 square window, to balance locality and statistical reliability. The information entropy distribution map of the entire enhanced expansion mode is calculated, with each position corresponding to a local entropy value, forming an entropy distribution matrix of the same dimension as the original matrix. The spatial region with an information entropy value greater than a set threshold is identified. These regions contain relatively rich information content. Adjacent high-entropy regions are merged through connectivity analysis to eliminate isolated points and connect disconnected regions. Regions with sufficient area and high information entropy density are selected as the entropy concentrated domain. These regions represent the most information-rich and important spatial positions in the enhanced expansion mode.

[0057] The node information capacity measurement is performed in the entropy concentrated domain to obtain a capacity peak value. In the identified entropy concentrated domain, the information capacity that each spatial position can carry and process is calculated. Information capacity is defined as the maximum carrying capacity of the position in the information processing and transmission process, which is calculated by the product of local information entropy and enhancement intensity. The calculation of information capacity considers the information richness and enhancement effect intensity of the position. Positions with high entropy values and high enhancement intensities have greater information capacity. The information capacity of all positions in the entropy concentrated domain is calculated to generate a capacity distribution map for peak detection and analysis. Local maximum value detection method is used to identify the peak points in the capacity distribution. The peak detection requires that the capacity value of the position is greater than the capacity values of all adjacent positions. The parameters for peak detection include minimum peak height and minimum peak spacing to ensure that the detected peak values have sufficient significance and independence. All capacity peak points that meet the detection conditions are extracted, and the coordinate position and capacity value of each peak point are recorded. The peak points are sorted according to the capacity size, with the largest capacity peak point having the highest importance and priority.

[0058] The capacity peak value is matched and screened by entropy value correlation to determine the enhanced key node. The identified capacity peak value point is correlated and analyzed with the information entropy value of its corresponding spatial position, and the optimal enhanced key node candidate is determined by matching and screening. The entropy capacity correlation strength of each capacity peak value point is calculated, and the correlation strength reflects the capacity size corresponding to unit information entropy. Set the correlation strength threshold, and keep the peak value points with correlation strength greater than the threshold as the candidate nodes. The spatial distribution uniformity of the candidate nodes is analyzed, and the nearest neighbor distance statistical method is used to calculate the distance distribution characteristics between nodes. Select a subset of peak value points that are relatively uniform in spatial distribution and have high correlation strength to avoid excessive aggregation of nodes in certain local areas. Calculate the comprehensive score of the candidate nodes, considering the three factors of capacity, entropy value and correlation strength. The setting of weight coefficient needs to be adjusted according to the specific application requirements, and generally the capacity weight is the largest, the entropy weight is the second, and the correlation strength weight is the smallest. According to the comprehensive score, the top K candidate nodes with the highest score are selected as the final enhanced key nodes. The spatial coordinates of the enhanced key nodes are recorded to provide position information for control coefficient calculation.

[0059] The enhanced control coefficient is determined according to the enhanced key node. Using the identified enhanced key node information, the control weight coefficient corresponding to each node is calculated by the influence function and weight distribution method. The influence range and influence strength of each enhanced key node in the system are calculated, and the influence range is determined by the spatial distance decay function. The influence weight of the node is calculated by the Gaussian influence function, and the influence weight decreases exponentially with the increase of the distance. The setting of the influence radius parameter needs to balance the coverage range and localization degree of the node influence, and the radius is too large, which will lead to the dispersion of the node influence, and the radius is too small, which will lead to insufficient influence range. The enhanced strength of the node is multiplied by the influence weight to obtain the control coefficient, and the control coefficient reflects the importance of the node in the system control. All control coefficients are normalized to ensure that the sum of the coefficients is equal to the preset standard value, and the relative importance of the control coefficients is allocated through normalization. The synergistic effect between nodes is calculated, and the synergistic effect reflects the mutual cooperation and enhancement effect of different nodes in space. The single node control coefficient and the synergistic effect between nodes are integrated to form a complete set of enhanced control parameters, which contains the weight information of all important control positions in the system.

[0060] In some embodiments, the constructing an enhanced regulation sequence using the enhanced control coefficient comprises: performing topological mapping transformation on the enhanced control coefficient to generate a set of mapped nodes; constructing a hierarchical transmission link based on the set of mapped nodes; performing sequence reconstruction coding on the hierarchical transmission link to obtain a coded transmission sequence; and time-sequencing the coded transmission sequence to arrange an enhanced regulation sequence.

[0061] The enhanced control coefficients are topologically mapped to generate a set of mapped nodes. The enhanced control coefficients are mapped from the numerical space to the node representation in the graph topology, and the structured organization of control parameters is achieved through topological mapping. Each control coefficient is assigned a unique node identifier to ensure the uniqueness and identifiability of the node in the topological space. The coordinate position of the node in the topological space is determined according to the numerical size of the control coefficient, and the numerical size of the control coefficient is converted into the distance of the node from the origin, while the spatial separation of the node is achieved through the angular distribution. The topological distance between the mapped nodes is calculated, which reflects the relative position relationship of the nodes in the mapped space. The adjacency relationship between nodes is determined according to the topological distance and the similarity threshold, and the nodes with a distance less than the threshold are connected. A complete set of mapped nodes is generated, each node containing a node identifier and an original control coefficient value. The mapped node set is topologically checked and optimized to ensure the rationality of the node distribution and the effectiveness of the connection relationship.

[0062] Based on the mapped node set, a hierarchical transmission link is constructed. The topology structure and control coefficient information of the mapped node set are used to construct a link connection network with hierarchical transmission characteristics. The enhanced control sequence is a control instruction sequence executed in time sequence. When the personnel vital signs are detected, the high-precision positioning sensor is first started, then the communication device is activated to send the position information, and finally the camera angle is adjusted for detailed observation, forming an ordered sensor coordination control process. The nodes are hierarchically divided according to the size of the control coefficient, and the nodes with similar coefficient values are grouped into the same level, forming a multi-level node organization structure. The number of levels is set, and the number of nodes contained in each level is allocated through the control coefficient sorting. The connection rules between levels are constructed, allowing the upper nodes to connect to multiple nodes in the lower layer, but limiting the connection of the lower nodes to the upper layer. The weight of the connection between levels is calculated, which is based on the node control coefficient and the topological distance. A complete hierarchical transmission link network is constructed, which contains complete information such as node level allocation, inter-level connection relationship and connection weight. The transmission efficiency and coverage of the link network are calculated to ensure that information and control instructions can be effectively propagated in the network.

[0063] The hierarchical transmission link is sequentially reconstructed and encoded to obtain an encoded transmission sequence. The complex network structure of the hierarchical transmission link is converted into a linear sequence representation, and the network information is stored compactly and transmitted efficiently through sequence encoding. A depth-first traversal method is used to systematically access the link network, starting from the root node of the highest level and sequentially accessing all nodes in the network in descending order of weight. The node sequence and edge sequence accessed during traversal are recorded to ensure that the traversal path covers all important connection relationships in the network. The accessed nodes are encoded, and the encoding includes node identification, level number, control coefficient, and connection information. The accessed connection edges are encoded, and the key information of the connection starting node, termination node, and connection weight is recorded. The node encoding and edge encoding are merged according to the access order to form a complete encoded transmission sequence, which contains all the topology and weight information of the network structure. The encoded sequence is compressed and optimized to remove redundant information and use an efficient encoding format to reduce storage space requirements. The integrity of the sequence is checked to ensure that important network information is not lost during encoding.

[0064] The encoded transmission sequence is time-sequenced to form an enhanced control sequence. The encoded transmission sequence is converted into a control instruction sequence that is executed in chronological order, and the control actions are coordinated and optimized through time sequencing. The execution priority is determined according to the control coefficient and level information of the nodes in the encoded sequence, and nodes with high control coefficients and high levels have higher execution priority. The execution time of each node's control action is calculated, and the formula is ti = T_base + (1-Ci) x T_delay, where ti is the execution time of node i, T_base is the base execution time, Ci is the normalized control coefficient, and T_delay is the delay time parameter. The setting of the execution time ensures that important nodes can be executed first, and appropriate time intervals are allocated to nodes of different importance. The nodes are time-sequenced according to the calculated execution time to generate an execution queue in chronological order. Synchronization control instructions are inserted into the execution queue to ensure that the control actions of the lower levels are executed after the completion of the control actions of the upper levels, maintaining the correctness of the control logic. The final enhanced control sequence is generated, which contains control action descriptions and corresponding execution times. The control sequence contains multiple types of control operation instructions such as node activation, weight adjustment, connection control, and state monitoring. The execution feasibility of the control sequence is analyzed and optimized in time to ensure that all control actions can be completed within the preset time constraints.

[0065] In step S160, hierarchical enhancement data is output based on the enhanced control sequence and the multi-dimensional projection feature map, and multi-modal data enhancement based on feature space alignment is completed.

[0066] In some embodiments, the outputting hierarchical enhancement data based on the enhancement control sequence and the multi-dimensional projection feature map comprises: hierarchically peeling off feature density layers from the multi-dimensional projection feature map according to the enhancement control sequence; performing back gradient tracking on the feature density layers to obtain gradient propagation paths; performing feature aggregation enhancement along the gradient propagation paths to generate enhanced feature clusters; and determining hierarchical enhancement data according to the distribution intensity of the enhanced feature clusters.

[0067] The multi-dimensional projection feature map is hierarchically peeled off to extract feature density layers according to the enhancement control sequence. According to the hierarchical information and execution sequence in the enhancement control sequence, the nodes and edges of the multi-dimensional projection feature map are decomposed and peeled off according to the hierarchical structure. Hierarchical peeling is a process of separating feature data layer by layer according to importance levels. When the radar sensor detects multiple target echoes and the optical sensor can only identify part of the targets, the high confidence targets, medium confidence targets and low confidence targets can be extracted into different density layers through hierarchical peeling. According to the hierarchical attribution relationship of the nodes in the control sequence, the feature map nodes are assigned to the corresponding density layers. The density distribution of the nodes in each density layer is calculated, and a simplified density estimation method is used to calculate the density function of the spatial position. The boundary features of each density layer are calculated, and the boundary regions with rapid density changes are identified through density gradient analysis. The isodensity line method is used to determine the boundary profile of each layer, and the positions with the same density value are connected to form the interlayer boundary line. The hierarchical peeling is performed in the order from the outer layer to the inner layer, and each peeling operation removes a complete density layer. The basic information of each density layer is recorded, including the number of nodes in the layer and the density distribution range. The hierarchical peeling process maintains the topological connection relationship of the feature map, ensuring that the layers after peeling still maintain the integrity of the internal structure.

[0068] The reverse gradient tracking is performed on the feature density layers to obtain the gradient propagation paths. In each feature density layer, the gradient field of the density distribution is calculated and the path tracking is performed in the opposite direction of the gradient. The reverse gradient tracking is a path finding method that traces back from the high-density area to the low-density area. When the laser radar detects the building edge features and the visible light camera captures the texture features at the same time, the reverse gradient tracking can find the optimal path for these features to converge from the scattered distribution to the key node. The spatial gradient of the density function is calculated, and the gradient direction points to the direction where the density increases the fastest, and the reverse gradient points to the direction where the density decreases the fastest. The reverse gradient tracking is started from the highest density point of each density layer, and the tracking direction is the opposite direction of the gradient. Numerical methods are used to calculate the gradient tracking to obtain continuous tracking paths. The tracking termination conditions are set, including the gradient modulus being less than a threshold, reaching the layer boundary, or reaching the maximum tracking length limit. The path point sequence passed during the tracking process is recorded, and each path point contains the spatial coordinates and the corresponding density value. The gradient tracking operation is repeated for all density layers to obtain the gradient propagation path network covering the entire feature map. The gradient propagation path network reflects the main propagation direction and connection mode of the features in each density layer.

[0069] Feature aggregation and enhancement are performed along the gradient propagation paths to generate enhanced feature clusters. The features on the paths are sequentially aggregated and synergistically enhanced using the gradient propagation path network. Feature aggregation and enhancement is a processing technique that amplifies similar features in a cluster. When multiple parameters such as temperature, humidity, and wind speed are detected by weather sensors in a certain area, these related abnormal signals can be combined into a strengthened weather warning feature cluster through feature aggregation and enhancement. For each gradient propagation path, the weighted aggregation of the features on the path is calculated. The calculation of the weight coefficient considers the density value and the gradient strength of the path point, and the path point with high density and large gradient strength obtains a larger weight coefficient and plays a dominant role in the aggregation process. The aggregated features are subjected to a nonlinear enhancement transformation, and a power-law form is used to achieve adaptive intensity adjustment. The selection of the enhancement index affects the strength of the enhancement effect, which can be adjusted according to application requirements. Path points with similar enhanced features are spatially clustered, and clustering methods are used to group the path points. Each cluster forms an enhanced feature cluster, which contains feature sets and cluster center coordinates, etc. The internal aggregation degree of each feature cluster is calculated to ensure the quality and effectiveness of the clustering results. The formation of the feature cluster realizes the centralized management and synergistic enhancement of similar features, improving the efficiency and effectiveness of the enhancement processing.

[0070] The hierarchical enhancement data is determined according to the distribution intensity of the enhanced feature clusters. Based on the intensity distribution characteristics of the enhanced feature clusters, the unmanned aerial vehicle multi-modal enhancement data is divided into a hierarchical structure of different levels. The overall intensity of each enhanced feature cluster is calculated, and the intensity index S=(1 / N)Σ|G(Fi)|, wherein S is the cluster intensity, N is the number of features in the cluster, and G(Fi) is the i-th enhanced feature value in the cluster. The feature clusters are sorted according to the cluster intensity, and a plurality of intensity thresholds are set to distribute the feature clusters to different levels. The threshold setting combines the unmanned aerial vehicle load limit and real-time processing capability to ensure that the data distribution of each level is relatively balanced. The feature clusters in each level are integrated, and the feature data of the clusters in the same level are merged to form a corresponding level data set. The final hierarchical enhancement data output is generated, including a plurality of level data sets. The high-level data is suitable for key tasks such as disaster rescue and precise measurement, the medium-level data is suitable for routine inspection and monitoring, and the low-level data is suitable for remote operation scenarios with limited resources. The hierarchical enhancement data provides differentiated data support for different application scenarios, and realizes the reasonable matching of data quality and computing resources. By establishing the correspondence between the data level and the application demand, it is ensured that the key tasks can obtain high-quality data guarantee, and at the same time, the basic data processing capability can still be maintained under the condition of limited resources. Finally, the multi-modal data enhancement based on feature space alignment is completed, which provides a reliable data basis for unmanned aerial vehicle intelligent perception and decision-making.

[0071] In order to perform the multi-modal data enhancement method based on feature space alignment corresponding to the above-mentioned method embodiment, in order to realize the corresponding functions and technical effects. Referring to Figure 2 , Figure 2 The structure block diagram of the multi-modal data enhancement system 200 based on feature space alignment provided by the embodiment of the application is shown. For the convenience of description, only the part related to the embodiment is shown, and the multi-modal data enhancement system 200 based on feature space alignment provided by the embodiment of the application comprises: The data mining module 201 is used for collecting multi-modal original data streams, and applying reverse feature mining to the multi-modal original data streams to generate a feature representation library; The space alignment module 202 is used for constructing an adaptive weight atlas according to the feature representation library, performing dynamic calibration processing on the adaptive weight atlas to identify a space alignment path, deriving an alignment coefficient from the space alignment path to generate an alignment coefficient library, and performing deviation detection using the alignment coefficient library to generate an alignment imbalance index; The collaborative enhancement module 203 is used for performing cross-information entropy analysis on the feature representation library to establish a collaborative enhancement chain, using the collaborative enhancement chain to determine the information coupling degree of different modal data, using the information coupling degree to identify a collaborative enhancement mode, and amplifying the alignment imbalance index according to the collaborative enhancement mode to generate a complementary enhancement vector; The projection reconstruction module 204 is configured to perform multi-dimensional projection reconstruction on the complementary enhancement vector to generate a projection coding feature, perform diffusion time inversion on the projection coding feature to determine a probability distribution of a feature mapping, sample a mapping path from the probability distribution, derive a structure enhancement feature according to the mapping path, and obtain a multi-dimensional projection feature map. The enhancement control module 205 is configured to perform nonlinear enhancement processing on the alignment imbalance index and the complementary enhancement vector to generate an enhanced extension mode, identify an enhanced key node from the enhanced extension mode, determine an enhancement control coefficient according to the enhanced key node, and construct an enhancement control sequence by using the enhancement control coefficient. The data output module 206 is configured to output hierarchical enhancement data based on the enhancement control sequence and the multi-dimensional projection feature map, and complete multi-modal data enhancement based on feature space alignment.

[0072] The above-described multi-modal data enhancement system 200 based on feature space alignment can implement a multi-modal data enhancement method based on feature space alignment. The optional items in the above-described method embodiments are also applicable to this embodiment, and will not be described in detail here. The remaining content of the present embodiment can be referred to the content of the above-described method embodiments, and will not be described in detail in this embodiment.

[0073] The above-described only some or preferred embodiments of the present application, neither the text nor the drawings can limit the scope of protection of the present application, any equivalent structural transformation made by using the content of the present application and the drawings, or direct / indirect application in other related technical fields are included in the scope of protection of the present application.

Claims

1. A method of multi-modal data augmentation based on feature space alignment, characterized in that, The method comprises the following steps: Collecting a multi-modal original data stream, applying reverse feature mining to the multi-modal original data stream to generate a feature representation library; According to the feature representation library, an adaptive weight atlas is constructed, a dynamic calibration process is performed on the adaptive weight atlas to identify a spatial alignment path, an alignment coefficient library is derived from the spatial alignment path, and a bias detection is performed using the alignment coefficient library to generate an alignment imbalance index; Cross information entropy analysis is performed on the feature representation library to establish a synergistic enhancement chain, the information coupling degree of different modal data is determined using the synergistic enhancement chain, the synergistic enhancement mode is identified using the information coupling degree, the alignment imbalance index is amplified according to the synergistic enhancement mode to generate a complementary enhancement vector; The complementary enhancement vector is subjected to multi-dimensional projection reconstruction to generate a projection coding feature, and the probability distribution of feature mapping is determined by performing diffusion time inversion on the projection coding feature, a mapping path is reconstructed by sampling from the probability distribution, and a structure enhancement feature is derived from the mapping path to obtain a multi-dimensional projection feature map; The alignment imbalance index and the complementary enhancement vector are subjected to nonlinear enhancement processing to generate an enhanced expansion mode, an enhanced key node is identified from the enhanced expansion mode, an enhanced control coefficient is determined according to the enhanced key node, and an enhanced control sequence is constructed using the enhanced control coefficient; Based on the enhanced control sequence and the multi-dimensional projection feature map, hierarchical enhancement data is output, and multi-modal data enhancement based on feature space alignment is completed.

2. The method of claim 1, wherein, The method comprises the following steps: Reverse time tracing is performed on the multi-modal original data stream to obtain a causal precursor signal; Feature attractor projection is applied to the causal precursor signal to extract a singular trajectory; The singular trajectory is used as a feature activation mode to generate sparse coding; According to the sparse coding, a feature representation library is constructed.

3. The method of claim 1, wherein, The method comprises the following steps: The dynamic monitoring mode of the alignment coefficient library is activated to obtain a coefficient change trajectory; Abnormal fluctuation identification is performed on the coefficient change trajectory to establish a fluctuation type library, the fluctuation type library includes sudden fluctuation, gradual fluctuation and periodic fluctuation; The fluctuation type library is subjected to influence quantification processing to generate an alignment imbalance index.

4. The method of claim 1, wherein, The method comprises the following steps: Modality correlation analysis is performed through the synergistic enhancement chain to construct a modality correlation network; Similarity measurement is performed on the modality correlation network to obtain a modality similarity matrix; Based on the modality similarity matrix, mutual information calculation is performed to obtain a mutual information coefficient; The standardized value of the mutual information coefficient is used as the information coupling degree.

5. The method of claim 1, wherein, The method comprises the following steps: The projection coding feature is input into diffusion analysis to obtain a forward propagation trajectory; Time inversion is performed on the forward propagation trajectory to generate a reverse path; The intermediate state distribution is obtained by sampling on the reverse path; The intermediate state distribution is aggregated into a probability distribution of feature mapping.

6. The method of claim 1, wherein, The method comprises the following steps: Performing information entropy distribution analysis on the enhanced expansion mode to identify an entropy concentration domain; Performing node information capacity measurement in the entropy concentration domain to obtain a capacity peak value; Performing entropy value correlation matching screening on the capacity peak value to determine an enhanced key node.

7. The method of claim 1, wherein, The enhanced control coefficient is used to construct an enhanced control sequence, including: Topological mapping transformation is performed on the enhanced control coefficient to generate a mapping node set; A hierarchical transmission link is constructed based on the mapping node set; Sequence reconstruction coding is performed on the hierarchical transmission link to obtain an encoded transmission sequence; The encoded transmission sequence is time-sequenced and arranged as an enhanced control sequence.

8. The method of claim 1, wherein, The enhanced control sequence and the multi-dimensional projection feature map are used to output hierarchical enhancement data, including: The multi-dimensional projection feature map is hierarchically stripped according to the enhanced control sequence to extract a feature density layer; Reverse gradient tracking is performed on the feature density layer to obtain a gradient propagation path; Feature aggregation enhancement is performed along the gradient propagation path to generate an enhanced feature cluster; The distribution intensity of the enhanced feature cluster is used to determine hierarchical enhancement data.

9. The method of claim 4, wherein, The mutual information coefficient is obtained by performing mutual information calculation based on the modality similarity matrix, including: Joint distribution estimation is performed based on the modality similarity matrix to generate a joint probability table and an edge probability table; The edge probability table is used to perform independence test on the joint probability table to form a dependency index; Logarithmic transformation processing is performed on the dependency index to generate an information gain value; The information gain value is normalized to obtain a mutual information coefficient.

10. A multi-modal data augmentation system based on feature space alignment, characterized in that, It includes: A data mining module is used to collect multi-modal original data streams and perform reverse feature mining on the multi-modal original data streams to generate a feature representation library; A spatial alignment module is used to construct an adaptive weight atlas based on the feature representation library, perform dynamic calibration processing on the adaptive weight atlas to identify a spatial alignment path, derive an alignment coefficient from the spatial alignment path to generate an alignment coefficient library, and use the alignment coefficient library to perform bias detection to generate an alignment imbalance index; A synergistic enhancement module is used to perform cross-information entropy analysis on the feature representation library to establish a synergistic enhancement chain, use the synergistic enhancement chain to determine the information coupling degree of different modal data, use the information coupling degree to identify a synergistic enhancement mode, and use the synergistic enhancement mode to amplify the alignment imbalance index to generate a complementary enhancement vector; A projection reconstruction module is used to perform multi-dimensional projection reconstruction on the complementary enhancement vector to generate a projection coding feature, perform diffusion time inversion on the projection coding feature to determine the probability distribution of feature mapping, sample a reconstruction mapping path from the probability distribution, and derive structural enhancement features from the mapping path to obtain a multi-dimensional projection feature map; An enhanced control module is used to perform nonlinear enhancement processing on the alignment imbalance index and the complementary enhancement vector to generate an enhanced expansion mode, identify an enhanced key node from the enhanced expansion mode, determine an enhanced control coefficient based on the enhanced key node, and use the enhanced control coefficient to construct an enhanced control sequence; A data output module is used to output hierarchical enhancement data based on the enhanced control sequence and the multi-dimensional projection feature map to complete multi-modal data enhancement based on feature space alignment.

Images