Sensor adaptive system based on deep learning algorithm and big data analysis
Through deep learning algorithms and big data analysis technology, combined with spatial time-frequency analysis and adaptive positioning, the problems of low sensor data processing efficiency and insufficient positioning accuracy are solved, and adaptive optimization and high-accuracy positioning of the sensor system are achieved.
Patent Information
- Application Number
- CN202411428103.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Adaptive adjustment and optimization of deep learning algorithms in different application scenarios is difficult to achieve, sensor data processing is low, and noise and broken value processing is not thorough, resulting in insufficient positioning accuracy and system performance.
Design a sensor adaptive system based on deep learning algorithms and big data analysis, including cloud big data management, data cleaning and feature extraction, model training and optimization, using spatial time-frequency analysis, data fusion and adaptive positioning methods, through real-time Data feedback and model retraining realize dynamic monitoring and intelligent optimization of the system.
It improves the flexibility and accuracy of the sensor system in different scenarios, enhances the generalization ability of the algorithm, reduces manual intervention, improves system performance and application efficiency, and achieves high accuracy positioning of ground targets.
Smart Images

Figure CN118940228A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of sensor management and control, and in particular to a sensor adaptive system based on deep learning algorithm and big data analysis. Background Art
[0002] Nowadays, a large number of sensor devices are used in production scenarios. After collecting relevant information, these sensor devices transmit the sensor data to the connected edge nodes for aggregation and processing. After the edge nodes process the sensor data according to the business logic, they are packaged into data packets and uploaded to the cloud through the network base station for further aggregation and processing.
[0003] How to solve the problems of adaptive adjustment and optimization of deep learning algorithms in different application scenarios.
[0004] Therefore, a sensor adaptive system based on deep learning algorithm and big data analysis is needed to solve the above technical problems. Summary of the invention
[0005] In order to solve the above technical problems, the present invention provides a sensor adaptive system based on deep learning algorithm and big data analysis, including a cloud big data management unit, a big data analysis and processing unit, a big data feature fusion unit and a deep learning training unit; wherein, The cloud-based big data management unit collects ground data and satellite remote sensing data from sensors and devices as collected big data; The big data analysis and processing unit cleans and processes the collected big data, eliminates noise and fills in broken values; uses spatial time-frequency analysis to extract multi-scale feature information from the collected big data; The big data feature fusion unit: based on the feature information and historical data, integrates and obtains the total data set, divides the training set, the validation set, and the test set, and establishes a flexible target model; integrates the collected big data, historical data, and feature information, associates them using a data fusion algorithm, and obtains fused data; The deep learning training unit: trains a target model based on the fused data, and uses the target model to predict new data to provide a preliminary judgment result of the target location; establishes an adaptive environmental model based on the collected big data collected in real time, and uses an adaptive positioning method to fine-tune the preliminary judgment result to obtain a final result; feeds the final result back to the adaptive target model to retrain and optimize the target model.
[0006] As a further solution, the sensor includes a vibration sensor, a microwave sensor, and a radar sensor, and the device is specifically a satellite remote sensing receiver; The steps of collecting ground data and satellite remote sensing data from sensors and equipment as big data collection are specifically as follows: Select the installation location, comprehensively consider the data collection objectives and environmental requirements, and determine the installation points of the sensors and equipment; Assemble the sensors and devices at the installation point and use waterproof housing, anti-corrosion coating and anti-biological attachment materials as protective measures; Adjust the sensitivity of sensors, correct errors or correct instrument deviations, and calibrate the sensors and equipment; Start the sensors and devices to collect data, set parameters and sampling rate, and obtain the collected big data; Based on the collected big data, processing software is used to convert it into processable digital data.
[0007] As a further solution, the cleaning and processing specifically refers to data cleaning and broken value processing; The data cleaning includes outlier detection, noise elimination, and data normalization; Specifically, the outlier detection is to use standard deviation statistics to detect and exclude outliers in the collected big data, and data points based on thresholds or rules are regarded as outliers and replaced; Specifically, the noise elimination includes applying signal processing techniques including moving average filtering, median filtering and Kalman filtering to eliminate the noise in the collected big data; The data normalization is specifically to ensure that the collected big data and the measurement unit are in the same scale range based on minimum-maximum normalization processing; The broken value processing includes broken value detection, broken value filling, and multiple interpolation; The broken value detection is specifically to use statistical analysis and data visualization methods to detect the existence of broken values in the collected big data; The broken value filling is specifically to select one of the mean, median, mode filling, regression filling, interpolation method, and deep learning-based filling method as a filling method based on the characteristics of the collected big data and the type of broken values to fill the broken values; The multiple interpolation is specifically to fill in the broken values using a multiple interpolation method, specifically a matrix decomposition method, for the collected big data whose broken value ratio exceeds a preset threshold.
[0008] As a further solution, spatial time-frequency analysis is used to extract multi-scale feature information from the collected big data. Specifically, the steps are as follows: Using short-time Fourier transform method as the analysis method of the spatial time-frequency analysis; Applying the analysis method, using sliding windows of different sizes to perform short-time Fourier transform, and performing multi-scale analysis on the collected big data to explore the time-frequency characteristics of the signal; Based on the time-frequency characteristics as a data source, an extraction method including energy calculation, spectrum peak extraction, and wavelet coefficient statistics is adopted to extract the spectrum energy, frequency contour, and waveform shape characteristics of the signal to generate a feature set; Based on the feature set, principal component analysis and feature selection algorithms are used to reduce the dimension of features and retain important information as the feature information.
[0009] As a further solution, the steps of integrating and acquiring the total data set based on the feature information and historical data and dividing the data into the training set, the validation set and the test set are as follows: Using the feature information of previous periods as the historical data, and integrating the historical data and the feature information as a total data set; Perform another dimensionality reduction process on the features in the total data set based on linear discriminant analysis to reduce the dimension of the features and eliminate redundant information, and obtain the processed total data set, including categorical variables or non-numerical features; Perform label encoding based on the categorical variables or non-numerical features; The total data set is divided into a training set, a validation set and a test set.
[0010] As a further solution, the steps of establishing a flexible target model are as follows: Selecting a logistic regression model as the target model; Training the target model based on the training set, applying training algorithms and optimization methods to fit the training set, and learning patterns and regularities of the data; Using the validation set, verifying and tuning the trained target model, evaluating the performance of the target model on the validation set and generating an evaluation result, and adjusting model parameters and selecting features based on the evaluation result to optimize the target model; Using the test set, evaluating the optimized target model by calculating evaluation indicators including accuracy, precision, recall, and F1 score; The target model is deployed in practical applications to perform target prediction tasks.
[0011] As a further solution, the steps of using the data fusion algorithm to perform association and obtain fused data are specifically as follows: Integrate the collected big data, historical data and feature information to generate a unified data set; By evaluating the importance of features, data having important significance for the correlation results is selected from the unified data set to extract key features; Using a data fusion algorithm to associate, match, align and merge the key features to obtain associated data, the data fusion algorithm includes association based on key value matching, association based on similarity matching, and association based on time window; The associated data are merged to generate fused data through weighted averaging, decision rules, and aggregation functions.
[0012] As a further solution, the steps of training a target model based on the fused data and using the target model to predict new data and provide a preliminary determination result of the target location are specifically as follows: Dividing the fused data into a post-training set and a post-test set, and calling the target model; Based on the post-training set, the target model is trained, and based on an optimization method including cross-validation and grid search, the hyperparameters and model fit of the target model are adjusted; Using the post-test set, the target model is evaluated again according to evaluation indicators including mean square error, accuracy, and recall, and the best target model is selected based on the evaluation results; The target model is applied to the latest collected big data, the feature data to be predicted is input, and the trained target model is used to generate the corresponding target position prediction result as the preliminary judgment result.
[0013] As a further solution, based on the big data collected in real time, an adaptive environment model is established, and an adaptive positioning method is used to fine-tune the preliminary judgment result. The steps for obtaining the final result are as follows: Based on the collected big data, an adaptive environmental model is established using a Bayesian filter; Adopting adaptive positioning methods including weighted least square positioning, particle filter positioning, and conditional random field positioning to make a more accurate estimate of the preliminary judgment result and generate a fine-tuning result; Based on the fine-tuning result, the position information of the target position is more accurately reflected as a final result.
[0014] As a further solution, the steps of feeding back the final result to the adaptive target model and retraining and optimizing the target model are as follows: The final result is combined with the fusion data to construct a new training set, wherein the new training set includes feature data and corresponding target position labels; Using the original target model as the initial model; Retraining the target model using a new training set, inputting feature data and corresponding target location labels, and iteratively adjusting the parameters of the target model to more accurately predict the target location; Retraining and optimization iterations are performed based on the above steps, and the performance and accuracy of the target model are gradually improved through continuous feedback of final results, retraining and optimization.
[0015] Compared with the related art, the sensor adaptive system based on deep learning algorithm and big data analysis provided by the present invention has the following beneficial effects: The unique advantage of the present invention lies in its continuous dynamic monitoring, intelligent data analysis, and continuous feedback training and performance adjustment of data based on self-adjusting deep learning methods. Through this highly automated process, the overall performance of the system can be greatly improved while reducing costs and human intervention, allowing sensors to respond quickly and optimize performance in a constantly changing real environment, and achieving flexible and accurate adaptation and adjustment of deep learning algorithms in various scenarios, greatly improving the generalization ability of the algorithm and the actual application efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 A schematic diagram of a sensor adaptive system based on deep learning algorithm and big data analysis provided by the present invention; Figure 2 A schematic diagram of the steps for collecting big data provided by the present invention; Figure 3 This is a schematic diagram of data cleaning and broken value processing provided by the present invention. DETAILED DESCRIPTION
[0017] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.
[0018] like Figure 1 As shown, a sensor adaptive system based on deep learning algorithm and big data analysis includes a cloud big data management unit, a big data analysis and processing unit, a big data feature fusion unit and a deep learning training unit; wherein, The cloud-based big data management unit collects ground data and satellite remote sensing data from sensors and devices as collected big data; The big data analysis and processing unit cleans and processes the collected big data, eliminates noise and fills in broken values; uses spatial time-frequency analysis to extract multi-scale feature information from the collected big data; The big data feature fusion unit: based on the feature information and historical data, integrates and obtains the total data set, divides the training set, the validation set, and the test set, and establishes a flexible target model; integrates the collected big data, historical data, and feature information, associates them using a data fusion algorithm, and obtains fused data; The deep learning training unit: trains a target model based on the fused data, and uses the target model to predict new data to provide a preliminary judgment result of the target location; establishes an adaptive environmental model based on the collected big data collected in real time, and uses an adaptive positioning method to fine-tune the preliminary judgment result to obtain a final result; feeds the final result back to the adaptive target model to retrain and optimize the target model.
[0019] First, ground data and satellite remote sensing data are collected from different sensors and devices as acquisition big data. Then, the acquisition data is cleaned and processed to eliminate noise and fill broken values. Next, spatial time-frequency analysis technology is used to extract multi-scale feature information from the acquisition data. On this basis, the feature information and historical data are integrated to construct the total data set, which is divided into training set, validation set and test set to establish a flexible target model. Then, by integrating the acquisition big data, historical data and feature information, the data fusion algorithm is used to associate them to obtain fused data. Based on the fused data, the target model is trained and used to predict new data to obtain the preliminary judgment result of the target location. Then, based on the acquisition big data collected in real time, an adaptive environment model is established, and the adaptive positioning method is used to fine-tune the preliminary judgment result to obtain the final result. Finally, the final result is fed back to the adaptive target model, and the target model is retrained and optimized to continuously improve the model performance and accuracy.
[0020] The accuracy of ground target positioning can be improved by collecting data from multiple sensors and devices and combining it with historical data for comprehensive analysis. Data fusion algorithms can associate and integrate information from various data sources to obtain more accurate target location information. Using big data fusion methods, multi-scale feature information can be used to train the target model. By integrating the collection of big data, historical data and feature information, and continuously feeding the final results back to the target model for retraining and optimization, the performance and accuracy of the model can be gradually improved. The use of adaptive environmental models and adaptive positioning methods enables this method to adapt to the changes and complexity of the ground environment. By collecting data in real time and fine-tuning it according to the adaptive environmental model, a more accurate final result can be obtained.
[0021] like Figure 2 As shown, the sensors include vibration sensors, microwave sensors, radar sensors, and the equipment is specifically a satellite remote sensing receiver; Collecting ground data and satellite remote sensing data from sensors and equipment, as the steps of collecting big data are as follows: selecting the installation location, integrating the data collection objectives and environmental requirements, and establishing the installation points of sensors and equipment; Assemble sensors and equipment at the installation point and use waterproof housing, anti-corrosion coating, and anti-biological attachment materials as protective measures; Adjust sensor sensitivity, correct errors or correct instrument deviations, and calibrate sensors and equipment; Start sensors and devices to collect data, set parameters and sampling rates, and obtain collected big data; based on the collected big data, use processing software to convert it into processable digital data.
[0022] First, select a suitable installation location and determine the installation point of sensors and equipment by comprehensively considering the data collection objectives and environmental requirements. This helps ensure that the collected data can accurately reflect the location and status of the target and provide an effective input source. Secondly, assemble sensors and equipment at the installation point and use protective measures such as waterproof housing, anti-corrosion coating and anti-biological attachment materials to protect sensors and equipment from damage by the ground environment. Doing so can increase the durability and reliability of sensors and equipment and ensure continuous and stable data collection. Third, calibrate sensors and equipment by adjusting the sensitivity of sensors, correcting errors or correcting instrument deviations. The calibration process can reduce the error of sensors and equipment and improve the accuracy and reliability of data. Accurate data is the basis for subsequent data processing and fusion, which helps to improve the accuracy of positioning methods. In addition, based on the collected data, use processing software and other tools to convert it into processable digital data. Such digital data processing can help quickly analyze and extract feature information in the data, providing strong support for subsequent data fusion and model building. Proper data processing is the key to achieving efficient positioning and prediction. In summary, through the meticulous steps of collecting ground data and satellite remote sensing data, accurate and complete data sources can be obtained, bringing multiple benefits to a sensor adaptive system based on deep learning algorithms and big data analysis. These benefits include improving the accuracy of positioning and the performance of the model, enhancing the adaptability and stability of the method, and improving ground safety and application efficiency. Effective data collection and processing will provide strong technical support for ground target positioning and promote the development and application of the ground field.
[0023] like Figure 3 As shown, cleaning and processing specifically refers to data cleaning and broken value processing; Data cleaning includes outlier detection, noise elimination, and data normalization; Outlier detection specifically involves using standard deviation statistics to detect and exclude outliers in the collected big data. Data points based on thresholds or rules are considered outliers and replaced. Specifically, noise elimination involves applying signal processing techniques including moving average filtering, median filtering, and Kalman filtering to eliminate noise in the collected big data; Data normalization is specifically based on minimum-maximum normalization processing to ensure that the collected big data and the measurement unit are in the same scale range; Broken value processing includes broken value detection, broken value filling, and multiple interpolation; Specifically, broken value detection uses statistical analysis and data visualization methods to detect the existence of broken values in the collected big data; The broken value filling method is to select one of the mean, median, mode filling, regression filling, interpolation method, and deep learning-based filling method as a filling method based on the characteristics of the collected big data and the type of broken values to fill the broken values; Specifically, multiple interpolation involves filling in the broken values using a multiple interpolation method, specifically a matrix decomposition method, for collected big data whose broken value ratio exceeds a preset threshold.
[0024] First, data cleaning includes outlier detection, noise elimination, and data normalization. By using standard deviation statistics to detect and replace outliers, anomalies in the collected big data can be eliminated. Signal processing techniques, such as moving average filtering and median filtering, can be used to eliminate noise in the data. In addition, through data normalization methods, such as minimum-maximum normalization, data can be mapped to the same scale range to eliminate the effects caused by different units.
[0025] Secondly, broken value processing is an important part of data cleaning. It includes broken value detection, broken value filling and multiple interpolation. Statistical analysis and data visualization methods can be used to detect broken values in collected big data. For different types of missing values, appropriate filling methods can be selected, such as mean, median, mode filling, regression filling, interpolation and filling methods based on machine learning. For situations with a high proportion of broken values, multiple interpolation methods, such as matrix decomposition, can be used to fill in broken values.
[0026] By cleaning and processing data, the reliability, consistency and accuracy of data can be improved, providing a high-quality data foundation for subsequent data analysis and model training. Cleaning outliers, eliminating noise, normalizing data and processing broken values can help ensure data quality and provide more accurate data input for ground target positioning methods. The effective implementation of these steps will improve the performance and accuracy of positioning methods and provide beneficial support for ground safety and application effectiveness.
[0027] The steps of extracting multi-scale feature information from collected big data using spatial time-frequency analysis are as follows: The short-time Fourier transform method is used as the analysis method for spatial time-frequency analysis; Apply analytical methods, use sliding windows of different sizes to perform short-time Fourier transform, and perform multi-scale analysis on the acquired big data to explore the time-frequency characteristics of the signal; Based on the time-frequency characteristics as the data source, the extraction methods including energy calculation, spectrum peak extraction and wavelet coefficient statistics are adopted to extract the spectrum energy, frequency contour and waveform shape characteristics of the signal and generate a feature set; Based on the feature set, principal component analysis and feature selection algorithms are used to reduce the dimension of features and retain important information as feature information.
[0028] First, the use of short-time Fourier transform as an analysis method for spatial time-frequency analysis can effectively analyze the time-frequency characteristics of the signal. By applying short-time Fourier transform on sliding windows of different sizes, the changes of the signal at different times and frequencies can be explored. This helps capture the instantaneous spectral characteristics of the signal and allows multi-scale analysis of the signal.
[0029] Secondly, the multi-scale analysis method can be used to extract the signal's spectral energy, frequency profile, waveform shape and other features. By calculating the signal's energy, extracting the spectral peak and counting the wavelet coefficients, key time-frequency feature information can be obtained. This helps describe the signal's energy distribution, frequency characteristics and waveform shape in different time periods and frequency bands, thereby providing an important feature set for subsequent data analysis and model building.
[0030] Based on the extracted feature set, principal component analysis and feature selection algorithms can reduce the dimension of features and retain important information. Principal component analysis can transform high-dimensional features into low-dimensional feature space through linear transformation, reduce data redundancy, and retain the main variance in the data. Feature selection algorithms can help screen out the most useful features for the target task and further streamline the feature set. This helps improve the efficiency and accuracy of data analysis.
[0031] From an implementation perspective, the spatial time-frequency analysis method and multi-scale feature extraction steps can extract rich feature information from the collected big data. These feature information fully reveals the time-frequency characteristics of the signal and can provide valuable input for subsequent data analysis, model training, and target positioning. By reducing the dimension of the features and selecting important features, the efficiency of data processing and the interpretability of the model can also be improved. Therefore, the implementation of these steps will have a beneficial effect on a sensor adaptive system based on deep learning algorithms and big data analysis, improving the accuracy and reliability of positioning.
[0032] Based on feature information and historical data, the steps to integrate and obtain the total data set and divide the training set, validation set, and test set are as follows: The feature information of the past periods is used as historical data, and the historical data and feature information are integrated as the total data set; Perform another dimensionality reduction process on the features in the total data set based on linear discriminant analysis to reduce the dimension of the features and eliminate redundant information, and obtain the processed total data set, including categorical variables or non-numerical features; perform label encoding based on categorical variables or non-numerical features; The total data set is divided into a training set, a validation set, and a test set.
[0033] First, the feature information of the past period is used as historical data, and the historical data and current feature information are integrated into a total data set. This total data set will contain all the data used to train and evaluate the model.
[0034] Secondly, for the features in the total data set, we can use methods such as linear discriminant analysis to perform dimensionality reduction again. Dimensionality reduction can reduce the dimension of features and eliminate redundant information, and extract features that have an important impact on the classification task. In this way, we can obtain the processed total data set, which includes categorical variables or non-numerical features.
[0035] Next, if there are categorical variables or non-numeric features in the total dataset, label encoding is required. Label encoding is the process of converting categorical variables or non-numeric features into numerical representations. For example, methods such as one-hot encoding or label encoding can be used to convert categorical variables into numerical values so that deep learning models can process these features.
[0036] Finally, the processed total data set is divided into training set, validation set and test set. The common division ratio is that 10% of the data is used as the training set, and 10% of the data is used as the validation set and test set. The training set is used to learn and train the model parameters, the validation set is used to adjust the model's hyperparameters and evaluate the model's performance, and the test set is used to finally evaluate the model's performance on unseen data.
[0037] From an implementation perspective, integrating feature information and historical data into a total data set, and performing dimensionality reduction and label encoding processes can extract key features, convert data types, and provide preparation for modeling and evaluation. Dividing the total data set into training sets, validation sets, and test sets helps with model training, tuning, and evaluation, and can also evaluate the model's generalization ability on unknown data. Such step implementation can effectively utilize data, improve model performance, and reduce the risk of overfitting, thereby providing beneficial effects on the application and performance of ground target positioning methods.
[0038] The steps to build a flexible target model are as follows: Select the logistic regression model as the target model; Train the target model based on the training set, apply training algorithms and optimization methods to fit the training set, and learn the patterns and regularities of the data; Use the validation set to verify and tune the trained target model, evaluate the performance of the target model on the validation set and generate evaluation results. Based on the evaluation results, adjust model parameters and select features to optimize the target model. Use the test set to evaluate the optimized target model by calculating evaluation indicators including accuracy, precision, recall, and F1 score; Deploy the target model to actual applications to perform target prediction tasks.
[0039] First, choose a model that is suitable for the target problem, such as a logistic regression model. Logistic regression is a commonly used classification algorithm that performs well when dealing with binary classification problems. Choosing a suitable model can be decided based on the problem requirements, data characteristics, and previous experience and knowledge.
[0040] Secondly, the target model is trained using the training set. The model is fitted to the training set using training algorithms and optimization methods, and the patterns and regularities of the data are learned. During the training process, optimization algorithms such as gradient descent can be used to iteratively update the model parameters to minimize the loss function, thereby improving the accuracy of the model.
[0041] Next, use the validation set to verify and tune the trained target model. By evaluating the performance of the target model on the validation set, you can understand the generalization ability and effect of the model. Based on the evaluation results, you can adjust model parameters, select features, and perform other optimization strategies to improve the performance and adaptability of the model.
[0042] Then, the optimized target model is finally evaluated using the test set. By calculating evaluation metrics such as accuracy, precision, recall, and F1 score, the predictive power and performance of the model can be objectively evaluated. This evaluation will provide a more comprehensive understanding of the model's performance on unseen data.
[0043] Finally, the optimized target model is deployed to actual applications to perform target prediction tasks. This can be done by integrating the model into a real-time or offline system and using the model to classify, predict or make decisions on unknown data. After the model is deployed, it can also be monitored and iteratively improved to ensure the continued effectiveness and performance stability of the model.
[0044] The steps of using data fusion algorithm to associate and obtain fused data are as follows: Integrate the collected big data, historical data and feature information to generate a unified data set; Through feature importance evaluation, data with important significance to the correlation results is selected from the unified data set to extract key features; Use data fusion algorithms to associate, match, align and merge key features to obtain associated data. Data fusion algorithms include association based on key value matching, association based on similarity matching, and association based on time windows. Through weighted average, decision rules and aggregation functions, the related data are merged to generate fused data.
[0045] First, collect big data, historical data and feature information to generate a unified data set. This can be achieved by integrating, cleaning and converting data from different data sources. Ensure the uniformity and consistency of data formats for subsequent association and fusion operations.
[0046] Secondly, through feature importance evaluation, data that is important to the association results is selected from the unified data set, and key features are extracted. Feature importance evaluation can be based on statistical analysis, deep learning models and other methods. By selecting features with high information content and important influence on the target task, the redundancy of the data can be reduced and the accuracy of the association results can be improved.
[0047] Next, the data fusion algorithm is used to perform association operations. Data fusion algorithms can include association based on key value matching, association based on similarity matching, and association based on time windows. These algorithms match, align, and merge key features according to the association rules and matching conditions of the data, thereby realizing the association between different data sources.
[0048] Finally, the associated data is combined to generate fused data through weighted average, decision rules or aggregation functions. This can be weighted average of the associated data to obtain a comprehensive result; or data decision making through decision rules, based on the weights of different data sources; or aggregation functions such as summation and averaging can be applied to aggregate the associated data into an overall result.
[0049] From an implementation perspective, using data fusion algorithms to associate and obtain fused data can provide more comprehensive, accurate and decision-making information. This process can solve problems such as data heterogeneity, fragmentation and inconsistency by integrating multiple data sources, extracting key features and applying fusion algorithms, thereby providing a more reliable and useful data foundation for applications such as ground target positioning.
[0050] The steps of training the target model based on the fused data and using the target model to predict new data and provide a preliminary judgment result of the target location are as follows: Divide the fused data into post-training set and post-test set, and call the target model; Based on the post-training set, the target model is trained, and the hyperparameters and model fit of the target model are adjusted based on optimization methods including cross-validation and grid search; Using the post-test set, the target model is evaluated again according to evaluation indicators including mean square error, accuracy, and recall, and the best target model is selected based on the evaluation results; Apply the target model to the latest collected big data, input the feature data to be predicted, and use the trained target model to generate the corresponding target position prediction result as the preliminary judgment result.
[0051] First, the fusion data is divided into a training set and a test set according to a certain ratio. The data of is used as the training set, and the data of is used as the test set. This ensures that the training set has enough data to train the target model, and the test set can be used to evaluate the prediction ability of the model.
[0052] Next, the target model is trained using the training set. Using the features of the fused data as input, the target model is fitted to the training set to learn the patterns and regularities of the data. Techniques such as cross-validation can be applied to evaluate the performance of the model on the training set, and hyperparameters can be optimized based on methods such as grid search to improve the model's fit and generalization ability.
[0053] Then, the trained target model is evaluated using the test set. The features of the test set are input into the target model for prediction and compared with the true target position of the test set. By calculating evaluation indicators such as mean square error, precision, and recall, the prediction accuracy and performance of the model can be objectively evaluated. Based on the evaluation results, the target model with the best performance is selected.
[0054] Finally, the trained target model is applied to the latest collected big data. The feature data to be predicted is input into the target model, and the model is used to predict the target position and generate a preliminary judgment result. This result can be used to preliminarily locate the target position and provide a reference for subsequent analysis, decision-making and further processing.
[0055] From an implementation perspective, by using fused data to train the target model and make predictions, a more accurate and reliable preliminary judgment result of the target location can be provided with the help of diverse data information. This step can make full use of the fused data and the trained model to provide preliminary prediction and decision support for ground target positioning.
[0056] Based on the real-time collected big data, an adaptive environmental model is established, and the adaptive positioning method is used to fine-tune the preliminary judgment results. The steps to obtain the final result are as follows: Based on the collected big data, an adaptive environmental model is established using Bayesian filters; Adopting adaptive positioning methods including weighted least squares positioning, particle filter positioning, and conditional random field positioning to make a more accurate estimate of the preliminary judgment results and generate fine-tuning results; Based on the fine-tuning results, the final result more accurately reflects the position information of the target position.
[0057] First, use the collected big data to build an adaptive environment model. This can be achieved through methods such as Bayesian filtering. Bayesian filtering is a recursive probabilistic filtering method that can perform state estimation and a posteriori updates based on prior information and real-time collected data to build an adaptive environment model. The adaptive environment model can help understand the position and motion state of the target at different time points.
[0058] Next, an adaptive positioning method is used to fine-tune the preliminary judgment results. Adaptive positioning methods include weighted least squares positioning, particle filter positioning, conditional random field positioning, etc. These methods can update and correct the initial position estimate based on real-time collected data and adaptive environmental models to obtain a more accurate target position estimate. The adaptive positioning method takes into account different environmental factors and data characteristics and can better adapt to different scenarios and conditions.
[0059] Finally, based on the fine-tuning result, it is used as the final position estimation result to reflect the information of the target location. The fine-tuning result takes into account the influence of the adaptive environmental model and real-time data, and can more accurately reflect the state and changes of the target location. This step will provide a more precise and reliable estimate of the target location, providing a more accurate basis for subsequent analysis, decision-making and operation.
[0060] From an implementation perspective, building an adaptive environmental model based on real-time collected data and using an adaptive positioning method to fine-tune the preliminary judgment results can improve the precision and accuracy of target position estimation. This step combines the establishment of an environmental model with the application of an adaptive positioning method, making full use of real-time data and previous information to provide a more reliable and accurate final result for ground target positioning.
[0061] The final result is fed back to the adaptive target model. The steps for retraining and optimizing the target model are as follows: The final result is combined with the fusion data to construct a new training set, which includes feature data and corresponding target position labels; Use the original target model as the initial model; Retrain the target model using the new training set, input feature data and corresponding target location labels, and iteratively adjust the parameters of the target model to make it predict the target location more accurately; Based on the above steps, retraining and optimization iterations are performed to gradually improve the performance and accuracy of the target model through continuous feedback of the final results, retraining and optimization.
[0062] First, the final position estimation result is merged with the fusion data to construct a new training set. The new training set includes feature data and corresponding target location labels. Feature data can include collected big data, historical data, and other related features, while the target location label is the final position estimation result.
[0063] Next, the original target model is used as the initial model. The initial model can be a previously trained model, or a model that has undergone a series of training and optimization.
[0064] Then, the target model is retrained using the new training set. The feature data of the new training set and the corresponding target location label are input into the target model, and the parameters of the model are iteratively adjusted to make it predict the target location more accurately. This can be done using common deep learning algorithms such as neural networks, support vector machines, etc., and using optimization methods to update the model's weights and biases.
[0065] Retrain and optimize iteratively based on the above steps. By continuously feeding back the final results, retraining and optimizing, the performance and accuracy of the target model can be gradually improved. This step can be repeated multiple times until satisfactory performance indicators are achieved.
[0066] From an implementation perspective, feeding the final results back into the adaptive target model for retraining and optimization can continuously improve the model's predictive ability and accuracy. By using the final results as training data, the model can learn more accurate target locations and adaptive environmental patterns, thereby improving the reliability of predictions. This step effectively utilizes feedback information and applies it to model updates and optimizations, which helps improve the overall performance of the target positioning system.
[0067] Working principle: First, the location, motion trajectory and other relevant data of ground targets are collected from the ground and air through devices such as vibration sensors, microwave sensors, radar sensors and satellite remote sensing receivers. These data are cleaned and processed, including outlier detection, noise elimination, data normalization and broken value processing. Next, multi-scale feature information is extracted from the cleaned data using techniques such as spatial time-frequency analysis. These feature information are integrated with historical data to construct a total data set, which is divided into training set, validation set and test set. A suitable target model, such as a logistic regression model, is selected on the training set, and trained and optimized to improve the performance and prediction ability of the model. The data fusion algorithm associates and merges the collected data, historical data and feature information to generate fused data. Using the trained target model, new data is predicted to obtain a preliminary judgment result of the target location. Then, an adaptive environmental model is established, and the adaptive positioning method is used to fine-tune the preliminary judgment result in combination with the real-time collected data to obtain a more accurate final result. The final result is fed back to the adaptive target model, which is retrained and optimized to gradually improve the performance and accuracy of the model. This cyclic process is continuously iterated, allowing the model to better adapt to changes in ground targets and complex environments. By integrating multiple key steps such as big data fusion, feature extraction, model training, and adaptive environment modeling, this method can achieve accurate ground target positioning and prediction, helping to improve ground safety and application effectiveness.
[0068] The above are only embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A sensor adaptive system based on deep learning algorithm and big data analysis, characterized in that: It includes cloud big data management unit, big data analysis and processing unit, big data feature fusion unit and deep learning training unit; among them, The cloud-based big data management unit collects ground data and satellite remote sensing data from sensors and devices as collected big data; The big data analysis and processing unit cleans and processes the collected big data, eliminates noise and fills in broken values; uses spatial time-frequency analysis to extract multi-scale feature information from the collected big data; The big data feature fusion unit: based on the feature information and historical data, integrates and obtains the total data set, divides the training set, the validation set, and the test set, and establishes a flexible target model; integrates the collected big data, historical data, and feature information, associates them using a data fusion algorithm, and obtains fused data; The deep learning training unit: trains a target model based on the fused data, and uses the target model to predict new data to provide a preliminary judgment result of the target location; establishes an adaptive environmental model based on the collected big data collected in real time, and uses an adaptive positioning method to fine-tune the preliminary judgment result to obtain a final result; feeds the final result back to the adaptive target model to retrain and optimize the target model.
2. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The sensors include vibration sensors, microwave sensors, and radar sensors, and the equipment is specifically a satellite remote sensing receiver; Collecting ground data and satellite remote sensing data from sensors and devices. The specific steps for collecting big data are: Select the installation location, comprehensively consider the data collection objectives and environmental requirements, and determine the installation points of the sensors and equipment; Assemble the sensors and devices at the installation point and use waterproof housing, anti-corrosion coating and anti-biological attachment materials as protective measures; Adjust the sensitivity of sensors, correct errors or correct instrument deviations, and calibrate the sensors and equipment; Start the sensors and devices to collect data, set parameters and sampling rate, and obtain the collected big data; Based on the collected big data, processing software is used to convert it into processable digital data.
3. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The cleaning and processing specifically refers to: data cleaning and broken value processing; The data cleaning includes outlier detection, noise elimination, and data normalization; Specifically, the outlier detection is to use standard deviation statistics to detect and exclude outliers in the collected big data, and data points based on thresholds or rules are regarded as outliers and replaced; Specifically, the noise elimination includes applying signal processing techniques including moving average filtering, median filtering and Kalman filtering to eliminate the noise in the collected big data; The data normalization is specifically to ensure that the collected big data and the measurement unit are in the same scale range based on minimum-maximum normalization processing; The broken value processing includes broken value detection, broken value filling, and multiple interpolation; The broken value detection is specifically to use statistical analysis and data visualization methods to detect the existence of broken values in the collected big data; The broken value filling is specifically to select one of the mean, median, mode filling, regression filling, interpolation method, and deep learning-based filling method as a filling method based on the characteristics of the collected big data and the type of broken values to fill the broken values; The multiple interpolation is specifically to fill in the broken values using a multiple interpolation method, specifically a matrix decomposition method, for the collected big data whose broken value ratio exceeds a preset threshold.
4. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The steps of extracting multi-scale feature information from the collected big data by using spatial time-frequency analysis are as follows: Using short-time Fourier transform method as the analysis method of the spatial time-frequency analysis; Applying the analysis method, using sliding windows of different sizes to perform short-time Fourier transform, and performing multi-scale analysis on the collected big data to explore the time-frequency characteristics of the signal; Based on the time-frequency characteristics as a data source, an extraction method including energy calculation, spectrum peak extraction, and wavelet coefficient statistics is adopted to extract the spectrum energy, frequency contour, and waveform shape characteristics of the signal to generate a feature set; Based on the feature set, principal component analysis and feature selection algorithms are used to reduce the dimension of features and retain important information as the feature information.
5. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The steps of integrating and acquiring the total data set based on the feature information and historical data and dividing the data set into the training set, the validation set and the test set are specifically as follows: Using the feature information of previous periods as the historical data, and integrating the historical data and the feature information as a total data set; Perform another dimensionality reduction process on the features in the total data set based on linear discriminant analysis to reduce the dimension of the features and eliminate redundant information, and obtain the processed total data set, including categorical variables or non-numerical features; Perform label encoding based on the categorical variables or non-numerical features; The total data set is divided into a training set, a validation set and a test set.
6. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The steps of establishing a flexible target model are specifically as follows: Selecting a logistic regression model as the target model; Training the target model based on the training set, applying training algorithms and optimization methods to fit the training set, and learning patterns and regularities of the data; Using the validation set, verifying and tuning the trained target model, evaluating the performance of the target model on the validation set and generating an evaluation result, and adjusting model parameters and selecting features based on the evaluation result to optimize the target model; Using the test set, evaluating the optimized target model by calculating evaluation indicators including accuracy, precision, recall, and F1 score; The target model is deployed in practical applications to perform target prediction tasks.
7. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The steps of using the data fusion algorithm to perform association and obtain fused data are specifically as follows: Integrate the collected big data, historical data and feature information to generate a unified data set; By evaluating the importance of features, data having important significance for the correlation results is selected from the unified data set to extract key features; Using a data fusion algorithm to associate, match, align and merge the key features to obtain associated data, the data fusion algorithm includes association based on key value matching, association based on similarity matching, and association based on time window; The associated data are merged to generate fused data through weighted averaging, decision rules, and aggregation functions.
8. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The step of training the target model based on the fused data and using the target model to predict new data and provide a preliminary determination result of the target position is specifically as follows: Dividing the fused data into a post-training set and a post-test set, and calling the target model; Based on the post-training set, the target model is trained, and based on an optimization method including cross-validation and grid search, the hyperparameters and model fit of the target model are adjusted; Using the post-test set, the target model is evaluated again according to evaluation indicators including mean square error, accuracy, and recall, and the best target model is selected based on the evaluation results; The target model is applied to the latest collected big data, the feature data to be predicted is input, and the trained target model is used to generate the corresponding target position prediction result as the preliminary judgment result.
9. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The steps of establishing an adaptive environment model based on the collected big data collected in real time, and fine-tuning the preliminary judgment result by using an adaptive positioning method to obtain the final result are specifically as follows: Based on the collected big data, an adaptive environmental model is established using a Bayesian filter; Adopting adaptive positioning methods including weighted least square positioning, particle filter positioning, and conditional random field positioning to make a more accurate estimate of the preliminary judgment result and generate a fine-tuning result; Based on the fine-tuning result, the position information of the target position is more accurately reflected as a final result.
10. The sensor adaptive system based on deep learning algorithm and big data analysis according to claim 1, characterized in that: The final result is fed back to the adaptive target model, and the steps of retraining and optimizing the target model are specifically as follows: The final result is combined with the fusion data to construct a new training set, wherein the new training set includes feature data and corresponding target position labels; Using the original target model as the initial model; Retraining the target model using a new training set, inputting feature data and corresponding target location labels, and iteratively adjusting the parameters of the target model to more accurately predict the target location; Retraining and optimization iterations are performed based on the above steps, and the performance and accuracy of the target model are gradually improved through continuous feedback of final results, retraining and optimization.
Citation Information
Patent Citations
Marine target positioning method based on big data fusion
CN116702093A
Anti-interference air intelligent sensing method based on weak target
CN118585899A
Cited By
Hydrogen long-distance pipeline sensor optimization method based on artificial intelligence algorithm
CN119249281A
Big data cleaning method based on deep learning and data feature matching
CN119884608A
Fire early warning and emergency processing system for lithium battery of electric bicycle
CN120299163A
Intelligent self-adaptive environment perception sensor
CN120489240A
