42 results about "High dimensional data sets" patented technology

The invention discloses a personalized design intelligent interaction method based on topological optimization, and the method comprises the steps: S1, constructing a user behavior monitoring system based on multi-modal data collection, and generating a high-dimensional data set containing time sequence features; s2, based on an adaptive deep neural network model, performing dynamic prediction and hierarchical modeling on the personalized demand of the user; s3, generating an optimal solution of the design parameters in real time through a dynamic topology reconstruction algorithm; s4, iteratively updating a design scheme generation rule through a reinforcement learning mechanism; s5, constructing a multi-objective design space exploration and optimization framework supported by the neural network, and rapidly screening and optimizing design parameters; s6, dynamic updating and local optimization of the design scheme are achieved through multi-dimensional data mapping and parameterization regulation and control means; and S7, establishing a data-driven continuous learning and evolution mechanism, continuously optimizing the user demand prediction model and designing an optimization algorithm. The method has the advantages of being high in dynamic adaptability, high in intelligent level and high in personalized meeting precision.

The invention relates to the technical field of internet of things federated learning, in particular to a federated learning method oriented to internet of things privacy and heterogeneous data and an internet of vehicles privacy data protection method and system, differential privacy is introduced into federated learning, Gaussian noise meeting the differential privacy is added to a client to be uploaded to local parameters of a server, and meanwhile, the local parameters of the client are added to the internet of things privacy and heterogeneous data. In order to prevent excessive accumulation of privacy cost in model iteration training, noise disturbance is added to parameters uploaded by a local model through dynamic differential privacy, and the privacy and availability of the model are balanced to the maximum extent by using an exponential attenuation mechanism of Gaussian noise; the Wasserstein distance between the local parameter and the global parameter is calculated to serve as a regularization item to update the local model, the generalization ability of the model to Non-IID data is improved, and the accuracy of the global model is improved; and when high-dimensional data is processed, dimension reduction processing is performed on a high-dimensional data set by combining PCA with federated learning, so that the data set consistency of each client is improved while the local model training speed is improved.

The invention discloses a data dynamic partition storage method and system based on adaptive clustering, and relates to the field of data processing. The method comprises the following steps: S1, extracting multi-scale geometric features of a high-dimensional data set, calculating a local curvature and generating a curvature feature matrix; s2, constructing a feature distance matrix and a similar matrix based on the curvature feature matrix, and generating a low-dimensional embedding matrix; s3, executing self-optimization clustering according to the low-dimensional embedded matrix, determining a cluster number through singular value distribution, and generating an initial cluster; s4, calculating a stability factor of each cluster, and triggering cluster splitting or merging operation according to a threshold value to form updated cluster division; and S5, performing incremental processing on newly added data points, obtaining a new point local curvature through local curvature gradient correction, mapping the new point local curvature to a low-dimensional space, dynamically deciding affiliation based on a cluster radius, and updating partitions. Through multi-scale feature extraction, self-optimization clustering and incremental updating mechanisms, high-dimensional data clustering precision, robustness and calculation efficiency are improved.

The invention provides a battery health state prediction method and system, and the method comprises the steps: obtaining a high-dimensional data set in a battery discharge process, and carrying out the normalization processing of the high-dimensional data set, and obtaining standardized data; analyzing the correlation between the standardized data and the battery capacity by using a Spearman correlation coefficient to obtain a target data set; inputting the target data set into a width auto-encoder, extracting feature nodes and enhanced nodes of the width auto-encoder, and obtaining intermediate feature representation; and inputting the intermediate feature representation into a random configuration network fused with an intuitionistic fuzzy mechanism, and generating a battery health state preliminary prediction result, thereby alleviating the technical problem of low accuracy of the existing battery health state prediction.

The invention provides a high-dimensional abnormal data detection method. The objective of the invention is to improve the high-dimensional data anomaly detection effect. The method comprises the following steps: constructing M isolated forests according to position codes of M first ants in the current iteration; based on high-dimensional abnormal data detection, evaluating the effectiveness of the M isolated forests, and determining fitness values corresponding to the M first ants of the current iteration; determining elite ants of the current iteration according to the fitness values corresponding to the M first ants of the current iteration and the fitness values corresponding to the M second ants of the previous iteration; iterating for multiple times according to the steps, and determining a final target position code according to the position code corresponding to the elite ant determined by the last round of iteration; and according to the final target position code, constructing an anomaly detection model, and based on a data segmentation mode corresponding to the anomaly detection model, segmenting the high-dimensional data set to obtain an anomaly detection result.

The invention relates to a K-nearest neighbor search method based on cluster structure optimization and a triangular inequality strategy, which comprises the following steps: preliminarily dividing a to-be-searched sample set into M clusters by adopting a k-means + + clustering algorithm, and obtaining cluster structure information of the sample set; calculating the representative distance of each sample in each cluster, constructing an unqualified cluster judgment rule based on the representative distance, and judging all clusters to obtain unqualified clusters; based on a k-means + + clustering algorithm and an unqualified cluster judgment rule, decomposing each unqualified cluster into a plurality of qualified sub-clusters, updating M and cluster structure information, and obtaining a sample set after structure updating; and based on the training sample set after structure updating, performing K-nearest neighbor search on a to-be-queried sample by adopting a triangular inequality check strategy. The method is suitable for rapid neighbor search under the condition of a large-scale high-dimensional data set, and can meet the equipment fault diagnosis requirement under a complex industrial scene.

Systems and methods are disclosed for identifying relationships between complex datasets and / or high-dimensional datasets for predictive modeling. The method includes clustering data associated with one or more entities in a first dataset based on distance data; clustering the data associated with the one or more entities in a second dataset based on longitudinal data; consolidating the first dataset and the second dataset into a third dataset based on weights assigned to one or more edges between one or more nodes in the first dataset and the second dataset; generating a diagnosis space indicating a condition of the one or more entities based on the third dataset; and determining, via input of the diagnosis space into a machine learning model, an optimization of a weighting scheme for assigning weights to one or more features within the longitudinal data.

The invention discloses a sliding bearing service life prediction method, and relates to the technical field of sliding bearing intelligent monitoring, and the method comprises the steps: an edge calculation node uses a quantum decryption algorithm to decrypt multi-modal time series data, and carries out the fractional order chaotic dynamics analysis, thereby achieving the prediction of the service life of a sliding bearing. Performing phase-space reconstruction on the multi-modal time sequence data, extracting fractal dimensions of phase-space attractors, and fusing chemical and crystal data to a quantum entanglement space by adopting a quantum variational auto-encoder to generate a high-dimensional data set fusing macroscopic and microscopic features; constructing a quantum enhanced life prediction model, performing parameter calibration by adopting quantum Bayesian optimization, and predicting a bearing health index according to the high-dimensional data set; according to the invention, the quantum-enhanced life prediction model is constructed through the quantum variational auto-encoder, and the model parameters are dynamically calibrated in combination with the quantum Bayesian optimization algorithm, so that the accuracy and reliability of bearing life prediction are improved.

The invention discloses a cardiovascular prediction method assisted by double-layer feature selection, and the method comprises the steps: collecting data sets which comprise a Cardiovascular data set, a Stalog data set, a Heart data set, and a Z-Alizadeh data set; preprocessing the data set; screening out the feature with the highest relevancy through double-layer feature selection; constructing a prediction model based on a machine learning algorithm; and inputting various health data of the patient into the cardiovascular prediction model to predict whether the patient has the risk of cardiovascular diseases. According to the method, a double-layer feature selection technology is introduced, so that the negative influence of redundant and irrelevant features in a high-dimensional data set on the model is avoided, and the prediction accuracy and stability are improved. Through automatic optimization of hyper-parameters, the model can automatically find an optimal parameter combination on different data sets, and the adaptability and prediction performance of the model are improved. In combination with precise training and optimization of a machine learning algorithm, the risk that a traditional model is prone to over-fitting or under-fitting on a small-scale data set is avoided, and the robustness of the model is ensured.

The invention discloses a data-driven packaged multi-physical digital twin system and a construction method thereof. The method comprises the following steps: firstly, acquiring a multi-physical-field thermal-force simulation high-dimensional data set of a physical device through a high-precision finite element, and then carrying out compression dimension reduction on the high-dimensional data set in a spatial dimension; a solving problem of a differential equation of a physical field is converted into a mapping relation between an input parameter and a low-dimensional time coefficient, and the mapping relation is learned by using an LSTM neural network. And finally, predicting the high-dimensional multi-physical field distribution of the real electronic device at the next moment through the multi-physical field state data at the current moment and the input control variable by utilizing the trained neural network, and completing the feedback control of the real physical device. Compared with a common numerical simulation method and a device monitoring means, the digital twin system calibration method combines the order reduction method and the neural network, calibrates the digital twin system through the measured real temperature value, can calculate the multi-physical field information of the physical device in real time, and maintains long-term accuracy.

The application discloses a kind of personalized design intelligent interaction methods based on topology optimization, including S1, construct the user behavior monitoring system based on multi-modal data acquisition, generate high-dimensional data set containing time sequence characteristics;S2, based on adaptive deep neural network model, dynamically predict and hierarchical modeling to user personalized demand;S3, the optimal solution of design parameter is generated in real time by dynamic topology reconstruction algorithm;S4, through reinforcement learning mechanism, iteratively update design scheme generation rule;S5, construct neural network supported multi-objective design space exploration and optimization framework, quickly filter and optimize design parameter;S6, through multidimensional data mapping and parameterization control means, realize the dynamic update and local optimization of design scheme;S7, establish data-driven continuous learning and evolution mechanism, constantly optimize user demand prediction model and design optimization algorithm.The application has the advantages of strong dynamic adaptability, high intelligent level and high personalized satisfaction precision.

The application relates to the technical field of optical communication device testing and control, and discloses a DPMZM modulator fast calibration system and method based on deep learning, which solves the problems of slow speed of an existing DPMZM calibration method, periodic multi-value mapping of deep learning calibration, ambiguity of symmetry symbols, and mismatch of a loss function, and the like. The method constructs a high-dimensional data set through a segmented decoupling scanning strategy, performs physical symmetry correction and preprocessing on labels, designs a one-dimensional convolutional neural network with multi-scale feature aggregation, proposes a physical perception hybrid loss function and adopts a progressive training strategy, and finally embeds the trained model into a calibration system to realize parameter reasoning and physical voltage restoration, thereby completing fast and accurate calibration of the DPMZM.

The present application provides a battery health status prediction method and system, including: obtaining a high-dimensional data set during the battery discharge process, and normalizing the high-dimensional data set to obtain standardized data; using the Spearman correlation coefficient to analyze the correlation between the standardized data and the battery capacity to obtain a target data set; inputting the target data set into a width autoencoder, extracting the feature nodes and enhancement nodes of the width autoencoder, and obtaining an intermediate feature representation; inputting the intermediate feature representation into a random configuration network that integrates an intuitionistic fuzzy mechanism to generate a preliminary prediction result of the battery health status, thereby alleviating the technical problem of low accuracy of existing battery health status prediction.

The present application proposes a data processing method, firstly obtaining similarity data of a data set, wherein the similarity data represents the similarity between data points in the data set, and then determining the similarity density and similarity relationship data of each data point in the data set based on the similarity data; finally clustering the data set based on the similarity density and similarity relationship data of each data point in the data set. Since the data processing method clusters the similarity density and similarity relationship data of each data point in the data set determined by the similarity data of the data set, and does not cluster based on the distance between the data, the method is used to cluster high-dimensional data sets with relatively sparse data distribution, and the quality of the clustering results obtained is better. The present application also proposes a data processing device, an electronic device, and a computer storage medium.

The invention relates to the field of data processing, in particular to a high-dimensional data visualization method and device, a high-dimensional data model training method and device, equipment and a medium, and the model training method comprises the steps that a plurality of training samples are acquired, and each training sample comprises a high-dimensional data set and a corresponding historical successful visualization result; for each high-dimensional data set, calculating k different similarity matrixes of the high-dimensional data set; aiming at each similarity matrix of the high-dimensional data set, taking the similarity matrix as a fully-connected graph with a weight, and extracting graph position coding information of nodes in the fully-connected graph from the similarity matrix by adopting a spectral method; and performing model training by taking k different similarity matrixes of the high-dimensional data set and the corresponding graph position coding information as input data of the model and taking a historical successful visualization result corresponding to the high-dimensional data set as supervision information of the model to obtain a trained high-dimensional data visualization model. The technical scheme can be more efficient and higher in generalization performance.

This invention discloses an approximate nearest neighbor connection optimization method for encrypted databases based on PM# trees, belonging to the field of approximate nearest neighbor connection optimization technology for encrypted databases. It solves the problems of low retrieval efficiency and insecure retrieval processes in existing technologies. The method includes: projecting a high-dimensional dataset into a low-dimensional space using hash projection to obtain a low-dimensional dataset; encrypting the high-dimensional dataset to obtain an encrypted high-dimensional dataset; deleting redundant nodes generated during node splitting in the PM# tree to obtain a PM# tree, and using the PM# tree to build an index on the high-dimensional dataset to obtain an index file; encrypting the index file to obtain an encrypted index; and performing an approximate nearest neighbor connection query on the high-dimensional dataset based on the encrypted index, the low-dimensional dataset, and the encrypted high-dimensional dataset to obtain the query results. This achieves encryption of the high-dimensional dataset while reducing computational load, thus accelerating retrieval speed and improving retrieval quality.

The invention discloses an unsupervised dimensionality reduction visualization method for cell image data. The method comprises the following steps: acquiring an unlabeled cell image high-dimensional data set; processing the high-dimensional data set by adopting an unsupervised dimension reduction algorithm to generate low-dimensional embedding representation; generating a first visual chart based on the low-dimensional embedded representation to display the distribution of the data in the low-dimensional space; performing unsupervised clustering analysis on the data points in the low-dimensional embedding representation, and identifying at least one clustering center point; generating a second visual chart based on a clustering analysis result, and marking a clustering center point in an identifiable manner; according to the method, the clustering center point can be accurately identified, and a visual data distribution overview is provided for a user.

The invention relates to the field of artificial intelligence, particularly discloses a high-dimensional evolutionary feature selection method fusing filtering and packaging strategies, and aims at high-dimensional, small-sample and unbalanced complex data to realize minimum discriminant feature subset screening through a multi-stage hierarchical search framework from coarse to fine and from global to local. The method comprises the steps of generating a high-quality initial population by adopting core-marginal probability sampling and preferentially entering evolution based on double filtering type integrated score sorting of symmetric uncertainty SU and ReliefF; an environment-aware unimproved counter is introduced in the global search stage, the feature adding probability and the feature removing number are dynamically adjusted to balance exploration and development, and repeated evaluation is reduced in combination with fitness cache; and when search stops, binary particle swarm optimization neighborhood refinement is triggered, local high-strength discrete search is realized by Sigmoid mapping, and global optimum is backfilled and updated. Experiments prove that the method obtains a high F1-score and a significant compression feature number on a plurality of high-dimensional data sets.

The application relates to a high-dimensional perception data dimension reduction method in a multi-hop time delay sensitive network, which comprises the following steps: taking a certain specific time window as a benchmark, reading data from each device to form a high-dimensional data set; for the high-dimensional data of each device, a new feature set is formed; a PCA algorithm is used to reduce the dimension of the selected features; an adaptive feature selection (AFS) algorithm is used to further optimize the features after dimension reduction; and the feature data after adaptive feature selection is used for network transmission, so that the data dimension is reduced while the key features of the data are maintained, the transmission time delay and the network load are reduced, the PCA and the AFS are comprehensively used, the advantages of the PCA and the AFS in dimension reduction and feature selection are fully exerted, the data after dimension reduction retains important information and reduces the data volume, and an effective solution is provided for high-dimensional perception data transmission in a multi-hop time delay sensitive network.

The invention relates to an unsupervised feature selection method based on multi-stage learning optimization, and belongs to the technical field of high-dimensional data preprocessing. Performing standardization preprocessing on the obtained original high-dimensional data set, initializing a feature state vector for the high-dimensional data set after standardization preprocessing, and setting a feature state of a preset proportion as a selection state and the rest as an elimination state; setting the state of a preset number of features in the selected state as an elimination state, generating a candidate feature subset, carrying out multi-stage optimization learning, changing the feature state through iteration, and calculating the score of the candidate feature subset by using an unsupervised evaluation index to obtain an optimal feature subset; and performing clustering based on the optimal feature subset to obtain a clustering label, and performing a subsequent classification task based on the clustering label. The method aims at screening out a most representative low-dimensional feature subset from high-dimensional data, and the performance of an unsupervised learning task is improved while the data dimension is reduced.

The invention relates to a multi-frequency and total cross-section-based pulverized coal concentration measurement method, which comprises the following steps of: acquiring multi-frequency and total cross-section microwave attenuation data through a microwave sensor, and constructing a high-dimensional data set; performing global feature extraction on the high-dimensional data set by using a PCA algorithm to obtain a data set after dimension reduction; performing local optimal parameter combination screening on the data set after dimension reduction through an OMP algorithm; and constructing a pulverized coal concentration prediction model, inputting measurement data, and predicting the pulverized coal concentration in real time. The method has the beneficial effects that a multi-frequency and total cross-section measurement method is adopted, a true value is taken as a target, and an optimal subset is gradually selected in an iteration mode, so that a result approaches to an optimal solution, a large amount of invalid calculation is avoided, the calculation amount can be effectively reduced, and the risk of calculation bottleneck is avoided. Moreover, compared with the single-frequency linear measurement, the multi-frequency and total cross-section measurement uses more information in the physical space and the frequency domain.

The invention relates to the technical field of computer and big data management, and discloses a data interaction method of a handheld tablet personal computer, which comprises the following steps: receiving a big data query instruction, and obtaining a high-dimensional data set; a three-dimensional point cloud is generated through graph neural network dimension reduction embedding; constructing and rendering a dynamic interactive three-dimensional topological mapping model; responding to the multi-finger gesture operation to adjust the spatial layout and the clustering structure of the data entity in real time; and dynamically calculating and visually coding the association strength between the data. The system comprises a query receiving module, a data acquisition module, a dimension reduction processing module, a model construction module, a rendering module, a gesture recognition module, an association calculation module, a visual coding module and the like. According to the method, through three-dimensional space mapping and multi-finger gesture interaction, the information density and association visibility are remarkably improved, local detail expansion, spatial memory anchor points and multi-user cooperation are supported, and efficient and visual big data exploration is achieved.

The invention discloses a new energy power generation equipment adaptive control system based on multi-scale data fusion, and relates to the technical field of power generation control, and the system comprises the steps: obtaining a multi-source power data set corresponding to each composite fuel type; performing multi-scale feature convolution on the multi-source power data set, and outputting a multi-source high-dimensional data set; fusing the multi-source high-dimensional data set to generate a fused high-dimensional data set; receiving equipment fault information, and generating equipment control parameter items; performing feature dimension reduction on the fused high-dimensional data set, and outputting a fused low-dimensional data set; and training the equipment adaptive control model, and performing adaptive control on the new energy power generation equipment. The technical problems that in the prior art, new energy power generation equipment is poor in adaptability and unstable in power output are solved, and the technical effects that the adaptability of the equipment is remarkably improved, and unstable power output is effectively reduced are achieved.

The present invention relates to a method for predicting the number of entity clusters in knowledge services based on deep learning, belonging to the technical field of data mining applications and information processing. This method is to solve the technical problem of how to effectively predict the number of entity clusters under a large amount of literature and data in network knowledge service applications, and improve the accuracy and efficiency of information data mining and pushing. By combining an autoencoder with the Monte Carlo method, it can accurately and automatically estimate the number of clusters in a high-dimensional data set, effectively overcome the technical defect that the current number of clusters needs to be estimated manually, significantly improve the information recommendation ability of the system in aspects such as hot topic recommendation, search, and question answering, and recommend more relevant information to users. The present invention has broad application prospects in the fields of text mining and fine-grained knowledge services, etc.

The present invention discloses a method for reducing the dimension of streaming high-dimensional data for real-time monitoring of network public opinion, comprising the steps of obtaining an initial streaming high-dimensional data set for real-time monitoring of network public opinion; using the obtained data set to obtain an initial embedding function through a parameterized nonlinear dimensionality reduction method; creating a new data embedding process, an embedding function updating process, and an embedding updating process; setting a new data generation source, and updating the created new data embedding process, embedding function updating process, and embedding updating process through the generated new data; using the updated result to replace the obtained initial data set and the initial embedding function; repeating the above steps until no new data is generated, thereby completing the dimensionality reduction processing of streaming high-dimensional data for real-time monitoring of network public opinion; the method of the present invention can achieve higher embedding quality; at the same time, it proposes to constrain the updating of the embedding function to improve the temporal stability of the embedding result; the accuracy of the method of the present invention is improved and can meet data processing requirements.

The invention discloses an industrial large space temperature and humidity uniformity optimization method based on a POD-mGPR proxy model, and belongs to the field of industrial environment intelligent regulation and control, the method fuses computational fluid mechanics (CFD), intrinsic orthogonal decomposition (POD) and multivariate Gaussian process regression (mGPR), a high-dimensional data set is generated through CFD simulation, after a main mode is extracted through POD dimensionality reduction, the mGPR is used for constructing the proxy model, and the main mode is extracted through the mGPR; rapid and accurate prediction of the whole-space temperature and humidity field is realized; defining a non-uniformity coefficient, and establishing quantitative mapping of a control variable and a uniformity improvement effect; and finally, screening key variables through Sobol global sensitivity analysis, and solving an optimal control parameter combination in combination with a genetic algorithm. According to the method, the problems of insufficient perception, lack of quantitative feedback in control and low high-dimensional optimization efficiency of a traditional method are solved, the transformation from local experience control to global quantitative control is realized, and the temperature and humidity uniformity and the energy utilization efficiency are improved.

A kind of double-agent model optimization method of wind snow-removal robot pipeline structure parameter, specific steps are as follows: (1) according to the initial design of wind snow-removal robot pipeline, determine the design variable and value range of pipeline structure parameter;(2) establish high-dimensional data set;(3) establish low-dimensional data set;(4) select a proxy model to obtain high-dimensional proxy model A and low-dimensional proxy model B;(5) generate active learning candidate sample pool;(6) calculate the under-learning degree score of each candidate learning point;(7) establish active learning point set;(8) carry out CFD simulation to obtain updated data set, continue to train model A, when the prediction accuracy of model A, the prediction accuracy of model B and the inconsistency degree of model A model B prediction result all reach specified threshold, stop active learning.The present application provides an efficient, reliable and engineering applicable method for the optimization of complex pipeline parameters with multiple structural variables and multiple outlet flow responses.

The application relates to the technical field of text dimension reduction, and discloses a local analysis text data dimension reduction method based on complete supervision, wherein a high-bit data set is segmented through a K-neighbor rule to form a plurality of mutual overlapping local parts, geometric information of different category data in a local neighborhood is explored, a complete supervision local target function is constructed through a complete supervision local analysis algorithm, a local arrangement matrix is solved, same data points in a local part are as close as possible, different categories are as separated as possible in the dimension reduction process, the local neighbor geometric structure can be effectively learned, the system classification capability is improved, global projection matrixes are obtained by comprehensively considering geometric arrangement information of each local part and using an eigenvalue solving mode, and thus low-dimensional data after dimension reduction of the high-dimensional data set can be solved, so that the problem of external samples can be effectively solved.

The application provides an information entropy calculation method and device for wafer processing machine stability evaluation. The method comprises the following steps: obtaining full high-dimensional data set collected by FDC system, each data point in the full high-dimensional data set corresponding to a set of process parameter values, and having multiple dimension characteristic information; performing normalization preprocessing on the full high-dimensional data set to eliminate the dimensional difference between parameters; traversing the full high-dimensional data set, taking each data point as a target data point, calculating the distance between the target data point and the remaining data points, and selecting the K nearest data points to form a neighbor set of the target data point; and calculating the information entropy value of the full high-dimensional data set based on the neighbor set of all target data points. The application uses information entropy as a measurement index, converts the complex high-dimensional parameter running state into a single comparable numerical value, and can intuitively and quantitatively represent the wafer processing machine running stability.

The invention provides an information entropy calculation method and equipment for stability evaluation of a wafer processing machine, and the method comprises the steps: obtaining a full-amount high-dimensional data set collected by an FDC system, and enabling each data point in the full-amount high-dimensional data set to correspond to a group of technological parameter values, and to have feature information of multiple dimensions; performing normalization preprocessing on the full-amount high-dimensional data set to eliminate dimensional differences among parameters; traversing the full-amount high-dimensional data set, taking each data point as a target data point, calculating the distance between the target data point and other data points, and selecting K nearest data points to form a neighbor set of the target data points; and calculating an information entropy value of the full-amount high-dimensional data set based on the neighbor set of all the target data points. According to the method, the information entropy is used as a measurement index, the complex high-dimensional parameter operation state is converted into a single comparable numerical value, and the operation stability of the wafer processing machine can be visually and quantitatively represented.