76 results about "Outlier removal" patented technology

The invention provides a virtual DPU power plant simulation fault restoration method and system based on digital twinning, and relates to the technical field of digital twinning, and the method comprises the steps: carrying out the preprocessing of collected DPU power plant operation data, including noise reduction, time sequence alignment and abnormal point elimination, carrying out the data quality evaluation, training a fault feature mapping model based on the processed data, and carrying out the fault restoration of the DPU power plant. The model is used for recognizing abnormal clusters in real time, a fault evolution path is searched and determined in combination with a conditional random field and a Monte Carlo tree, an optimal path is determined through a particle filtering algorithm, fault root causes are determined in combination with causal analysis, spectral clustering and a Bayesian network, and a fault diagnosis report is generated.

The invention discloses a marine environment short-term prediction method and system based on a conditional adversarial network, and particularly relates to the field of marine environment short-term prediction, and the method comprises the steps: S1, obtaining multi-dimensional original data covering a marine physical environment and a remote sensing field; s2, performing space-time registration and abnormal value elimination on the multi-dimensional original data, and filling missing values by adopting a moving average method to obtain standardized data; s3, fusing the standardized data into coupling features reflecting the ocean dynamic process, and constructing spatial-temporal feature codes; s4, constructing a conditional adversarial network model based on the spatio-temporal feature codes, driving the conditional adversarial network model to learn a real data distribution rule through a loss function, generating prediction features meeting spatio-temporal conditions, and completing training; according to the method, the three-dimensional spatial-temporal feature tensor reflecting the ocean dynamic process is obtained through multi-source data fusion and coupling feature engineering, and the characterization capability of data and the description precision of a physical mechanism are improved.

The invention discloses a method for predicting shield tunneling parameters under a composite stratum, which comprises the following steps of: collecting and sorting shield tunneling historical data under the composite stratum, including geological parameters and construction parameters, and performing abnormal value elimination, missing value filling and data standardization; the geological parameters and the geometric parameters are combined according to the tunneling ring number, and a quantitative geological matrix of each ring is generated; the geometric parameters represent the position of the soil layer relative to the tunnel excavation face by adopting 1-5 codes, and input dimension unification is ensured through zero filling alignment; a CNN-Attention-SVR model is constructed, and the CNN-Attention-SVR model is trained; new geological data are input into the trained model, tunneling parameters are predicted, and construction adjustment is guided; reliable technical support and decision basis are provided for tunnel construction under the composite stratum condition, and therefore the overall safety and economical efficiency of shield construction are improved.

The invention discloses a spectral gas concentration inversion method based on a CNN-Transform fusion model, and belongs to the technical field of trace gas detection and spectral signal processing. The method comprises the following steps: acquiring second harmonic spectrum data of target gas through a TDLAS (tunable diode laser absorption spectroscopy) system; carrying out abnormal value elimination and smooth filtering processing on the acquired signal, and constructing a standard input vector; a depth model fusing a one-dimensional convolutional neural network (CNN) and a Transform architecture is designed, the CNN is used for extracting local spectrogram features, and the Transform is used for modeling a global dependency relationship; and inputting the training set and the verification set into the model for training optimization, and finally realizing high-precision inversion of the sample gas concentration of the test set. Experimental results show that the method is superior to an existing model in the aspects of fitting precision, robustness and error control, and has good physical consistency and engineering popularization value.

The invention relates to the technical field of iron and steel smelting, in particular to a continuous casting quality data processing method based on multi-modal cognitive reasoning, which comprises the following steps of: synchronously acquiring multi-source sensor data, performing time alignment, format standardization, noise filtering and normalization processing, extracting image / thermal image spatial features through CNN (Convolutional Neural Network), and calculating the continuous casting quality data according to the image / thermal image spatial features. The LSTM is combined with frequency domain analysis to extract vibration / sound time sequence features, cross-modal alignment fusion is realized through feature vectorization, quality prediction, anomaly detection and real-time deployment are realized based on a deep neural network and an attention mechanism by fusing process knowledge, and then slab defect causes are accurately identified by integrating a multi-modal reasoning result. Through time synchronization, format unification, noise filtering and data filling, the problem of heterogeneity of multi-source sensor data is solved, and time dislocation and noise interference are eliminated; abnormal data are cleaned by an outlier removal technology, the reliability of input data and the model robustness are improved, and efficient cooperative processing of multi-modal data is ensured.

An outlier removal method for removing an outlier included in training data that has data of an explanatory variable and an objective variable used for machine learning. The method includes calculating prediction errors by repeating, a predetermined number of times, division of the training data into teaching data and test data, creation of a regression model representing a correlation between the explanatory variable and the objective variable using the teaching data, and calculation of a prediction error using the test data on the created regression model, calculating a distribution by extracting, for each data included in the training data, prediction errors when using that data as the test data from prediction errors obtained in the calculating prediction errors, and obtaining, for each data included in the training data, an index value characterizing a distribution of the extracted prediction errors, determining an outlier by determining whether each data is an outlier based on the index value of each data obtained in the calculating a distribution, and removing an outlier by removing data determined to be an outlier in the determining an outlier.

The invention provides a wire distance calculation method fusing regular component extraction and clustering growth, and belongs to the technical field of power equipment monitoring, and the method comprises the steps: obtaining point cloud data of a transformer substation, carrying out the noise and outlier removal and ground point cloud elimination processing, and obtaining the residual point cloud data containing power transformation facilities and various wires; a deep learning model is utilized to extract components with regular structures from the residual point cloud data, and various wires and power transformation facilities connected with the wires are preliminarily labeled and identified; then classifying various types of wires, cutting out a point cloud area where the power transformation facility is located, and further refining and extracting a power transformation facility point cloud connected with the wires by applying a clustering growth algorithm; and finally, calculating the distance between the wires, the distance between the wires and the ground and the distance between the wires and the power transformation facility, and generating an analysis report for operation, maintenance and monitoring of a power transformation transmission line. According to the method, the problems of high difficulty and incapability of accurately extracting the lead and the power transformation facility in the existing method are solved.

Aspects of the invention include training an optimal interpretable decision tree for regression using mixed-integer linear programming techniques. A non-limiting example computer-implemented method includes receiving, using a processor, input data that includes time-series data. The method further includes training, using a binary mixed-integer linear program of the processor, an ODT for regression based on the input data. During the training process one or more outliers are filtered out by a linear loss model that minimizes training loss and outlier loss.

The invention discloses a multi-vehicle-type trajectory collaborative prediction method based on Rhodeus deus optimized hierarchical game, and the method comprises the steps: S1, constructing a multi-vehicle-type initial three-dimensional data matrix, carrying out the interpolation compensation and outlier elimination, and carrying out the unified coding and indexing; differential normalization processing is carried out; s2, constructing a layered skeleton network comprising a basic dynamics layer and a disturbance behavior layer, and obtaining preliminary prediction; s3, establishing a multi-vehicle-type coupling objective function, scene penalty and a dynamic radius mechanism, and outputting a global solution by adopting a rhodeus deus deus algorithm; s4, performing hierarchical game residual error correction on the global optimal trajectory to obtain a hierarchical game residual error; s5, coupling the residual error with a rhodeus deus algorithm, synchronously and finely adjusting the trajectory, and outputting an optimal multi-vehicle type prediction trajectory; and S6, applying the optimal multi-vehicle-type prediction trajectory to short-term trajectory prediction deployment of the autonomous vehicle, completing comprehensive prediction and scheduling of different vehicle types, and realizing high-precision trajectory prediction in a multi-vehicle-type mixed driving scene.

The invention provides a three-dimensional reconstruction method and system, and the method comprises the steps: obtaining a distortion coefficient and camera internal parameters through employing a first picture set of a to-be-reconstructed object collected by a camera; performing distortion correction on the first picture contained in the first picture set through the distortion coefficient to obtain a second picture set; calculating a sparse point cloud, and calculating a camera extrinsic parameter and a first depth map of a second picture included in the second picture set; outlier removal is carried out on the first depth map; inputting the outlier-removed first depth map into a depth completion network model for depth completion to obtain an optimized second depth map; and constructing a final three-dimensional model of the to-be-reconstructed object according to the depth information of the second depth map, the camera internal reference, the camera external reference and the sparse point cloud. According to the scheme, the more complete second depth map is obtained through the outlier removal operation and the depth completion network model, and then the final three-dimensional model is constructed by using the second depth map, so that the precision of dense reconstruction and the effect of three-dimensional reconstruction are improved.

The invention provides a refrigerator fault prediction method based on a gradient boosting tree. The method comprises the steps of data acquisition, missing value processing and abnormal value removal preprocessing. Carrying out feature coding, temperature difference feature calculation and high-dimensional transformation feature extraction to obtain data features; performing time window flattening operation on the data features, performing resistance feature selection and recursive feature elimination, and screening out an optimal feature subset; training an xgboost gradient boosting tree model by using the optimal feature subset, and outputting a fault prediction model when the accuracy rate accords with a preset value; and finally, whether the refrigerator breaks down or not is judged through the fault prediction model. According to the application, comprehensive monitoring and fault prediction of the running state of the refrigerator are realized. The prediction accuracy and generalization ability of the model are improved by improving the data quality and enhancing the model performance; real-time prediction is achieved, and potential faults of the refrigerator can be found in time; and the accuracy and adaptability of prediction are improved by dynamically adjusting the fault threshold.

The invention provides a method for predicting a memory effect of a natural gas hydrate, belongs to the technical field of intelligent modeling and prediction of the natural gas hydrate, and aims to realize accurate modeling and intelligent identification of a memory effect behavior in a hydrate formation and dissociation process. The method comprises the following steps: S1, collecting experimental data, including a plurality of variable characteristics influencing the memory effect, such as synthesis temperature, dissociation pressure, synthesis pressure, synthesis-decomposition cycle index, decomposition temperature and the like, and constructing a training data set by taking nucleation time as a dependent variable; s2, preprocessing the experimental data, completing missing value filling, abnormal value elimination and standardization processing, and obtaining a standardized sample; s3, constructing a prediction model based on support vector regression, random forest, XGBoost, polynomial regression and other algorithms, and improving prediction performance by adjusting model hyper-parameters; and S4, a graphic visual interface is constructed based on Python and PyQt5, a user can input experimental conditions and select a model, and a system automatically predicts a memory effect and judges whether a nucleation promoting behavior exists according to the memory effect. The method integrates key modules such as feature engineering, active learning, hyper-parameter optimization and visual interaction, has high precision, high adaptability and good expansibility, and is suitable for various thermodynamic application scenes such as natural gas hydrate phase change behavior prediction, energy storage and transportation optimization, carbon sequestration process regulation and control and the like.

A method for performing core-to-log depth matching includes receiving input data. The input data includes core data and well log data. The method also includes performing an autonomous data preprocessing procedure to standardize the core data and the well log data to determine correlations between the core data and the well log data. The method also includes performing an autonomous outlier removal procedure to address differences in acquisition methods and measurement principles of the core data and the well log data. The method also includes automatically determining normalized cross-correlations between measurements derived from the core data and measurements derived from the well log data. The method also includes automatically shifting the measurements derived from the core data to a new depth position based upon a maximum of the normalized cross-correlations.

The invention relates to an electricity consumption information data cleaning and filtering method and system, and the method comprises the steps: dividing collected electricity consumption data into an end point data set, a middle data set and a tail end data set according to properties, and carrying out the basic abnormal value removal and missing data filling; converting the time domain data of each data set into a frequency domain pattern capable of representing the fluctuation mode of the time domain data through discrete Fourier transform; performing correlation analysis from top to bottom based on a graph, screening an intermediate data set by utilizing the graph of the endpoint data set, and classifying and screening an end data set by utilizing the screened intermediate data set graph; performing block processing on the screened end data set, extracting core features of the end data set by separating an input effect, a load effect and associated components, and dividing the core features into weak, medium and strong categories according to the fluctuation intensity of the features; carrying out inverse transformation on each processed graphic expression, and outputting cleaned and filtered high-quality time domain power utilization data; the accuracy of data cleaning can be improved, and the data value can be deeply mined.

This invention proposes an efficient system and method for processing objective data from helicopter flight simulators. The system includes a data preprocessing module for preprocessing acquired complete flight data, including data fusion and time-series correction; a data segmentation module for segmenting the raw simulator objective data based on helicopter configuration, control variable changes, and flight status to obtain segmented data; a batch processing module for providing a batch processing entry point for multiple sets of segmented data, simultaneously importing multiple sets of segmented data and multiple data parameters into one or more other processing modules for processing; an outlier removal module for identifying and removing outliers from the batch-processed segmented data; a curve fitting module for correcting the data after outlier removal; and a filtering module for recovering the true information as much as possible from the segmented data containing interference.

The present invention relates to an obstacle detection method based on lidar point cloud clustering. First, point cloud preprocessing is carried out, mainly filtering the point cloud through outlier removal and downsampling. Then, a linear fitting algorithm is used to separate the ground point cloud and the obstacle point cloud. After that, Euclidean clustering is adopted to achieve the separation of obstacles, improving the accuracy of obstacle detection. Finally, a 3D bounding box fitting is performed on the clustered point cloud, facilitating the further driving control of the unmanned vehicle after obstacle perception, for subsequent work such as obstacle avoidance and tracking.

This invention discloses an industrial carbon emission detection and prediction method and system. It collects multi-source heterogeneous data from industrial production processes, and sequentially performs standardization, noise removal, outlier removal, and data fusion on the multi-source heterogeneous data to obtain initial carbon emission data. A hybrid detection and prediction model is constructed, employing an encoder to extract multi-factor correlation features from the multi-source data and a long short-term memory network to extract temporal fluctuation features of the carbon emission data. An attention mechanism is introduced to weight the correlation features and temporal fluctuation features. An improved IWOA whale optimization algorithm is used to optimize the hyperparameters of the hybrid detection and prediction model to obtain a target hybrid detection and prediction model. The initial carbon emission data is input into the target hybrid detection and prediction model, and the prediction results are output, including the current carbon emission detection value and the carbon emission prediction sequence. This improves the accuracy of carbon emission prediction results and the efficiency of carbon emission detection.

The present invention belongs to the technical field of oil exploration information processing, and specifically discloses a method, system, device and medium for fusing near-surface and mid-deep Q fields based on mask bodies. The method includes: generating a near-surface Q field F1 and a mid-deep Q field F2, subtracting the two Q fields after weighting to form a data volume F3, and obtaining the data volume F3 after processing to remove outliers from the data volume F3 ” , for the data volume F3 ” perform mask transformation to respectively obtain mask bodies F 1 / 0 , F 0 / 1 , and the data boundaries where the 0 and 1 values in the mask bodies F 1 / 0 , F 0 / 1 are located are ideal fusion surfaces; multiply the points corresponding to the near-surface Q field F1 and the mid-deep Q field F2 by the mask bodies F 1 / 0 , F 0 / 1 respectively to obtain complementary data volumes F 上 and data volume F 下 , and finally add the data volume F 上 and the data volume F 下 . The present invention can realize intelligent and efficient fusion of near-surface and mid-deep Q fields, with high fusion accuracy and high efficiency. The present invention is applicable to the fusion of near-surface and mid-deep Q fields and the establishment of a full-layer Q field.

This invention discloses a method and system for detecting internal thread parameters based on laser point cloud analysis. Addressing the shortcomings of incomplete results and slow speed of traditional internal thread measurement methods, this device can quickly acquire point cloud information of the internal thread to be measured. Within the system, outlier removal, cylindrical fitting, and sawtooth wave fitting operations are performed on this point cloud information to determine the major diameter, minor diameter, pitch diameter, pitch, and thread angle parameters of the internal thread.

The invention provides a coal mine extremely shallow goaf geophysical prospecting data analysis method based on a transient electromagnetic method, which is characterized by comprising the following steps of: optimizing data acquisition parameters and correcting terrain; data preprocessing and noise suppression are carried out, multi-stage preprocessing is carried out on the original data, and preprocessing comprises outlier elimination, filtering noise reduction, null drift correction and background field deduction; time-depth conversion and resistivity inversion are carried out; quantitative analysis of goaf response characteristics; carrying out three-dimensional data fusion and visual modeling, integrating multi-measuring-line data and geological information, and constructing a goaf space distribution model; and the conclusion reliability is improved through field drilling verification and error source analysis. Aiming at the detection difficulty of an extremely shallow goaf, electromagnetic response details of a shallow medium are captured through small coil high-frequency excitation and microsecond-level signal acquisition. The terrain laser correction and the whole-period apparent resistivity inversion are combined, so that the tiny holes are accurately identified; and through drilling entity verification and error simulation, the misjudgment rate is remarkably reduced.

The present invention relates to the field of data processing technology, and discloses a production feature classification method for rubber extrusion molding equipment based on a CART decision tree, comprising the following steps: S1, data processing: performing data cleaning, outlier removal, data standardization, and normalization on the production line equipment data in an industrial scenario, and performing data analysis based on business knowledge; S2, key time series data labeling: labeling key time periods based on actual production and business knowledge, and associating data with business knowledge. This production feature classification method for rubber extrusion molding equipment based on a CART decision tree, combined with business knowledge and advanced data analysis and machine learning techniques, intelligently extracts key features. Compared with traditional methods, we can more comprehensively capture subtle changes in production patterns; the model can be dynamically adjusted based on the latest production data to ensure classification accuracy and meet the real-time requirements of industrial production.

A machine learning-based synthesis runtime prediction method includes collecting initial training data from a database, selecting featured training data from the initial training data useful for predicting runtimes, building a machine learning model for predicting the runtimes based on the featured training data, measuring a loss and an accuracy of the machine learning model, performing standardization and / or normalization on the featured training data of the training data to generate updated training data if the loss and / or the accuracy fails to meet predefined criteria, performing clustering at least once on the updated training data to generate clustered training data, identifying at least one outlier from the clustered training data, removing the at least one outlier to generate filtered training data, and preprocessing, training and testing the machine learning model based on the filtered training data until the loss and the accuracy meet the predefined criteria.

The invention provides an outlier removal method and an outlier removal device, which can properly remove outliers. The method comprises: an evaluation index calculation step for repeating a predetermined number of times a model creation step for dividing learning data into training data and test data and creating a regression model using the training data, a first evaluation index calculation step for calculating a first evaluation index, and a second evaluation index calculation step for calculating a second evaluation index; the first evaluation index operation process uses test data to operate an evaluation index on the regression model and set the evaluation index as a first evaluation index, and the second evaluation index operation process uses training data to operate an evaluation index on the regression model and set the evaluation index as a second evaluation index; an outlier determination step for determining, on the basis of the calculation result of the evaluation index calculation step, whether or not the data included in the learning data is an outlier, using both the first evaluation index and the second evaluation index when the data is used as test data; and an outlier removal step for removing the data determined as an outlier in the outlier determination step.

This invention relates to a method for generating visualized early warning information based on sensor data. The method includes: collecting multi-source sensor data from various UAVs, and performing outlier removal, missing value repair, and standardization to obtain preprocessed data; inputting the preprocessed data into a long short-term memory network to obtain risk values, predicted local state values, and early warning levels; generating sparse state features based on risk values, spatial proximity, and task area overlap, and uploading them to a collaborative decision-making center; updating the collaborative graph by the collaborative decision-making center to generate enhanced state representations and global states, further obtaining joint action values; predicting trajectories, calculating collision probabilities, generating visualized early warning information based on enhanced state representations and predicted local state values, and determining and issuing joint actions to relevant UAVs when action triggering conditions are met.

The present invention relates to a time series cross-calibration method and system for a synthetic aperture radar. The method comprises: screening a target area for time series cross-calibration; generating SAR image data pairs at each moment within a time range; extracting a median matrix of a data slice at a single moment using a window median method, obtaining a median matrix array, performing robust regression fitting with outlier removal, and generating a radiation calibration coefficient array; processing the radiation calibration coefficient array according to a final reference value of the radiation calibration coefficient, and obtaining a final radiation calibration coefficient. By performing robust regression fitting with outlier removal on the median matrix array, interference from outliers is avoided; calculating the radiation calibration coefficient corresponding to each single moment, using the correlation of the radiation calibration coefficients between different moments as a reference, combining the radiation calibration coefficient value measured at the moment to be calibrated and the radiation calibration coefficient value at the previous moment, and determining the final reference value of the radiation calibration coefficient, thereby improving calibration accuracy.

The application discloses a point cloud outlier removal method, a point cloud processing method, a device and related equipment. The method comprises the following steps: dividing a point cloud to be processed into multiple layers along a reference coordinate axis; determining whether to remove each point in each layer of the point cloud based on the bounding box of the point, and obtaining a second point cloud; obtaining the average nearest neighbor Euclidean distance of the second point cloud, that is, the nearest neighbor Euclidean distance of each point; and determining whether to remove each point in the second point cloud based on the difference between the nearest neighbor Euclidean distance of the point and the average nearest neighbor Euclidean distance, and the standard deviation of the nearest neighbor Euclidean distance of each point in the second point cloud and the average nearest neighbor Euclidean distance. Through the above steps, the application can remove the outliers in the point cloud more intuitively and conveniently while retaining the detailed features.

Systems and methods for iterative removal of outlier data from spectrometry data to determine one or more of a particle baseline and a detection threshold for nanoparticles are described. Ion signal intensity values that exceed an outlier threshold value associated with a sum of a first multiple of an average of the count distribution of ion signal intensity and a first multiple of a standard deviation of the count distribution of ion signal intensity are iteratively removed from the raw data set until no outliers remain, providing a background data set. A nanoparticle baseline intensity value is set as a sum of a second multiple of an average of the background data set and a second multiple of a standard deviation of the background data set to differentiate between signal intensity values that are associated with background interference and that are associated with the presence of nanoparticles in the sample.

The present invention discloses a method for removing outliers and generating reflectivity and normals based on multi-angle illumination images, which belongs to the fields of optics, digital image processing, self-supervised deep learning, etc. First, images under illumination from different directions are obtained; the images are read and normalized to obtain a multi-channel image array; a spatial coordinate system is established according to the position of the light source and the camera, and multiple light sources form a light source direction matrix; a neural network model is built, and a loss function is selected according to the characteristics of the outliers; each channel of the image array and the light source direction matrix are respectively input into the neural network model in pairs, and a back-propagation algorithm is used for iterative optimization to obtain a normal matrix with reflectivity scaling; the normal with reflectivity scaling is obtained by obtaining the vector modulus and normalizing it to obtain the reflectivity and normal of each channel, and the average value of the normal of each channel is the final normal. The present invention can be applied to solving the normal of an object, and can effectively reduce errors such as those caused by highlights, shadows, noise, etc.

This invention provides a zero-sample online fault diagnosis method based on a multi-resolution diffusion model. First, a source domain dataset U is constructed from data collected by a traction drive control simulation platform. S The target domain dataset U is constructed from data collected during actual train operation. T The fault category sets of the source and target domains do not overlap. A multi-resolution diffusion model is used to process the data, achieving outlier removal, missing data imputation, and standardization. A category-attribute description matrix A' is constructed, and fault features f from the source domain data are extracted using deep incremental supervised principal component analysis. i s Feature B s With attribute description a i The mapping relationship between them. Online acquisition of target domain data D. t Then, update the attribute descriptions of the fault categories and construct a new sparse matrix A. * The model parameters are then fine-tuned. Finally, Bayesian inference is used to perform real-time fault diagnosis on zero-sample data in the target domain, and the optimal fault category is inferred based on the maximum posterior probability, thus achieving accurate diagnosis and prediction of the target domain data.

The invention provides a data sampling and abnormal point removing method and system based on reinforcement learning. The method comprises the following steps: extracting multi-modal features, constructing corresponding relation features, extracting key points and descriptors thereof from two input images, generating an initial corresponding relation set, and performing coordinate normalization processing; constructing a reinforcement learning agent, and training the agent to enable the agent to learn an optimal strategy of data sampling and outlier removal; constructing a feature point relation graph, and dynamically segmenting normal matching points and outliers based on a clustering result represented by nodes and a confidence threshold; and carrying out multi-scale geometric verification on the screened high-confidence matching points, eliminating local inconsistent points, estimating an affine transformation matrix, and introducing illumination invariance constraint to optimize the image alignment precision. According to the technical scheme, through interaction with the environment, the reinforcement learning model can learn that the most representative strategy is selected under specific conditions, and the utilization efficiency of data and the performance of image matching are improved.