26 results about "Outlier removal" patented technology

The invention relates to an electricity consumption information data cleaning and filtering method and system, and the method comprises the steps: dividing collected electricity consumption data into an end point data set, a middle data set and a tail end data set according to properties, and carrying out the basic abnormal value removal and missing data filling; converting the time domain data of each data set into a frequency domain pattern capable of representing the fluctuation mode of the time domain data through discrete Fourier transform; performing correlation analysis from top to bottom based on a graph, screening an intermediate data set by utilizing the graph of the endpoint data set, and classifying and screening an end data set by utilizing the screened intermediate data set graph; performing block processing on the screened end data set, extracting core features of the end data set by separating an input effect, a load effect and associated components, and dividing the core features into weak, medium and strong categories according to the fluctuation intensity of the features; carrying out inverse transformation on each processed graphic expression, and outputting cleaned and filtered high-quality time domain power utilization data; the accuracy of data cleaning can be improved, and the data value can be deeply mined.

This invention proposes an efficient system and method for processing objective data from helicopter flight simulators. The system includes a data preprocessing module for preprocessing acquired complete flight data, including data fusion and time-series correction; a data segmentation module for segmenting the raw simulator objective data based on helicopter configuration, control variable changes, and flight status to obtain segmented data; a batch processing module for providing a batch processing entry point for multiple sets of segmented data, simultaneously importing multiple sets of segmented data and multiple data parameters into one or more other processing modules for processing; an outlier removal module for identifying and removing outliers from the batch-processed segmented data; a curve fitting module for correcting the data after outlier removal; and a filtering module for recovering the true information as much as possible from the segmented data containing interference.

This invention discloses an industrial carbon emission detection and prediction method and system. It collects multi-source heterogeneous data from industrial production processes, and sequentially performs standardization, noise removal, outlier removal, and data fusion on the multi-source heterogeneous data to obtain initial carbon emission data. A hybrid detection and prediction model is constructed, employing an encoder to extract multi-factor correlation features from the multi-source data and a long short-term memory network to extract temporal fluctuation features of the carbon emission data. An attention mechanism is introduced to weight the correlation features and temporal fluctuation features. An improved IWOA whale optimization algorithm is used to optimize the hyperparameters of the hybrid detection and prediction model to obtain a target hybrid detection and prediction model. The initial carbon emission data is input into the target hybrid detection and prediction model, and the prediction results are output, including the current carbon emission detection value and the carbon emission prediction sequence. This improves the accuracy of carbon emission prediction results and the efficiency of carbon emission detection.

This invention discloses a method and system for detecting internal thread parameters based on laser point cloud analysis. Addressing the shortcomings of incomplete results and slow speed of traditional internal thread measurement methods, this device can quickly acquire point cloud information of the internal thread to be measured. Within the system, outlier removal, cylindrical fitting, and sawtooth wave fitting operations are performed on this point cloud information to determine the major diameter, minor diameter, pitch diameter, pitch, and thread angle parameters of the internal thread.

This invention relates to a method for generating visualized early warning information based on sensor data. The method includes: collecting multi-source sensor data from various UAVs, and performing outlier removal, missing value repair, and standardization to obtain preprocessed data; inputting the preprocessed data into a long short-term memory network to obtain risk values, predicted local state values, and early warning levels; generating sparse state features based on risk values, spatial proximity, and task area overlap, and uploading them to a collaborative decision-making center; updating the collaborative graph by the collaborative decision-making center to generate enhanced state representations and global states, further obtaining joint action values; predicting trajectories, calculating collision probabilities, generating visualized early warning information based on enhanced state representations and predicted local state values, and determining and issuing joint actions to relevant UAVs when action triggering conditions are met.

The application discloses a point cloud outlier removal method, a point cloud processing method, a device and related equipment. The method comprises the following steps: dividing a point cloud to be processed into multiple layers along a reference coordinate axis; determining whether to remove each point in each layer of the point cloud based on the bounding box of the point, and obtaining a second point cloud; obtaining the average nearest neighbor Euclidean distance of the second point cloud, that is, the nearest neighbor Euclidean distance of each point; and determining whether to remove each point in the second point cloud based on the difference between the nearest neighbor Euclidean distance of the point and the average nearest neighbor Euclidean distance, and the standard deviation of the nearest neighbor Euclidean distance of each point in the second point cloud and the average nearest neighbor Euclidean distance. Through the above steps, the application can remove the outliers in the point cloud more intuitively and conveniently while retaining the detailed features.

The invention belongs to the technical field of multi-channel spliced spectrum background deduction methods, and particularly relates to a laser plasma multi-channel spliced spectrum background deduction method, which comprises the following steps of: acquiring multi-channel original spectrum data, and performing preprocessing operations of wavelength resampling, noise filtering and abnormal point elimination on the original spectrum data to obtain a background deduction result; obtaining preprocessed spectral data; calculating multi-domain statistical characteristics based on the preprocessed spectral data, and generating a multi-domain auto-covariance structural element length field; the multi-domain statistical characteristics comprise the first-order difference, the second-order difference, the first-order difference variance, the second-order difference variance and the first-order and second-order difference covariance of the spectrum and the neighborhood of the spectrum; and performing multi-scale morphological opening operation on the preprocessed spectral data to obtain multi-scale upper and lower envelopes, determining a fusion weight based on a local amplitude unbalance degree, and fusing the upper and lower envelopes to obtain an initial morphological background envelope.

An example apparatus is to clip a value of an activation associated with a layer of a floating-point version of a machine learning model to determine a clipped value of the activation, the value of the activation based on calibration data applied to the floating-point version of the machine learning model. The example apparatus is also to determine, using the clipped value of the activation, a quantization factor to quantize activations associated with a corresponding layer of a fixed-point version of the machine learning model. The example apparatus is further to configure the fixed-point version of the machine learning model on a device using the quantization factor.

This invention provides a substation equipment installation guidance and control method and system based on semantic perception and EKF algorithm, belonging to the field of substation equipment installation guidance and control technology. To address the technical problems of current substation equipment installation guidance methods relying on manual experience, resulting in poor installation accuracy, dependence on stringent auxiliary markers, and inability to adapt to unstructured installation environments, this invention preprocesses the raw point cloud data of the acquired equipment, including voxel downsampling and statistical outlier removal, filtering out dust and noise from the construction site. The unordered point cloud is classified into base, equipment, ground, or background, and then subjected to perception processing and semantic segmentation. In the segmented point cloud data, the flange plane is extracted using a random sampling consistency algorithm, and the bolt hole center coordinates are identified using a clustering algorithm. Coarse and fine registration strategies are employed for virtual-real registration operations. This invention is applied to substation equipment installation guidance.

The application relates to a multi-modal environment perception and post-fusion method of a sweeping robot, and relates to mobile robot environment perception. Geometric processing is first performed on depth point cloud data: voxel down-sampling, ROI cropping and outlier removal are performed on the point cloud by a CUDA parallel preprocessing unit, a ground surface is segmented by using a RANSAC algorithm with strong constraint of a gravity direction normal vector, and obstacle clustering is completed in combination with an Euclidean clustering algorithm; in parallel, two-dimensional semantic features are extracted from RGB image data by using a deep learning target detection model; the geometric and semantic information is input into a multi-modal cascade fusion and classification module, cross-modal space mapping is established, visual semantics is injected into geometric point cloud, and garbage and obstacles are accurately distinguished by using a multi-constraint cascade reasoning strategy; AED time sequence tracking and output are executed, a correlation measurement based on an aggregated Euclidean distance is introduced, real-time perception output with low delay and high robustness is realized on an embedded platform, and the problems of low obstacle detection and garbage misjudgment are solved.

This invention provides a time series data cleaning method and apparatus. The method includes: acquiring initial sequence data of an initial dataset, the initial sequence data including trend sequence data, periodic sequence data, and residual sequence data; performing a periodic component rationality judgment based on the initial sequence data to obtain target residual sequence data; performing anomaly detection on the target residual sequence data to obtain an outlier index of the initial dataset and performing an outlier removal operation to obtain the target dataset; and performing missing value imputation on the target dataset to complete the initial dataset cleaning. This invention uses a periodic component rationality judgment to analyze whether the periodic components of the sequence data are accurate and optimizes the residual sequence data. This significantly reduces the complexity of time series data while improving the accuracy of outlier identification during data cleaning, thus improving the accuracy, efficiency, and reliability of data cleaning, resulting in cleaner, more complete, and usable data after cleaning.

The invention relates to outlier removal for transformer network quantization. An example device will clip (494) a value of an activation associated with a layer of a floating point version of a machine learning model (410) to determine a clipped value of the activation, the value of the activation based on calibration data (455) applied to the floating point version of the machine learning model (410). The example device will also determine (496) a quantization factor (460) using the clipped value of the activation to quantify an activation associated with a corresponding layer of a fixed point version of the machine learning model (415). The example apparatus will further configure (425) the fixed point version of the machine learning model (415) on a device (430) using the quantization factor (460).

This invention relates to the fields of precision instrument manufacturing and precision testing and measurement technology, specifically a microstructure parameter measurement method based on a UNet network. First, addressing the presence of numerous prominent textures and scratches in the background region of the image of the test part, a region segmentation method based on a UNet network is employed to improve segmentation accuracy and overcome the influence of noise. Then, a dataset is collected and image enhancement is performed to improve the model's generalization ability. Horizontal and vertical flipping enriches the pose of the microstructure, and random variations in the HSV color gamut enhance the model's robustness to different lighting environments. After segmenting the target region, edge points are extracted and then fitted. Due to the presence of processing defects, outliers need to be removed before fitting. For micropore structures, outlier removal involves connecting edge sequence points to remove protrusions, and finally selecting the region with the largest connected component as the repaired feature region. For microgroove structures, outlier removal involves using the RANSAC algorithm for multiple iterations to filter out all interior points satisfying the "correct solution," and then using the data that best fits as input for least squares fitting, which can better estimate model parameters. This invention can improve the visual measurement accuracy of micro-holes and micro-grooves at the micrometer to millimeter level in workpieces.

This invention relates to the field of power load forecasting technology, specifically to a machine learning-based accurate power load forecasting system, comprising: a data processing module, which receives power load-related data through a multi-source data interface and performs data normalization, outlier removal, and missing value completion operations. This invention combines outlier removal and missing value completion to improve the quality of basic data and provide reliable data support for subsequent forecasts. Through multi-dimensional feature extraction and scientific screening, it comprehensively captures the temporal characteristics, correlation impact characteristics, and abrupt change characteristics of the load, strengthening the correlation between features and load and improving the model's generalization ability. By constructing a basic model library covering the entire time scale, it adaptively selects model combinations and dynamically allocates weights based on load data characteristics, improving the model's adaptability to different load fluctuation scenarios and enhancing the predictive specificity.

The invention discloses a drill rod thread area defect identification method and system based on three-dimensional point cloud deep learning. The method comprises the following steps: acquiring three-dimensional point cloud data of a drill rod thread area; performing statistical outlier removal, voxel downsampling, random resampling to preset fixed points, zero centralization and unit scale normalization processing on the point clouds in sequence to obtain standard point clouds; and inputting the standard point cloud into a pre-trained point cloud deep learning model for forward reasoning, and outputting a defect category and confidence. The point cloud deep learning model adopts a Point Net architecture and comprises an input level T-Net, and the T-Net is used for learning a spatial rigid transformation matrix and aligning input point clouds to reduce the influence of attitude difference on an identification result. And model training can be carried out based on the marked point cloud data set, and the engineering availability is improved in combination with visual rechecking.

We provide the system. [Solution] Means for collecting regional information data, wind power information data, and ecosystem information data, Methods for preprocessing collected data, such as imputing missing values, removing outliers, and standardizing data shape, A means for applying a generative AI model to evaluate candidate locations using preprocessed data and calculating a score for each candidate location, A means for generating a list of construction candidate sites sorted in order of priority based on the calculated score, and transmitting the list to the user's terminal, A system that includes this.

The invention discloses a sea-gas interface momentum flux calculation method coupled with boundary layer meteorological conditions, and relates to the technical field of marine atmospheric boundary layer flux observation, comprising: performing threshold screening and outlier removal processing on turbulence flux elements required by vortex correlation method calculation, and performing coordinate rotation on three-dimensional wind speed; processing the obtained sea surface fluctuation data to obtain a sea wave spectrum and wave parameters; on the basis of the Monin-Obhoff similarity theory, the Monin-Obhoff length, the atmospheric stability and the wind speed U10 of 10 meters on the sea are calculated; based on the data, splitting the friction speed into a wind speed item and a wave mixing item for fitting, and finally obtaining momentum flux; on the basis of guaranteeing the flux calculation precision, the requirement for an observation instrument is lowered, accurate calculation of the momentum flux of the air-sea boundary layer is achieved, and the requirements for momentum flux data in the fields of marine meteorological research, weather forecast, marine resource development and the like are met.

This invention proposes a frame synchronization and light intensity sampling outlier removal method based on a DLP projection system. The method includes: acquiring the original image to be projected and extracting a structural feature vector; calculating the structural predicted light intensity of the original image based on the structural feature vector; acquiring at least one light intensity sample value within a fixed time window after the original image is projected to form a light intensity sequence; calculating the average light intensity of the light intensity sequence, and combining the structural predicted light intensity and the structural feature vector, introducing a structural response correction term and a structural adaptability tolerance term to determine whether the sampling of the current frame is successfully synchronized with the pattern projection; if the sampling of the current frame is successfully synchronized, constructing an outlier removal threshold, and identifying and removing outliers from the light intensity sequence; calculating the variance of the remaining sampling sequence after outlier removal; and weighted fusing the arithmetic mean with the median of the remaining sequence to finally generate a light intensity output value and submit it to the image reconstruction module.

This invention discloses an outlier removal method for solder plate point cloud processing. First, an image of the solder plate point cloud to be processed is acquired. The solder plate plane is segmented, and equations for each plane are fitted. Then, the positions of the theoretical intersection lines of the solder plates are determined based on the equations. Based on the positions of the theoretical intersection lines and the coordinates of each point in the point cloud image, the solder plate plane to which each point belongs is determined. Finally, the distance from each point to its corresponding solder plate plane is determined, and a distance threshold and processing range are set. If the distance is less than the threshold, the point is retained; otherwise, it is identified as an outlier and deleted. This invention incorporates the shape information of the solder plate plane during processing, making the denoising process simpler and faster. It can remove not only generalized outliers but also has good removal effects on other types of outlier noise points.

The application provides a COD soft measurement method for water quality purification plants, and relates to the field of environmental monitoring. The method comprises the following steps: S1: data acquisition and dynamic heterogeneous graph preprocessing: collecting historical easily measured water quality parameter data and corresponding COD measured values, and performing preprocessing; the preprocessing comprises data enhancement, missing value filling, outlier removal and data normalization; S2: two-stage feature optimization: performing feature construction and screening on the preprocessed data to generate a key feature set; S3: integrated prediction model construction and parameter optimization: based on the key feature set and the corresponding COD measured values, an integrated learning model is constructed and trained as a COD soft measurement model; S4: scene prediction application: inputting the real-time collected influent easily measured water quality parameters into the trained COD soft measurement model to output the COD prediction value of the influent; a high-performance and high-robustness integrated prediction model is established, and the real-time prediction of the influent COD at a second level and high precision is realized.

The invention provides a fault processing method, device and equipment for a dry type slag remover of a thermal power plant and a medium. The method comprises the steps that multiple types of sensors are deployed for key fault points of the dry type slag remover to collect time sequence data in real time; preprocessing the collected data, including abnormal value removal and standardization, and missing value filling; inputting the preprocessed data into a CNN-LSTM model for training to capture spatial association and time sequence trends among different sensors, obtaining a trained fault prediction model according to a preset convergence condition, and outputting a fault type, a confidence coefficient and an abnormal parameter; and according to the fault type and the abnormal parameter output by the model, determining a corresponding fault level and sending out corresponding alarm information according to a preset reference threshold value of each sensor. According to the method, the fault type of the dry type slag remover can be quickly and accurately diagnosed, effective treatment measures are taken, the operation reliability of the slag remover is improved, the maintenance cost is reduced, and stable operation of a thermal power plant is guaranteed.

The application provides a kind of array point cloud real-time outlier removal method and system based on sliding completion, it is related to point cloud outlier processing field.The initial input multiple frame array point cloud is preprocessed, incomplete point cloud is generated and stored in sliding window;Incomplete point cloud of each frame of subsequent input is obtained, the removed point is regarded as the feature point of current frame, and the union of point cloud in sliding window is regarded as the point cloud to be filled;A completion strategy based on array discrete probability model is designed to screen the feature points;The screened feature points are filled into the incomplete point cloud of current frame, which is the denoised point cloud of current frame, and the denoising of one frame of point cloud is completed;After the denoising of current frame point cloud is completed, the corresponding incomplete point cloud is added to the tail of sliding window, and the point cloud at the head of sliding window is removed, to realize the dynamic update of sliding window.The application overcomes the problem that the prior art has weak adaptability and low denoising precision when processing array point cloud outliers.

The invention provides a multi-time scale energy load prediction method for a large-scale public building. The method comprises the following steps: carrying out normalization and abnormal value elimination preprocessing on original power load data; clustering the preprocessed load data through multi-scale dynamic time warping, and dividing typical load modes; constructing a multi-dimensional time feature set for each typical load mode, evaluating feature importance based on a random forest replacement mechanism, and screening key features; and finally, for each load mode, constructing and training a hybrid prediction model fusing the time sequence convolutional network and the improved attention mechanism based on the key features, and realizing multi-step power load prediction. According to the method, the load modes are finely divided and key features are dynamically screened, so that the accuracy of load prediction and the model generalization ability in different operation scenes are effectively improved, and reliable technical support is provided for intelligent energy management of large-scale public buildings.

This invention discloses an automatic generation method for coastal salt marsh vegetation samples based on phenological characteristics and outlier removal. The method includes: acquiring a multi-temporal remote sensing image time-series dataset of the study area; calculating time-series curves of various vegetation indices after preprocessing; constructing phenological characteristic rules for the target salt marsh vegetation based on the time-series curves of the vegetation indices; performing pixel-by-pixel judgment on each pixel of the study area using the phenological characteristic rules to select pixels that conform to the phenological characteristics of various vegetation types, forming initial candidate sample areas for each type of vegetation; constraining the initial candidate sample areas using a priori spatial mask to obtain purified candidate sample areas; generating a pure automated sample set after traversing all candidate samples; and performing a hierarchical classification process based on the generated pure automated sample set. This invention can combine temporal remote sensing characteristics with a hierarchical classification strategy to achieve high-precision identification and long-term temporal change monitoring of coastal salt marsh vegetation.

This invention discloses a self-supervised learning prediction method and system for spatiotemporal meteorological-power data, primarily applied to the field of new energy power generation prediction. The method first acquires historical power, historical meteorological data, and future meteorological forecast data from wind farms, performing spatiotemporal interpolation for completion, 3σ outlier removal, Min-Max normalization, and spatial grid alignment preprocessing. Then, it constructs a meteorological-power pre-training framework based on physical simulation, incorporating time-series frequency alignment prediction. This framework includes a core model pre-trained using virtual wind farm data, a time-series alignment network, an adaptive multi-scale spatiotemporal fusion module, and a dual-branch feature extraction network. Next, a self-supervised calibration gated fusion module dynamically fuses spatiotemporal and temporal features. Finally, labeled data is used to fine-tune the complete model and optimize parameters. The system then outputs ultra-short-term and medium-short-term wind power predictions based on the input data. This invention significantly improves prediction accuracy and generalization ability, providing reliable support for power system dispatching.

The invention belongs to the technical field of data processing, and particularly relates to a data processing method for scientific and technological achievement transformation value evaluation. According to the method, through multi-source heterogeneous data acquisition, four-dimension data of scientific and technological innovation, market demands, risk factors and implementation support are acquired; performing timeliness correction on the feature data after unstructured data conversion, abnormal value removal and reconstruction preprocessing; and constructing an evaluation model based on a BP neural network, synchronously optimizing model parameters and a structure by adopting an improved dream optimization algorithm, and finally outputting a comprehensive potential score to judge an application value. According to the method, the problems of difficulty in multi-source data integration, poor data quality and timeliness, insufficient model precision and the like in existing evaluation are solved, the comprehensiveness, reliability and scientificity of evaluation are improved, and effective support is provided for scientific and technological achievement conversion decision making.