23 results about "Noisy data" patented technology

This invention proposes a method for observing ocean waves based on a binocular camera, belonging to the field of image processing technology. The method includes: S1: The binocular camera captures ocean wave images, generating ocean wave point cloud data at the current time t; S2: The ocean wave point cloud data in S1 is converted to ocean wave point cloud data in a geodetic coordinate system; S3: The ocean wave data in S2 undergoes quality screening and optimization, including: selecting the region of interest, quality screening, converting noisy data points to null values, null value filling, and Gaussian filtering; the quality screening classifies the ocean wave data into three levels: excellent, good, and poor; S4: Based on the excellent and good quality ocean wave point cloud data, the wave height, wavelength, and period at that time are calculated; S5: S1-S4 are repeated to obtain multiple frames of ocean wave point cloud data at different times, and the effective wave height, wavelength, period, and wave spectrum are calculated. This method can fill in null values, making the overall data more reasonable and the obtained elevation data more accurate.

The application provides a model training method and a model training device, and relates to the technical field of computers. The method comprises the following steps: obtaining target data; determining a sampling time in a training iteration, and performing disturbance sampling on the target data based on the sampling time to obtain noisy data; inputting the noisy data and the sampling time into a backbone network of an initial large language model to perform deep feature interaction and fusion, obtaining a hidden vector sequence output by the backbone network, and mapping the hidden vector sequence back to discrete codebook indexes of each modality through a decoding head to obtain predicted data; calculating a loss based on the predicted data and the target data to obtain a target loss; updating model parameters of the initial large language model based on the target loss, and obtaining a large language model that has completed training under the condition that a training termination condition is met. The application solves the problem that the prior art in the related art cannot simultaneously consider multi-modal understanding, generation and retrieval capabilities under a unified architecture.

This invention discloses a visual reconstruction method and system for remote sensing interpretation based on a generative diffusion model. The method includes: acquiring high-resolution and low-resolution remote sensing image data; adding different levels of Gaussian noise to the training data using a forward stochastic differential equation until pure Gaussian noise data is obtained; training a noise conditional scoring network to predict the scores corresponding to these noisy data; adding noise to the low-resolution image using a forward stochastic differential equation to finally obtain pure Gaussian noise; using a trained neural network to guide the random noise to gradually converge and generate a super-resolution remote sensing image; rapidly identifying land cover types on the generated remote sensing image; and delineating land cover patches on the original remote sensing image and assigning patch information based on the identified land cover categories. This invention achieves a super-resolution effect from low resolution without changing the land cover types and patch boundaries, thereby reducing interpretation costs and improving interpretation efficiency.

This invention relates to the field of oil and gas exploration and development technology, and discloses a method and apparatus for well logging data completion based on a generative deep learning model. The method progressively adds Gaussian noise to missing well logging data to be restored, obtaining noisy well logging data. The data to be restored includes known well logging data and missing data, while the noisy well logging data includes known well logging data and noisy data. Based on a created observation mask, zeros are filled into the missing well logging data to be restored and the known well logging data of the noisy well logging data, respectively, to obtain first mask data and second mask data. The first mask data and second mask data are concatenated to construct a hybrid tensor. Based on the complete logging data corresponding to the data to be restored, the hybrid tensor, and the trained generative deep learning model, the noisy well logging data is denoised to obtain the completed well logging data. This invention can effectively improve the accuracy of well logging data completion.

The application discloses a panoramic video view angle prediction method based on a neural network, belongs to the technical field of image processing, and is used for interactive video processing, comprising the following steps: acquiring a panoramic video of an existing public data set, then constructing a panoramic video view angle prediction model based on structured attention to perform training, and predicting a saliency prediction view angle of the panoramic video based on the panoramic video view angle prediction model based on structured attention which has completed the training. The panoramic video view angle prediction model based on structured attention is constructed, original sparse and noisy data is converted into more robust supervision signals, the deviation of single local fixation is overcome, comprehensive modeling of scene saliency and accurate prediction of user view angle are realized, and the accuracy and robustness of view angle prediction in virtual reality application are improved.

A method and device for generating a training dataset of a bio-signal denoising artificial intelligence (AI) model are provided. According to an embodiment, the method includes receiving first bio-signal data measured in a non-noise-suppressed environment and second bio-signal data measured in a noise-suppressed environment, performing component analysis of the first bio-signal data, and determining noise component data by separating target component data from the first bio-signal data, generating third bio-signal data by combining the noise component data with the second bio-signal data, and determining the second bio-signal data as ground truth data of the training dataset and determining the third bio-signal data as noisy data of the training dataset to determine the training dataset.

This invention discloses a visualization and interactive device for geophysical data evaluation, relating to the field of data processing technology. It includes an intent input parsing module for acquiring and parsing user input information to obtain preliminary geological intent features; an example feature extraction module for generating a high-dimensional feature vector representing the user's geological intent as the geological intent feature vector; multi-dimensional features including the internal physical property statistical features, external geometric morphology features, spatial context relationship features, and derived attribute features of the example area; a matching degree generation visualization module for calculating the similarity between the geological intent feature vector and each geophysical data unit within the target work area, generating a three-dimensional geological similarity volume; and an iterative optimization module for receiving user feedback based on the visualization results. This invention significantly improves the accuracy of target identification and effectively avoids the problems of missing key targets or interference from noisy data.

A system and method for prediction of failures and optimization, that can provide solution available for unsupervised learning models based on limited data that can predict different types of failure and pre-failure instances. The solution provides improvement upon previous methods of labelling by marking certain days data ahead of failure as belonging to failure data which will result in reduction of noisy data and improves good working condition data. The present invention helps with improved data quality due to labelling as the proposed method models complex distributions of feature vectors accurately and are better at finding deviations from normal data distribution which is used for detecting failures. The novel solution help to analyse and categorise the type of failures for PC Pumps currently deployed in CBM Fields for which failure days in advance can be predicted.

This application discloses a data denoising method and related equipment. The method utilizes artificial intelligence technology to denoise data. Any one of the target denoising operations performed on noisy data includes: generating distribution information corresponding to the target denoising operation based on first and second prediction information. The first prediction information indicates the predicted noise between the second noisy data and clean data, and the second prediction information indicates the square of the predicted noise between the second noisy data and clean data, or the square of the predicted distance between the first prediction information and the actual noise, where the actual noise includes the actual noise between the second noisy data and clean data; and sampling the denoised data from the distribution space pointed to by the distribution information. Since the distribution information corresponding to the denoising operation is not directly learned, the number of denoising operations performed on the noisy data is not constrained by the training phase, improving the flexibility of the inference phase.

The embodiment of the invention provides a content generation method and related equipment, and the method comprises the steps: indicating the execution of a calling reasoning model through business indication information related to a content generation business, when de-noising processing is carried out on corresponding input content coding characteristics of the noisy data content under a time step tj in the T time steps under the constraint of the first data content, obtaining a cache type for cache multiplexing according to content analysis parameters indicated by a cache multiplexing strategy related to the time step tj; and if the cache type is an inter-step cache type, multiplexing a predicted noise feature corresponding to a time step ti having an inter-step multiplexing relationship with the time step tj, performing denoising processing on an input content coding feature corresponding to the time step tj, and performing content generation based on an output content coding feature corresponding to the time step tj obtained by the denoising processing, the second data content is obtained. Therefore, the model reasoning speed can be effectively improved, and waste of computing resources is avoided.

Methods, apparatus, devices, and media for updating a decoder model are provided. In one method, a reference image is acquired, having first reference features determined by an encoder model. Based on noisy data and the reference image, second reference features of the reference image are determined, the second reference features being different from the first reference features. A predicted reference image associated with the reference image is determined by a decoder model corresponding to the encoder model, based on the second reference features. The decoder model is updated based on the difference between the reference image and the predicted reference image. Using some implementations of this disclosure, adding noise information to the second reference features can reduce the quality of the second reference features. In this way, the decoder model can be updated in a direction that enables the decoder model to generate higher-quality images from lower-quality features.

This invention discloses a method for identifying stony glaciers based on machine learning. The method comprises the following steps: S1: Interpreting different geomorphic features on stony glacier optical images to obtain identification results; S2: Enhancing the reliability of different types of stony glaciers using the InSAR method to obtain vector data of residual and complete stony glaciers; S3: Slicing images based on the vector data, masking non-stony glacier areas to obtain images with only stony glacier features, and cleaning noisy data; S4: Importing the InSAR-enhanced dataset into RF, SVM, LR, DT, KNN, and ResNet models for training, obtaining the accuracy of different machine learning models in identifying stony glaciers, and obtaining the optimal machine learning model. The method proposed in this invention overcomes the problems of low efficiency and high subjectivity in existing stony glacier state classification, thus achieving automatic classification of complete and residual states of stony glaciers. Furthermore, this method can compensate for the lack of complete and residual state attributes in many existing stony glacier catalogs.

This invention provides a data processing method, apparatus, device, computer program product, and computer-readable storage medium. The method considers that the preference data used in DPO training may have noisy labels that negatively impact training performance. It proposes a noise-aware metric to indicate the probability that the preference label of a pair of preference data is a noisy label, and incorporates this metric into the DPO training objective function. This incorporates the quality of the preference data into the DPO training process, thereby identifying and mitigating the impact of noisy preference data on DPO training performance. This method effectively identifies noisy data in the preference data used in DPO training and effectively reduces the impact of noisy data on training performance by introducing a noise-aware metric into the DPO objective function, thus improving DPO training performance. Furthermore, this method is widely applicable to various large models, including LLM and DM, to improve the DPO training performance of these large models.

Noisy data parsed from raw data can be received. The noisy data indicates first transaction events determined to be noise in the raw data. Using the noisy data, a first detection model can be trained to assign anomaly event indicators to second transaction events. The first detection model can receive an anomaly record. The anomaly record can indicate at least a portion of anomalous transaction events identified in runtime data. The first detection mode can assign the anomaly event indicators to the anomalous transaction events. The anomaly event indicators can indicate levels of severity of the anomalous transaction events identified in the runtime data.

A data processing method, applied in the field of artificial intelligence, includes: acquiring noisy data; performing denoising on the noisy data in multiple steps sequentially using a first diffusion model to obtain a denoising result corresponding to each step, wherein the first step is either the first step or an intermediate step; using the first diffusion model and a second diffusion model to predict noise in the denoised result corresponding to the first step, respectively, to obtain a first prediction result and a second prediction result; the maximum number of denoising steps of the first diffusion model is less than the maximum number of denoising steps of the second diffusion model; and updating the first diffusion model based on the difference between the first prediction result and the second prediction result. This application can improve the training efficiency of student models, reduce training time overhead, and enable student models to have better model performance.

This application discloses a coal facies logging classification method, system, equipment, storage medium, and program product, belonging to the field of coalfield geological exploration technology. First, by selecting reference wells and collecting multi-source data, a reliable data foundation is provided for coal facies classification, avoiding the limitations of subjective assignment. Second, by evaluating the correlation between coal facies parameters and logging data, logging data categories that conform to geological laws are selected, effectively eliminating noisy data irrelevant to coal facies types and improving the effectiveness of the training dataset used for model training. By determining the significance between logging data and coal facies types, a reliable statistical correlation between the logging data used for model training and coal facies types is ensured, improving the quality of model training and the accuracy of prediction. Finally, by training a coal facies evaluation model and applying it to prediction wells, the objectivity and accuracy of coal facies classification are guaranteed. This solves the technical problems of strong subjectivity and low classification efficiency in existing technologies.

This invention relates to the fields of artificial intelligence and machine learning, and discloses a method for noisy labeled image recognition based on active forgetting and noise suppression. The method first uses a Gaussian mixture model to dynamically partition training samples into clean and noisy subsets. For clean samples, basic supervised learning is maintained; for noisy samples, a dual regularization mechanism is introduced in parallel: one is an active forgetting module based on negative cross-entropy loss, which erases erroneous knowledge internalized by the model; the other is a noise suppression module based on complementary labels, which prevents further assimilation of noisy data by the model through a negative learning strategy. Both work together as plug-and-play regularization terms to construct a unified end-to-end optimization objective. This method effectively overcomes the limitations of traditional paradigms, exhibiting superior robustness and generalization performance on both synthetic and real noise benchmarks, providing a new technical direction for building highly reliable deep learning models.

This invention relates to a credit report generation method and apparatus based on a multi-index context bus, applied in the field of artificial intelligence technology. The method includes: generating micro-contexts through three-dimensional filtering of "entity-spatiotemporal-semantic," physically isolating noisy data, effectively solving the "lost in the middle" effect of long text input in large models, and significantly improving the recall rate and inference accuracy of key risk information; replacing full parameter fine-tuning with "configurable expert skill bodies," achieving decoupling of business logic and model, allowing credit policy adjustments to take effect immediately with only configuration updates, avoiding catastrophic forgetting, and reducing model maintenance costs; and significantly reducing memory usage and IO overhead by utilizing pointer-based context transmission and lifecycle management, effectively supporting high-concurrency credit inference scenarios.

This application discloses a clustering method, apparatus, processor, and electronic device for customer information. The method is applied in the field of big data technology and includes: S101, calculating the error function for each customer information in a customer information set to obtain a first error function, and determining the customer information corresponding to the first error function as a first centroid; S102, determining a first candidate centroid set based on the determined centroids and other customer information, and determining the next centroid based on the first candidate centroid set, repeating step S102 until K centroids are determined; S103, clustering the customer information set based on the K centroids to obtain the target clustering result. This application solves the problem in related technologies where, when clustering customer information and randomly selecting centroids for clustering, discrete points or noisy data in the customer information may be used as centroids, leading to inaccurate clustering results.

This application discloses a fault diagnosis method for equipment components based on a noise residual fusion strategy, belonging to the field of fault diagnosis technology. The method includes: acquiring noisy condition monitoring data collected from mechanical equipment operating in a noisy environment; performing standardized preprocessing to unify data distribution and eliminate dimensional differences; inputting the preprocessed noisy data into a trained denoising-diagnosis joint model; first, filtering noise through a signal denoising model to obtain denoised data; integrating the noise residual and denoised data into a comprehensive fusion feature through a fusion module; and then using a diagnostic model to complete fault prediction. This application effectively suppresses noise interference through a signal denoising model, avoids the loss of important fault information through a fusion module, and achieves collaborative optimization of denoising and diagnosis through cascaded logic in the joint model. This solves the problems of poor collaborative effect and easy information loss in existing technologies, improving the robustness and accuracy of fault diagnosis in noisy environments, and is applicable to fault monitoring and diagnosis of various industrial equipment.

This invention discloses a method for identifying transformer-household relationships based on single-step active annotation. It requires no additional equipment and can directly process noisy data, while achieving accuracy close to manual screening. The process is simple and easy to deploy; single-step active annotation eliminates the need for multiple rounds of interaction, reducing annotation management complexity. It can also be integrated with existing smart grid platforms, implemented through software, enhancing its engineering application value. It achieves almost the same recognition accuracy with only a quarter of the cost of manual annotation. Compared to unsupervised clustering combined with simple uncertainty labeling, this invention improves accuracy by over 20% when the annotation budget coefficient is 0.2. It exhibits good robustness and generalization ability, directly handling data containing missing, asynchronous, and anomalous elements without manual data cleaning. Sample selection balances representativeness and diversity, effectively covering transformer areas of different types and sizes. Especially in scenarios with uneven user numbers, its accuracy approaches that of fully supervised algorithms.

This invention provides a real-time monitoring and intelligent dispatching system for special transport vehicles based on multi-source sensor fusion. By standardizing and hierarchically fusing multi-source data, fused data is obtained, avoiding monitoring distortion caused by outliers and noisy data. The hierarchical fusion strategy solves the problems of simple data overlay and unresolved conflicts through a progressive integration approach. The output fused data possesses accuracy and consistency, and can be directly used as a reliable basis for monitoring and dispatching. By comparing the fused data with preset thresholds for each dimension in real time, the management level and direction are determined based on the comprehensive comparison results, achieving hierarchical and precise management. According to the management level and direction, combined with the fused data, the demand dispatching information for special transport vehicles is determined sequentially, enabling dynamic adjustment of risk-based dispatching for risky states and efficiency-based dispatching for normal states. Furthermore, the dispatching scheme is formulated based on the fused data, making it more practical.

This invention relates to the field of computer system technology based on specific computational models, and discloses a screening system and method for tumor therapeutic drugs, comprising: a modality-specific tokenization engine for generating a first token for data content and a quality characterization token for data credibility; a dynamic sequence assembly module for assembling the first token and the quality characterization token for available modalities; and a sequence fusion Transformer module for collaboratively processing the assembled token sequence and adjusting the dependence on the first token according to the quality characterization token, and finally generating a prediction output. This invention solves the technical problem that specific computational models cannot perceive quality differences when processing data uniformly. By enabling the model to simultaneously know the data content and data credibility during fusion, it achieves autonomous distrust of low-quality data input by the model, avoiding the drowning out of key signals by noisy data.