131 results about "Noisy data" patented technology

The invention discloses a data security sharing method and system, and relates to the technical field of data sharing. The method comprises the steps that a data provider uploads original data, a privacy budget value is calculated through a differential privacy algorithm, noise is added to obtain noisy data, and the noisy data is processed through an anonymous privacy protection algorithm based on maximum dissimilarity degree clustering to obtain desensitized data; encrypting the desensitized data by adopting a block chain decentralization-based ciphertext policy attribute-based encryption method, obtaining an encrypted data ciphertext, uploading the encrypted data ciphertext to a cloud server, and obtaining a content addressing hash value; encrypting the hash value through an elliptic curve encryption algorithm, and storing the encrypted hash value to a block chain account book; and when an access request of a data requester is received, the cloud server verifies the authority by using a non-interactive zero-knowledge proof protocol and returns an encrypted hash value after passing the verification, and the requester decrypts to obtain the hash value and decrypts the encrypted ciphertext to obtain desensitized data, thereby realizing data sharing.

The invention provides an industrial time sequence generation method based on a time sequence decomposition conditional diffusion model. The method comprises the steps that industrial time sequence data are collected and preprocessed to obtain a data set; introducing conditional variables to construct a conditional diffusion model, and injecting noise through forward diffusion; designing a time sequence decomposition and reconstruction UNet module, performing feature extraction on noisy data and conditional variables to obtain an intermediate state, performing time sequence decomposition on the intermediate state, and performing feature reconstruction by using a decoder; the Sinkhorn distance is used as a regularization term to be fused into conditional noise prediction loss to construct a loss function for training; and randomly generating pure Gaussian noise, inputting the pure Gaussian noise into the trained TDA-CDM model, obtaining predicted noise of the current time step, calculating noisy data of the next time step, performing cyclic operation until a time sequence without noise is obtained, and accelerating sampling by using a denoising diffusion implicit model in circulation. According to the method, coexisting multi-scale dynamic features in the complex industrial MTS can be carefully and effectively captured and restored, and the comprehensive quality of generated data is remarkably improved.

Embodiments described herein provide systems and methods for noise-robust contrastive learning. In view of the need for a noise-robust learning system, embodiments described herein provides a contrastive learning mechanism that combats noise by learning robust representations of the noisy data samples. Specifically, the training images are projected into a low-dimensional subspace, and the geometric structure of the subspace is regularized with: (1) a consistency contrastive loss that enforces images with perturbations to have similar embeddings; and (2) a prototypical contrastive loss augmented with a predetermined learning principle, which encourages the embedding for a linearly-interpolated input to have the same linear relationship with respect to the class prototypes. The low-dimensional embeddings are also trained to reconstruct the high-dimensional features, which preserves the learned information and regularizes the classifier.

The invention discloses a seismic strong background noise removal method based on a two-channel network, and relates to the field of geophysical exploration data processing, and the method comprises the steps: selecting a noiseless signal and background noise when the noiseless signal and the background noise are not excited from a single-channel seismic record after a seismic source is excited, carrying out the superposition, generating noisy data, and carrying out the normalization processing; constructing a dual-channel constraint denoising network composed of a global constraint sub-network and a denoising sub-network; optimizing network parameters through the training sample set; and performing global constraint denoising and slice local denoising on the noisy seismic data by using the trained model, and finally splicing to obtain a complete denoising result. According to the method, the splicing effect problem caused by traditional blocking processing is effectively solved, the strong background noise removal effect is remarkably improved, and seamless high-quality denoising with the complete edge is achieved.

The invention relates to the field of field groundwater magnetic resonance detection methods, in particular to an augmentation system and method for field groundwater magnetic resonance detection small sample data. Comprising a generation network configured to receive random noise and generate augmented data based on the random noise; the discrimination network comprises a time domain discrimination network and a frequency domain discrimination network; the generative network and the time domain discrimination network form a first discrimination channel, the generative network and the frequency domain discrimination network form a second discrimination channel, and in the second discrimination channel, augmented data generated by the generative network and the noisy data are jointly input into the frequency domain discrimination network; the frequency domain discrimination network is used for discriminating whether the source of input data is a generation network or noisy data in a frequency domain. According to the method, a magnetic resonance detection small sample noisy data set is established, and multi-scale features in an amplitude spectrum and a phase spectrum are captured through the frequency domain discrimination network; the collaborative driving generation network generates a high-quality sample which is close to small sample noisy data in waveform form and spectral characteristics.

The invention discloses a driving decision model optimization method and electronic equipment, and relates to the technical field of automatic driving, and the method comprises the steps: carrying out the denoising processing through a training diffusion model according to first driving perception data and first noisy data, so as to generate a first driving path point sequence of the first driving perception data at a future moment; therefore, abundant and diversified driving tracks are generated by utilizing the strong complex multi-modal distribution modeling capability of the diffusion model; using a first diffusion model to generate a plurality of second driving path point sequences at future moments according to the second driving perception data so as to construct an evaluator training sample, and training an evaluator model to select an optimal driving path point sequence from the plurality of second driving path point sequences, according to the method, the diffusion model and the evaluator model obtained through training are utilized to construct the driving decision-making model, through joint training of the evaluator model and the diffusion model, the decision-making ability of the driving decision-making model for generating candidate driving tracks and selecting the optimal driving track is enhanced, and the safety of automatic driving is improved.

The invention provides an aeromagnetic data noise reduction method of a noise reduction auto-encoder based on adversarial regularization, and the method comprises the steps: constructing a simulation target signal library, and designing an adversarial regularization noise reduction auto-encoder model structure composed of a noise simulator-encoder-decoder-discriminator structure; inputting the clean signal into a noise simulator to generate noisy data, inputting the noisy data into a decoder-encoder, and outputting a de-noised signal; secondly, the de-noised signal is input into a discriminator, and the discriminator generates a score for the de-noised signal; the reconstruction loss and the adversarial loss form an adversarial loss function; then, the model is trained; and finally, inputting an actually measured signal into the trained model, and carrying out aeromagnetic signal denoising. According to the technical scheme, the technical problems that in the prior art, when an aeromagnetic interference compensation method is used for processing non-linear, time-varying and atypical interference, the compensation precision is low, manual modeling is depended on, and the generalization ability is weak are solved.

The invention discloses a robust target recognition method and device based on noise analysis and a storage medium thereof. The method comprises the following steps: acquiring original noisy data to establish an original data set, screening the original data set based on a K-nearest neighbor method in cooperation with Jensen-Shannon divergence, and respectively establishing a net data subset and a noise data subset; corresponding weight parameters and gradient parameters are obtained in the training process of the network, responses of the net data subset and the noise data subset are calculated, and corresponding sensitivity and contribution degree are obtained; the sensitivity and the contribution degree are fused, and the overall importance score of the parameters is obtained through integration; and improving a loss function in a pruning criterion of Taylor expansion through the overall importance score, and building a pruning model. According to the method, information in the feature space and information in the prediction space are fused, clean samples and noise samples in an original data set are accurately divided, and the reliability of weight parameter evaluation samples in the pruning process of the pruning model is improved.

The invention provides a soil moisture sensor data quality inspection and interpolation method, which comprises the following steps: screening multivariable soil moisture time sequence data based on a preset core physical feature list, carrying out abnormal value detection through physical rule constraint and an isolation forest algorithm, and carrying out data labeling by creating a complete time axis; generating a training sample from the preprocessed data through sliding window sampling, and performing deep feature learning by using a denoising network based on a space-time diffusion probability model; performing interpolation on missing values in the original data by using the trained space-time diffusion probability model, generating a noisy data sample through a forward noise adding process, and performing conditional data interpolation based on a known observation value and a mask matrix in a reverse denoising process to generate a preliminary interpolation result; and performing post-processing correction on the preliminary interpolation result, wherein the post-processing correction comprises clamping correction based on a monthly historical range and correction based on interlayer physical logic.

Noisy data parsed from raw data can be received. The noisy data indicates first transaction events determined to be noise in the raw data. Using the noisy data, a first detection model can be trained to assign anomaly event indicators to second transaction events. The first detection model can receive an anomaly record. The anomaly record can indicate at least a portion of anomalous transaction events identified in runtime data. The first detection mode can assign the anomaly event indicators to the anomalous transaction events. The anomaly event indicators can indicate levels of severity of the anomalous transaction events identified in the runtime data.

The present invention discloses a device (100), a system (200), and a method (300) for analysing a document received from one or more users. The invention includes a system (200) for document analysis. The system (200) comprises a user interface (101) to interact with users. The user interface (101) generates queries and receives documents from users. The documents are then transmitted to a device (100) equipped with processors (102) for analysis. The processors (102) extract key sections from the document, prioritize them, remove noisy data, and generate a summary. An interactive tool (105) within the user interface (101) facilitates user feedback on the analysis. The user feedback is used to update a keyword dictionary (104) at predetermined intervals, allowing the system (200) to continuously improve its document analysis capabilities.

The invention relates to a spatio-temporal data rule extraction method based on spatio-temporal Fourier expert mixing, belongs to the technical field of urban spatio-temporal data processing, solves the problem that global stability characteristics and local disturbance characteristics driven by real physical rules are difficult to effectively describe in the prior art, and comprises the steps that S1, a spatio-temporal data acquisition module acquires historical data information; s2, establishing a space-time Fourier expert hybrid network, and carrying out expert mixing to obtain mixed features; s3, using adaptive group normalization as a conditional fusion module to obtain fusion features; s4, establishing a space-time Fourier attention mechanism module to obtain output features; s5, establishing a diffusion model for performing back diffusion on the noisy data in combination with the noise estimation network to obtain predicted denoised data, and performing training to obtain a trained diffusion model; and S6, performing sampling to obtain spatio-temporal data, inputting the spatio-temporal data into the trained diffusion model, and obtaining an extracted causal law for urban traffic flow prediction.

The application discloses a method for discovering control equations from scarce and noisy data based on physics. The application combines a physical information neural network and a sparse regression method to discover the partial differential control equations of a dynamic system from scarce and noisy data. Firstly, the size of the candidate function library is effectively reduced through a dimension verification method. Then, the powerful nonlinear fitting capability and automatic differentiation characteristics of a deep neural network are used to model a physical system and calculate candidate functions. Finally, the form of the control equation and the coefficients of the equation are obtained through sparse regression, and the coefficients are fine-tuned through a DNN. The method can not only discover control equations from data, but also obtain a network model to realize response prediction of a dynamic system. The method is simple, efficient, high-precision and highly universal, and can be widely applied to physical knowledge mining, modeling and reasoning of complex dynamic systems.

This invention proposes a method for observing ocean waves based on a binocular camera, belonging to the field of image processing technology. The method includes: S1: The binocular camera captures ocean wave images, generating ocean wave point cloud data at the current time t; S2: The ocean wave point cloud data in S1 is converted to ocean wave point cloud data in a geodetic coordinate system; S3: The ocean wave data in S2 undergoes quality screening and optimization, including: selecting the region of interest, quality screening, converting noisy data points to null values, null value filling, and Gaussian filtering; the quality screening classifies the ocean wave data into three levels: excellent, good, and poor; S4: Based on the excellent and good quality ocean wave point cloud data, the wave height, wavelength, and period at that time are calculated; S5: S1-S4 are repeated to obtain multiple frames of ocean wave point cloud data at different times, and the effective wave height, wavelength, period, and wave spectrum are calculated. This method can fill in null values, making the overall data more reasonable and the obtained elevation data more accurate.

The invention belongs to the technical field of biomedical engineering, and discloses a gene regulatory network optimization method based on a diffusion model, which comprises the following steps: acquiring gene data of cells under a steady state condition, and constructing a gene expression matrix according to the gene data; injecting Gaussian noise into the gene expression matrix based on a diffusion model method to generate a series of noisy data sequences; performing noise estimation and structure estimation on the noisy data sequence by a noise estimator and a structure estimator based on a gene regulation and control network, and performing reverse denoising processing according to the noise estimation and the structure estimation to obtain gene structure estimation after reverse denoising; performing structure optimization on the gene structure estimation after reverse denoising by adopting an acyclic constraint function and a regularization substitution method; and outputting the optimized gene structure estimation. According to the method, the regulation and control relation between the genes is accurately recognized from high-dimensional gene expression data, and the modeling precision of the regulation and control relation between the genes is improved.

The invention discloses a low-cost imaging sensor noise modeling method and system. According to the method, calibration data are collected and processed, sensor noise is decomposed into time-invariant noise, stripe noise and pixel-level noise components, and modeling calibration is carried out on the time-invariant noise, the stripe noise and the pixel-level noise components respectively; the method comprises the following steps: constructing a neural proxy network taking random noise, sensitivity and exposure time as input conditions, and training the neural proxy network by using a high-precision pixel-level noise sample, so that the neural proxy network can generate synthetic noise adaptive to sensor characteristics and imaging parameters; a random mixing strategy is adopted to fuse synthetic noise generated by the network and real sampling noise, then the synthetic noise and the real sampling noise are superposed with other calibration noise components and signal correlation noise, a complete synthetic noise field is generated, and finally the complete synthetic noise field and a clean image are synthesized into vivid noisy data used for training a denoising network. According to the method, the complex noise characteristics of the low-cost sensor can be accurately described, and the authenticity and diversity of the noise are effectively considered.

The application provides a model training method and a model training device, and relates to the technical field of computers. The method comprises the following steps: obtaining target data; determining a sampling time in a training iteration, and performing disturbance sampling on the target data based on the sampling time to obtain noisy data; inputting the noisy data and the sampling time into a backbone network of an initial large language model to perform deep feature interaction and fusion, obtaining a hidden vector sequence output by the backbone network, and mapping the hidden vector sequence back to discrete codebook indexes of each modality through a decoding head to obtain predicted data; calculating a loss based on the predicted data and the target data to obtain a target loss; updating model parameters of the initial large language model based on the target loss, and obtaining a large language model that has completed training under the condition that a training termination condition is met. The application solves the problem that the prior art in the related art cannot simultaneously consider multi-modal understanding, generation and retrieval capabilities under a unified architecture.

The invention discloses a panoramic video view angle prediction method based on a neural network, which belongs to the technical field of image processing, is used for interactive video processing, and comprises the following steps: acquiring a panoramic video of an existing public data set, and then constructing a panoramic video view angle prediction model based on structured attention for training; and predicting a saliency prediction view angle of the panoramic video based on the trained structured attention-based panoramic video view angle prediction model. According to the method, the panoramic video view angle prediction model based on the structured attention is constructed, and original sparse and noised data is converted into a more robust supervision signal; the deviation of single local fixation is overcome; comprehensive modeling of scene saliency and accurate prediction of a user visual angle are realized, and the accuracy and robustness of visual angle prediction in virtual reality application are improved.

This invention discloses a visual reconstruction method and system for remote sensing interpretation based on a generative diffusion model. The method includes: acquiring high-resolution and low-resolution remote sensing image data; adding different levels of Gaussian noise to the training data using a forward stochastic differential equation until pure Gaussian noise data is obtained; training a noise conditional scoring network to predict the scores corresponding to these noisy data; adding noise to the low-resolution image using a forward stochastic differential equation to finally obtain pure Gaussian noise; using a trained neural network to guide the random noise to gradually converge and generate a super-resolution remote sensing image; rapidly identifying land cover types on the generated remote sensing image; and delineating land cover patches on the original remote sensing image and assigning patch information based on the identified land cover categories. This invention achieves a super-resolution effect from low resolution without changing the land cover types and patch boundaries, thereby reducing interpretation costs and improving interpretation efficiency.

A system and method for learning a predictive function that can automatically learn different operating modes for a multi-modal system and predict the number of operating states for a multi-modal system and additionally the detailed structure for each state. Once learned, the predictive function (model) can be used to determine a mode of a new sample (an asset). Based on the determined components that maximize a log likelihood function, a mode of the new sample is detected into the model via dependency graphs. One aspect includes enforcing a lower bound for the number of sample points to form an operational mode for an asset. While a mode relates to sample points which maximizes like log-likelihood, an ability is provided to remove artifact modes due to noisy data by considering a sufficient sample data condition and maximizing log-likelihood. Domain knowledge can be incorporated into the model via dependency graphs.

The invention relates to the comprehensive technical field of map making, in particular to a building simplification method and system based on a condition graph diffusion model, noise data are input into a pre-trained diffusion model, building graph structure data to be processed are used as generation conditions to be injected into the diffusion model, and the generation conditions are generated; enabling the diffusion model to carry out the denoising of the noise data based on the generation condition, and obtaining the simplified data of the structure of the building drawing, the diffusion model takes building graph structure data of a first scale as a generation condition and building graph structure data of a second scale as a generation target in a training process, correlation characteristics between noisy target data and condition data are learned through forward noise adding operation, and the first scale is larger than the second scale. According to the method, the generation type simplification is carried out by understanding the overall and detail morphological characteristics of the building based on the diffusion model, and the overall quality of building simplification in drawing is improved.

A data denoising method and a related device are provided. According to the method, an artificial intelligence technology may be used to perform denoising on data, and any target denoising operation in at least one denoising operation performed on noisy data includes: generating, based on first prediction information and second prediction information, distribution information corresponding to the target denoising operation, where the first prediction information indicates predicted noise between second noisy data and clean data, the second prediction information indicates a square of the predicted noise between the second noisy data and the clean data or indicates a square of a predicted distance between the first prediction information and actual noise, and the actual noise includes actual noise between the second noisy data and the clean data; and sampling denoised data in distribution space to which the distribution information points.

The invention discloses a cloud model-based tailing dam rainfall stability risk assessment method. The method comprises the following steps of S1, establishing a complete index system; s2, carrying out denoising preprocessing on the monitoring data by adopting a variational mode decomposition (VMD) algorithm; s3, carrying out weight analysis, and determining a combined weight; and S4, constructing a cloud model, determining a standard cloud of the cloud model, and judging a risk level. According to the method, the VMD algorithm is innovatively introduced to carry out denoising reconstruction on the monitoring data, original noisy data does not need to be stored, only effective signal components are reserved, risk level distribution characteristics can be described only through expected value-entropy-hyper-entropy parameterized representation of the cloud model and by using three groups of core parameters, dependence of a traditional method on full-amount historical data is replaced, and the method is high in reliability and high in reliability. The problem that the size of the buffer area is difficult to match the data size is thoroughly solved; under the mechanism, the risk evolution key information can be completely reserved with lower storage cost, and risk misjudgment caused by data loss can be avoided.

The application belongs to the field of nuclear magnetic resonance sounding signal noise suppression methods, and is a ground magnetic resonance multi-type noise denoising network construction system and denoising method, which comprises a magnetic resonance signal construction module, a plurality of groups of magnetic resonance effective signals and environmental noise are simulated, and a noisy data set affected by three types of noise is constructed; a denoising neural network building module, the magnetic resonance signal generated by the magnetic resonance signal construction module is trained and optimized based on three types of noisy data sets and noise data sets, three denoising networks for different types of noise are obtained, a classification and discrimination model adopts a support vector machine method to judge the noise type according to the characteristics of different noises, and outputs the judgment result to the corresponding denoising network model in the denoising neural network building module. The application can reduce the scale of the label data amount, shorten the training time of the model, and remove the noise in a targeted manner, so that the dependence of the network model on data diversity can be reduced.

This invention relates to the field of oil and gas exploration and development technology, and discloses a method and apparatus for well logging data completion based on a generative deep learning model. The method progressively adds Gaussian noise to missing well logging data to be restored, obtaining noisy well logging data. The data to be restored includes known well logging data and missing data, while the noisy well logging data includes known well logging data and noisy data. Based on a created observation mask, zeros are filled into the missing well logging data to be restored and the known well logging data of the noisy well logging data, respectively, to obtain first mask data and second mask data. The first mask data and second mask data are concatenated to construct a hybrid tensor. Based on the complete logging data corresponding to the data to be restored, the hybrid tensor, and the trained generative deep learning model, the noisy well logging data is denoised to obtain the completed well logging data. This invention can effectively improve the accuracy of well logging data completion.

The application discloses a panoramic video view angle prediction method based on a neural network, belongs to the technical field of image processing, and is used for interactive video processing, comprising the following steps: acquiring a panoramic video of an existing public data set, then constructing a panoramic video view angle prediction model based on structured attention to perform training, and predicting a saliency prediction view angle of the panoramic video based on the panoramic video view angle prediction model based on structured attention which has completed the training. The panoramic video view angle prediction model based on structured attention is constructed, original sparse and noisy data is converted into more robust supervision signals, the deviation of single local fixation is overcome, comprehensive modeling of scene saliency and accurate prediction of user view angle are realized, and the accuracy and robustness of view angle prediction in virtual reality application are improved.

A method and device for generating a training dataset of a bio-signal denoising artificial intelligence (AI) model are provided. According to an embodiment, the method includes receiving first bio-signal data measured in a non-noise-suppressed environment and second bio-signal data measured in a noise-suppressed environment, performing component analysis of the first bio-signal data, and determining noise component data by separating target component data from the first bio-signal data, generating third bio-signal data by combining the noise component data with the second bio-signal data, and determining the second bio-signal data as ground truth data of the training dataset and determining the third bio-signal data as noisy data of the training dataset to determine the training dataset.

The invention discloses a diffusion model migration method based on prediction residual guidance and a related device, and relates to the technical field of diffusion model knowledge migration, and the method comprises the steps: taking noise data as input at each denoising time step, and carrying out denoising on the noise data; determining a first prediction noise, a second prediction noise and a third prediction noise by using the basic model, the adaptive model and the target model respectively, calculating a deviation between the first prediction noise and the second prediction noise to obtain a prediction residual error, performing weighted summation on the third prediction noise and the prediction residual error to obtain a guide prediction noise, and outputting the guide prediction noise. And on the basis of the noise data and the guide prediction noise, noise data of the next denoising time step is calculated until the last denoising time step is reached, and an output result of the target model is obtained, and the output result is a picture, a video, a voice or a text. According to the method, the knowledge migration of the diffusion model can be completed on the premise of not accessing original training data and not training.

The application discloses a self-supervised seismic denoising method and system fusing a noise estimation module and a channel attention mechanism, and belongs to the field of oil and gas exploration and seismic data processing, and solves the problems of poor actual noise removal effect and easy removal of original data in the prior art. The application slices and makes a dataset for noisy seismic data; inputs the noisy data into a noise estimation module to extract noise features, and then inputs the noise features into a blind spot network denoising integrated with a channel attention mechanism; trains the data by inputting the data into branches with an inflation convolution module and branches with an inflation Transform module, respectively, saves the model weight after the training is completed, and uses the trained model to perform denoising processing on input noisy seismic data, and outputs denoised seismic data. The application is used for seismic data noise removal.

This invention discloses a visualization and interactive device for geophysical data evaluation, relating to the field of data processing technology. It includes an intent input parsing module for acquiring and parsing user input information to obtain preliminary geological intent features; an example feature extraction module for generating a high-dimensional feature vector representing the user's geological intent as the geological intent feature vector; multi-dimensional features including the internal physical property statistical features, external geometric morphology features, spatial context relationship features, and derived attribute features of the example area; a matching degree generation visualization module for calculating the similarity between the geological intent feature vector and each geophysical data unit within the target work area, generating a three-dimensional geological similarity volume; and an iterative optimization module for receiving user feedback based on the visualization results. This invention significantly improves the accuracy of target identification and effectively avoids the problems of missing key targets or interference from noisy data.