Big Data Cleaning Method Based on Deep Learning and Data Feature Matching

Abstract

The present invention relates to the technical field of data cleaning and processing, and specifically relates to a big data cleaning method based on deep learning and data feature matching, which includes the following steps: constructing a three-dimensional electromagnetic pollution topology map through an electromagnetic fingerprint map generator; activating an anti-interference cleaning channel according to the dynamic changes of the electromagnetic pollution topology map, and each channel generates an instruction set including channel configuration; receiving the channel configuration instruction set, performing dual operations of data pulse waveform reconstruction and feature repair, and outputting a repaired clean data block; writing the clean data block back to the original storage medium. The present invention realizes the precise recovery of pulse distortion caused by electromagnetic interference. Compared with traditional methods based on low-pass filtering or interpolation repair, it can more effectively remove non-linear interference components, ensure the integrity of data signals, and improve the fidelity of the recovered data.

Description

Technical Field

[0001] The present invention relates to the technical field of data cleaning and processing, and particularly to a big data cleaning method based on deep learning and data feature matching. Background Art

[0002] In the modern information storage and communication fields, data integrity and reliability face various complex challenges. Especially in high-noise electromagnetic environments, unstable storage media, and high-speed data interaction scenarios, traditional data cleaning and error correction methods often struggle to ensure accurate data recovery and long-term availability.

[0003] Currently, the mainstream data processing methods for electromagnetic interference and storage media noise mainly include low-pass filtering, interpolation reconstruction, error correction coding (ECC), data redundant storage (RAID), etc. However, these methods have obvious limitations in practical applications: Traditional low-pass filtering or interpolation algorithms are difficult to effectively remove high-frequency non-linear interference, resulting in signal distortion still existing and affecting data accuracy. Methods such as ECC usually rely on fixed redundant bits for error correction, but in a strong interference environment, data errors beyond the ECC error correction ability will accumulate, ultimately leading to data irrecoverability. Summary of the Invention

[0004] The present invention provides a big data cleaning method based on deep learning and data feature matching.

[0005] The big data cleaning method based on deep learning and data feature matching includes the following steps:

[0006] S1: Real-time monitor the electromagnetic interference intensity distribution of the data acquisition environment, and construct a three-dimensional electromagnetic pollution topological map through an electromagnetic fingerprint map generator;

[0007] S2: According to the dynamic changes of the electromagnetic pollution topological map, activate the anti-interference cleaning channels, and each channel generates a channel configuration instruction set including frequency band isolation parameters, signal regeneration coefficients, and quantum annealing initial conditions;

[0008] S3: Receive the channel configuration instruction set, and perform the dual operations of data pulse waveform reconstruction and feature repair. Among them, waveform reconstruction is based on reverse compensation of electromagnetic radiation residue traces, and the compensation parameters are dynamically adjusted by the signal regeneration coefficient. Feature repair uses a controlled quantum annealing algorithm to eliminate hidden bit flips, and the initial temperature of the annealing process is specified by the channel configuration instruction set, and output the cleaned data block after repair;

[0009] S4: Write the cleaned data block back to the original storage medium.

[0010] Optionally, the S1 specifically includes:

[0011] S11, Deployment of multi - band electromagnetic sensor arrays: Deploy multi - band electromagnetic sensors distributively in the data acquisition area, synchronously collect time - domain and frequency - domain electromagnetic radiation signals in the environment, and generate an original electromagnetic fingerprint sequence;

[0012] S12, Spatiotemporal feature fusion coding: Align timestamps and map spatial coordinates for the original electromagnetic fingerprint sequence, and extract spatiotemporal correlation features of electromagnetic interference through a pre - trained convolutional neural network (CNN). The input layer of the convolutional neural network receives spectral slice data, and the output layer generates a three - dimensional feature tensor including electromagnetic interference intensity, interference frequency, and spatial coordinates;

[0013] S13, Dynamic topology modeling: Input the three - dimensional feature tensor into an electromagnetic fingerprint map generator, model the interference propagation path between sensor nodes based on a graph neural network (GNN), and generate a three - dimensional electromagnetic pollution topology map through node embedding update. The vertices of the three - dimensional electromagnetic pollution topology map represent electromagnetic pollution intensity values, and the edge weights represent the interference coupling coefficients of adjacent regions;

[0014] According to the real - time data stream of the sensor, adopt a sliding window mechanism to update the topology map, trigger resampling of abnormal regions by comparing the map difference degrees of adjacent windows, and ensure that the map refresh rate matches the change rate of the electromagnetic environment.

[0015] Optionally, S2 specifically includes:

[0016] S21, Dynamic change detection and channel triggering: Monitor the change situation of the electromagnetic pollution topology map in real - time, and activate the anti - interference cleaning channel according to the triggering conditions:

[0017] Intensity mutation detection: When the electromagnetic interference intensity of a certain sensor changes violently within a short time and exceeds the preset change threshold, the system will identify this point as an abnormal area and trigger the cleaning channel.

[0018] Coupling anomaly detection: If the electromagnetic interference coupling degree between two sensors exceeds the set interference diffusion threshold, indicating that interference may spread in space, the system will take cleaning measures for this area.

[0019] Frequency conflict detection: If the main interference frequency of a certain sensor falls into the working frequency band range of the data signal, which may affect the normal data transmission, the system will isolate it.

[0020] S22, Band isolation parameter calculation: For the activated cleaning channel, calculate the band isolation parameters: By analyzing the set of sensors affected by interference, determine the affected frequency bands, and then generate a binary parameter vector, which represents the frequency bands that the current cleaning channel should shield, ensuring that the subsequent data cleaning process avoids the influence of interference frequencies;

[0021] S23, Dynamic adjustment of signal regeneration coefficient: During the cleaning process, dynamically adjust the intensity of signal compensation:

[0022] Analyze historical interference data through a traditional long short-term memory model to predict the compensation coefficient required for current signal recovery. The long short-term memory model receives historical interference intensity data within a predetermined time window, calculates the regeneration coefficient of the current signal, and uses an activation function to constrain its output;

[0023] S24, Setting the initial conditions of quantum annealing: Based on the overall interference level of the current electromagnetic environment, set the initial temperature of quantum annealing.

[0024] Optionally, the triggering conditions include:

[0025] Intensity mutation detection: When the electromagnetic interference intensity of a certain sensor changes violently within a short period of time and exceeds the intensity change threshold, it will be identified as an abnormal area and trigger the cleaning channel;

[0026] Coupling anomaly detection: If the electromagnetic interference coupling degree between two sensors exceeds the set spatial interference diffusion threshold, it indicates that interference is spreading in space, that is, cleaning measures are taken for this area;

[0027] Frequency conflict detection: If the main interference frequency of a certain sensor falls within the working frequency band range of the data signal, isolate this area.

[0028] Optionally, the initial temperature of quantum annealing in S24 includes calculating the global interference entropy within the entire monitoring area, evaluating the overall complexity of interference, and determining the initial temperature of the quantum annealing algorithm according to the magnitude of the global interference entropy. The more complex the interference, the higher the initial temperature.

[0029] Optionally, S2 further includes instruction set encapsulation and transmission, encapsulating the frequency band isolation parameters, signal regeneration coefficient, and quantum annealing initial conditions into a channel configuration instruction set, and distributing the channel configuration instruction set to the corresponding cleaning channels through a communication bus.

[0030] Optionally, S3 specifically includes:

[0031] S31, Pulse waveform reverse compensation and reconstruction, eliminating the influence of electromagnetic interference on the original data pulse signal and reconstructing the signal;

[0032] S32, Quantum annealing feature repair, repairs the reconstructed signal at the binary logic level to eliminate hidden data damage, including converting the reconstructed signal into a binary bit sequence and constructing an optimized energy function to measure the correctness of the signal logic bits; calculating the flip energy of each bit, considering the influence of data noise, and determining the ideal logic state at the bit level; evaluating the coupling relationship between different bits through the spatio-temporal correlation coupling strength, and calculating the mutual influence strength between bits; adopting the quantum annealing algorithm, based on the set initial temperature, to optimize the energy state of the entire bit sequence.

[0033] S33, Through the quantum annealing process, obtain the optimal logic bit sequence and convert it back to the repaired time-domain signal.

[0034] Optionally, S31 specifically includes collecting the original data pulse signal and pre-extracting the typical interference residual pulse template through the electromagnetic fingerprint spectrum; calculating the propagation delay of the interference signal to determine the interference contribution of different sensors to the data source; dynamically adjusting the compensation amplitude through the signal regeneration coefficient, constructing an inverse compensation model based on the convolution operation, and generating a compensated signal to make it close to the ideal signal waveform under the interference-free condition.

[0035] Optionally, S33 further includes dividing the repaired time-domain signal into blocks according to the preset data block standard to form structured data that meets the application requirements; calculating the error between the repaired data block and the ideal data block, and ensuring that the error is within the preset fault tolerance threshold, and outputting the final clean data block.

[0036] Optionally, S4 includes using the physical signal injection method to write the clean data block back to the original storage medium according to the physical coding rules of the storage medium (such as the P / E cycle characteristics of NAND flash).

[0037] Advantages of the present invention:

[0038] In the present invention, through the electromagnetic pollution topology modeling and dynamic cleaning channel activation mechanism, the electromagnetic interference sources in the environment can be accurately detected, and the affected data can be waveform reconstructed and feature repaired based on spatio-temporal feature extraction. Especially, the pulse waveform reconstruction based on convolution inverse compensation can dynamically adjust the signal compensation amplitude, and combined with the interference residual template in the electromagnetic fingerprint spectrum, it can achieve the accurate recovery of the pulse distortion caused by electromagnetic interference. Compared with the traditional methods based on low-pass filtering or interpolation repair, the present invention can more effectively remove the non-linear interference components, ensure the integrity of the data signal, and improve the fidelity of the recovered data.

[0039] The present invention adopts a controlled quantum annealing feature repair technology. By constructing an optimized energy function that includes single-bit flip energy and spatio-temporal coupling strength, and combining the characteristics of the storage medium and the changes in the electromagnetic environment, dynamic correction at the data logic level is achieved. Compared with traditional error correction algorithms based on bit error rate (BER) detection, the present invention can find the optimal data bit sequence through a global optimization strategy in a high-interference environment, improving the robustness of data logic recovery. In addition, based on the adaptive adjustment of the annealing temperature based on global interference entropy, the error correction ability for complex interference patterns can be enhanced in a high-noise environment, and the computational overhead can be reduced in a low-noise environment, optimizing the computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only those of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0041] Figure 1 Schematic flowchart of the cleaning method according to an embodiment of the present invention;

[0042] Figure 2 Schematic diagram of generating a channel configuration instruction set according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The present invention will be described in detail below with reference to the drawings and specific embodiments. At the same time, it should be noted here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments. For some well-known technologies, those skilled in the art can also adopt other alternative methods for implementation; and the drawings are only for more specifically describing the embodiments, and are not intended to specifically limit the present invention.

[0044] It should be pointed out that in the specification, it is mentioned that "an embodiment", "embodiments", "exemplary embodiments", "some embodiments", etc. indicate that the described embodiments may include specific features, structures or characteristics, but not necessarily every embodiment includes the specific feature, structure or characteristic. In addition, when combining an embodiment to describe a specific feature, structure or characteristic, implementing such a feature, structure or characteristic in combination with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the relevant art.

[0045] Generally, terms can be understood at least in part from their use in context. For example, depending at least in part on the context, the term "one or more" as used herein can be used to describe any feature, structure, or property in the singular sense, or can be used to describe a combination of features, structures, or properties in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey a set of exclusive factors, but rather can alternatively, at least in part depending on the context, allow for the existence of other factors that are not necessarily explicitly described.

[0046] As Figure 1 - Figure 2 shown, a big data cleaning method based on deep learning and data feature matching includes the following steps:

[0047] S1: Real-time monitor the electromagnetic interference intensity distribution of the data acquisition environment, and construct a three-dimensional electromagnetic pollution topology map through an electromagnetic fingerprint map generator;

[0048] S2: According to the dynamic changes of the electromagnetic pollution topology map, activate the anti-interference cleaning channels, and each channel generates a channel configuration instruction set including frequency band isolation parameters, signal regeneration coefficients, and quantum annealing initial conditions;

[0049] S3: Receive the channel configuration instruction set, and perform the dual operations of data pulse waveform reconstruction and feature repair. Among them, the waveform reconstruction is based on the reverse compensation of the electromagnetic radiation residue traces, and the compensation parameters are dynamically adjusted by the signal regeneration coefficient. The feature repair uses a controlled quantum annealing algorithm to eliminate hidden bit flips (repair data features through the quantum annealing algorithm, match the feature distribution of the original data, and eliminate hidden errors). The initial temperature of the annealing process is specified by the channel configuration instruction set, and output the cleaned data block after repair;

[0050] S4: Write back the cleaned data block to the original storage medium.

[0051] S1 specifically includes:

[0052] S11. Deploy multiple multi-band electromagnetic sensors in the data acquisition area, and each sensor (the th sensor) represents the electromagnetic radiation signal collected at time as a frequency domain slice :

[0053] , where represents the radiation intensity value of the sensor in the th frequency band, represents the frequency domain slice collected by the sensor at time

[0054] S12. The frequency domain slices collected by the sensors Input the pre-trained convolutional neural network (CNN) to extract spatio-temporal features. The output of the CNN is a three-dimensional feature tensor , where represents the electromagnetic interference intensity value of the sensor at time , represents the main interference frequency of the sensor at time , represents the spatial coordinates of the sensor , represents the three-dimensional feature tensor generated by the sensor at time ;

[0055] The loss function of is defined as:

[0056] , where is the frequency domain slice reconstructed by the CNN, is the network weight, is the regularization coefficient.

[0057] S13, Dynamic topology modeling: Input the three-dimensional feature tensor into the electromagnetic fingerprint map generator, and construct a dynamic three-dimensional electromagnetic pollution topology map based on the graph neural network. Define the graph structure , where the vertex set represents the electromagnetic pollution state of the sensor node; the edge set represents the interference coupling relationship between nodes, represents the edge between sensors , represents the interference coupling coefficient between sensors , is determined by the following formula:

[0058] ;

[0059] where is the attenuation coefficient of the frequency difference, represents the Euclidean distance between sensors. The node embedding update is expressed as:

[0060] ;

[0061] where represents the embedding vector of node at the th layer, represents the embedding vector of node at the The embedding vector of the layer, denotes the set of neighbors of node , is the weight matrix of the -th layer, is the activation function.

[0062] Online graph calibration: The topological map is updated using a sliding window mechanism. Define the window size as , and the window sliding step size as . For each window , calculate the graph difference degree between adjacent windows :

[0063] , where denotes the final embedding vector of node at the -th layer. If ( is the difference degree threshold), an abnormal area resampling is triggered, the sensor data is updated, and the topological map is regenerated.

[0064] Low-noise environment (stable electromagnetic environment): τ1 ∈ [0.01, 0.05]

[0065] Medium-noise environment (general industrial scenario): τ1 ∈ [0.05, 0.1]

[0066] High-noise environment (strong electromagnetic interference area, such as near radar stations and wireless base stations): τ ∈ [0.1, 0.2].

[0067] S2 specifically includes:

[0068] S21, dynamic change detection and channel triggering: The vertex set V(t) and edge set E(t) of the electromagnetic pollution topological map are monitored in real time. When any of the following conditions is met, the anti-interference cleaning channel is activated:

[0069] Intensity mutation detection: If the electromagnetic interference intensity of vertex (the vertex corresponding to sensor at time ) satisfies , monitor the vertex set and edge set of the electromagnetic pollution topological map in real time. When any of the following conditions is met, the anti-interference cleaning channel is activated: is the intensity change threshold, with a value of 0.3, is the monitoring interval;

[0070] Coupling anomaly detection: If the coupling coefficient of edge is​ Meet , where is the spatial interference diffusion threshold, and β ∈ [10 −2 , 10 −1 ;

[0071] Frequency conflict detection: If the main interference frequency of vertex (the main interference frequency observed by the sensor at time ) falls within the data signal frequency band range .

[0072] S22, Band isolation parameter calculation: For the activated cleaning channel, generate the band isolation parameter , represents the band isolation parameter vector of the th cleaning channel, which is a binary parameter vector, where:

[0073] , where is the th band isolation parameter vector, represents the set of abnormal sensors associated with the th channel, is the boundary value of the th band, represents "exists", represents "such that", means that there exists at least one sensor belonging to the set of abnormal sensors affected by interference , means that the main interference frequency of this sensor falls within the range of the th band.

[0074] S23, Signal regeneration coefficient dynamic adjustment: Generate the signal regeneration coefficient through a pre-trained LSTM (Long Short-Term Memory Network) model, whose input is the historical interference intensity sequence , indicating to obtain historical sequences of interference intensity for time steps, where

[0075] , where and are the model weights and biases, and the Sigmoid function constrains the output to the range [0,1];

[0076] S24. Quantum annealing initial condition setting: According to the vertex set of the global interference entropy set the initial annealing temperature :

[0077] ;

[0078] ;

[0079] Among them, is the preset maximum temperature, is the entropy sensitivity coefficient, taking empirical values: 0.5 - 1.2, is the total interference intensity of all sensors, is the total number of sensors.

[0080] Encapsulate the above parameters into a channel configuration instruction set , and distribute it to the execution unit of the corresponding cleaning channel through a low-latency bus.

[0081] S3 specifically includes:

[0082] S31. Pulse waveform reverse compensation reconstruction: Perform electromagnetic radiation residue trace elimination on the original data pulse signal , and the reconstructed waveform signal is expressed as:

[0083] ;

[0084] Among them, represents the convolution operation, is the Dirac pulse function, is the interference residue pulse template (pre-extracted through electromagnetic fingerprint spectrum), represents the sensor to the data source location of the propagation delay ( is the electromagnetic wave speed, taking the value 3.0×10 8 m / s), , is the reference interference intensity reference value (set based on hardware specifications, generally taking ), is the signal regeneration coefficient, dynamically adjusting the compensation amplitude, is the spatial coordinate of the data source, represents the Euclidean distance between the sensor and the data source.

[0085] S32. Quantum annealing feature repair: Convert the reconstructed pulse signal into a binary bit sequence , is the total length of the binary bit sequence, and constructs a controlled quantum annealing energy function :

[0086] ;

[0087] wherein, represents the single-site bit flip energy, , wherein, is the ideal logic level, is the noise variance, represents the spatio-temporal correlation coupling strength, , wherein, is the signal coherence time, with a value of 10–100 ms, is the set of correlated bit pairs, determined by the pulse timing correlation, is the th bit of the binary bit sequence;

[0088] The logical high level (1) takes values: 3.3V, 5V, 1.8V (depending on the circuit standard)

[0089] The logical low level (0) takes the value: 0V;

[0090] Perform controlled annealing based on the initial temperature :

[0091] , wherein, the annealing temperature decays according to , is the annealing time constant, synchronized with the hardware clock period, represents the quantum annealing transition probability;

[0092] S33, clean data block generation: Obtain the optimal bit sequence through quantum annealing , convert it into a repaired time-domain signal , and divide it into data blocks , satisfying:

[0093] ;

[0094] wherein, is the ideal data block template, is the matrix Frobenius norm, is the optimal bit sequence after annealing optimization, is the th data block after repair, takes the value .

[0095] The extraction of the interference residual pulse template is based on the electromagnetic fingerprint spectrum, specifically as follows:

[0096] Record the electromagnetic radiation signals within the target area, including intensity, frequency, and time characteristics.

[0097] Perform short-time Fourier transform on the collected data to extract the time-frequency distribution characteristics of electromagnetic interference in each frequency band.

[0098] Match the historical interference patterns through the established electromagnetic fingerprint database to identify typical interference sources (power frequency noise, radio frequency interference).

[0099] Select highly correlated interference patterns and use deconvolution technology to restore the morphology of the original interference pulses.

[0100] Adopt the clustering method to perform dimensionality reduction on different interference templates and store them in the interference fingerprint library for use in the real-time cleaning stage.

[0101] Adopt the physical signal injection method to inject clean data blocks Write them back to the original storage medium according to the physical coding rules of the storage medium (such as the P / E cycle characteristics of NAND flash memory).

[0102] The present invention covers any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the present invention. For the public to have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention. However, those skilled in the art can fully understand the present invention without the description of these details. Additionally, to avoid unnecessary confusion to the essence of the present invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0103] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A big data cleaning method based on deep learning and data feature matching, characterized in that: The following steps are involved: S1: Real-time monitoring of the electromagnetic interference intensity distribution of the data collection environment, and construction of a three-dimensional electromagnetic pollution topology map through an electromagnetic fingerprint generator; S2: According to the dynamic changes of the electromagnetic pollution topology map, the anti-interference cleaning channel is activated, and each channel generates a channel configuration instruction set including frequency band isolation parameters, signal regeneration coefficients, and quantum annealing initial conditions; S3: Receive channel configuration instruction set, perform dual operations of data pulse waveform reconstruction and feature repair, where waveform reconstruction is based on reverse compensation of residual traces of electromagnetic radiation, compensation parameters are dynamically adjusted by signal regeneration coefficient, feature repair uses controlled quantum annealing algorithm to eliminate implicit bit flips, the initial temperature of the annealing process is specified by the channel configuration instruction set, and outputs the repaired clean data block, specifically including: S31, pulse waveform reverse compensation reconstruction, eliminating the influence of electromagnetic interference on the original data pulse signal, and reconstructing the signal; S32, quantum annealing feature repair, repairs the reconstructed signal at the binary logic level to eliminate hidden data damage, including converting the reconstructed signal into a binary bit sequence and constructing an optimized energy function to measure the correctness of the signal logic bit; calculating the flip energy of each bit, considering the impact of data noise, and determining the ideal logic state at the bit level; evaluating the coupling relationship between different bits through the spatiotemporal correlation coupling strength, and calculating the mutual influence strength between bits; using the quantum annealing algorithm, based on the set initial temperature, optimize the energy state of the entire bit sequence; S33, obtaining the optimal logic bit sequence through the quantum annealing process and converting it back into a repaired time domain signal; S4: Writing the clean data blocks back to the original storage medium.

2. The big data cleaning method based on deep learning and data feature matching according to claim 1 is characterized in that: The S1 specifically includes: S11, multi-band electromagnetic sensor array deployment: multi-band electromagnetic sensors are distributed in the data collection area to synchronously collect time-domain and frequency-domain electromagnetic radiation signals in the environment to generate original electromagnetic fingerprint sequences; S12, spatiotemporal feature fusion coding: timestamp alignment and spatial coordinate mapping of the original electromagnetic fingerprint sequence are performed, and the spatiotemporal correlation features of electromagnetic interference are extracted through a pre-trained convolutional neural network, where the input layer of the convolutional neural network receives spectrum slice data, and the output layer generates a three-dimensional feature tensor including electromagnetic interference intensity, interference frequency, and spatial coordinates; S13, dynamic topology modeling: input the three-dimensional feature tensor into the electromagnetic fingerprint generator, model the interference propagation path between sensor nodes based on the graph neural network, and generate a three-dimensional electromagnetic pollution topology map through node embedding update. The vertices of the three-dimensional electromagnetic pollution topology map represent the electromagnetic pollution intensity values, and the edge weights represent the interference coupling coefficients of adjacent areas.

3. The big data cleaning method based on deep learning and data feature matching according to claim 2 is characterized in that: The S2 specifically includes: S21, dynamic change detection and channel triggering: real-time monitoring of changes in the electromagnetic pollution topology map, and activation of the anti-interference cleaning channel according to the trigger conditions: Intensity mutation detection: When the electromagnetic interference intensity of a sensor node changes dramatically in a short period of time and exceeds the preset change threshold, the sensor node is identified as an abnormal area and the cleaning channel is triggered; Coupling anomaly detection: If the electromagnetic interference coupling between two sensors exceeds the set interference diffusion threshold, it indicates that the interference may propagate in space, and the system will take cleaning measures for this area; Frequency conflict detection: If the main interference frequency of a sensor falls into the working frequency band of the data signal, it may affect the normality of data transmission, and the system will isolate it; S22, frequency band isolation parameter calculation: for the activated cleaning channel, the frequency band isolation parameter is calculated: by analyzing the set of interfered sensors, the affected frequency band is determined, and then a binary parameter vector is generated. The binary parameter vector represents the frequency band that should be shielded by the current cleaning channel, ensuring that the subsequent data cleaning process is free from the influence of the interference frequency; S23, Dynamic adjustment of signal regeneration coefficient: During the cleaning process, dynamically adjust the intensity of signal compensation: The traditional long short-term memory model is used to analyze historical interference data and predict the compensation coefficient required for the current signal recovery. The long short-term memory model receives the historical interference intensity data within a predetermined time window, calculates the regeneration coefficient of the current signal, and constrains its output using an activation function. S24, setting of initial conditions for quantum annealing: setting the initial temperature of quantum annealing based on the overall interference level of the current electromagnetic environment.

4. The big data cleaning method based on deep learning and data feature matching according to claim 3 is characterized in that: The trigger conditions include: Intensity mutation detection: When the electromagnetic interference intensity of a sensor changes dramatically in a short period of time and exceeds the intensity change threshold, it will be identified as an abnormal area and trigger the cleaning channel; Coupling anomaly detection: If the electromagnetic interference coupling between two sensors exceeds the set spatial interference diffusion threshold, it indicates that the interference is propagating in space, and cleaning measures are taken for this area; Frequency conflict detection: If the main interference frequency of a sensor falls into the working frequency band of the data signal, this area will be isolated.

5. The big data cleaning method based on deep learning and data feature matching according to claim 4 is characterized in that: The initial temperature of the quantum annealing in S24 includes calculating the global interference entropy in the entire monitoring area, evaluating the overall complexity of the interference, and determining the initial temperature of the quantum annealing algorithm according to the size of the global interference entropy. The more complex the interference, the higher the initial temperature.

6. The big data cleaning method based on deep learning and data feature matching according to claim 5 is characterized in that: The S2 also includes instruction set packaging and transmission, which packages the frequency band isolation parameters, signal regeneration coefficients and quantum annealing initial conditions into a channel configuration instruction set, and the channel configuration instruction set is distributed to the corresponding cleaning channel through the communication bus.

7. The big data cleaning method based on deep learning and data feature matching according to claim 1 is characterized in that: The S31 specifically includes collecting raw data pulse signals and pre-extracting typical interference residual pulse templates through electromagnetic fingerprints; Calculate the propagation delay of the interference signal and determine the interference contribution of different sensors to the data source; dynamically adjust the compensation amplitude through the signal regeneration coefficient, build an inverse compensation model based on convolution operation, and generate a compensated signal to make it close to the ideal signal waveform in the absence of interference.

8. The big data cleaning method based on deep learning and data feature matching according to claim 1 is characterized in that: The S33 also includes dividing the repaired time domain signal into blocks according to a preset data block standard to form structured data that meets application requirements; Calculate the error between the repaired data block and the ideal data block, ensure that the error is within the preset fault tolerance threshold, and output the final clean data block.

9. The big data cleaning method based on deep learning and data feature matching according to claim 1 is characterized in that ,The S4 includes adopting a physical signal injection method to write the clean data block back to the original storage medium according to the physical encoding rules of the storage medium.

Images