419 results about "Data filtering" patented technology

The invention discloses an SLAM-oriented multi-sensor space-time synchronization and online calibration method, and relates to the technical field of computer vision and robot perception. Comprising the steps that S100, initialization is carried out, and an initial pose and an external parameter transformation matrix initial value are obtained through coarse alignment; s200, data acquisition and preprocessing: acquiring IMU data, LiDAR data and Odom data in real time, and performing data filtering, distortion removal, time sequence correction and point cloud downsampling processing on the acquired data; and S300, performing time synchronization processing, estimating the time offset of the LiDAR and Odom data in real time, and performing unified correction to generate a unified timestamp. According to the method, an online time offset estimation and compensation mechanism with an IMU as a main clock is established, accurate synchronization of sensor data of different frequencies is achieved, and the problem of data alignment caused by sampling frequency difference and clock drift is effectively solved; and meanwhile, tight coupling fusion is realized by using an IMU pre-integration factor, a LiDAR residual factor and an Odom constraint factor, so that the space-time consistency of multi-sensor data is remarkably improved.

The invention discloses an abnormal water quality ion trend deviation early warning system based on an unsupervised confrontation anomaly detection model. The abnormal water quality ion trend deviation early warning system comprises a water quality ion data acquisition module which generates an ion concentration data sequence in real time; the data preprocessing module is used for carrying out abnormal data filtering, missing completion and smoothing processing on the sequence data; the feature extraction module extracts sequence trends and statistical features to generate multi-dimensional feature data; the USAD model construction module is used for training the normal working condition data to obtain an anomaly detection model; the real-time trend deviation analysis module reconstructs and analyzes the real-time characteristic data by using a USAD model to generate a trend evaluation result; the abnormity judgment module judges the trend deviation abnormity according to a preset threshold value; the early warning module automatically generates abnormal early warning information; and the output module visualizes the abnormal indexes in real time. According to the invention, real-time and high-confidence early warning of water quality ion trend deviation is realized, and the early monitoring precision and response speed of water quality abnormal events are obviously improved.

The invention discloses a PID parameter optimization system based on offline system identification, and the system comprises a data selection module which is used for screening modeling data in a historical database; the data preprocessing module is used for performing real data complementation and data filtering on the screened modeling data; the model system identification module is used for establishing a controlled object model by using the preprocessed data; and the PID parameter optimization module is used for performing off-line optimization of PID parameters on the controlled object model. The defects in the prior art can be overcome, any on-line closed-loop test is not needed, and the optimal PID parameter is output at a time.

The invention belongs to the technical field of data filtering query, and particularly relates to a dynamic SQL-based parameter data transmission filtering query mode for a low-code visual component, which comprises the following steps of: S1, constructing a low-code visual large-screen component and a background service API (Application Program Interface); s2, interactively filtering the linkage demand by the visualized large-screen component, and realizing dynamic parameter transmission of the SQL parameter transmission query request; s3, providing a visual operation interface by using the visual large screen component; s4, calling the data source identifier stored in the step S2 by the SQL parameter analysis constructor generation component; and S5, performing parameter extraction on the structured filtering condition data by the SQL parameter analysis constructor. According to the overall structure provided by the embodiment of the invention, the SQL parameters are transmitted during data query, and then the back-end service constructs the filtering and screening conditions meeting the conditions based on the dynamic SQL parameter transmission mode to query the SQL and query the database to obtain the data, so that accurate filtering and screening query of the data can be realized; the problem that a traditional large screen cannot perform database level filtering and screening is solved.

The invention relates to the technical field of power grid data privacy protection, in particular to a smart power grid data privacy protection system and method based on a homomorphic encryption technology. The system protects data privacy of an intelligent power grid based on a homomorphic encryption technology, and comprises eight modules: a data acquisition module is connected with an intelligent electric meter in a multi-protocol manner in a 15-minute period to adopt electric data; the data filtering and preprocessing module optimizes data quality; the homomorphic encryption module generates a ciphertext capable of being added and multiplied by using an FHE algorithm; the multi-level key management module manages multiple types of keys in the whole life cycle; the secure transmission module transmits the ciphertext by means of a hybrid protocol; the data analysis module decrypts and analyzes the ciphertext in the data center; the decryption module decrypts after authorization; and the auditing and monitoring module records operation and monitors abnormity. According to the invention, it is ensured that the power use data is kept in an encrypted state in the whole life cycle of collection, transmission and analysis, the data is available and invisible, and the data privacy protection level and quantum attack resistance of the smart grid are improved.

The invention discloses a CO2 well flooding oil casing pressure intelligent control and anti-freezing and anti-blocking system, and relates to the technical field of oil and gas field development engineering.The CO2 well flooding oil casing pressure intelligent control and anti-freezing and anti-blocking system comprises the steps that a controller starts a self-checking program, loads a preset target pressure value, a preset temperature early warning threshold value and a preset ice blocking critical temperature parameter, completes initial configuration, and sends the initial configuration to the controller; zero calibration and precision verification are carried out on the front-end pressure sensor, the rear-end pressure sensor and the temperature sensor, front-end pressure data, rear-end pressure data and temperature data are collected, data filtering and abnormal value removing processing are carried out, and the real-time temperature data and CO2 physical property parameters are obtained. According to the invention, hardware self-inspection and sensor precision calibration are realized through the controller initialization module, the data acquisition module completes multi-source data synchronous acquisition and filtering processing, and the dynamic risk assessment module based on an improved Span-Wagner equation and a fuzzy logic algorithm can accurately calculate a CO2 phase change critical point and output a quantitative risk index. The problems of energy waste and untimely freezing and blocking prevention caused by a traditional fixed threshold value method are solved.

Disclosed are methods, systems, and other implementations, including a unified complex-valued deep learning framework (AFT-Net), which determines the k-space domain to image domain mapping for MRI reconstruction and allows incorporation of existing deep learning models. Embodiments include a computer-implemented method for reconstructing images that includes obtaining resonance (MR) k-space data resulting from a scan performed by an MRI scanner on tissue of a patient, with the MR k-space data including complex-valued data, and processing, by a complex-valued machine learning image reconstruction system, the complex-valued data of the MR k-space data to generate image data representing features of the MR k-space data. The processing may include performing data filtering operations, by one or more machine learning filter blocks implemented according to a CU-Net architecture realized using one or more convolutional neural networks (CNN) configured for complex data processing, on data that is based on the k-space data.

The invention relates to the technical field of data filtering, and provides a sliding window robust adaptive cubature Kalman filtering method and device and a medium, and the method comprises the steps: obtaining an observation vector of a sensor for a tracked target at a current moment; estimating a current state and an error covariance matrix by using a volume Kalman filtering algorithm based on the observation vector and the previous moment information; performing reverse smoothing on an estimation result at each moment in the sliding window by using a volume Kalman smoother to obtain a state smooth vector and a corresponding matrix; and finally, determining a current noise covariance matrix by variational Bayesian inference based on the information, wherein the current noise covariance matrix is used for correcting a prediction error covariance matrix at the next moment. According to the embodiment of the invention, the sliding window mechanism and the volume Kalman smoother are introduced to carry out reverse smoothing on the historical state, and the variational Bayesian inference is combined to carry out online joint estimation on the time-varying noise covariance matrix, so that the uncertainty of the model and the filtering divergence caused by noise abrupt change can be effectively inhibited, and the state estimation precision and robustness are improved.

The invention discloses a coherent accumulation measurement method of an S-mode low-power signal and related equipment, and relates to the field of aviation communication signal measurement. Comprising the following steps: performing noise statistics and baseline calibration on digital I / Q sampling data filtered and amplified by an analog front end to obtain a noise mean value and a noise standard deviation; setting a detection threshold value based on the noise standard deviation, and performing sliding correlation on the net signal and the S-mode lead pulse reference template to capture the lead pulse; taking the capturing moment as a starting point, intercepting a preset-frame-length signal sequence covering the S-mode leading pulse and the data segment from the net signal, and distinguishing an S-mode complete frame and an A / C-mode short pulse string through dual-threshold verification; and performing coherent accumulation on the I-path and Q-path sampling points in the complete frame according to a complex number form to obtain a coherent accumulation result, and calculating the average power of the S-mode low-power signal according to the coherent accumulation result. Accurate measurement of an extremely weak S-mode response signal can be realized under a noise background.

The invention relates to a question and answer processing method for a data chart, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: in response to an optimization request for a first answer result, displaying an optimization interface; the first answer result is generated based on the first natural language question; the optimization interface comprises a first data screening condition of the data chart obtained by analyzing a first natural language question; responding to an editing operation aiming at the first data screening condition in the optimization interface; editing the first data screening condition according to the editing operation to obtain a second data screening condition; according to a second data screening condition, constructing and displaying a second natural language question; and in response to a confirmation operation for the second natural language question, displaying a second answer result corresponding to the second natural language question. By adopting the method, the question and answer processing efficiency and accuracy can be improved.

The invention relates to a multi-agent-driven remote sensing image intelligent processing algorithm autonomous training verification system. The system comprises a task input and large model agent planning module, a data screening and preprocessing module, an algorithm model screening and super parameter configuration module, a model training and testing module and a comparative analysis and report generation module. According to the method, the whole process of data preparation, algorithm matching, parameter configuration, parallel training, test verification and report generation can be intelligently completed through multi-agent cooperative work, manual intervention is remarkably reduced, the model research and development efficiency and quality are improved, and it is ensured that the process is traceable and the result is reproducible.

Eye tracking is performed by determining an initial pupil position of a user in relation to a lens situated in front of the user, detecting a change in pupil position in relation to the lens to an updated pupil position in relation to the lens, and determining that the updated pupil position is outside a bounding box associated with the lens. The updated pupil position is a replacement pupil position with a replacement pupil position within the bounding box associated with the lens, and the updated pupil position is utilized for eye-tracking functionality. Eye tracking is also performed by determining that a first pixel associated with a gaze direction is outside a visibility region, identifying a replacement pixel within the visibility region, determining an updated gaze angle based on the replacement pixel, and performing eye tracking using the updated gaze angle.

The invention relates to the technical field of big data processing, and discloses a cloud computing and edge computing-based big data co-processing method and a cloud server, which are applied to a co-processing system consisting of an edge node cluster and the cloud server. And obtaining target data through a preprocessing strategy consisting of data filtering, format conversion, increment extraction and lightweight compression. According to the method, the compression ratio larger than or equal to 62% is achieved in combination with the improved LZ4 algorithm, only differential data with service value is transmitted, the invalid data transmission amount is greatly reduced, and the network bandwidth occupation pressure is remarkably reduced; and a dynamic task scheduling link, which is fused with a genetic-reinforcement learning hybrid algorithm of bidirectional LSTM load prediction, can accurately predict edge node computing power fluctuation, dynamically divide task types, match optimal processing nodes, control real-time task response delay within a reasonable range, and realize maximization of computing power resource utilization rate at the same time.

The invention provides a data storage method, a road test system and a computer readable storage medium. The method comprises the steps that an acquisition module acquires test data; the processing module filters the test data according to the data filtering model to obtain first filtering data; the cloud server trains a data filtering model according to the first filtering data and the test data to obtain updated model parameters; the processing module filters the test data according to the updated model parameters and the data filtering model to obtain second filtering data; the storage module stores the second filtering data. According to the method, the data storage amount of the test vehicle in the road test process can be reduced, and the data storage efficiency is improved.

This disclosure provides a data processing method, system, apparatus, device, and storage medium, relating to the field of data processing technology, and particularly to the field of autonomous driving technology. The specific implementation scheme is as follows: receiving a data acquisition request for a target service sent by a client; in response to receiving the data acquisition request, obtaining a target data matching model set for the target service; performing data filtering processing on tagged takeover data based on the filtering items of the target data matching model; wherein the content represented by the tags belongs to the candidate content of the filtering items; and feeding back the processing result of the data filtering processing to the client. It is evident that this scheme can improve data acquisition efficiency.

The invention belongs to the field of intelligent security and protection, and relates to an intelligent decision support method for frontier public opinion and multi-source data fusion, which comprises the following steps: S1, collecting preset type public opinion data and preset type basic multi-source data related to frontier protection, and removing public opinion data and multi-source data irrelevant to frontier protection by adopting a data filtering algorithm to obtain a data filtering algorithm; primary data screening is completed; s2, performing semantic and spatial dimension alignment on the primarily screened public opinion data and multi-source data, and cleaning the aligned data to process data conflicts to obtain an associated fusion data set; s3, extracting a corresponding data resolution, collecting frequency class decision demand indexes, and performing priority ranking on the demand indexes; s4, integrating the scheduled data with the public opinion data to generate a decision support report; and S5, collecting application effect feedback of the decision support report, and optimizing data acquisition key points and fusion parameters in the associated fusion data set based on the feedback.

The application discloses a ground penetrating radar three-dimensional imaging display method and radar. The method comprises the following steps: making the antenna move at a constant speed in a straight line, and transmitting a pulse to the ground at each distance node; receiving the pulse reflected by the ground features to form a return signal; weighting and averaging a plurality of return signals of the same ground feature at the same distance node to form a new return signal; arranging the new return signal in time sequence to form a radar scanning data matrix; performing Fourier transform on the radar scanning data matrix to obtain a frequency spectrum; performing frequency domain filtering processing on the frequency spectrum; performing inverse Fourier transform on the filtered frequency spectrum to obtain a processed radar scanning data matrix; and synthesizing the processed radar scanning data matrix to obtain a three-dimensional display of the ground features, so that the return signal corresponding to the ground features is more prominent, the calculation amount of data filtering is greatly reduced, and the speed and accuracy of the three-dimensional imaging display are improved.

The application discloses a privacy set intersection method based on homomorphic encryption and an optimized filter, a readable medium and an electronic device; wherein the method comprises: in the initialization stage: the sender side and the receiver side jointly negotiate a hash function and generate a semi-homomorphic encryption key pair, and simultaneously create a filter system and a hash function group; in the data interaction stage: based on the hash function and the filter system, data filtering of a private set of the sender and a private set of the receiver is realized, a set Y' is generated at the receiver side, and a set X' is generated at the sender side; in the hash replacement sorting stage, a comparable hash table is generated based on the sets Y' and X', and a replacement data set is generated at the receiver side; in the set segmentation and polynomial generation stage: the problem is converted into a mathematical polynomial solution; in the polynomial calculation and intersection solution stage, the intersection is calculated through homomorphic encryption security. Through the method, the intersection of two private sets can be efficiently and securely calculated.

The invention provides a SaaS-based multi-party system data pushing method and system and a storage medium, and the method comprises the steps: obtaining a differentiated pushing rule pre-configured by a target tenant, the differentiated pushing rule comprising a field selection rule for a target interface, a data filtering rule and a data format conversion rule; obtaining original data to be sent to a target tenant, and performing field screening on the original data based on a field selection rule to obtain first intermediate data; performing content filtering on the first intermediate data based on a data filtering rule to obtain second intermediate data; performing format conversion on the second intermediate data based on a data format conversion rule to obtain to-be-transmitted data meeting the customized format requirement of the target tenant; and sending the to-be-transmitted data to a receiving system corresponding to the target tenant. According to the invention, multi-party system data pushing can be carried out safely and efficiently at low cost.

The invention relates to the technical field of data processing, and discloses a copper product full life cycle-oriented carbon footprint big data warehouse table design method, which comprises the following steps of: carrying out abnormal data filtering on an original carbon footprint data set of a copper product full life cycle to obtain standardized carbon footprint data; based on the standardized carbon footprint data, performing time series data storage optimization on the table structure of the big data warehouse to obtain a dynamic partition table structure; according to the standardized carbon footprint data, performing data mapping on the dynamic partition table structure to obtain an initial data table; traceability information embedding is carried out on the initial data table, and a carbon footprint big data warehouse table is obtained; performing storage structure reconstruction on the carbon footprint big data warehouse table to obtain an efficient storage data warehouse; carrying out life cycle carbon emission analysis on the efficient storage data warehouse to obtain a structured carbon footprint analysis report; according to the invention, the efficiency of carbon footprint data processing can be improved.

The invention provides an aircraft dynamic derivative test device in a strong interference environment, which relates to the technical field of aircraft aerodynamic characteristic tests and comprises an interference flow field establishing device, a dynamic derivative measuring device and a synchronous control module. The disturbing flow field establishing device is composed of a shock wave generator, a supporting connecting piece and a wind tunnel attack angle mechanism, the shock wave generator is connected with the wind tunnel attack angle mechanism through the supporting connecting piece, + / -15-degree pitching and + / -10-degree yawing motion is achieved based on the wind tunnel attack angle mechanism, and multiple types of strong disturbing flow fields can be established by replacing the shock wave generator; the dynamic derivative measuring device is installed in a plug-in mechanism of a wind tunnel, based on a forced vibration test method, a model is driven by a high-precision servo motor to vibrate slightly, precise acquisition is realized by matching with a high-frequency dynamic balance and a data filtering unit, and the displacement of the model relative to a shock wave generator can be adjusted through the plug-in mechanism. According to the scheme, a real strong interference environment can be accurately reproduced, and the dynamic derivative measurement accuracy is improved.

The application provides a liquid level measurement method and device, a storage medium and a computer device. The method comprises the following steps: collecting a first liquid level data set of a liquid level object to be detected by a plurality of liquid level sensors; performing outlier filtering on the first liquid level data set to obtain a second liquid level data set from which abnormal values are removed; obtaining a highest frequency liquid level value, a liquid level median value and a liquid level average value from the second liquid level data set; and obtaining an actual liquid level value of the liquid level object to be detected according to the highest frequency liquid level value, the liquid level median value, the liquid level average value and a preset liquid level value judgment mode. The application can accurately measure the liquid level value of the liquid level object to be detected in a dynamic state.

This invention discloses a device for measuring the roll diameter of an uncoiler head based on a laser rangefinder, belonging to the field of roll diameter measurement technology. The device includes an uncoiler, a lifting machine, an uncoil assembly, an automatic cleaning and detection assembly, and a roll diameter measurement and control system. The detection assembly integrates a laser rangefinder sensor and a blowpipe. The roll diameter measurement and control system includes a data acquisition module, a data analysis and processing module, and an execution control module. The data acquisition module completes multi-source data acquisition and analog-to-digital conversion. The data analysis and processing module performs data filtering, roll diameter calculation, and sensor contamination determination. The execution control module triggers automatic blowpipe cleaning, adjusts the tension and speed of the uncoiler in conjunction with the uncoiler, and coordinates the lifting machine's movement. The blowpipe can perform graded cleaning according to the contamination level. This device eliminates dust interference with detection, ensures stable roll diameter measurement data, and achieves automatic cleaning of the laser rangefinder sensor, constant tension closed-loop control of the uncoiler, and coordinated operation of the equipment.

This invention relates to the field of power edge computing technology, specifically to a data filtering and transmission method and system based on an edge gateway. The method includes the following steps: generating an original power quantity sequence; constructing a dynamic phase filtering matrix; performing projection comparison on the mapping matrix to accurately separate and extract intrinsic feature components; calculating degradation trend values ​​and determining dynamic transmission priorities; and performing communication scheduling and distribution on the feature components, with high-priority components sent immediately and low-priority components merged after delay. In this invention, by performing multi-dimensional projection comparison on high-frequency sampling points, components reflecting the actual operating status of power distribution equipment are accurately extracted, false mutations are eliminated, and real faults are retained. Transmission priorities are calculated by combining channel congestion and degradation trend values. Status information is classified and graded for scheduling, transmission, and merging, completely eliminating the omission of critical early warning information, reducing the occupation of limited network bandwidth resources by redundant and invalid messages, and ensuring long-term stable operation of cross-level communication.

The invention discloses a camping security method and system, terminal equipment and a storage medium, and belongs to the field of camping security, and the method comprises the steps: obtaining a candidate monitoring radar data set of a camping security monitoring area; determining target monitoring radar data from the candidate monitoring radar data set; determining target monitoring camera data based on the target monitoring radar data and the radar-camera mapping relation; obtaining a monitoring radar measurement value based on a radar conversion algorithm and the target monitoring radar data; obtaining a monitoring camera measurement value based on a camera conversion algorithm and the target monitoring camera data; obtaining a joint monitoring observation value based on the monitoring radar measurement value and the monitoring camera measurement value; obtaining target fusion pose data based on a monitoring data filtering updating algorithm and the combined monitoring observation value; and obtaining an environmental risk score of the camping security monitoring area based on the target fusion pose data and an environmental risk scoring algorithm, and carrying out alarm processing based on the environmental risk score. Therefore, the camping security reliability is improved.

The invention relates to a big data information collection method. The method comprises the following steps: collecting retrieval data, browsing data, consulting data and auxiliary data of multiple users in an application system; invalid data removal and noise data filtering are carried out on the collected data, invalid data removal comprises null value and meaningless data removal, duplicate data de-duplication and format standardization processing, and noise data filtering comprises regular filtering and statistical filtering; and constructing an invalid and noise data judgment model based on the processed valid data and invalid and noise data. According to the method, through data preprocessing and double screening of invalid and noise data judgment models, invalid data and noise data can be effectively eliminated, so that the accuracy of data information collection is improved; through automatic judgment of invalid and noise data judgment models and an incremental updating mechanism, the models are continuously optimized along with data accumulation, the manual screening workload of data information collection can be reduced, and the misjudgment rate of automatic judgment is reduced.

The application discloses a kind of class imbalance table data processing methods based on contrast constraint diffusion generation, belong to data processing technical field, it includes using binary classification table dataset to generate model training, obtain the generation model based on noise generation minority class's synthetic latent representation, generate model is randomly generated with several noise inputs, diffusion modeling branch is denoised to each noise and obtains the synthetic latent representation of minority class of each noise, using decoding module is decoded to each synthetic latent representation, obtains synthetic table sample;Data filtering is carried out to all synthetic table samples, then all synthetic table samples after filtering are merged with table dataset, and enhanced imbalance table data are obtained.The application introduces contrast learning branch in the process of generating minority class sample, effectively overcome the deficiency existing in class imbalance table data processing of existing oversampling method and general generation model.

The invention discloses a query method, device and equipment and a computer readable storage medium, and is applied to the technical field of computers, and the query method comprises the following steps: generating a query page according to configuration information of a structured query template; the structured query template comprises a query condition placeholder and a query result field definition; generating a scheduling task according to the configuration information of the data processing task parameters; the data processing task parameters comprise basic configuration parameters, scheduling configuration parameters, data filtering parameters and script configuration parameters; when the scheduling task is triggered, starting a data processing task to obtain a physical table; the physical table comprises normalized processing results; and when a statistical query triggered based on the query page is received, performing data query in the physical table to obtain a query result. According to the method, the development threshold is reduced, non-technical personnel can participate, and the function delivery period is remarkably shortened; the large-scale data query can be met, the real-time calculation bottleneck can be avoided, and the query efficiency is greatly improved.

The invention relates to a data screening method and device based on hardware processing capacity and a medium. The invention provides a data screening method based on hardware processing capability. The data screening method comprises the following steps: acquiring N pieces of data to be processed and a target quantity K from a large language model; dividing the N data into G data groups based on the hardware processing capability, the hardware processing capability comprising a maximum number M of data that can be processed in parallel by a hardware computing unit; screening out candidate data from each data group by using the hardware computing unit based on a preset group screening ratio P, and merging the candidate data of all the data groups into a candidate data set; and screening the target number K of target data from the candidate data set by using the hardware computing unit.