172 results about "Data reduction" patented technology

The invention provides a privacy protection federated distillation and backdoor defense method for large model fine tuning, and belongs to the technical field of artificial intelligence security and federated learning, and the method comprises the steps: 1, carrying out the distillation and core representation extraction of a data set based on contribution degree weighted federated pre-training and local neural feature function matching; step 2, self-adaptive noise back door defense processing based on multi-feature fusion; according to the method, a dataset distillation mechanism based on neural feature function matching and a self-adaptive noise defense strategy are adopted, so that effective balance of the large model among data simplification, privacy protection and backdoor defense robustness is realized; the method is of great significance in improving the safety and reliability of an artificial intelligence system in a distributed environment.

A system and method for adaptive data processing combining compression and encryption. The system analyzes input data characteristics, compares probability distributions, and creates a transformation matrix to convert data into a dyadic distribution. It generates a main data stream of transformed data and a secondary stream of transformation information. The system dynamically selects and applies processing techniques, including transformation, encoding, compression, and encryption algorithms, based on analyzed characteristics and real-time performance metrics. It compresses the main data stream using Huffman coding and implements security measures to protect the output. A feedback loop monitors technique effectiveness, updates a knowledge base, and influences future selections. The system can operate in lossless, lossy, or modified lossless modes, adapting to different application requirements. This approach offers an efficient solution for scenarios where both data reduction and security are critical concerns.

An adaptive transformation matrix generation is used simultaneous compression and encryption of data. The system continuously monitors input data streams to detect changes in distribution patterns, dynamically updating transformation matrices in response to these changes. Using performance evaluation criteria, the system selects and deploys optimal matrices that maintain both compression efficiency and cryptographic security as data characteristics evolve. The system implements configurable adaptation policies that govern when and how matrices are updated, ensuring system stability while maximizing performance. By employing sliding window analysis and distribution variance metrics, the system can quantify data drift and trigger appropriate adaptations. The system maintains compatibility with various operational modes including lossless, lossy, and modified lossless configurations, applying mode-specific optimization criteria to each scenario. This adaptive approach offers a resilient solution for data transmission and storage scenarios where both data reduction and security must be maintained across diverse and evolving data streams.

The invention discloses a power distribution cabinet remote monitoring system based on wireless communication, and relates to the technical field of remote monitoring, and the system comprises a communication scheduling module which is used for receiving an uploaded data packet, carrying out link evaluation through an uploading link optimization method in combination with a real-time network condition, and carrying out link optimization according to an actual link condition and the priority of the uploaded data; selecting an initial uploading link and uploading data; the Bluetooth self-healing module is used for evaluating the link quality according to the link quality data in a data uploading process, dynamically switching an active link and a standby link according to a link quality evaluation result, and carrying out Bluetooth task transmission; the remote control module is used for carrying out data reduction by adopting a multilayer decryption mechanism and a mixed format analysis mechanism based on the encrypted Bluetooth task data, evaluating the operation state of the power distribution cabinet, generating a control instruction through a fuzzy logic control method according to an operation state evaluation result and executing the control instruction; through a dynamic link switching mechanism of the Bluetooth self-healing module, continuous and stable transmission of monitoring data of the power distribution cabinet is ensured.

The invention discloses a cyberspace advanced long-term threat (APT) trapping method based on a graph neural network, a storage medium and a cyberspace advanced long-term threat trapping device, and relates to the technical field of computers, and the method comprises the steps: carrying out the dimension reduction of original encrypted traffic data based on a hierarchical clustering algorithm, retaining an abnormal traffic cluster, and removing normal traffic redundant data; designing an E-ResGAT model to perform encrypted semantic modeling, and introducing a residual connection technology to retain original traffic features; and a dynamic trapping-protocol inference closed-loop system is further constructed, and a honeypot environment is utilized to simulate attack sample interaction behaviors and capture traffic. According to the scheme, efficiency balance of encrypted traffic data reduction and semantic reasoning is realized through collaborative optimization of hierarchical clustering and a graph neural network, the capture capability of the model on a hidden attack mode is enhanced in combination with a residual attention mechanism, and a multi-dimensional analysis basis is provided for high-security scenes such as a military private network; and an innovative solution is provided for improving the security defense capability of a military information system.

A method for greatly reducing data storage requirements of autonomous robots capable of product inventory is described. Instead using high resolution images for label identification and position tracking, a lower resolution map can be searched. Human or machines can identify position of labels, and via a reverse mapping, the corresponding position of a label on the high resolution image can be identified. Except for those portions of the high resolution image showing a label, most of the high resolution image can be discarded. Further processing on the limited image subset can be used to read the labels.

The invention discloses a big data advertisement label classification system based on AI analysis, particularly relates to the field of advertisement data processing and label intelligent classification, and is used for solving the problems of interest identification distortion and released label drift caused by coarse granularity of an end-side theme abstract. Judging the combination credibility according to two parameters of a curvature amplitude index and a path stability potential so as to remodel a pseudo-continuous trajectory, and deducing a fine grain interest label under mixed attention modulation; according to the overall architecture, information faults after abstract compression are broken through, audience portraits are close to real-time interests and do not drift along with data reduction, a delivery end bids accurately according to the audience portraits, repeated correction around distortion labels is not needed any more, budget waste is greatly reduced, and exposure frequency distribution tends to be balanced; the progressive resonance graph and the label tree mapping library are kept synchronous, the real-time bidding engine always anchors the latest interest coordinate, the advertisement experience meets the user demand, and the brand security risk is inhibited along with the decline of the false bidding.

The invention provides a Bluetooth scanning method and related equipment. The method can be applied to an intelligent interconnection scene. After broadcast data is obtained through a Bluetooth chip configured with a filter, filtering configuration information matched with the broadcast data is determined, and the filtering configuration information comprises address information of a callback function used for processing the broadcast data. And transmitting the broadcast data to a callback function according to the address information. Therefore, the callback function matched with the broadcast data can be accurately used for processing the broadcast data, the processing flow is simplified, the efficiency is improved, and the power consumption is reduced. In some embodiments, the broadcast data is subjected to de-duplication processing through the hard filtering module, and the broadcast data is transmitted to the callback function after being determined to be not repeated broadcast, so that the computing power and the power consumption are saved. In some embodiments, after all the broadcast data needing to be scanned by the operating system are transmitted to the corresponding callback function, scanning of the operating system is stopped, the number of broadcasts scanned by the operating system can be reduced as much as possible, and computing power and power consumption are saved.

A data processing method is provided. The implementation is: for each of a plurality of data blocks included in data to be processed, dividing the data block into a plurality of first sub-blocks corresponding to a plurality of computing unit groups respectively; for each first sub-block of the plurality of first sub-blocks, dividing the first sub-block, based on a target computing unit group corresponding to the first sub-block, into a plurality of second sub-blocks corresponding to the plurality of computing units in the target computing unit group, respectively; determining, by processing each second sub-block utilizing the corresponding computing unit of the second sub-block, a plurality of first processing results output by the plurality of computing units respectively; and determining a processing result of the first sub-block by performing a data reduction operation on the plurality of first processing results utilizing the target computing unit group.

A system and method for cross-stream asymmetric enhancement combines machine learning-driven asymmetric codebook generation with dyadic distribution algorithms to enable simultaneous optimization of compression efficiency, cryptographic security, and error correction capability. The system analyzes input data characteristics and initializes multiple specialized ML models to generate stream-specific asymmetric codebooks optimized for different objectives. Enhanced dyadic distribution processing creates three pre-conditioned data streams that are processed through parallel asymmetric transformation pipelines: compression-optimized for maximum data reduction, security-optimized for cryptographic strength, and error-correction-optimized for robust recovery capability. Cross-stream optimization coordinates the multiple processing paths to ensure overall system coherence while maintaining individual stream objectives. The system supports multiple operating modes including ultra-high compression using only the primary stream, broadcast quality using primary and secondary streams, and archival mode using all streams for lossless reconstruction. The system supports graduated access control that enables different reconstruction quality levels based on available stream combinations.

The invention relates to the technical field of space flight and aviation, in particular to a low-delay communication control system and method for an analog machine in a fixed flight training course. The control system comprises a multi-source sensor fusion module, a data simplification processing module, a dynamic time delay optimization communication framework, a distributed intelligent decision-making terminal, a virtual-real mapping synchronous controller, a milli delay regulation and control module and an intelligent sensing alarm interaction module. According to the invention, 4G, 5G and satellite double links are integrated, an optimal link is dynamically selected by adopting a Q-Learning algorithm, millisecond-level time delay control is realized, transmission reliability is improved through a packet loss rate compensation algorithm, multi-sensor data is fused based on an FPGA hardware accelerator and an EKF algorithm, and a data compression ratio is improved in combination with Huffman coding and Kalman filtering. The bandwidth requirement is greatly reduced, through software and hardware collaborative optimization, the end-to-end time delay is reduced to be within hundreds of milliseconds, the virtual-real synchronization precision is improved to the centimeter level, and the decision response speed is improved by two times.

The present application belongs to the technical field of communications. Disclosed are a data reduction method and apparatus, and a device and a computer-readable storage medium. The method is applied to at least one root node in a multi-dimensional direct-connect topology, wherein nodes in the multi-dimensional direct-connect topology are used for model training. The method comprises: receiving first data, wherein the first data is part of transmitted data, which is data transmitted between nodes in a model training process; obtaining second data, wherein the second data is part of first generated data, which is data generated by a root node in the model training process; performing reduction on the first data and the second data to obtain third data, wherein the third data is used by nodes in a multi-dimensional direct-connect topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct-connect topology, wherein the first node is a node in the multi-dimensional direct-connect topology other than the root node. The method can improve the communication efficiency in a data reduction process.

The invention discloses a column storage-based big data compression and query optimization method, which relates to the technical field of big data compression and query optimization, and comprises the following steps of: performing sliding window division on original data of column storage to obtain a plurality of data windows; calculating a feature vector based on the value distribution characteristics in the multi-data window, wherein the feature vector is used for representing a local statistical rule of the data; performing dynamic merging according to the similarity of the feature vectors of the adjacent data windows to form a compressed block; selecting a corresponding compression coding mode based on the data type and the distribution characteristic of the compression block, and generating a metadata index containing a minimum value, a maximum value, a coding type and a compression size; after receiving a query request, mapping a predicate condition to a compressed coding space, and executing non-decompressed filtering judgment on a compressed block by utilizing the metadata index and a compressed coding type; and determining a decompression range based on a filtering result of the compressed blocks, and only performing local decompression and data reduction on the possibly hit compressed blocks.

The embodiment of the invention provides a big data storage optimization method based on a zero-copy collaboration technology and related equipment, and belongs to the technical field of big data storage. The method comprises a zero-copy data transmission module used for directly writing a data stream into a front-end buffer area of an annular structure through a direct memory access technology, and adopting a zero-copy algorithm based on pointer offset to parallelly separate data of each channel in a multi-thread environment; the intelligent data reduction module is used for deleting duplicated data from the data and then compressing the duplicated data; and the hybrid storage management module is used for managing the hybrid partition storage architecture and dynamically adjusting the distribution of the data in the hybrid partition storage architecture according to the data access mode. The CPU copy frequency is reduced through the zero copy technology, the data transmission efficiency is remarkably improved by combining the data reduction and intelligent layering strategies, the occupied storage space is reduced, and meanwhile the data safety and compliance are guaranteed.

The invention discloses a data reduction method, device and equipment and a computer readable storage medium, and belongs to the technical field of communication. The method is applied to at least one root node in a multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology are used for model training, and the method comprises the steps that first data are received, the first data are partial data of transmission data, and the transmission data are data transmitted among the nodes in the model training process; second data is obtained, the second data is part of first generated data, and the first generated data is data generated by the root node in the model training process; the first data and the second data are reduced to obtain third data, and the third data are used for nodes in the multi-dimensional direct connection topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct connection topology, wherein the first node is a node except the root node in the multi-dimensional direct connection topology. According to the method, the communication efficiency in the data reduction process can be improved.

The present invention relates to a system and a method for measuring and analyzing minimal residual disease (MRD) in pediatric B-cell precursor acute lymphoblastic leukemia (B-ALL) using multiparameter flow cytometry (MPFC). The invention finds application in clinical diagnostics and hematology-oncology for quantifying MRD in B-ALL patients with high sensitivity and specificity, needed for risk stratification, monitoring treatment response, and informing therapeutic decisions. The system comprises interconnected subsystems including an acquisition subsystem with an MPFC instrument, a control and file generation subsystem, and an analytical subsystem. The analytical subsystem incorporates modules for sequential data reduction, automated data cleaning, automated unsupervised data clustering, and interactive cluster analysis. Key advantages include high MRD detection sensitivity (e.g., 10⁻⁵ or 0.001%) and high specificity, without reliance on reference samples or supervised machine learning models, making it applicable in laboratories with different measuring equipment and using different panels of antibodies for identification of leukemic cells.

The invention is suitable for the technical field of flight height prediction, and particularly relates to a flight height prediction method based on a neural network, and the method comprises the steps: collecting sensor data of a passenger plane, and inhibiting outlier data points through a dynamic truncation mechanism based on a distribution quantile threshold; dividing the sensor data by adopting a time sequence slice division strategy to obtain a training set and a verification set, and constructing a prediction feature set based on the training set and the verification set; and constructing a flight height prediction model, training the flight height prediction model through the prediction feature set, and predicting the flight height based on the trained flight height prediction model. According to the invention, a normalization technology is applied to carry out data reduction on a flight time sequence signal, a parallel network structure comprising a multi-scale convolution kernel feature extraction layer and an attention weight distribution module is designed, and gradient iteration is carried out on a Huber loss function through an optimizer for dynamically adjusting a learning rate. And the characterization capability of the model on the nonlinear change mode of the flight state is obviously improved.

The invention relates to the technical field of data storage management, in particular to a method and system for improving data storage and processing efficiency based on a large model, and the method comprises the following steps: constructing a distributed storage system, introducing a heterogeneous storage management mechanism, and adopting a data reduction technology; the method has the beneficial effects that intelligent compression and duplicate removal are performed on storage data through a large model, and the storage space requirement is remarkably reduced; secondly, analyzing a data access mode by using a large model, realizing dynamic data-tiering and life cycle management, and optimizing storage resource allocation; in the aspect of data processing, the large model is used for constructing an efficient data index system, and the data retrieval and analysis process is accelerated; meanwhile, data preprocessing and feature extraction are carried out through a large model, and the performance of big data analysis and machine learning tasks is improved.

A method for providing process data of a device in an industrial automation environment to a computer system. In one embodiment, the method includes the following steps: executing a process data model on the device for generating estimated process data; determining that the estimated process data deviates from the real process data by more than a threshold value; and only if the estimated process data deviates from the real process data by more than the threshold value: transmitting information representing the real process data from the device to the computer system.

A system and method for adaptive data processing combining compression and encryption. The system analyzes input data characteristics, compares probability distributions, and creates a transformation matrix to convert data into a dyadic distribution. It generates a main data stream of transformed data and a secondary stream of transformation information. The system dynamically selects and applies processing techniques, including transformation, encoding, compression, and encryption algorithms, based on analyzed characteristics and real-time performance metrics. It compresses the main data stream using Huffman coding and implements security measures to protect the output. A feedback loop monitors technique effectiveness, updates a knowledge base, and influences future selections. The system can operate in lossless, lossy, or modified lossless modes, adapting to different application requirements. This approach offers an efficient solution for scenarios where both data reduction and security are critical concerns.

The invention discloses a tunnel face integrity evaluation method based on deep learning, and the method comprises the steps: firstly scanning a tunnel face through a laser radar carried on an unmanned plane, obtaining a point cloud file of the tunnel face, and carrying out the data registration, data noise reduction, data simplification and three-dimensional modeling of the point cloud file through a data processing technology; the method comprises the following steps: performing tunnel face image extraction and image preprocessing on a three-dimensional model obtained after data processing, performing crack identification through a YOLOv5 convolutional neural network, and performing enhancement and structure optimization on crack image features through operations such as image segmentation, image noise reduction, crack skeleton extraction, edge detection, crack skeleton line splicing and the like. And information such as crack width and crack length is obtained through the processed image. The face fracture density and the ratio of open, micro-open and closed fractures are calculated through the information, the face fracture density is used for distinguishing the completeness and local stability of the tunnel face rock mass, and the completeness and local stability of the tunnel face are evaluated more comprehensively through the ratio of the three fractures.

The invention belongs to the technical field of data reduction, and discloses a delta compression method and system for flexible feature matching, and the method comprises the steps: calculating a plurality of independent features for partitioned data; each feature is stored in an independent hash table; based on the characteristic that no correlation exists between the characteristics, a flexible characteristic matching scheme is used for randomly combining the characteristics to realize similarity detection; evaluating the benefit of executing delta compression coding on the similar data obtained through detection so as to ensure that relatively high reduction benefit is finally obtained; and performing data reduction through delta compression coding. According to the method, the problem that the features cannot be fully utilized in the existing similarity detection is solved, and the delta compression income is evaluated by utilizing the features. The method has extremely high flexibility, a plurality of parameters dynamically changing according to the characteristics of the data sets are set, and the effects of effectively working on different types of data sets and adjusting characteristic storage distribution and balancing overhead according to the current compression characteristics are achieved.

The application discloses a compressor blade phosphate aluminum coating precision grinding and polishing path generation method and system based on point cloud data processing, which comprises the following steps: obtaining point cloud data of the coating surface of the compressor blade by using a three-dimensional scanning device, filtering and denoising the point cloud data, and pre-processing the data, and aligning the pre-processed point cloud data with a robot machining reference coordinate system; based on the local curvature of the point cloud, the point cloud model is divided into multiple layers of contour slices by using a curvature adaptive slicing algorithm, wherein the slicing interval is negatively correlated with the local curvature radius; the contour points of each layer of slices are B-spline fitted to generate a continuous grinding and polishing path curve; the robot executes the grinding and polishing path curve, and integrates a real-time thickness detection device; when the thickness measurement value deviates from a preset threshold value, closed-loop feedback control is performed, and the grinding and polishing path and / or grinding and polishing parameters are dynamically compensated. The application has the advantages of high machining precision, high efficiency, high consistency and the like.

The invention discloses a converter tapping curve presetting method and device based on data mining. The converter tapping curve presetting method comprises the steps that converter tapping process data are collected; performing data preprocessing through data cleaning, data specification, data integration and data conversion methods, and establishing a sample set; randomly dividing a sample set into a training set, a verification set and a test set through random sampling; selecting a data mining method, establishing a data mining model, and training the data mining model by using a training set, so that the data mining model outputs a converter tapping curve according to the input converter tapping process data; verifying the data mining model by using a verification set; evaluating the data mining model using a test set; and outputting a converter tapping curve through the qualified data mining model, and issuing the converter tapping curve to a converter tilting mechanism to control converter tapping. According to the method, the converter tapping curve is intelligently preset in a data mining mode, tapping slag and molten steel rephosphorization caused by unreasonable tapping curve are reduced, and the automatic tapping effect of the converter is improved.

Devices, systems, methods, and processes for in-network computing using modular switch architecture are described herein. Endpoint devices generate data chunks and forward them to a network, comprising spine and leaf switches, for data reduction. Leaf switches act as conduits and forward the data chunks to a spine switch. The spine switch includes various line cards (e.g., one for each leaf switch) and a fabric element. The line cards may execute a stage of data reduction on the received data chunks or may forward the received data chunks directly to the fabric element. The fabric element executes a data reduction operation on the data received from the line cards and obtains a reduced output which is forwarded to the endpoint devices via the line cards and the leaf switches. Thus, a single-tier in-network computing topology is implemented to execute data reduction in a cost-effective, simple, and efficient manner.

A method of conducting operations for an IPG adapted to provide electrical pulses to tissue of a patient for medical therapy is provided. The IPG provides the medical therapy according to pulse parameter data executed according to an execution path. The method can include identifying a root node repeated within the execution path of a first set of the pulse parameter data, identifying a stimulation pulse pattern repeated in the first set between occurrences of the root node, replacing the repeated pattern with (a) a single instance and a recurrence count or (b) a pointer to a reference pattern to generate a second set of the pulse parameter data, and communicating the second set between the external controller and the IPG and / or storing the second set within a memory of the IPG.

The invention discloses a data storage method and device, equipment and a storage medium, and relates to the field of data security, and the method comprises the steps: obtaining initial target data from a preset database based on a target data query request of a user side; desensitizing the initial target data by using corresponding preset reversible desensitization execution logic according to a preset desensitization strategy to obtain desensitized target data; the desensitized target data is sent to the corresponding user side, so that the user side edits the desensitized target data, and edited desensitized data is obtained; and performing sensitive data reduction processing on the edited desensitized data by using the preset reversible desensitization execution logic to obtain reduced target data, and storing the reduced target data to the preset database. Therefore, the desensitization echo of the data can be realized, and the desensitization data can be prevented from covering the original data.

The invention discloses a container real-time migration method and system and a medium. The method comprises the steps that a deduplication method based on content definition partitioning is introduced into container real-time migration, highly redundant memory pages are effectively detected and removed, and an optimal data reduction algorithm is dynamically selected to improve migration performance by means of an adaptive data reduction technology. Experimental results show that the total migration time can be reduced by 75%, and stable performance can be kept under different working load and bandwidth conditions.

The invention provides a method for data reduction in the context of spectral CT imaging. The method comprises reducing the number of spectral channels of the acquired CT data by a reduced set of synthetic energy channels, each from a weighted combination of the original measured energy channel. The weights are selected in an optimization process in which an error metric associated with an output of a material decomposition procedure to be applied to the CT data is estimated, and the weights adjusted to minimize a value of the error metric. The error might for example be a noise estimate, and / or a bias estimate.

The invention relates to the technical field of cloud computing and data processing, and discloses an efficient data processing method and system based on cloud computing, and compared with a traditional method in which data quality is simply reduced or frequent retransmission is carried out, intelligent processing and data simplification are carried out locally, so that the transmission data volume is remarkably reduced, and the data transmission efficiency is improved. The real-time performance and the accuracy of the key information under the limited bandwidth are ensured, the communication congestion and the excessive consumption of airborne energy are avoided, the execution efficiency of the whole task and the cloud decision support capability are greatly improved, and the method and the system have remarkable advantages especially in emergency response and other dispute scenes.