124 results about "Data reduction" patented technology

The invention provides a privacy protection federated distillation and backdoor defense method for large model fine tuning, and belongs to the technical field of artificial intelligence security and federated learning, and the method comprises the steps: 1, carrying out the distillation and core representation extraction of a data set based on contribution degree weighted federated pre-training and local neural feature function matching; step 2, self-adaptive noise back door defense processing based on multi-feature fusion; according to the method, a dataset distillation mechanism based on neural feature function matching and a self-adaptive noise defense strategy are adopted, so that effective balance of the large model among data simplification, privacy protection and backdoor defense robustness is realized; the method is of great significance in improving the safety and reliability of an artificial intelligence system in a distributed environment.

A system and method for adaptive data processing combining compression and encryption. The system analyzes input data characteristics, compares probability distributions, and creates a transformation matrix to convert data into a dyadic distribution. It generates a main data stream of transformed data and a secondary stream of transformation information. The system dynamically selects and applies processing techniques, including transformation, encoding, compression, and encryption algorithms, based on analyzed characteristics and real-time performance metrics. It compresses the main data stream using Huffman coding and implements security measures to protect the output. A feedback loop monitors technique effectiveness, updates a knowledge base, and influences future selections. The system can operate in lossless, lossy, or modified lossless modes, adapting to different application requirements. This approach offers an efficient solution for scenarios where both data reduction and security are critical concerns.

A method for greatly reducing data storage requirements of autonomous robots capable of product inventory is described. Instead using high resolution images for label identification and position tracking, a lower resolution map can be searched. Human or machines can identify position of labels, and via a reverse mapping, the corresponding position of a label on the high resolution image can be identified. Except for those portions of the high resolution image showing a label, most of the high resolution image can be discarded. Further processing on the limited image subset can be used to read the labels.

The invention discloses a big data advertisement label classification system based on AI analysis, particularly relates to the field of advertisement data processing and label intelligent classification, and is used for solving the problems of interest identification distortion and released label drift caused by coarse granularity of an end-side theme abstract. Judging the combination credibility according to two parameters of a curvature amplitude index and a path stability potential so as to remodel a pseudo-continuous trajectory, and deducing a fine grain interest label under mixed attention modulation; according to the overall architecture, information faults after abstract compression are broken through, audience portraits are close to real-time interests and do not drift along with data reduction, a delivery end bids accurately according to the audience portraits, repeated correction around distortion labels is not needed any more, budget waste is greatly reduced, and exposure frequency distribution tends to be balanced; the progressive resonance graph and the label tree mapping library are kept synchronous, the real-time bidding engine always anchors the latest interest coordinate, the advertisement experience meets the user demand, and the brand security risk is inhibited along with the decline of the false bidding.

A system and method for cross-stream asymmetric enhancement combines machine learning-driven asymmetric codebook generation with dyadic distribution algorithms to enable simultaneous optimization of compression efficiency, cryptographic security, and error correction capability. The system analyzes input data characteristics and initializes multiple specialized ML models to generate stream-specific asymmetric codebooks optimized for different objectives. Enhanced dyadic distribution processing creates three pre-conditioned data streams that are processed through parallel asymmetric transformation pipelines: compression-optimized for maximum data reduction, security-optimized for cryptographic strength, and error-correction-optimized for robust recovery capability. Cross-stream optimization coordinates the multiple processing paths to ensure overall system coherence while maintaining individual stream objectives. The system supports multiple operating modes including ultra-high compression using only the primary stream, broadcast quality using primary and secondary streams, and archival mode using all streams for lossless reconstruction. The system supports graduated access control that enables different reconstruction quality levels based on available stream combinations.

The invention relates to the technical field of space flight and aviation, in particular to a low-delay communication control system and method for an analog machine in a fixed flight training course. The control system comprises a multi-source sensor fusion module, a data simplification processing module, a dynamic time delay optimization communication framework, a distributed intelligent decision-making terminal, a virtual-real mapping synchronous controller, a milli delay regulation and control module and an intelligent sensing alarm interaction module. According to the invention, 4G, 5G and satellite double links are integrated, an optimal link is dynamically selected by adopting a Q-Learning algorithm, millisecond-level time delay control is realized, transmission reliability is improved through a packet loss rate compensation algorithm, multi-sensor data is fused based on an FPGA hardware accelerator and an EKF algorithm, and a data compression ratio is improved in combination with Huffman coding and Kalman filtering. The bandwidth requirement is greatly reduced, through software and hardware collaborative optimization, the end-to-end time delay is reduced to be within hundreds of milliseconds, the virtual-real synchronization precision is improved to the centimeter level, and the decision response speed is improved by two times.

The present application belongs to the technical field of communications. Disclosed are a data reduction method and apparatus, and a device and a computer-readable storage medium. The method is applied to at least one root node in a multi-dimensional direct-connect topology, wherein nodes in the multi-dimensional direct-connect topology are used for model training. The method comprises: receiving first data, wherein the first data is part of transmitted data, which is data transmitted between nodes in a model training process; obtaining second data, wherein the second data is part of first generated data, which is data generated by a root node in the model training process; performing reduction on the first data and the second data to obtain third data, wherein the third data is used by nodes in a multi-dimensional direct-connect topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct-connect topology, wherein the first node is a node in the multi-dimensional direct-connect topology other than the root node. The method can improve the communication efficiency in a data reduction process.

The invention discloses a column storage-based big data compression and query optimization method, which relates to the technical field of big data compression and query optimization, and comprises the following steps of: performing sliding window division on original data of column storage to obtain a plurality of data windows; calculating a feature vector based on the value distribution characteristics in the multi-data window, wherein the feature vector is used for representing a local statistical rule of the data; performing dynamic merging according to the similarity of the feature vectors of the adjacent data windows to form a compressed block; selecting a corresponding compression coding mode based on the data type and the distribution characteristic of the compression block, and generating a metadata index containing a minimum value, a maximum value, a coding type and a compression size; after receiving a query request, mapping a predicate condition to a compressed coding space, and executing non-decompressed filtering judgment on a compressed block by utilizing the metadata index and a compressed coding type; and determining a decompression range based on a filtering result of the compressed blocks, and only performing local decompression and data reduction on the possibly hit compressed blocks.

The embodiment of the invention provides a big data storage optimization method based on a zero-copy collaboration technology and related equipment, and belongs to the technical field of big data storage. The method comprises a zero-copy data transmission module used for directly writing a data stream into a front-end buffer area of an annular structure through a direct memory access technology, and adopting a zero-copy algorithm based on pointer offset to parallelly separate data of each channel in a multi-thread environment; the intelligent data reduction module is used for deleting duplicated data from the data and then compressing the duplicated data; and the hybrid storage management module is used for managing the hybrid partition storage architecture and dynamically adjusting the distribution of the data in the hybrid partition storage architecture according to the data access mode. The CPU copy frequency is reduced through the zero copy technology, the data transmission efficiency is remarkably improved by combining the data reduction and intelligent layering strategies, the occupied storage space is reduced, and meanwhile the data safety and compliance are guaranteed.

The invention discloses a data reduction method, device and equipment and a computer readable storage medium, and belongs to the technical field of communication. The method is applied to at least one root node in a multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology are used for model training, and the method comprises the steps that first data are received, the first data are partial data of transmission data, and the transmission data are data transmitted among the nodes in the model training process; second data is obtained, the second data is part of first generated data, and the first generated data is data generated by the root node in the model training process; the first data and the second data are reduced to obtain third data, and the third data are used for nodes in the multi-dimensional direct connection topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct connection topology, wherein the first node is a node except the root node in the multi-dimensional direct connection topology. According to the method, the communication efficiency in the data reduction process can be improved.

The present invention relates to a system and a method for measuring and analyzing minimal residual disease (MRD) in pediatric B-cell precursor acute lymphoblastic leukemia (B-ALL) using multiparameter flow cytometry (MPFC). The invention finds application in clinical diagnostics and hematology-oncology for quantifying MRD in B-ALL patients with high sensitivity and specificity, needed for risk stratification, monitoring treatment response, and informing therapeutic decisions. The system comprises interconnected subsystems including an acquisition subsystem with an MPFC instrument, a control and file generation subsystem, and an analytical subsystem. The analytical subsystem incorporates modules for sequential data reduction, automated data cleaning, automated unsupervised data clustering, and interactive cluster analysis. Key advantages include high MRD detection sensitivity (e.g., 10⁻⁵ or 0.001%) and high specificity, without reliance on reference samples or supervised machine learning models, making it applicable in laboratories with different measuring equipment and using different panels of antibodies for identification of leukemic cells.

A method for providing process data of a device in an industrial automation environment to a computer system. In one embodiment, the method includes the following steps: executing a process data model on the device for generating estimated process data; determining that the estimated process data deviates from the real process data by more than a threshold value; and only if the estimated process data deviates from the real process data by more than the threshold value: transmitting information representing the real process data from the device to the computer system.

The invention discloses a tunnel face integrity evaluation method based on deep learning, and the method comprises the steps: firstly scanning a tunnel face through a laser radar carried on an unmanned plane, obtaining a point cloud file of the tunnel face, and carrying out the data registration, data noise reduction, data simplification and three-dimensional modeling of the point cloud file through a data processing technology; the method comprises the following steps: performing tunnel face image extraction and image preprocessing on a three-dimensional model obtained after data processing, performing crack identification through a YOLOv5 convolutional neural network, and performing enhancement and structure optimization on crack image features through operations such as image segmentation, image noise reduction, crack skeleton extraction, edge detection, crack skeleton line splicing and the like. And information such as crack width and crack length is obtained through the processed image. The face fracture density and the ratio of open, micro-open and closed fractures are calculated through the information, the face fracture density is used for distinguishing the completeness and local stability of the tunnel face rock mass, and the completeness and local stability of the tunnel face are evaluated more comprehensively through the ratio of the three fractures.

The invention belongs to the technical field of data reduction, and discloses a delta compression method and system for flexible feature matching, and the method comprises the steps: calculating a plurality of independent features for partitioned data; each feature is stored in an independent hash table; based on the characteristic that no correlation exists between the characteristics, a flexible characteristic matching scheme is used for randomly combining the characteristics to realize similarity detection; evaluating the benefit of executing delta compression coding on the similar data obtained through detection so as to ensure that relatively high reduction benefit is finally obtained; and performing data reduction through delta compression coding. According to the method, the problem that the features cannot be fully utilized in the existing similarity detection is solved, and the delta compression income is evaluated by utilizing the features. The method has extremely high flexibility, a plurality of parameters dynamically changing according to the characteristics of the data sets are set, and the effects of effectively working on different types of data sets and adjusting characteristic storage distribution and balancing overhead according to the current compression characteristics are achieved.

The application discloses a compressor blade phosphate aluminum coating precision grinding and polishing path generation method and system based on point cloud data processing, which comprises the following steps: obtaining point cloud data of the coating surface of the compressor blade by using a three-dimensional scanning device, filtering and denoising the point cloud data, and pre-processing the data, and aligning the pre-processed point cloud data with a robot machining reference coordinate system; based on the local curvature of the point cloud, the point cloud model is divided into multiple layers of contour slices by using a curvature adaptive slicing algorithm, wherein the slicing interval is negatively correlated with the local curvature radius; the contour points of each layer of slices are B-spline fitted to generate a continuous grinding and polishing path curve; the robot executes the grinding and polishing path curve, and integrates a real-time thickness detection device; when the thickness measurement value deviates from a preset threshold value, closed-loop feedback control is performed, and the grinding and polishing path and / or grinding and polishing parameters are dynamically compensated. The application has the advantages of high machining precision, high efficiency, high consistency and the like.

Devices, systems, methods, and processes for in-network computing using modular switch architecture are described herein. Endpoint devices generate data chunks and forward them to a network, comprising spine and leaf switches, for data reduction. Leaf switches act as conduits and forward the data chunks to a spine switch. The spine switch includes various line cards (e.g., one for each leaf switch) and a fabric element. The line cards may execute a stage of data reduction on the received data chunks or may forward the received data chunks directly to the fabric element. The fabric element executes a data reduction operation on the data received from the line cards and obtains a reduced output which is forwarded to the endpoint devices via the line cards and the leaf switches. Thus, a single-tier in-network computing topology is implemented to execute data reduction in a cost-effective, simple, and efficient manner.

A method of conducting operations for an IPG adapted to provide electrical pulses to tissue of a patient for medical therapy is provided. The IPG provides the medical therapy according to pulse parameter data executed according to an execution path. The method can include identifying a root node repeated within the execution path of a first set of the pulse parameter data, identifying a stimulation pulse pattern repeated in the first set between occurrences of the root node, replacing the repeated pattern with (a) a single instance and a recurrence count or (b) a pointer to a reference pattern to generate a second set of the pulse parameter data, and communicating the second set between the external controller and the IPG and / or storing the second set within a memory of the IPG.

The invention discloses a container real-time migration method and system and a medium. The method comprises the steps that a deduplication method based on content definition partitioning is introduced into container real-time migration, highly redundant memory pages are effectively detected and removed, and an optimal data reduction algorithm is dynamically selected to improve migration performance by means of an adaptive data reduction technology. Experimental results show that the total migration time can be reduced by 75%, and stable performance can be kept under different working load and bandwidth conditions.

The invention provides a method for data reduction in the context of spectral CT imaging. The method comprises reducing the number of spectral channels of the acquired CT data by a reduced set of synthetic energy channels, each from a weighted combination of the original measured energy channel. The weights are selected in an optimization process in which an error metric associated with an output of a material decomposition procedure to be applied to the CT data is estimated, and the weights adjusted to minimize a value of the error metric. The error might for example be a noise estimate, and / or a bias estimate.

The invention relates to the technical field of cloud computing and data processing, and discloses an efficient data processing method and system based on cloud computing, and compared with a traditional method in which data quality is simply reduced or frequent retransmission is carried out, intelligent processing and data simplification are carried out locally, so that the transmission data volume is remarkably reduced, and the data transmission efficiency is improved. The real-time performance and the accuracy of the key information under the limited bandwidth are ensured, the communication congestion and the excessive consumption of airborne energy are avoided, the execution efficiency of the whole task and the cloud decision support capability are greatly improved, and the method and the system have remarkable advantages especially in emergency response and other dispute scenes.

The invention discloses a matrix data protocol processing device and method. According to the embodiment of the invention, a cooperative effect is achieved under a closed loop of tile loading, hardware grouping rearrangement, parallel protocol and fixed prefix slot single-beat write-back; according to the method, higher throughput, lower end-to-end delay, less data migration / instruction quantity and better energy efficiency are obtained under the same example and resource constraint, and consistent and efficient hardware support is provided for a matrix protocol in the row / column direction through a unified instruction interface.

This invention discloses a fine-grained data reduction method and system for Tar archive files, relating to the field of data storage and version management technology. The method includes the following steps: S1: Initialize performance statistics variables; set fingerprint index, feature index, pathname index, and incremental index; S2: Receive multiple backup versions of Tar archive files in the input directory, and process each backup version in ascending order of backup time; S3: Perform semantic-aware block segmentation on the current Tar archive file backup version; S4: Calculate SHA-256 fingerprints for all generated header blocks, file blocks, and CDC blocks, and identify duplicate and unique blocks using the fingerprint index; S5: Perform a hybrid metadata and feature value deduplication method on unique blocks; S6: Perform incremental compression or local compression on selected blocks, and update performance statistics variables; S7: Save the compressed blocks to a container for persistent storage, and synchronously update the index information. This invention achieves efficient data reduction.

A point cloud data reduction device is provided that can compress point cloud data at a high compression rate while suppressing loss of necessary information. [Solution] A point cloud discrimination unit 5 compares target point cloud data with base point cloud data and classifies the target point cloud into a fixed point cloud and a non-fixed point cloud. The point cloud discrimination unit 5 sets point clouds at corners or edges in the fixed point cloud as an exceptional point cloud, and sets point clouds in clusters obtained by clustering the fixed point cloud that represent thin objects narrower than a predetermined width or objects smaller than a predetermined size as exceptional point clouds. A downsampling unit 6 compresses the target point cloud data by downsampling point clouds from the fixed point cloud that are not set as exceptional point clouds.

The invention discloses an electric energy metering box abnormal electricity utilization detection method and system based on edge intelligence, and particularly relates to the technical field of electricity utilization detection, comprising the steps of S1, deploying edge computing nodes in an electric energy metering box during edge computing node deployment, S2, multi-dimensional electric energy data processing, S3, preprocessing, S4, exception processing, S5, exception judgment, S6, verification and S7, and S2, multi-dimensional electric energy data processing, multi-dimensional electric energy data processing, multi-dimensional electric energy data processing, multi-dimensional electric energy data processing, multi-dimensional electric energy data processing, multi-dimensional electric energy data processing, multi-dimensional electric energy data processing and multi-dimensional electric energy data processing. And when the trigger processing is performed, field alarm is triggered. Through edge calculation and data burden reduction, a traditional cloud centralized processing mode is thoroughly changed, a lightweight model is deployed on the edge side and real-time preprocessing is executed, the dimension of original data is directly reduced, end-to-end top-speed response is realized, the problem of slow response caused by large data volume and complex model is solved, and the method is suitable for large-scale popularization and application. And the timeliness of abnormal power utilization detection is greatly improved.

The application discloses a kind of based on ST-GAN's multi-source data dynamic risk early warning method and system, the method includes the following steps: S1. constructing the spatiotemporal generation confrontation network suitable for spatiotemporal data characteristics, utilize the network to generate extreme precipitation data, realize spatiotemporal unbalanced data reduction, obtain the equalization data of final output;S2. the equalization data of final output with high-dimensional space variable is carried out multi-source data fusion, and nonlinear spatiotemporal information conversion equation, and by local linearization obtains linear approximation model, to predict future time series;S3. under the present situation that extreme rainfall data amount is relatively insufficient, neural network based on dual learning theory accurately learns the parameter of nonlinear spatiotemporal conversion, estimates extreme weather event.The application is through spatiotemporal generation confrontation network (ST-GAN) and multi-source data fusion engine, significantly improves the precision and efficiency of natural disaster warning.

A method for inventory monitoring and an autonomous robot for inventory monitoring are provided. The method includes detecting one or more shelf labels from one or more images, where the one or more images are captured by one or more cameras on an autonomous robot. The method further includes obtaining one or more bounding boxes based on the one or more shelf labels detected from the one or more images, where each bounding box encloses one or more facings of a same product, and associating the one or more bounding boxes with the one or more shelf labels detected from the one or more images.

A system and method for simultaneous compression and encryption of data. The system analyzes input data to determine its properties and creates a transformation matrix based on these properties. Using this matrix, the input data is transformed into a modified distribution, generating a main data stream of transformed data and a secondary stream of transformation information. The main data stream is compressed, and both streams are combined into a single output. The system implements security measures to protect against various attacks, including side-channel vulnerabilities. By using a dyadic distribution algorithm, the system achieves both compression and encryption in a single pass over the data, offering significant efficiency gains. The system can operate in both lossless and lossy modes, providing flexibility for different application requirements. This approach offers a unique solution for data transmission and storage scenarios where both data reduction and security are critical concerns.

The embodiment of the invention discloses a test vector generation method and device and electronic equipment, and relates to the technical field of data processing, and the method comprises the steps: obtaining a standard test interface language STIL file and a configuration file required by a to-be-tested chip scanning SCAN test; testing information in the STIL file is extracted, and configuration information in the configuration file is extracted; carrying out formatting processing on the test information based on the configuration information so as to enable the test information to accord with an SCAN standard test protocol; and converting the formatted test information to generate a test vector. Therefore, conversion from the STIL file to the test vector file can be realized without participation of an EDA simulation tool, and the efficiency of test vector development is greatly improved. Furthermore, by configuring data, the finally output test vector is simplified, and the occupation of storage resources and the debugging difficulty are reduced.

An adaptive transformation matrix generation is used simultaneous compression and encryption of data. The system continuously monitors input data streams to detect changes in distribution patterns, dynamically updating transformation matrices in response to these changes. Using performance evaluation criteria, the system selects and deploys optimal matrices that maintain both compression efficiency and cryptographic security as data characteristics evolve. The system implements configurable adaptation policies that govern when and how matrices are updated, ensuring system stability while maximizing performance. By employing sliding window analysis and distribution variance metrics, the system can quantify data drift and trigger appropriate adaptations. The system maintains compatibility with various operational modes including lossless, lossy, and modified lossless configurations, applying mode-specific optimization criteria to each scenario. This adaptive approach offers a resilient solution for data transmission and storage scenarios where both data reduction and security must be maintained across diverse and evolving data streams.

The present invention relates to the generation and storage and / or transmission of images of a collection area in which one or more arthropods may be located, for the purpose of detecting, identifying and / or quantifying arthropods. The present invention also relates to a computer-implemented method, to an apparatus and to a computer program.