219 results about "Data reduction" patented technology

A digital auxiliary repair method for an ancient building. The method comprises: point cloud data acquisition, point cloud model establishment, point cloud model optimization and model printing, which involve: scanning and photographing a damaged ancient building by using a three-dimensional scanner and an industrial camera, so as to obtain three-dimensional point cloud data of the ancient building; performing preprocessing, such as data registration, data denoising, data simplification and data segmentation, on the point cloud data to obtain data for establishing a three-dimensional model; optimizing the three-dimensional model; and by means of a 3D printer, obtaining a mold for a damage position of the ancient building, and on the basis of requirements, using the mold to perform molding to obtain a wooden or brick-mixed workpiece.

The invention provides a privacy protection federated distillation and backdoor defense method for large model fine tuning, and belongs to the technical field of artificial intelligence security and federated learning, and the method comprises the steps: 1, carrying out the distillation and core representation extraction of a data set based on contribution degree weighted federated pre-training and local neural feature function matching; step 2, self-adaptive noise back door defense processing based on multi-feature fusion; according to the method, a dataset distillation mechanism based on neural feature function matching and a self-adaptive noise defense strategy are adopted, so that effective balance of the large model among data simplification, privacy protection and backdoor defense robustness is realized; the method is of great significance in improving the safety and reliability of an artificial intelligence system in a distributed environment.

A system for adaptive data compression uses content-aware analysis and dynamic feedback to optimize compression quality. An adaptive quantization subsystem analyzes content characteristics of input data and determines appropriate quantization parameters. A bit allocation engine distributes available bits across different portions of the input data based on the analyzed characteristics. A quality assessment subsystem monitors the compressed output and generates parameter adjustment signals based on measured quality metrics. A feedback control subsystem then modifies the quantization parameters in response to these signals. The modified parameters are used to create optimized compressed output data from the input dataset. This dynamic, content-aware approach enables improved compression quality while maintaining efficient data reduction.

A system and method for adaptive data processing combining compression and encryption. The system analyzes input data characteristics, compares probability distributions, and creates a transformation matrix to convert data into a dyadic distribution. It generates a main data stream of transformed data and a secondary stream of transformation information. The system dynamically selects and applies processing techniques, including transformation, encoding, compression, and encryption algorithms, based on analyzed characteristics and real-time performance metrics. It compresses the main data stream using Huffman coding and implements security measures to protect the output. A feedback loop monitors technique effectiveness, updates a knowledge base, and influences future selections. The system can operate in lossless, lossy, or modified lossless modes, adapting to different application requirements. This approach offers an efficient solution for scenarios where both data reduction and security are critical concerns.

An adaptive transformation matrix generation is used simultaneous compression and encryption of data. The system continuously monitors input data streams to detect changes in distribution patterns, dynamically updating transformation matrices in response to these changes. Using performance evaluation criteria, the system selects and deploys optimal matrices that maintain both compression efficiency and cryptographic security as data characteristics evolve. The system implements configurable adaptation policies that govern when and how matrices are updated, ensuring system stability while maximizing performance. By employing sliding window analysis and distribution variance metrics, the system can quantify data drift and trigger appropriate adaptations. The system maintains compatibility with various operational modes including lossless, lossy, and modified lossless configurations, applying mode-specific optimization criteria to each scenario. This adaptive approach offers a resilient solution for data transmission and storage scenarios where both data reduction and security must be maintained across diverse and evolving data streams.

The invention discloses a power distribution cabinet remote monitoring system based on wireless communication, and relates to the technical field of remote monitoring, and the system comprises a communication scheduling module which is used for receiving an uploaded data packet, carrying out link evaluation through an uploading link optimization method in combination with a real-time network condition, and carrying out link optimization according to an actual link condition and the priority of the uploaded data; selecting an initial uploading link and uploading data; the Bluetooth self-healing module is used for evaluating the link quality according to the link quality data in a data uploading process, dynamically switching an active link and a standby link according to a link quality evaluation result, and carrying out Bluetooth task transmission; the remote control module is used for carrying out data reduction by adopting a multilayer decryption mechanism and a mixed format analysis mechanism based on the encrypted Bluetooth task data, evaluating the operation state of the power distribution cabinet, generating a control instruction through a fuzzy logic control method according to an operation state evaluation result and executing the control instruction; through a dynamic link switching mechanism of the Bluetooth self-healing module, continuous and stable transmission of monitoring data of the power distribution cabinet is ensured.

The invention discloses a cyberspace advanced long-term threat (APT) trapping method based on a graph neural network, a storage medium and a cyberspace advanced long-term threat trapping device, and relates to the technical field of computers, and the method comprises the steps: carrying out the dimension reduction of original encrypted traffic data based on a hierarchical clustering algorithm, retaining an abnormal traffic cluster, and removing normal traffic redundant data; designing an E-ResGAT model to perform encrypted semantic modeling, and introducing a residual connection technology to retain original traffic features; and a dynamic trapping-protocol inference closed-loop system is further constructed, and a honeypot environment is utilized to simulate attack sample interaction behaviors and capture traffic. According to the scheme, efficiency balance of encrypted traffic data reduction and semantic reasoning is realized through collaborative optimization of hierarchical clustering and a graph neural network, the capture capability of the model on a hidden attack mode is enhanced in combination with a residual attention mechanism, and a multi-dimensional analysis basis is provided for high-security scenes such as a military private network; and an innovative solution is provided for improving the security defense capability of a military information system.

The invention provides a sensitive data protection system and method based on pseudonymous encryption, and relates to the field of sensitive data protection, and the system comprises the following modules: a dynamic pseudonym generation module which is used for receiving original sensitive data in real time, and generating a dynamic pseudonym based on a quantum random number seed and a timeliness noise factor; and the regulation self-adaptive decision module is connected with the dynamic pseudonym generation module. According to the method, a quantum random number seed # imgabs0 #, a timeliness noise factor # imgabs1 # and a life cycle lt are adopted; the dynamic pseudonym is unpredictable and is short-term effective, the replay attack risk is remarkably reduced, key synthesis and data reduction are realized through an SGX secure enclave in a GDPR mode, the execution success rate of the forgotten right is ensured, and differential privacy k = 3, delta = 0.1 and Bloom Filter are adopted for desensitization in a CCPA mode, so that the reidentification risk is reduced.

A method for greatly reducing data storage requirements of autonomous robots capable of product inventory is described. Instead using high resolution images for label identification and position tracking, a lower resolution map can be searched. Human or machines can identify position of labels, and via a reverse mapping, the corresponding position of a label on the high resolution image can be identified. Except for those portions of the high resolution image showing a label, most of the high resolution image can be discarded. Further processing on the limited image subset can be used to read the labels.

The invention discloses a nuclear power system file layout identification and structured information extraction method and system, and the method comprises the steps: collecting first object data, and classifying the first object data; and carrying out first preprocessing on the classification result, and carrying out layout analysis and data reduction on the preprocessing result. And performing text vectorization and semantic analysis according to the restored data to obtain structured output. According to the nuclear power system file layout identification and structured information extraction method and system provided by the invention, the file classification accuracy and efficiency are improved, and meanwhile, the manual correction requirement is remarkably reduced. And the processing capability of the scanned file is enhanced, and the stability in complex layout processing is also obviously enhanced. The automatic restoration of the hierarchical structure of the file is realized, and the manual intervention time is greatly reduced. And efficient identification and restoration of table contents, efficient text semantic extraction and clustering and generation of high-quality structured output are realized.

A system and method for simultaneous compression and encryption of data. The system analyzes input data to determine its properties and creates a transformation matrix based on these properties. Using this matrix, the input data is transformed into a modified distribution, generating a main data stream of transformed data and a secondary stream of transformation information. The main data stream is compressed, and both streams are combined into a single output. The system implements security measures to protect against various attacks, including side-channel vulnerabilities. By using a dyadic distribution algorithm, the system achieves both compression and encryption in a single pass over the data, offering significant efficiency gains. The system can operate in both lossless and lossy modes, providing flexibility for different application requirements. This approach offers a unique solution for data transmission and storage scenarios where both data reduction and security are critical concerns.

The invention discloses a big data advertisement label classification system based on AI analysis, particularly relates to the field of advertisement data processing and label intelligent classification, and is used for solving the problems of interest identification distortion and released label drift caused by coarse granularity of an end-side theme abstract. Judging the combination credibility according to two parameters of a curvature amplitude index and a path stability potential so as to remodel a pseudo-continuous trajectory, and deducing a fine grain interest label under mixed attention modulation; according to the overall architecture, information faults after abstract compression are broken through, audience portraits are close to real-time interests and do not drift along with data reduction, a delivery end bids accurately according to the audience portraits, repeated correction around distortion labels is not needed any more, budget waste is greatly reduced, and exposure frequency distribution tends to be balanced; the progressive resonance graph and the label tree mapping library are kept synchronous, the real-time bidding engine always anchors the latest interest coordinate, the advertisement experience meets the user demand, and the brand security risk is inhibited along with the decline of the false bidding.

The invention provides a Bluetooth scanning method and related equipment. The method can be applied to an intelligent interconnection scene. After broadcast data is obtained through a Bluetooth chip configured with a filter, filtering configuration information matched with the broadcast data is determined, and the filtering configuration information comprises address information of a callback function used for processing the broadcast data. And transmitting the broadcast data to a callback function according to the address information. Therefore, the callback function matched with the broadcast data can be accurately used for processing the broadcast data, the processing flow is simplified, the efficiency is improved, and the power consumption is reduced. In some embodiments, the broadcast data is subjected to de-duplication processing through the hard filtering module, and the broadcast data is transmitted to the callback function after being determined to be not repeated broadcast, so that the computing power and the power consumption are saved. In some embodiments, after all the broadcast data needing to be scanned by the operating system are transmitted to the corresponding callback function, scanning of the operating system is stopped, the number of broadcasts scanned by the operating system can be reduced as much as possible, and computing power and power consumption are saved.

A data processing method is provided. The implementation is: for each of a plurality of data blocks included in data to be processed, dividing the data block into a plurality of first sub-blocks corresponding to a plurality of computing unit groups respectively; for each first sub-block of the plurality of first sub-blocks, dividing the first sub-block, based on a target computing unit group corresponding to the first sub-block, into a plurality of second sub-blocks corresponding to the plurality of computing units in the target computing unit group, respectively; determining, by processing each second sub-block utilizing the corresponding computing unit of the second sub-block, a plurality of first processing results output by the plurality of computing units respectively; and determining a processing result of the first sub-block by performing a data reduction operation on the plurality of first processing results utilizing the target computing unit group.

A system and method for cross-stream asymmetric enhancement combines machine learning-driven asymmetric codebook generation with dyadic distribution algorithms to enable simultaneous optimization of compression efficiency, cryptographic security, and error correction capability. The system analyzes input data characteristics and initializes multiple specialized ML models to generate stream-specific asymmetric codebooks optimized for different objectives. Enhanced dyadic distribution processing creates three pre-conditioned data streams that are processed through parallel asymmetric transformation pipelines: compression-optimized for maximum data reduction, security-optimized for cryptographic strength, and error-correction-optimized for robust recovery capability. Cross-stream optimization coordinates the multiple processing paths to ensure overall system coherence while maintaining individual stream objectives. The system supports multiple operating modes including ultra-high compression using only the primary stream, broadcast quality using primary and secondary streams, and archival mode using all streams for lossless reconstruction. The system supports graduated access control that enables different reconstruction quality levels based on available stream combinations.

The invention relates to the technical field of space flight and aviation, in particular to a low-delay communication control system and method for an analog machine in a fixed flight training course. The control system comprises a multi-source sensor fusion module, a data simplification processing module, a dynamic time delay optimization communication framework, a distributed intelligent decision-making terminal, a virtual-real mapping synchronous controller, a milli delay regulation and control module and an intelligent sensing alarm interaction module. According to the invention, 4G, 5G and satellite double links are integrated, an optimal link is dynamically selected by adopting a Q-Learning algorithm, millisecond-level time delay control is realized, transmission reliability is improved through a packet loss rate compensation algorithm, multi-sensor data is fused based on an FPGA hardware accelerator and an EKF algorithm, and a data compression ratio is improved in combination with Huffman coding and Kalman filtering. The bandwidth requirement is greatly reduced, through software and hardware collaborative optimization, the end-to-end time delay is reduced to be within hundreds of milliseconds, the virtual-real synchronization precision is improved to the centimeter level, and the decision response speed is improved by two times.

The present application belongs to the technical field of communications. Disclosed are a data reduction method and apparatus, and a device and a computer-readable storage medium. The method is applied to at least one root node in a multi-dimensional direct-connect topology, wherein nodes in the multi-dimensional direct-connect topology are used for model training. The method comprises: receiving first data, wherein the first data is part of transmitted data, which is data transmitted between nodes in a model training process; obtaining second data, wherein the second data is part of first generated data, which is data generated by a root node in the model training process; performing reduction on the first data and the second data to obtain third data, wherein the third data is used by nodes in a multi-dimensional direct-connect topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct-connect topology, wherein the first node is a node in the multi-dimensional direct-connect topology other than the root node. The method can improve the communication efficiency in a data reduction process.

The invention discloses a column storage-based big data compression and query optimization method, which relates to the technical field of big data compression and query optimization, and comprises the following steps of: performing sliding window division on original data of column storage to obtain a plurality of data windows; calculating a feature vector based on the value distribution characteristics in the multi-data window, wherein the feature vector is used for representing a local statistical rule of the data; performing dynamic merging according to the similarity of the feature vectors of the adjacent data windows to form a compressed block; selecting a corresponding compression coding mode based on the data type and the distribution characteristic of the compression block, and generating a metadata index containing a minimum value, a maximum value, a coding type and a compression size; after receiving a query request, mapping a predicate condition to a compressed coding space, and executing non-decompressed filtering judgment on a compressed block by utilizing the metadata index and a compressed coding type; and determining a decompression range based on a filtering result of the compressed blocks, and only performing local decompression and data reduction on the possibly hit compressed blocks.

The embodiment of the invention provides a big data storage optimization method based on a zero-copy collaboration technology and related equipment, and belongs to the technical field of big data storage. The method comprises a zero-copy data transmission module used for directly writing a data stream into a front-end buffer area of an annular structure through a direct memory access technology, and adopting a zero-copy algorithm based on pointer offset to parallelly separate data of each channel in a multi-thread environment; the intelligent data reduction module is used for deleting duplicated data from the data and then compressing the duplicated data; and the hybrid storage management module is used for managing the hybrid partition storage architecture and dynamically adjusting the distribution of the data in the hybrid partition storage architecture according to the data access mode. The CPU copy frequency is reduced through the zero copy technology, the data transmission efficiency is remarkably improved by combining the data reduction and intelligent layering strategies, the occupied storage space is reduced, and meanwhile the data safety and compliance are guaranteed.

The invention discloses a data reduction method, device and equipment and a computer readable storage medium, and belongs to the technical field of communication. The method is applied to at least one root node in a multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology are used for model training, and the method comprises the steps that first data are received, the first data are partial data of transmission data, and the transmission data are data transmitted among the nodes in the model training process; second data is obtained, the second data is part of first generated data, and the first generated data is data generated by the root node in the model training process; the first data and the second data are reduced to obtain third data, and the third data are used for nodes in the multi-dimensional direct connection topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct connection topology, wherein the first node is a node except the root node in the multi-dimensional direct connection topology. According to the method, the communication efficiency in the data reduction process can be improved.

The invention relates to a road and bridge pavement crack detection device applied to the related technical field of crack detection, which can realize identification of pavement surface cracks and slight recesses at the same time, when the pavement is flat and has no slight recess, the bottoms of a plurality of recess detection units are on the same straight line, at the moment, the data of two laser range finders tend to be unchanged, and the detection result is accurate. When a slight recess exists, the bottom of the recess measuring unit at the recess is relatively low, so that the corresponding light display plate moves downwards, at the moment, data of one laser range finder is reduced, the main camera and the auxiliary camera can be controlled to photograph at the same time, and two pieces of photographing information are combined into one group of data, so that on-line marking of the slight recess is realized; and meanwhile, toner can fall off at the auxiliary marking pipe, marking of the sunken part of the road surface is achieved, and compared with the mode that detection is conducted only through photographing in the prior art, the probability that hidden cracks are detected out is greatly improved, and then the influence of the hidden cracks on traffic and the road service life is effectively reduced.

A hardware acceleration device is equipped with circuitry to perform one or more reversible data transformations on data based on a request and compress the transformed data to produce compressed transformed data. The hardware acceleration device generates an output containing the compressed transformed data and transformation metadata indicating the set of reversible data transformations applied to the compressed transformed data.

A system and method for adaptive data processing combining compression and encryption. The system analyzes input data characteristics, compares probability distributions, and creates a transformation matrix to convert data into a dyadic distribution. It generates a main data stream of transformed data and a secondary stream of transformation information. The system dynamically selects and applies processing techniques, including transformation, encoding, compression, and encryption algorithms, based on analyzed characteristics and real-time performance metrics. It compresses the main data stream using Huffman coding and implements security measures to protect the output. A feedback loop monitors technique effectiveness, updates a knowledge base, and influences future selections. The system can operate in lossless, lossy, or modified lossless modes, adapting to different application requirements. This approach offers an efficient solution for scenarios where both data reduction and security are critical concerns.

The invention provides a flow field reconstruction method and system for aircraft design, and the method comprises the steps: carrying out the simulation calculation of an aircraft under different working conditions, and constructing a full-field data reduced-order model and a sparse measurement point data reduced-order model based on a simulation result; establishing a coefficient correlation model between the full-field data reduced-order model and the sparse measuring point data reduced-order model; performing a wind tunnel test on the aircraft, and based on a test result, obtaining a coefficient of the full-field data reduced-order model through the sparse measurement point data reduced-order model and the coefficient correlation model; and substituting the coefficient of the full-field data reduced-order model into the full-field data reduced-order model to obtain a reconstructed flow field. According to the method, the value of the test result to the aircraft design process is improved, the test result is converted into a model-based field data form from a traditional text-based scalar data form by combining numerical simulation and test, and extension from a discrete working condition to a continuous working condition is realized.

The present invention relates to a system and a method for measuring and analyzing minimal residual disease (MRD) in pediatric B-cell precursor acute lymphoblastic leukemia (B-ALL) using multiparameter flow cytometry (MPFC). The invention finds application in clinical diagnostics and hematology-oncology for quantifying MRD in B-ALL patients with high sensitivity and specificity, needed for risk stratification, monitoring treatment response, and informing therapeutic decisions. The system comprises interconnected subsystems including an acquisition subsystem with an MPFC instrument, a control and file generation subsystem, and an analytical subsystem. The analytical subsystem incorporates modules for sequential data reduction, automated data cleaning, automated unsupervised data clustering, and interactive cluster analysis. Key advantages include high MRD detection sensitivity (e.g., 10⁻⁵ or 0.001%) and high specificity, without reliance on reference samples or supervised machine learning models, making it applicable in laboratories with different measuring equipment and using different panels of antibodies for identification of leukemic cells.

The present invention discloses a method for intelligent monitoring and fault diagnosis of power plants based on multi-source data fusion, which relates to the technical field of equipment fault analysis, including multi-source data collection, preprocessing and data reduction; data fusion of the reduced data to construct a fault analysis model; fault diagnosis output, and feedback adjustment of the model. The method for intelligent monitoring and fault diagnosis of power plants based on multi-source data fusion provided by the present invention organizes the pre-processed multi-source data into a high-dimensional data matrix, and uses the SVD-POD algorithm for order reduction processing, thereby reducing the data dimension and redundant information and increasing the processing speed. The feedback optimization diagnosis model realizes effective response to unknown working conditions and fault modes, ensures the accuracy and timeliness of fault diagnosis, has adaptive capabilities, can be dynamically adjusted according to new data, and improves the intelligence level and adaptability of the system.

A data reduction method, apparatus, and system are provided, and belong to the field of data processing. One example method includes: obtaining a first address; and updating, based on popularity of the first address, an entry corresponding to the first address in a hot data table, where popularity of a storage address in the hot data table is greater than popularity of a storage address in a first cache table, and the first cache table is used for data reduction.

The invention discloses a metro train speed measurement method based on a millimeter-wave radar, and the method comprises the steps: S1, millimeter-wave radar equipment installation and calibration: enabling the millimeter-wave radar to be fixedly installed at the outer side of a train cab through a tool clamp, testing and measuring the installation posture of the millimeter-wave radar, and employing a gyro machine as the basis of dynamic calibration; step S2, radar signal acquisition and processing: millimeter wave radar message data are analyzed, interference of a subway line environment is reduced from the aspects of radar working parameter tuning and beam radiation optimization, and conditions are set to trigger high-frequency data acquisition; s3, data preprocessing and data analysis: preprocessing the collected data from the aspects of noise values, outliers and data reduction, identifying a detection target by adopting a density-based spatial clustering algorithm, and outputting target point cloud data; and S4, calculating the train speed and outputting the train speed in real time. According to the invention, the millimeter-wave radar is installed and the detected millimeter-wave radar point cloud data is analyzed in real time, so that accurate measurement of the train running speed is realized.

The invention is suitable for the technical field of flight height prediction, and particularly relates to a flight height prediction method based on a neural network, and the method comprises the steps: collecting sensor data of a passenger plane, and inhibiting outlier data points through a dynamic truncation mechanism based on a distribution quantile threshold; dividing the sensor data by adopting a time sequence slice division strategy to obtain a training set and a verification set, and constructing a prediction feature set based on the training set and the verification set; and constructing a flight height prediction model, training the flight height prediction model through the prediction feature set, and predicting the flight height based on the trained flight height prediction model. According to the invention, a normalization technology is applied to carry out data reduction on a flight time sequence signal, a parallel network structure comprising a multi-scale convolution kernel feature extraction layer and an attention weight distribution module is designed, and gradient iteration is carried out on a Huber loss function through an optimizer for dynamically adjusting a learning rate. And the characterization capability of the model on the nonlinear change mode of the flight state is obviously improved.

The invention relates to the technical field of data storage management, in particular to a method and system for improving data storage and processing efficiency based on a large model, and the method comprises the following steps: constructing a distributed storage system, introducing a heterogeneous storage management mechanism, and adopting a data reduction technology; the method has the beneficial effects that intelligent compression and duplicate removal are performed on storage data through a large model, and the storage space requirement is remarkably reduced; secondly, analyzing a data access mode by using a large model, realizing dynamic data-tiering and life cycle management, and optimizing storage resource allocation; in the aspect of data processing, the large model is used for constructing an efficient data index system, and the data retrieval and analysis process is accelerated; meanwhile, data preprocessing and feature extraction are carried out through a large model, and the performance of big data analysis and machine learning tasks is improved.