325 results about "Data reduction" patented technology

A digital auxiliary repair method for an ancient building. The method comprises: point cloud data acquisition, point cloud model establishment, point cloud model optimization and model printing, which involve: scanning and photographing a damaged ancient building by using a three-dimensional scanner and an industrial camera, so as to obtain three-dimensional point cloud data of the ancient building; performing preprocessing, such as data registration, data denoising, data simplification and data segmentation, on the point cloud data to obtain data for establishing a three-dimensional model; optimizing the three-dimensional model; and by means of a 3D printer, obtaining a mold for a damage position of the ancient building, and on the basis of requirements, using the mold to perform molding to obtain a wooden or brick-mixed workpiece.

Machine learning model scalability with distributed multi-layer processing is disclosed herein. A method for processing and deploying machine learning models that enhances scalability and efficiency by executing a subset of a neural network on each of a plurality of interconnected processing units. The method involves partitioning compute tasks across these processing units to reduce latency, including broadcast and reduction processes for inputs and outputs. It also includes managing the allocation of samples in a batch to specific master processing units within the distributed arrangement and synchronizing computation between fully connected layers within each processing unit. Additionally, the method implements data reduction during the transfer of data across the processing units, wherein data is accumulated with a current processing unit's partial sum as it is transferred to the destination processing unit.

The invention provides a privacy protection federated distillation and backdoor defense method for large model fine tuning, and belongs to the technical field of artificial intelligence security and federated learning, and the method comprises the steps: 1, carrying out the distillation and core representation extraction of a data set based on contribution degree weighted federated pre-training and local neural feature function matching; step 2, self-adaptive noise back door defense processing based on multi-feature fusion; according to the method, a dataset distillation mechanism based on neural feature function matching and a self-adaptive noise defense strategy are adopted, so that effective balance of the large model among data simplification, privacy protection and backdoor defense robustness is realized; the method is of great significance in improving the safety and reliability of an artificial intelligence system in a distributed environment.

A system for adaptive data compression uses content-aware analysis and dynamic feedback to optimize compression quality. An adaptive quantization subsystem analyzes content characteristics of input data and determines appropriate quantization parameters. A bit allocation engine distributes available bits across different portions of the input data based on the analyzed characteristics. A quality assessment subsystem monitors the compressed output and generates parameter adjustment signals based on measured quality metrics. A feedback control subsystem then modifies the quantization parameters in response to these signals. The modified parameters are used to create optimized compressed output data from the input dataset. This dynamic, content-aware approach enables improved compression quality while maintaining efficient data reduction.

A system and method for adaptive data processing combining compression and encryption. The system analyzes input data characteristics, compares probability distributions, and creates a transformation matrix to convert data into a dyadic distribution. It generates a main data stream of transformed data and a secondary stream of transformation information. The system dynamically selects and applies processing techniques, including transformation, encoding, compression, and encryption algorithms, based on analyzed characteristics and real-time performance metrics. It compresses the main data stream using Huffman coding and implements security measures to protect the output. A feedback loop monitors technique effectiveness, updates a knowledge base, and influences future selections. The system can operate in lossless, lossy, or modified lossless modes, adapting to different application requirements. This approach offers an efficient solution for scenarios where both data reduction and security are critical concerns.

A method to operate a customer-centric portable personal vehicle seat position system includes: generating pressure sensor profiles in response to a user seated on a vehicle seat of a first vehicle having a seat base and a seat back; correlating the pressure sensor profiles to individual user tasks; performing data reduction of the pressure sensor profiles after correlation to the individual user tasks; and encoding. multiple seat comfort preference profiles.

An adaptive transformation matrix generation is used simultaneous compression and encryption of data. The system continuously monitors input data streams to detect changes in distribution patterns, dynamically updating transformation matrices in response to these changes. Using performance evaluation criteria, the system selects and deploys optimal matrices that maintain both compression efficiency and cryptographic security as data characteristics evolve. The system implements configurable adaptation policies that govern when and how matrices are updated, ensuring system stability while maximizing performance. By employing sliding window analysis and distribution variance metrics, the system can quantify data drift and trigger appropriate adaptations. The system maintains compatibility with various operational modes including lossless, lossy, and modified lossless configurations, applying mode-specific optimization criteria to each scenario. This adaptive approach offers a resilient solution for data transmission and storage scenarios where both data reduction and security must be maintained across diverse and evolving data streams.

The invention relates to a multi-sensor data fusion method and device, equipment and a medium, and the method comprises the steps: obtaining data and GPS information of a plurality of sensors such as a binocular camera and a radar, taking the data and GPS information as input data, and carrying out the preprocessing, including one or more of image color space conversion, filtering and denoising, and data density reduction; further processing the preprocessed data, wherein the processing comprises one or more of target detection and tracking, coordinate system conversion, target UID distribution and aerial view angle projection; a precision time protocol (PTP) is adopted to synchronize processed data of a plurality of sensors, and the time is kept consistent; a central point of a road intersection is selected, GPS information is recorded, a fusion coordinate system is established, sensor and target position information is calculated according to the GPS information of a sensor, the information is mapped to the fusion coordinate system, and target, association speed, distance and other information and UID are screened and merged according to an overlap ratio threshold. According to the invention, modeling display of the whole intersection is realized, and a reliable basis is provided for traffic management decisions.

The invention relates to a lightweight method and system for a power dispatching model under meta-universe and a storage medium, and the method comprises the following steps: carrying out the intelligent data simplification of a three-dimensional model based on a deep learning adaptive reduction compression technology in combination with an advanced texture mapping and curve thinning algorithm, so as to optimize the details of the model and reduce the data size, thereby achieving the intelligent data simplification of the three-dimensional model; meanwhile, key features are reserved; through power grid cascade loading and visual elimination, display details of a power grid model are dynamically adjusted according to the visual angle of an observer, and model resource management and dynamic scheduling are realized through asynchronous loading and rendering; in combination with an intelligent preloading and prediction mechanism, a multi-level-of-detail technology and a progressive loading technology are applied to realize rapid loading and rendering of the model, and resource allocation and detail management are refined. According to the invention, the lightweight processing of the three-dimensional model of the power dispatching element universe can be realized, and the goals of real-time efficient rendering and rapid interaction of meteorological data, power grid operation information and a plant station equipment model are achieved.

The invention discloses a power distribution cabinet remote monitoring system based on wireless communication, and relates to the technical field of remote monitoring, and the system comprises a communication scheduling module which is used for receiving an uploaded data packet, carrying out link evaluation through an uploading link optimization method in combination with a real-time network condition, and carrying out link optimization according to an actual link condition and the priority of the uploaded data; selecting an initial uploading link and uploading data; the Bluetooth self-healing module is used for evaluating the link quality according to the link quality data in a data uploading process, dynamically switching an active link and a standby link according to a link quality evaluation result, and carrying out Bluetooth task transmission; the remote control module is used for carrying out data reduction by adopting a multilayer decryption mechanism and a mixed format analysis mechanism based on the encrypted Bluetooth task data, evaluating the operation state of the power distribution cabinet, generating a control instruction through a fuzzy logic control method according to an operation state evaluation result and executing the control instruction; through a dynamic link switching mechanism of the Bluetooth self-healing module, continuous and stable transmission of monitoring data of the power distribution cabinet is ensured.

The invention provides a coal mine roadway point cloud data simplification method and device, a storage medium and electronic equipment, and the method comprises the steps: constructing a point cloud topological structure for roadway original point cloud data through a kd tree, setting a domain radius, selecting core points, recording index values, and then estimating a normal vector of each core point through a principal component analysis method and a plane fitting method; the method comprises the following steps: constructing a cylinder field at a core point by using a normal vector, then accelerating the search of a ball field of the core point by using a kd tree, calculating whether each point in the ball field is a point in the cylinder field, finally calculating the center of mass of the points in the cylinder field, and keeping the point closest to the center of mass, so that a constructed new point cloud is a simplified point cloud. The thickness of the point cloud is reduced while the point cloud is simplified, so that the problem that an existing roadway point cloud simplification method does not consider the thickness of the point cloud during point cloud simplification and cannot meet the precision requirement of similar deformation monitoring based on the roadway point cloud is solved.

The invention discloses a cyberspace advanced long-term threat (APT) trapping method based on a graph neural network, a storage medium and a cyberspace advanced long-term threat trapping device, and relates to the technical field of computers, and the method comprises the steps: carrying out the dimension reduction of original encrypted traffic data based on a hierarchical clustering algorithm, retaining an abnormal traffic cluster, and removing normal traffic redundant data; designing an E-ResGAT model to perform encrypted semantic modeling, and introducing a residual connection technology to retain original traffic features; and a dynamic trapping-protocol inference closed-loop system is further constructed, and a honeypot environment is utilized to simulate attack sample interaction behaviors and capture traffic. According to the scheme, efficiency balance of encrypted traffic data reduction and semantic reasoning is realized through collaborative optimization of hierarchical clustering and a graph neural network, the capture capability of the model on a hidden attack mode is enhanced in combination with a residual attention mechanism, and a multi-dimensional analysis basis is provided for high-security scenes such as a military private network; and an innovative solution is provided for improving the security defense capability of a military information system.

The invention provides a sensitive data protection system and method based on pseudonymous encryption, and relates to the field of sensitive data protection, and the system comprises the following modules: a dynamic pseudonym generation module which is used for receiving original sensitive data in real time, and generating a dynamic pseudonym based on a quantum random number seed and a timeliness noise factor; and the regulation self-adaptive decision module is connected with the dynamic pseudonym generation module. According to the method, a quantum random number seed # imgabs0 #, a timeliness noise factor # imgabs1 # and a life cycle lt are adopted; the dynamic pseudonym is unpredictable and is short-term effective, the replay attack risk is remarkably reduced, key synthesis and data reduction are realized through an SGX secure enclave in a GDPR mode, the execution success rate of the forgotten right is ensured, and differential privacy k = 3, delta = 0.1 and Bloom Filter are adopted for desensitization in a CCPA mode, so that the reidentification risk is reduced.

A method for greatly reducing data storage requirements of autonomous robots capable of product inventory is described. Instead using high resolution images for label identification and position tracking, a lower resolution map can be searched. Human or machines can identify position of labels, and via a reverse mapping, the corresponding position of a label on the high resolution image can be identified. Except for those portions of the high resolution image showing a label, most of the high resolution image can be discarded. Further processing on the limited image subset can be used to read the labels.

The invention discloses a nuclear power system file layout identification and structured information extraction method and system, and the method comprises the steps: collecting first object data, and classifying the first object data; and carrying out first preprocessing on the classification result, and carrying out layout analysis and data reduction on the preprocessing result. And performing text vectorization and semantic analysis according to the restored data to obtain structured output. According to the nuclear power system file layout identification and structured information extraction method and system provided by the invention, the file classification accuracy and efficiency are improved, and meanwhile, the manual correction requirement is remarkably reduced. And the processing capability of the scanned file is enhanced, and the stability in complex layout processing is also obviously enhanced. The automatic restoration of the hierarchical structure of the file is realized, and the manual intervention time is greatly reduced. And efficient identification and restoration of table contents, efficient text semantic extraction and clustering and generation of high-quality structured output are realized.

The invention discloses a set communication control method and device for distributed training and a medium, and the method comprises the steps: reducing data on all calculation nodes in a first cluster to a specified calculation node when the difference of the number of calculation nodes between any two clusters in clusters participating in data reduction is within a preset range, the number of the specified computing nodes is the same as the number of the computing nodes in the second cluster with the minimum number of the computing nodes. Controlling the specified computing node and the computing nodes in the second cluster to perform data reduction; and controlling data synchronization between the specified computing node and other nodes except the specified computing node in the first cluster. Therefore, other clusters except the cluster with the minimum number of computational nodes are reduced once in the cluster, the data are reduced to the specified computational nodes with the number the same as that of the minimum nodes in each cluster, it is guaranteed that the nodes of each cluster are the same during cross-cluster reduction, and it is avoided that some nodes are reduced with multiple nodes at the same time; and the set communication overhead is reduced.

A system and method for simultaneous compression and encryption of data. The system analyzes input data to determine its properties and creates a transformation matrix based on these properties. Using this matrix, the input data is transformed into a modified distribution, generating a main data stream of transformed data and a secondary stream of transformation information. The main data stream is compressed, and both streams are combined into a single output. The system implements security measures to protect against various attacks, including side-channel vulnerabilities. By using a dyadic distribution algorithm, the system achieves both compression and encryption in a single pass over the data, offering significant efficiency gains. The system can operate in both lossless and lossy modes, providing flexibility for different application requirements. This approach offers a unique solution for data transmission and storage scenarios where both data reduction and security are critical concerns.

The invention discloses an agricultural multi-source knowledge migration method based on a graph model, and belongs to the technical field of agricultural information, and the method comprises the steps: carrying out the reduction, feature selection and feature fusion of agricultural literature data; the data processing efficiency and accuracy are improved by utilizing agricultural data reduction based on self-organizing mapping and feature selection and fusion based on spatial mapping; constructing a knowledge migration network based on a graph model, and performing knowledge semantic extraction and knowledge migration relationship mining; agricultural multi-carrier meta-knowledge acquisition and information feedback are adjusted, and meta-knowledge acquisition and network weight adjustment are optimized. According to the method, large-scale and heterogeneous agricultural data can be effectively processed, effective migration of knowledge is realized, the performance of agricultural learning tasks is greatly improved, and the method is of great significance to improvement of agricultural production efficiency.

The invention discloses a big data advertisement label classification system based on AI analysis, particularly relates to the field of advertisement data processing and label intelligent classification, and is used for solving the problems of interest identification distortion and released label drift caused by coarse granularity of an end-side theme abstract. Judging the combination credibility according to two parameters of a curvature amplitude index and a path stability potential so as to remodel a pseudo-continuous trajectory, and deducing a fine grain interest label under mixed attention modulation; according to the overall architecture, information faults after abstract compression are broken through, audience portraits are close to real-time interests and do not drift along with data reduction, a delivery end bids accurately according to the audience portraits, repeated correction around distortion labels is not needed any more, budget waste is greatly reduced, and exposure frequency distribution tends to be balanced; the progressive resonance graph and the label tree mapping library are kept synchronous, the real-time bidding engine always anchors the latest interest coordinate, the advertisement experience meets the user demand, and the brand security risk is inhibited along with the decline of the false bidding.

The invention provides a Bluetooth scanning method and related equipment. The method can be applied to an intelligent interconnection scene. After broadcast data is obtained through a Bluetooth chip configured with a filter, filtering configuration information matched with the broadcast data is determined, and the filtering configuration information comprises address information of a callback function used for processing the broadcast data. And transmitting the broadcast data to a callback function according to the address information. Therefore, the callback function matched with the broadcast data can be accurately used for processing the broadcast data, the processing flow is simplified, the efficiency is improved, and the power consumption is reduced. In some embodiments, the broadcast data is subjected to de-duplication processing through the hard filtering module, and the broadcast data is transmitted to the callback function after being determined to be not repeated broadcast, so that the computing power and the power consumption are saved. In some embodiments, after all the broadcast data needing to be scanned by the operating system are transmitted to the corresponding callback function, scanning of the operating system is stopped, the number of broadcasts scanned by the operating system can be reduced as much as possible, and computing power and power consumption are saved.

A data processing method is provided. The implementation is: for each of a plurality of data blocks included in data to be processed, dividing the data block into a plurality of first sub-blocks corresponding to a plurality of computing unit groups respectively; for each first sub-block of the plurality of first sub-blocks, dividing the first sub-block, based on a target computing unit group corresponding to the first sub-block, into a plurality of second sub-blocks corresponding to the plurality of computing units in the target computing unit group, respectively; determining, by processing each second sub-block utilizing the corresponding computing unit of the second sub-block, a plurality of first processing results output by the plurality of computing units respectively; and determining a processing result of the first sub-block by performing a data reduction operation on the plurality of first processing results utilizing the target computing unit group.

A system and method for cross-stream asymmetric enhancement combines machine learning-driven asymmetric codebook generation with dyadic distribution algorithms to enable simultaneous optimization of compression efficiency, cryptographic security, and error correction capability. The system analyzes input data characteristics and initializes multiple specialized ML models to generate stream-specific asymmetric codebooks optimized for different objectives. Enhanced dyadic distribution processing creates three pre-conditioned data streams that are processed through parallel asymmetric transformation pipelines: compression-optimized for maximum data reduction, security-optimized for cryptographic strength, and error-correction-optimized for robust recovery capability. Cross-stream optimization coordinates the multiple processing paths to ensure overall system coherence while maintaining individual stream objectives. The system supports multiple operating modes including ultra-high compression using only the primary stream, broadcast quality using primary and secondary streams, and archival mode using all streams for lossless reconstruction. The system supports graduated access control that enables different reconstruction quality levels based on available stream combinations.

The invention relates to the technical field of space flight and aviation, in particular to a low-delay communication control system and method for an analog machine in a fixed flight training course. The control system comprises a multi-source sensor fusion module, a data simplification processing module, a dynamic time delay optimization communication framework, a distributed intelligent decision-making terminal, a virtual-real mapping synchronous controller, a milli delay regulation and control module and an intelligent sensing alarm interaction module. According to the invention, 4G, 5G and satellite double links are integrated, an optimal link is dynamically selected by adopting a Q-Learning algorithm, millisecond-level time delay control is realized, transmission reliability is improved through a packet loss rate compensation algorithm, multi-sensor data is fused based on an FPGA hardware accelerator and an EKF algorithm, and a data compression ratio is improved in combination with Huffman coding and Kalman filtering. The bandwidth requirement is greatly reduced, through software and hardware collaborative optimization, the end-to-end time delay is reduced to be within hundreds of milliseconds, the virtual-real synchronization precision is improved to the centimeter level, and the decision response speed is improved by two times.

The present application belongs to the technical field of communications. Disclosed are a data reduction method and apparatus, and a device and a computer-readable storage medium. The method is applied to at least one root node in a multi-dimensional direct-connect topology, wherein nodes in the multi-dimensional direct-connect topology are used for model training. The method comprises: receiving first data, wherein the first data is part of transmitted data, which is data transmitted between nodes in a model training process; obtaining second data, wherein the second data is part of first generated data, which is data generated by a root node in the model training process; performing reduction on the first data and the second data to obtain third data, wherein the third data is used by nodes in a multi-dimensional direct-connect topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct-connect topology, wherein the first node is a node in the multi-dimensional direct-connect topology other than the root node. The method can improve the communication efficiency in a data reduction process.

A semiconductor device, referred to herein as a Globally Interconnected Operations (GIO) layer, provides global operations in the form of global data reduction for one or more PE arrays. The GIO layer includes processing elements that perform global data reduction on processing results from one or more PE arrays. The GIO layer includes connectors that allow it to be arranged in a 3D stack with one or more PE arrays, for example, on top of or beneath a PE array. This allows reduction operations to be implemented across PE arrays using an efficient topology with superior flexibility, scalability, latency and / or power characteristics that is customizable for particular use cases at assembly time, without requiring costly and time-consuming redesign of PE arrays, and without being constrained by particular PE array designs.

The invention discloses a column storage-based big data compression and query optimization method, which relates to the technical field of big data compression and query optimization, and comprises the following steps of: performing sliding window division on original data of column storage to obtain a plurality of data windows; calculating a feature vector based on the value distribution characteristics in the multi-data window, wherein the feature vector is used for representing a local statistical rule of the data; performing dynamic merging according to the similarity of the feature vectors of the adjacent data windows to form a compressed block; selecting a corresponding compression coding mode based on the data type and the distribution characteristic of the compression block, and generating a metadata index containing a minimum value, a maximum value, a coding type and a compression size; after receiving a query request, mapping a predicate condition to a compressed coding space, and executing non-decompressed filtering judgment on a compressed block by utilizing the metadata index and a compressed coding type; and determining a decompression range based on a filtering result of the compressed blocks, and only performing local decompression and data reduction on the possibly hit compressed blocks.

The invention discloses a concentration index extraction method and device, electronic equipment and a medium. The method comprises the following steps: acquiring af7 and af8 dual-channel electroencephalogram signals in front of a forehead; calculating the fuzzy entropy of the dual-channel electroencephalogram signal by adopting a preset sliding window method to obtain a dual-channel electroencephalogram signal fuzzy entropy value; smoothing the dual-channel electroencephalogram signal fuzzy entropy value to obtain a target dual-channel electroencephalogram signal fuzzy entropy value; and performing weighted calculation on the fuzzy entropy value of the target dual-channel electroencephalogram signal to obtain a concentration index. According to the concentration degree extraction method, only af7 and af8 double-channel data need to be collected, channel data are simplified, and the possibility and convenience of electroencephalogram signal collection are improved; and meanwhile, the method adopted by the invention does not need to perform frequency division processing on the data, so that the anti-interference capability of the data is further improved, and the accuracy of obtaining the concentration index is improved.

The embodiment of the invention provides a big data storage optimization method based on a zero-copy collaboration technology and related equipment, and belongs to the technical field of big data storage. The method comprises a zero-copy data transmission module used for directly writing a data stream into a front-end buffer area of an annular structure through a direct memory access technology, and adopting a zero-copy algorithm based on pointer offset to parallelly separate data of each channel in a multi-thread environment; the intelligent data reduction module is used for deleting duplicated data from the data and then compressing the duplicated data; and the hybrid storage management module is used for managing the hybrid partition storage architecture and dynamically adjusting the distribution of the data in the hybrid partition storage architecture according to the data access mode. The CPU copy frequency is reduced through the zero copy technology, the data transmission efficiency is remarkably improved by combining the data reduction and intelligent layering strategies, the occupied storage space is reduced, and meanwhile the data safety and compliance are guaranteed.

The invention discloses a data reduction method, device and equipment and a computer readable storage medium, and belongs to the technical field of communication. The method is applied to at least one root node in a multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology are used for model training, and the method comprises the steps that first data are received, the first data are partial data of transmission data, and the transmission data are data transmitted among the nodes in the model training process; second data is obtained, the second data is part of first generated data, and the first generated data is data generated by the root node in the model training process; the first data and the second data are reduced to obtain third data, and the third data are used for nodes in the multi-dimensional direct connection topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct connection topology, wherein the first node is a node except the root node in the multi-dimensional direct connection topology. According to the method, the communication efficiency in the data reduction process can be improved.

The invention relates to a road and bridge pavement crack detection device applied to the related technical field of crack detection, which can realize identification of pavement surface cracks and slight recesses at the same time, when the pavement is flat and has no slight recess, the bottoms of a plurality of recess detection units are on the same straight line, at the moment, the data of two laser range finders tend to be unchanged, and the detection result is accurate. When a slight recess exists, the bottom of the recess measuring unit at the recess is relatively low, so that the corresponding light display plate moves downwards, at the moment, data of one laser range finder is reduced, the main camera and the auxiliary camera can be controlled to photograph at the same time, and two pieces of photographing information are combined into one group of data, so that on-line marking of the slight recess is realized; and meanwhile, toner can fall off at the auxiliary marking pipe, marking of the sunken part of the road surface is achieved, and compared with the mode that detection is conducted only through photographing in the prior art, the probability that hidden cracks are detected out is greatly improved, and then the influence of the hidden cracks on traffic and the road service life is effectively reduced.