197 results about "Data compaction" patented technology

The invention provides an industrial safety risk monitoring and early warning system and method for an autonomous planning intelligent agent, relates to the technical field of industrial safety production, and solves various limitation problems still existing in an existing safety risk monitoring scheme. In the system, an environment sensing layer is used for collecting multi-source sensing data in an industrial production environment in real time; the edge calculation layer is used for carrying out primary processing on the multi-source sensing data and executing data compression and key information extraction so as to generate key analysis data; the cloud analysis layer is used for performing deep intelligent analysis on the key analysis data, executing security risk assessment and risk diffusion prediction, and dynamically generating and adjusting an early warning decision through an autonomous planning agent; and the action execution layer is used for executing a corresponding safety action instruction according to the early warning decision and feeding back an execution result to the cloud analysis layer to realize system optimization. According to the invention, the risk early warning accuracy and the emergency disposal timeliness can be improved finally.

The invention discloses a virtual power plant operation management and control method based on a block chain, original power data are compressed into Hash fingerprints to be chained through a collaborative architecture of an edge computing layer and an AI pre-decision module, the block chain network load is effectively reduced, and a dynamic elastic PBFT consensus mechanism adjusts the scale of verification nodes in real time based on transaction types, so that the efficiency is improved. And compared with a traditional fixed node consensus mechanism, the network communication overhead is reduced. And a hash value asynchronous pre-verification technology is adopted, so that the interaction rate of the multi-dimensional power data is improved, and the efficiency is improved compared with that of a conventional data uplink mode. Through the decoupling design of the intelligent contract execution flow and the consensus verification flow and in combination with the pre-decision data cache queue, the resource scheduling instruction issuing time delay is reduced. The block chain transaction throughput is improved and is effectively improved compared with a standard alliance chain architecture, and the technical problems that the consensus mechanism processing transaction speed of an existing block chain is limited and the high-frequency real-time data interaction requirement of a virtual power plant is difficult to meet are successfully solved.

A computer system for compacting video data. The system acquires a video stream, reduces redundancy through pre-processing, and analyzes the stream to identify patterns and irregularities. It detects spatial or temporal anomalies in the video and produces three outputs: a conditioned video stream based on statistical analysis, an error stream reflecting adjustments made during conditioning, and an anomaly meta-stream containing metadata about detected anomalies. The system communicates with one or more remote systems to synchronize and negotiate a compatible compression codebook, optionally exchanging compact updates that represent differences between local and remote codebooks. The conditioned video stream is then compressed using the agreed codebook. The system outputs a compacted representation of the video that includes the compressed stream, the error stream, and the anomaly metadata, supporting efficient storage or transmission while maintaining the ability to detect, trace, and reconstruct key information within the video.

The invention provides an edge computing remote medical task offloading method based on an improved PPO algorithm, and the method comprises the steps: dynamically selecting an optimal offloading mode through a PPO-MED intelligent decision module according to the characteristics of a medical task and a system resource state, and enabling the offloading mode to comprise a local computing offloading mode (LCO), an edge server offloading mode (ESO) and an edge intelligent preprocessing offloading mode (EIPO); a PPO-MED algorithm models a medical task unloading problem into a Markov decision process, and an unloading strategy is optimized through deep reinforcement learning. The improved PPO-MED algorithm realizes intelligent decision, can dynamically select the optimal unloading strategy according to the real-time system state, task urgency and resource availability, significantly improves the adaptive capacity and service quality of the system in a complex environment, compresses the original medical data into a plurality of orders of magnitude, and improves the reliability of the system. On the premise that the diagnosis accuracy is guaranteed, the transmission delay is remarkably reduced, and the method is particularly suitable for remote diagnosis application of basic medical institutions under the limited bandwidth condition.

A system and method for data compaction with adaptive codebook statistical estimates. Training data sets determine sourceblock frequencies while a dynamic mismatch probability system continuously refines estimates based on observed patterns. Context-aware handling selects appropriate secondary encoding methods for different data types (text, binary, image, executable). Machine learning models predict optimal mismatch probabilities from extracted features. Edge-optimized training enables codebook development on resource-constrained devices with intelligent resource management. Differential updates transmit only changes between codebook versions, minimizing bandwidth usage. Federated learning enables multiple devices to contribute to shared codebooks while maintaining data privacy. A secure synchronization protocol with authentication and verification ensures codebook consistency. The distributed maintenance method provides systematic validation, conflict resolution, and optimization across device networks, enabling efficient encoding across heterogeneous systems.

The invention relates to the technical field of chemical production monitoring, and discloses a cloud computing-based chemical production monitoring system, which is characterized in that an on-site data acquisition module is used for acquiring chemical production data in real time and carrying out filtering and denoising processing to obtain preprocessed data; the network transmission module is used for encrypting the transmitted data and reducing the data transmission quantity by adopting a data compression technology; the cloud computing platform module is used for dispersedly storing the data on a plurality of storage nodes by utilizing a distributed storage technology, classifying the data, analyzing the data in real time by adopting a streaming computing technology, detecting the abnormal condition of the data, and immediately triggering an alarm mechanism when the abnormal condition exceeds a preset safety threshold value; the monitoring application module is used for displaying various parameters and equipment operation states, and providing decision suggestions for production optimization in combination with a chemical production process and equipment; the production efficiency can be improved, the production cost can be reduced, the product quality can be improved, and the market competitiveness of enterprises can be enhanced.

A system and method for data compaction and encryption of anonymized data records. A dataset may be pre-processed by dividing into sourceblocks at reasonable intervals and tallying each sourceblock's frequency, creating a tally record of tokens and count values. This tally record may then be anonymized and transmitted to a data deconstruction engine which combined with a library manager creates a codebook and performs optimization techniques on the codebook. The data deconstruction engine and library manager may be distributed across multiple nodes or devices. The received anonymized tally record may be parsed into individual tokens by identifying the tokens with the highest count value. The tokens may then be sent descending order of count value to the library manger where each token may be assigned a codeword. A half-backed codebook is then created using the tokens and each token's unique codeword, before sending the half-backed codebook to a system user.

The invention discloses a complex environment high-precision positioning system based on multi-signal-source cooperation, and belongs to the technical field of cooperative positioning, and the system specifically comprises the steps: receiving satellite, cellular base station and wireless access point signals based on a multi-source signal extraction module, and extracting carrier phase, propagation time delay and multipath reflection intensity distribution; the reflection weight generation module is used for identifying environmental characteristics according to multipath reflection intensity and generating a fusion weight set containing satellite phase dominant factors and the like; the measurement value constraint conversion module converts the measurement value into three types of distance constraint quantities; the constraint fusion field construction module carries out nonlinear superposition on the three types of constraint quantities according to weights to generate a three-dimensional probability density field; the elevation coordinate resolving module fuses the air pressure data compression resolving space, and positioning coordinates are obtained through iterative convergence; according to the invention, through cooperation of multi-source signals, the positioning precision and reliability in a complex environment are effectively improved.

The invention discloses an NVMe storage array virtualization device based on an FPGA (Field Programmable Gate Array), which is characterized in that single I / O (Input / Output) virtualization (SR-IOV) is realized in the FPGA, and a plurality of NVMe virtual functions which can be directly accessed by a virtual machine are externally provided; mapping from a virtual volume logic address to a physical storage address is completed in hardware, striping management and RAID verification calculation are carried out on a plurality of NVMe solid state disks at the rear end, and storage resource pooling is achieved; the partial reconfigurable characteristic of the FPGA is utilized, I / O scheduling, data compression and prefetching strategies are dynamically adjusted, and the adaptability and performance efficiency of the system are improved. Hardware unloading of the whole I / O path is completed in the FPGA, intervention of a host CPU is not needed, delay is remarkably reduced, the throughput capacity is improved, service quality isolation and elastic expansion in a multi-tenant environment are supported, and an effective technical path is provided for constructing a next-generation high-performance and low-delay storage infrastructure.

The invention discloses an electric vehicle transportation state real-time monitoring method based on multi-sensor fusion, and belongs to the field of electric vehicle transportation state real-time monitoring. A comprehensive state index is obtained through integration of the multi-source data fusion model; acquiring a data transmission priority in the comprehensive state index, and if the data transmission priority is higher than a communication bandwidth threshold, allocating an additional transmission time sequence to obtain a dynamic communication scheduling sequence; carrying out resource allocation on the vehicle-mounted communication system according to the dynamic communication scheduling sequence, and judging whether key safety data such as battery overheating early warning is in the first position of the sequence or not so as to determine a real-time transmission path; and monitoring a communication congestion index in the real-time transmission path, and if the communication congestion index exceeds a threshold value, activating a standby data compression mechanism to obtain a reliable transmission data packet, and sending the reliable transmission data packet to a monitoring center.

The invention discloses an unmanned aerial vehicle group collaborative path planning method and system, a product and a medium, and relates to the field of non-electrical variable control or adjustment systems.The method comprises the steps that environmental data such as terrain height, obstacle distribution and dynamic targets in an unmanned aerial vehicle detection range are collected and converted into a grid unit space occupation state, a search space is divided into three-dimensional grids, and the three-dimensional grids are divided into three-dimensional data; and calculating an unexplored probability value of each unit, extracting time dimension change data of the value and gradient information between the units, identifying an environment dynamic change area, determining boundary features and generating path node data. Node data is compressed into lightweight path information containing coordinates, priority and time sequence constraints, and a track sequence is generated and distributed by combining the position of the unmanned aerial vehicle. By implementing the method, the detection efficiency of the unmanned aerial vehicle can be improved.

Provided is a data compaction method that includes: determining, by an electronic device, a plurality of Sorted String Table (SSTable) files on which a compaction operation is to be performed among SSTable files stored in a storage device connected to the electronic device, the data in the SSTable files stored in the storage device is stored based on a Log Structured Merge Tree (LSM-Tree) mechanism; based on receiving the plurality of SSTable files from the storage device, performing, by the electronic device, merge sorting on data in data blocks of the plurality of SSTable files based on indexes of the data in the data blocks of the plurality of SSTable files, to obtain reconstructed data blocks; and sending, by the electronic device, the reconstructed data blocks to the storage device, the reconstructed data blocks being used by the storage device to obtain reconstructed SSTable files.

The invention discloses a million-account-level distillation mixed inspection method based on SAAS financial software, and the method achieves the cross-enterprise mixed inspection and correction and knowledge closed-loop feedback system through the establishment of a voucher abnormal distillation mechanism, solves the auditing efficiency problem of million-account-level financial vouchers under an SAAS platform, and is suitable for the scenes of account service, finance and tax auditing and the like. At the auditing efficiency level, the to-be-audited data is compressed through an abnormal voucher distillation mechanism, and the exponential-level shortening of the auditing period of millions of enterprises is realized in cooperation with batch processing; in the aspect of auditing quality, industry depth rules cover scenes such as cost allocation of the manufacturing industry and income confirmation of the service industry, and quantitative criteria are provided for differential indexes; the correction efficiency is fundamentally broken through, and a correction result conduction mechanism in the same industry is established for the first time; the knowledge precipitation system realizes automatic conversion from artificial experience to reusable rules, so that finance and tax policy changes can respond within 24 hours.

A system and method for data compaction and encryption of anonymized data records. A dataset may be pre-processed by dividing into sourceblocks at reasonable intervals and tallying each sourceblock's frequency, creating a tally record of tokens and count values. This tally record may then be anonymized and transmitted to a data deconstruction engine which combined with a library manager creates a codebook and performs optimization techniques on the codebook. The data deconstruction engine and library manager may be distributed across multiple nodes or devices. The received anonymized tally record may be parsed into individual tokens by identifying the tokens with the highest count value. The tokens may then be sent descending order of count value to the library manger where each token may be assigned a codeword. A half-backed codebook is then created using the tokens and each token's unique codeword, before sending the half-backed codebook to a system user.

A method includes obtaining, by a first communication apparatus, training data used to train a dictionary used for data compression. The method also includes determining a group of to-be-updated subsets in the dictionary based on the training data. The method further includes determining dictionary update information corresponding to the group of to-be-updated subsets. The method additionally includes sending the dictionary update information.

This specification relates to file format-based transparent encryption tailored for big data. In some aspects, a method includes receiving a write request including a table with one or more columns to be stored in a storage device; compressing table data in a unit of block, wherein each column includes a number of blocks; generating a column key for each column and a block key for each block including sensitive information; encrypting (i) each block including sensitive information with a corresponding block key and (ii) rest of blocks in each column with a corresponding column key; generating wrapped keys for the column keys and block keys and storing the wrapped keys into a key file; and storing the encrypted blocks of each column into a data file in a data folder of the storage device.

The application discloses a kind of low-code application quick construction method, including the following steps: data preparation, application construction, application release, construction is completed, finally generates application module;First, data preparation is carried out, obtains the construction resource data required for application construction, is compressed and uploaded to construction module, then construction module decompresses data and passes through visual construction page, business logic and data structure, to design page design application function in the way of drag and drop.By this method, non-professional personnel can also directly build the application module that oneself needs, improve the application building speed and user-friendly.

A combined satellite and mobile weather forecasting system may include a satellite payload having a plurality of sensors configured to capture atmospheric and environmental data including atmospheric radiances and vertical soundings and a receiver for receiving data from other satellites. The satellite payload may also include a processing unit configured to condense the captured atmospheric and environmental data and the received data from other satellites into initialization data to generate a weather forecast. The satellite payload may include a transmission device configured to communicate with a mobile cellular device and to transmit the initialization data. The mobile cellular device may be configured to receive the initialization data from the transmission device of the satellite payload to generate the weather forecast using the initialization data.

The invention relates to the technical field of Internet of Things, and particularly discloses an Internet of Things data acquisition method and system, and the method comprises the steps: constructing a three-stage data acquisition architecture composed of a main control node, edge acquisition nodes and terminal sensing equipment; the terminal sensing equipment dynamically adjusts own data acquisition frequency based on a built-in activeness evaluation module; the edge acquisition node receives the data sent by the terminal sensing device and executes distributed data verification and data compression processing; the main control node schedules the processed data uploaded by the edge acquisition node according to the network load condition and the data priority; and the main control node performs aggregation analysis on the received data, and optimizes the acquisition strategy of the whole network based on the analysis result. The acquisition frequency can be automatically adjusted according to the equipment state and the data change, invalid data acquisition is reduced, and the equipment energy consumption is reduced. According to the distributed data verification mechanism, the integrity and non-tampering property of the data are ensured, and the credibility of the data is improved.

A server apparatus disclosed in the present document may comprise a communication circuit, a memory, a time series database, and at least one processor. The memory may store instructions that cause the server apparatus to: store, in a first table, a label included in time series data and a first hash value corresponding to the label; receive the time series data from an external device through the communication circuit; generate a second hash value by hashing the label of the received time series data; check whether the second hash value is included in the first table; if the second hash value is included in the first table, store, in a second table, the time series data from which the label has been removed and the second hash value; check data prior to a reference time, from among the time series data stored in the second table; and compress a designated data range from among the data prior to the reference time and store same in a third table.

A system and method for lossy precompression for data compaction using automated model monitoring and training, wherein statistical analyses of test datasets are used to determine if the probability distribution of two datasets are within a pre-determined range, and responsive to that determination new encoding and decoding algorithms may be retrained in order to produce new data sourceblocks, and pre-compression of data prior to processing and statistical analysis allows for the compaction of already compressed data into highly dense formats. The new data sourceblocks may then be processed and assigned new codewords which are compiled into an updated codebook which may be distributed back to encoding and decoding systems and devices.

The utility model discloses a large-capacity bandwidth self-adaptive transmission device based on a CPRI (common public radio interface) protocol, which comprises a data compression module, a data mapping cache module and a CPRI framing module which are sequentially connected along a data sending direction, and a CPRI deframing module, a data demapping cache module and a data decompression module which are sequentially arranged along a data receiving direction. According to the technical scheme provided by the embodiment of the invention, the total bandwidth of CPRI transmission data can be improved on the premise of ensuring the system EVM and the data precision, and the support and the flexibility of base frame segmentation mapping of data with different bandwidths are improved.

This invention discloses a centralized management method for a data center cluster, relating to the field of data management technology. The method includes acquiring the original dataset from the database cluster; deduplicating the original dataset based on the number of samples in the important score dataset to obtain a deduplicated dataset; randomly dividing the data within the deduplicated dataset based on the number of samples to obtain multiple granular spheres; classifying the data using the granular spheres; compressing and encrypting the classified data based on importance scores to obtain encrypted data; and migrating and storing the encrypted data based on importance scores. By deduplicating a large amount of data in the database, redundant data is reduced; by using granular spheres and calculating dispersion, the data becomes more compact; and by classifying the data and storing it according to its importance level, efficient data management is achieved. Furthermore, data compression reduces storage costs, and encryption protects data security, thus enabling efficient data management.

An example memory sub-system includes a memory device and a processing device, operatively coupled to the memory device. The processing device is configured to receive a memory access request specifying a logical address of a data item and a memory access operation to be performed with respect to the data item; produce a truncated logical address by applying a predefined mathematical transformation to the specified logical address; identifying, in an address translation table, an address translation table entry identified by the truncated logical address; and perform the memory access operation using a physical address specified by the address translation table entry.

The invention relates to the technical field of bridge detection, in particular to a steel-concrete composite bridge multistage damage identification method and system.The steel-concrete composite bridge multistage damage identification system comprises a data acquisition module, a data transmission module, a data analysis module and a decision module, and the data acquisition module comprises an acquisition unit, a preprocessing unit and a compression unit; the acquisition unit acquires bridge structure response and environment information in real time to obtain acquired data, the preprocessing unit preprocesses the acquired data to obtain processed data, the compression unit unifies and compresses the processed data to obtain compressed data, and the data analysis module decompresses and restores the processed data, analyzes the data and identifies the type and degree of bridge damage. And the decision module displays the damage position, degree and evolution trend in real time and generates a reinforcement decision, and the system realizes full-process automatic monitoring of the steel-concrete composite bridge from early warning to accurate quantitative evaluation through a hierarchical and multi-technology fused architecture, and can significantly improve the efficiency and reliability of bridge safety monitoring.

A method for managing metadata blocks of file system entities (FSEs), the method includes (i) determining, by a processing circuit of a storage system, a stress level that is non-linearly dependent upon a fullness of static FSE metadata blocks and a fullness of dynamic FSE metadata blocks, wherein the static FSE metadata blocks store FSE metadata items of a first type and the dynamic FSE metadata blocks store FSE metadata items of a second type and additional FSE metadata items of a first type; (ii) determining, based on the stress level and a fullness parameter, a probability of applying a conditional compression process on a new FSE metadata block; and (iii) applying the conditional compression process, based on the probability, on the new FSE metadata block.

A communication method and apparatus, which can flexibly adjust the degree of module parallelization, thereby achieving a balance between a service latency and resource occupancy. The method comprises: a first communication apparatus determining to update the number of first modules to a first number, and sending first information to a second communication apparatus, so as to instruct the update of the number of first modules; and after receiving the first information, the second communication apparatus configuring a first number of first modules on the basis of the first information, wherein each first module is used for realizing data compression / decompression, and can be, for example, an ROHC module in a PDCP entity. For example, the first communication apparatus can determine the first information on the basis of resource conditions of a sending end and / or a receiving end, wherein the resource conditions can comprise, for example, a QoS configuration file of a service, a service traffic condition, and a hardware utilization / load condition. As a first communication apparatus can send information used for updating the number of first modules, the number of first modules can be flexibly adjusted, thereby achieving a balance between a service latency and resource occupancy.

A system and method for data compaction and encryption of anonymized data records. A dataset may be pre-processed by dividing into a plurality of sourceblocks at all reasonable sourceblock lengths, and then counting how many times each sourceblock occurs in the dataset, resulting in a tally record of tokens and their count value. This tally record may then be anonymized and transmitted to a data deconstruction engine which combined with a library manager creates a codebook and performs optimization techniques on the codebook. The received anonymized tally record may be parsed into individual tokens by identifying the tokens with the highest count value. The tokens may then be sent, in descending order of count value, to the library manger where each token may be assigned a codeword. A half-backed codebook is then created using the tokens and each token's unique codeword, before sending the half-backed codebook to a system user.

The present invention discloses an intelligent fusion analysis method for multi-source confidential data based on an engineering management system, which relates to the technical field of information security analysis of intelligent engineering management systems, and comprises the following steps: S1, system initialization, S2, generation of a privacy mask, S3, confidential transmission of multi-source data, and S4, fault-tolerant confidential fusion analysis; the present application logically forms a group ring structure with intelligent terminal sensor equipment and an engineering management system control center, generates group secret parameters for generating message authentication codes, and when generating ciphertext data, even if the group password parameters are leaked, key plaintext information will not be leaked, thereby effectively enhancing data confidentiality; the application has a transmission fault-tolerant function, and even if a small number of intelligent terminal sensor equipment have problems in the network and fail to transmit data in time, as long as the system reaches a threshold value, it will not affect the normal intelligent fusion analysis of the system; the multi-dimensional data is compressed into a single data block, effectively reducing the communication overhead of data transmission.

The invention provides a data submission method, system and device and a storage medium, and the data submission method can effectively improve the data compression efficiency and facilitates the reduction of the data transmission and storage cost by carrying out column type compression processing on data needing to be submitted under data interfaces of different agent systems at the data level; on the data interface management level, comprehensive and systematic rule carding and integration can be carried out on different data interfaces through the established hot plug supervision rule base, online updating and real-time effectiveness of interface rules can be achieved, and shutdown adaptation of an entrusted party system is avoided; in the aspect of data storage, the row-level version number is configured for the data row in each submitted data, so that transmission type row-level version management of the historical submitted data is realized, and the data row of a specific version can be quickly positioned and processed when the historical submitted data is objection.