118 results about "Data compaction" patented technology

A system and method for data compaction and encryption of anonymized data records. A dataset may be pre-processed by dividing into sourceblocks at reasonable intervals and tallying each sourceblock's frequency, creating a tally record of tokens and count values. This tally record may then be anonymized and transmitted to a data deconstruction engine which combined with a library manager creates a codebook and performs optimization techniques on the codebook. The data deconstruction engine and library manager may be distributed across multiple nodes or devices. The received anonymized tally record may be parsed into individual tokens by identifying the tokens with the highest count value. The tokens may then be sent descending order of count value to the library manger where each token may be assigned a codeword. A half-backed codebook is then created using the tokens and each token's unique codeword, before sending the half-backed codebook to a system user.

The invention discloses an NVMe storage array virtualization device based on an FPGA (Field Programmable Gate Array), which is characterized in that single I / O (Input / Output) virtualization (SR-IOV) is realized in the FPGA, and a plurality of NVMe virtual functions which can be directly accessed by a virtual machine are externally provided; mapping from a virtual volume logic address to a physical storage address is completed in hardware, striping management and RAID verification calculation are carried out on a plurality of NVMe solid state disks at the rear end, and storage resource pooling is achieved; the partial reconfigurable characteristic of the FPGA is utilized, I / O scheduling, data compression and prefetching strategies are dynamically adjusted, and the adaptability and performance efficiency of the system are improved. Hardware unloading of the whole I / O path is completed in the FPGA, intervention of a host CPU is not needed, delay is remarkably reduced, the throughput capacity is improved, service quality isolation and elastic expansion in a multi-tenant environment are supported, and an effective technical path is provided for constructing a next-generation high-performance and low-delay storage infrastructure.

The invention discloses an electric vehicle transportation state real-time monitoring method based on multi-sensor fusion, and belongs to the field of electric vehicle transportation state real-time monitoring. A comprehensive state index is obtained through integration of the multi-source data fusion model; acquiring a data transmission priority in the comprehensive state index, and if the data transmission priority is higher than a communication bandwidth threshold, allocating an additional transmission time sequence to obtain a dynamic communication scheduling sequence; carrying out resource allocation on the vehicle-mounted communication system according to the dynamic communication scheduling sequence, and judging whether key safety data such as battery overheating early warning is in the first position of the sequence or not so as to determine a real-time transmission path; and monitoring a communication congestion index in the real-time transmission path, and if the communication congestion index exceeds a threshold value, activating a standby data compression mechanism to obtain a reliable transmission data packet, and sending the reliable transmission data packet to a monitoring center.

A method includes obtaining, by a first communication apparatus, training data used to train a dictionary used for data compression. The method also includes determining a group of to-be-updated subsets in the dictionary based on the training data. The method further includes determining dictionary update information corresponding to the group of to-be-updated subsets. The method additionally includes sending the dictionary update information.

This specification relates to file format-based transparent encryption tailored for big data. In some aspects, a method includes receiving a write request including a table with one or more columns to be stored in a storage device; compressing table data in a unit of block, wherein each column includes a number of blocks; generating a column key for each column and a block key for each block including sensitive information; encrypting (i) each block including sensitive information with a corresponding block key and (ii) rest of blocks in each column with a corresponding column key; generating wrapped keys for the column keys and block keys and storing the wrapped keys into a key file; and storing the encrypted blocks of each column into a data file in a data folder of the storage device.

The application discloses a kind of low-code application quick construction method, including the following steps: data preparation, application construction, application release, construction is completed, finally generates application module;First, data preparation is carried out, obtains the construction resource data required for application construction, is compressed and uploaded to construction module, then construction module decompresses data and passes through visual construction page, business logic and data structure, to design page design application function in the way of drag and drop.By this method, non-professional personnel can also directly build the application module that oneself needs, improve the application building speed and user-friendly.

A combined satellite and mobile weather forecasting system may include a satellite payload having a plurality of sensors configured to capture atmospheric and environmental data including atmospheric radiances and vertical soundings and a receiver for receiving data from other satellites. The satellite payload may also include a processing unit configured to condense the captured atmospheric and environmental data and the received data from other satellites into initialization data to generate a weather forecast. The satellite payload may include a transmission device configured to communicate with a mobile cellular device and to transmit the initialization data. The mobile cellular device may be configured to receive the initialization data from the transmission device of the satellite payload to generate the weather forecast using the initialization data.

The invention relates to the technical field of Internet of Things, and particularly discloses an Internet of Things data acquisition method and system, and the method comprises the steps: constructing a three-stage data acquisition architecture composed of a main control node, edge acquisition nodes and terminal sensing equipment; the terminal sensing equipment dynamically adjusts own data acquisition frequency based on a built-in activeness evaluation module; the edge acquisition node receives the data sent by the terminal sensing device and executes distributed data verification and data compression processing; the main control node schedules the processed data uploaded by the edge acquisition node according to the network load condition and the data priority; and the main control node performs aggregation analysis on the received data, and optimizes the acquisition strategy of the whole network based on the analysis result. The acquisition frequency can be automatically adjusted according to the equipment state and the data change, invalid data acquisition is reduced, and the equipment energy consumption is reduced. According to the distributed data verification mechanism, the integrity and non-tampering property of the data are ensured, and the credibility of the data is improved.

A server apparatus disclosed in the present document may comprise a communication circuit, a memory, a time series database, and at least one processor. The memory may store instructions that cause the server apparatus to: store, in a first table, a label included in time series data and a first hash value corresponding to the label; receive the time series data from an external device through the communication circuit; generate a second hash value by hashing the label of the received time series data; check whether the second hash value is included in the first table; if the second hash value is included in the first table, store, in a second table, the time series data from which the label has been removed and the second hash value; check data prior to a reference time, from among the time series data stored in the second table; and compress a designated data range from among the data prior to the reference time and store same in a third table.

This invention discloses a centralized management method for a data center cluster, relating to the field of data management technology. The method includes acquiring the original dataset from the database cluster; deduplicating the original dataset based on the number of samples in the important score dataset to obtain a deduplicated dataset; randomly dividing the data within the deduplicated dataset based on the number of samples to obtain multiple granular spheres; classifying the data using the granular spheres; compressing and encrypting the classified data based on importance scores to obtain encrypted data; and migrating and storing the encrypted data based on importance scores. By deduplicating a large amount of data in the database, redundant data is reduced; by using granular spheres and calculating dispersion, the data becomes more compact; and by classifying the data and storing it according to its importance level, efficient data management is achieved. Furthermore, data compression reduces storage costs, and encryption protects data security, thus enabling efficient data management.

A method for managing metadata blocks of file system entities (FSEs), the method includes (i) determining, by a processing circuit of a storage system, a stress level that is non-linearly dependent upon a fullness of static FSE metadata blocks and a fullness of dynamic FSE metadata blocks, wherein the static FSE metadata blocks store FSE metadata items of a first type and the dynamic FSE metadata blocks store FSE metadata items of a second type and additional FSE metadata items of a first type; (ii) determining, based on the stress level and a fullness parameter, a probability of applying a conditional compression process on a new FSE metadata block; and (iii) applying the conditional compression process, based on the probability, on the new FSE metadata block.

A communication method and apparatus, which can flexibly adjust the degree of module parallelization, thereby achieving a balance between a service latency and resource occupancy. The method comprises: a first communication apparatus determining to update the number of first modules to a first number, and sending first information to a second communication apparatus, so as to instruct the update of the number of first modules; and after receiving the first information, the second communication apparatus configuring a first number of first modules on the basis of the first information, wherein each first module is used for realizing data compression / decompression, and can be, for example, an ROHC module in a PDCP entity. For example, the first communication apparatus can determine the first information on the basis of resource conditions of a sending end and / or a receiving end, wherein the resource conditions can comprise, for example, a QoS configuration file of a service, a service traffic condition, and a hardware utilization / load condition. As a first communication apparatus can send information used for updating the number of first modules, the number of first modules can be flexibly adjusted, thereby achieving a balance between a service latency and resource occupancy.

The invention provides a data submission method, system and device and a storage medium, and the data submission method can effectively improve the data compression efficiency and facilitates the reduction of the data transmission and storage cost by carrying out column type compression processing on data needing to be submitted under data interfaces of different agent systems at the data level; on the data interface management level, comprehensive and systematic rule carding and integration can be carried out on different data interfaces through the established hot plug supervision rule base, online updating and real-time effectiveness of interface rules can be achieved, and shutdown adaptation of an entrusted party system is avoided; in the aspect of data storage, the row-level version number is configured for the data row in each submitted data, so that transmission type row-level version management of the historical submitted data is realized, and the data row of a specific version can be quickly positioned and processed when the historical submitted data is objection.

The invention discloses a regional power load prediction and scheduling service platform and method based on cloud edge collaboration, and relates to the technical field of big data processing. The regional power load prediction and scheduling service platform and method based on cloud edge collaboration comprises the steps of S1, collecting equipment operation data and instruction scheduling data of regional power for preprocessing operation, and compressing and storing the data to construct a regional power scheduling management database; s2, analyzing the synchronization state between the cloud and the edge node by analyzing the event time difference and the equipment state deviation; s3, evaluating the conflict degree between the cloud and the edge node by analyzing the equipment state, the load difference, the synchronous pressure and the response characteristics; and S4, according to the equipment state change, the frequency fluctuation and the conflict detection analysis result, performing oscillation risk quantitative evaluation. The problem of insufficient load adjustment stability caused by asynchronous data states, easy conflict of scheduling instructions and equipment oscillation in power load prediction in communication interruption and recovery scenes is solved.

The invention discloses a computing power sharing method, device and system of an air conditioner, a storage medium and a computer program product.The method comprises the steps that operation state data of the air conditioner and task parameters of a computing power task to be executed are obtained, and the operation state data comprise temperature control related data of the air conditioner and working state data of a compressor; the task parameters comprise computing power demand parameters; based on a preset thermal computing power balance strategy, computing power output configuration is determined according to the operation state data and the task parameters, and the thermal computing power balance strategy is used for distributing computing power resources on the premise that the air conditioner core temperature control function and equipment operation safety are guaranteed; and controlling the air conditioner to execute the to-be-executed computing power task according to the computing power output configuration to realize sharing of idle computing power of the air conditioner. According to the scheme, the air conditioner idle computing power is effectively activated, the conversion efficiency of the air conditioner idle computing power to revenue edge computing resources is improved, the air conditioner core temperature control function and equipment operation safety in the computing power sharing process are guaranteed, and air conditioner idle computing power sharing is achieved.

The disclosure relates to a communication method and system for converging a 5th-Generation (5G) communication system for supporting higher data rates beyond a 4th-Generation (4G) system with a technology for Internet of Things (IoT). The disclosure can be applied to smart services, such as smart home, smart building, smart city, smart car, connected car, health care, digital education, smart retail, security and safety services, based on 5G communication technology and IoT-related technology. The disclosure discloses a method and apparatus for reducing overhead of an Ethernet frame, a method and apparatus for efficiently performing a connection resume procedure of a terminal, and a method and apparatus for preventing data loss when performing an uplink user data compression procedure.

The invention relates to quantum key distribution, in particular to a passive optical network system based on continuous variables and a multi-user key distribution method, and the method comprises the following steps: a provider is used as a network center and is responsible for generating a randomly modulated continuous variable coherent state; the quantum broadcast channel is used for transmitting a coherent state from a provider to the passive optical beam splitter by taking an untrusted single-mode optical fiber as a medium; the passive optical beam splitter is a passive core device and is used for definitely distributing the coherent state into a plurality of parts according to energy and distributing the coherent state to a plurality of user terminals; the user terminal measures the orthogonal component of the received optical signal, extracts the quantum random number of the original code, and completes the receiving and analysis of the quantum information; the classical authentication channel is used for establishing a secure classical communication link between a provider and a user terminal and is used for key post-processing negotiation and compressing coordinated data into a final secure key according to a parameter estimation result and a security protocol; according to the invention, the defect that deterministic and simultaneous multi-user key distribution is difficult to realize can be overcome.

The present disclosure relates to a smartNIC-based network traffic acceleration device and method, the device comprising: a memory for storing at least one instruction for smartNIC-based network traffic acceleration; and a processor for performing an operation according to the instruction, wherein the processor performs network optimization in a WAN environment through smartNIC on the basis of a WAN optimizer using the smartnNIC, and lightweights data and compresses artificial intelligence data so as to optimize data transmission.

Preclusive data decompression failure techniques, such as may avoid data decompression failures using predictive decompression failure indications and / or configure communication devices for data transmission recovery before reordering window timer expiration, are described. Preclusive data compression failure operation may, for example, initiate compressor memory resets conditionally according to some aspects of the present disclosure, such as using one or more trigger events to predictively indicate decompression failure. Additionally or alternatively, preclusive data compression failure operation may provide for configuring communication devices for data transmission recovery before reordering window timer expiration, such as configuration for prioritization to provide data transmission recovery before reordering window timer expiration. Other aspects and features are also claimed and described.

The present application relates to the field of data storage, and more particularly to an agricultural data storage method for smart villages, which comprises: obtaining standardized agricultural environment data and farm operation log data by preprocessing and windowing agricultural environment raw data; obtaining an initial mutation intensity factor of data points by performing multidimensional minimum distance calculation on local historical baseline; obtaining an event semantic consistency factor by comprehensively analyzing the mutation dimension distribution of the agricultural environment data points and the operation log matching; obtaining a time series mutation weight factor by fusion and normalization processing of the initial mutation intensity factor and the event semantic consistency factor of the agricultural environment data points; and obtaining a representative data centroid by introducing a weighted K-means clustering of the time series mutation weight factor, and storing the data by compression through the representative data centroid, thereby solving the problem that key farm events are easily missed in the time series data compression process.

Embodiments of the inventive concept include solid state drive (SSD) multi-card adapters that can include multiple solid state drive cards, which can be incorporated into existing enterprise servers without major architectural changes, thereby enabling the server industry ecosystem to easily integrate evolving solid state drive technologies into servers. The SSD multi-card adapters can include an interface section between various solid state drive cards and drive connector types. The interface section can perform protocol translation, packet switching and routing, data encryption, data compression, management information aggregation, virtualization, and other functions.

The invention relates to an aerodynamic stability quantification method and system based on flow mechanism and deep learning coupling, and the method comprises the steps: compressing the high-dimensional point cloud data of a manufactured blade into a low-dimensional basic mode based on principal component analysis, and effectively relieving the variable dimension disaster problem caused by geometric uncertainty; and then, by taking the low-dimensional fundamental mode weight as input, directly embedding the key physical characteristic, namely the blade tip leakage flow rate ratio, into a deep neural network hidden layer and taking the gas compressor stability margin variable quantity as output, constructing a physical constraint and data driving collaborative optimization framework, and remarkably improving the prediction precision of the pneumatic stability of the gas compressor under the small sample condition. According to the method, a quantification tool with high precision and physical credibility is provided for solving the problem of performance degradation caused by geometric deviation of the CAES multi-stage axial flow compressor under the non-design working condition, and key technical support is provided for achieving robustness design of the CAES multi-stage axial flow compressor.

The invention relates to an operation and maintenance fault positioning method, an operation and maintenance fault positioning system and a storage medium. The operation and maintenance fault positioning method comprises the following steps: obtaining operation and maintenance summary data of different operation and maintenance fields, and compressing the operation and maintenance summary data into structured summary data; inputting the structured abstract data into a large language model inference engine for fault causal analysis to obtain initial verification information output by the large language model inference engine; sending the initial verification information to a policy scheduling agent node, so that the policy scheduling agent node sends a verification instruction to a target verification agent node corresponding to the initial verification information based on the initial verification information; and according to a verification result fed back by the target verification agent node, determining a target confidence coefficient of the initial verification information, and when the target confidence coefficient is greater than or equal to a target threshold value, generating a fault root cause report based on the initial verification information. The fault positioning accuracy can be improved, and the fault positioning timeliness is improved.

A method in an illustrative embodiment includes registering a storage system of a server with a programmable switch for transmitting and compressing data. The method further includes transmitting read data and write data associated with the storage system between the server and a client by the programmable switch. The method further includes determining a compression ratio of the read data and a compression ratio of the write data for a connection between the client and the programmable switch. The method further includes detecting a network attack based on the compression ratio of the read data and the compression ratio of the write data. In this way, abnormal data can be quickly and effectively identified without consuming a large quantity of resources, and whether a network attack exists can be determined, thereby maintaining the performance of a storage server while reducing the consumption of resources.

The invention discloses an anchor cable force monitoring method based on a fundamental frequency vibration method. The anchor cable force monitoring method comprises the following steps of sensor deployment and signal acquisition; the acquisition nodes separate interference components in the vibration signals through denoising, the interference components are transmitted to the main acquisition module through a wireless communication link after data compression, and the main acquisition module uploads the interference components to the monitoring platform; the wireless communication link is a LoRa wireless communication link, and the communication anti-interference capability is improved by adopting a channel frequency hopping and signal power adaptive adjustment mechanism; fundamental frequency extraction and cable force calculation are conducted, specifically, the monitoring platform extracts vibration signal fundamental frequency through a VMD-HHT fusion method, a cable force model containing a boundary correction coefficient is constructed based on string vibration, and the fundamental frequency and anchor cable parameters are combined to determine actual cable force; triggering multi-stage early warning by combining the cable force value, the cable force change rate and the abnormal condition of the vibration mode, pushing early warning information to a related system, and executing corresponding linkage measures; the invention further discloses a corresponding system. The anti-interference capability and the real-time performance and the accuracy of non-stationary vibration signal fundamental frequency extraction are improved; and a powerful guarantee is provided for accurate and reliable measurement of the cable force.

The application discloses a data table row-column dual storage conversion method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a first table; scanning, segmenting and compressing a second table to obtain a first data block; and storing the first data block into the first table to obtain a third table. The method can generate the first data block, i.e. a column storage data block, by segmenting the second table, save the first data block in the first table, and merge the first data block with the second table, so that the row storage information and the column storage information can exist in one table simultaneously, and the performance requirement of OLTP and OLAP services cannot be met by a single storage format.

The invention provides a sandbox data clouding method and system based on MQTT, electronic equipment and a medium, and belongs to the technical field of vehicle data acquisition and reporting, the method comprises the following steps: acquiring vehicle-mounted data, the vehicle-mounted data being transmitted to a cloud end after a TBox compresses and encrypts data acquired by a vehicle end controller through a vehicle cloud protocol; analyzing the vehicle-mounted data, and performing data cleaning and storage backup; and reporting the vehicle-mounted data to a sandbox through an MQTT protocol, wherein the MQTT protocol adopts a theme partition, a QoS strategy and a load balancing mechanism. According to the invention, through efficient cloud data analysis, cleaning and storage mechanisms, the processing requirements of massive vehicle-mounted data are met; according to the method, a reasonable theme partition, a QoS strategy and a load balancing mechanism are designed based on the MQTT protocol, low-delay and high-reliability clouding of the vehicle-mounted data is realized, and the requirements of the sandbox on supervision and analysis of the automatic driving data are met.

The invention discloses an intelligent robot multi-protocol communication and task execution method, system and device, and belongs to the technical field of intelligent robots. The method comprises two core parts of protocol conversion and instruction issuing and task execution and data feedback, reliable instruction transmission is ensured through multi-protocol adaptive selection, serialized compression and closed-loop confirmation, and efficient task execution is ensured through precise management and control of a state machine, hierarchical data compression and multi-scene fault tolerance. According to the invention, the problems of single protocol adaptation, redundant data volume, unreliable transmission, disordered state management, low efficiency of data uploading, lack of feedback closed loop, weak fault-tolerant capability and the like in the prior art are solved, and the communication efficiency and task execution reliability of the intelligent robot in a complex scene are remarkably improved.

The invention discloses a meat detection data chaining and credibility verification method, and relates to the field of block chain and food data safety. The data is compressed to 2KB / bar through feature extraction, and the uplink delay is 0.28 seconds; rapid traceability (0.5 second) of the segmented meat is realized through main and side chain association; the intelligent contract is automatically verified (the accuracy is greater than or equal to 99%), and manual intervention is not needed. The method solves the problems of slow uplink, difficult traceability and complicated verification of the existing scheme, ensures credibility and traceability of detection data, and is suitable for meat safety full-chain traceability management.

Metadata associated with a file is received, where the metadata indicates whether data included in the file is transformable or not. The metadata of the file is processed to determine that the metadata indicates that the data is transformable. The data is transformed in accordance with indication provided in the metadata during garbage collection in a computational storage.