346 results about "Lossless compression" patented technology

The invention discloses a data compression transmission method and system applied to a ferry inspection image, and the method comprises the steps: collecting and obtaining the ferry inspection image in real time, recognizing a key inspection target region in the image, and carrying out the segmentation and partitioning of the image; compressing the key inspection target area based on lossless compression coding; the quantization step size is dynamically adjusted by comparing statistical variances of background pixels between continuous frames, and lossy compression coding is carried out on a background area; based on the boundary distance between the key inspection target area and the background area, adaptive compression coding is carried out on the transition area; constructing a hierarchical data packet; and constructing a data transmission optimization model, dynamically allocating data transmission links, and obtaining a transmission scheme with the highest total transmission. The method has the advantages that efficient data compression transmission is realized by accurately segmenting the image area and adopting a lossless, lossy and adaptive compression technology, the overall transmission efficiency is improved through intelligent transmission optimization, and the definition and real-time performance of the inspection image are ensured.

The invention relates to the technical field of computer vision and image recognition, in particular to an image recognition method based on edge computing, which comprises the following steps: dynamically capturing an original image through a plurality of edge nodes, rejecting redundant regions through a multi-modal perception triggering mechanism, and establishing a cooperative processing group. Illumination equalization, noise filtering and resolution self-adaptive compression tasks are distributed according to dynamic role election, a standardized preprocessed image is generated, a lightweight convolutional neural network is operated in parallel to extract a dual-channel feature vector, and after entropy coding lossless compression and equipment identity tag and time sequence stamp attachment, the dual-channel feature vector is transmitted to a cloud end by adopting a lightweight encryption protocol. The cloud end analyzes the data packet, reconstructs a feature topological graph based on space-time relevance, loads a depth residual error recognition model to execute feature fusion and classification decision, feeds back and updates the weight of an edge node model, solves the problems of low collaborative efficiency and feature distortion, and improves the efficiency and precision of image recognition.

The invention relates to the technical field of electric power management and control, and discloses an electric power BIM type data GIS discount classification method and system, and the method comprises the following steps: 1, constructing an electric power industry exclusive classification system and semantic mapping, and building a three-stage classification frame based on a GIS and an electric power industry standard; 2, unifying a space coordinate reference and a topological relation, and verifying topological consistency between components and between the components and topography and roads through a GIS (Geographic Information System) tool; 3, multi-scale classification and dynamic LOD model generation are carried out, and LOD model details are automatically switched according to a GIS scale; 4, performing data format conversion and intelligent lightweight processing, performing lossless compression on key components, and performing simplified compression on secondary components; and step 5, performing classification verification and dynamic updating mechanism, and comparing whether BIM and GIS base map position classification is correct or not, thereby realizing data synchronization. According to a dynamic classification framework and an intelligent lightweight strategy in the power industry, the BIM model efficiently adapts to the spatial analysis requirement of the GIS, and the data fusion and application efficiency is improved.

The invention is suitable for the technical field of power transmission line alarm, and provides a power transmission line alarm method based on fault data flow dynamic analysis and lightweight compression, and the method comprises the steps: a terminal collects line state data in real time through a multi-mode sensor disposed on a power transmission line; the terminal carries out lightweight compression processing on the line state data, lossless compression is carried out on steady state data by using improved Huffman coding, and lossy compression is carried out on transient fault data by using combination of wavelet transform and sparse coding; the terminal uploads the compressed data to the cloud according to a preset priority scheduling rule; the cloud end automatically identifies a data format and decompresses the data format through a protocol non-inductive analysis engine, and reconstructs a transient waveform; performing space-time correlation analysis on the analyzed line data, meteorological data and a historical fault library, and calculating a fault confidence coefficient; and according to the fault diagnosis result, generating a grading alarm instruction, and transmitting the grading alarm instruction to the terminal for visualization, thereby improving the power grid fault response speed and the operation and maintenance intelligent level.

The invention discloses a time series data lossless compression method and system based on dynamic context awareness, and relates to the field of data lossless compression, and the method comprises the steps: extracting target data from a device port, and carrying out the preprocessing of the target data through a data processing technology according to the business dimension corresponding to the target data, constructing a structured data set based on a preprocessing result; capturing local time sequence features based on the structured data set, distributing attention weights in combination with a hidden state, generating a feature vector used for sensing a context, and outputting a dimension vector by using a three-layer full-connection network; and according to the dimension vector, analyzing the compression performance score value of each compression algorithm in the resource library, sorting the compression performance score values, and selecting the compression algorithm meeting the target requirement to compress the target data. According to the method, the global periodic features and the local mutation features of the time series data are analyzed through the space-time attention mechanism, and real-time driving algorithm switching is achieved.

A method for streaming dynamically changing user interface data from a server to a client, including resizing user interface data in accordance with a client display, repeatedly transmitting in the form of IP packets, the user interface data in a lossy compression format, for each transmission, if an acknowledgement is received that all packets have been received, then increase the number of IP packets in the next transmission, and if the increased number of IP packets is sufficient to transmit the user interface data in a lossless compression format, then retransmit previously transmitted user interface data in the lossless compression format, including partitioning previously transmitted user interface data into display strips, and for each strip, if the user interface data is unchanged, transmit the strip in the lossless compression format, and if the user interface data has changed, return to the repeatedly transmitting for the changed user interface data.

The invention discloses a multi-edge-node collaborative industrial energy consumption data compression and storage method and system, particularly relates to the technical field of data compression and storage, and realizes hierarchical compression processing by layering industrial energy consumption data acquisition equipment into a main node, a core slave node and a terminal slave node. According to the method, various compression algorithms are used for carrying out lossy or lossless compression on different types of data, and cooperative compression and data redundancy elimination between edge nodes are realized under the coordination of a main node by introducing a distance calculation mechanism of'node hierarchy + auxiliary parameters'. The system constructs a node correlation graph through data abstracts, optimizes a data transmission path based on reinforcement learning, and adopts a cold and hot data classification strategy to improve the storage efficiency. The data compression efficiency and reliability are improved, and the method is suitable for real-time processing and long-term storage of large-scale industrial energy consumption data.

This disclosure relates generally to encoding and decoding of 3-dimensional (3D) mesh and specifically to encoding and decoding order and dependency between 3D positions of 3D vertices and 2D positions of 2D texture coordinates of a 3D mesh. In particular, the various embodiments in this disclosure provide a novel method and device to enhance lossless compression of 3D mesh models. The disclosed method and device prioritize encoding texture coordinates before encoding 3D positions. The disclosed method and device further employ angle-preserving position prediction. The disclosed approaches aim to optimize the compression process of 3D meshes by leveraging an inherent correlation between 3D structures and their corresponding 2D texture mappings.

The invention relates to the technical field of robot navigation and autonomous positioning, and discloses a real-time multi-modal map generation method and system, and the method comprises the steps: synchronously obtaining the position and posture of equipment and a spatial point cloud through a tight coupling mechanism of a laser radar and an inertial measurement unit, and forming original data input; on the basis, an incremental point cloud registration technology is adopted to process a continuous point cloud flow, real-time organization and updating of the point cloud are realized through a dynamic spatial index architecture, and an accurate three-dimensional point cloud map is constructed. And inputting the three-dimensional point cloud into the probability octree model, analyzing a space occupation state by using a logarithm domain Bayesian updating mechanism, and generating a three-dimensional probability occupation map. And finally, performing spatial dimension conversion on the three-dimensional occupation map: screening an effective height interval along the vertical direction, and fusing a multi-level occupation probability to generate a two-dimensional grid map, thereby realizing lossless compression from three-dimensional environment information to two-dimensional navigation information.

The invention relates to the field of traffic storage optimization and traffic data compression, in particular to a multithreading-combined lossless compression method for fine-grained traffic, which comprises the following steps of: acquiring a traffic data file, extracting a header triple based on the traffic data file, generating an identifier according to the header triple, and transmitting the identifier to a server; aggregating the traffic data files with the same identifier to obtain data streams, and generating a sorted data stream size table according to the data volume of the data streams; creating threads according to system computing power, distributing the data flow to the thread with the minimum thread load according to the data flow size table and the thread load condition, and outputting a data flow distribution scheme; and performing fine-grained characterization and serialization on each data stream to obtain an integer sequence and a byte sequence, selecting a compression processing method according to redundancy characteristics of the integer sequence and the byte sequence, obtaining a redundancy-eliminated sequence, and writing the redundancy-eliminated sequence into a compressed file. The invention aims to provide high compression efficiency to reduce the storage cost while ensuring the data precision.

The invention discloses a parallelizable self-adaptive matrix lossless compression method, which comprises the following steps of: partitioning a first matrix with a larger scale into second matrixes with smaller scales, distributing a splitting process for the second matrixes, splitting all the second matrixes into a plurality of bit matrixes in parallel by the splitting process, and distributing compression threads for the bit matrixes. And adaptively selecting an optimal compression strategy from the compression strategy set by the compression thread, and finally, completing compression of all bit matrixes by the compression thread according to the determined compression strategy in parallel to form a compression matrix and a flag array as final compression data of the first matrix. Through compression process grading and step-by-step parallelization, the memory overhead in the compression process is effectively reduced, the compression speed is improved, an optimal compression strategy is selected through trial compression, self-adaptive compression is achieved, the data compression rate is effectively improved, and lossless compression of data is achieved by constructing a flag array.

The invention provides a lossless compression method and decompression method for power grid waveform data, equipment, a medium and a program product, and the method comprises the steps: carrying out the periodic feature analysis of the original sampling data of a power grid waveform, so as to determine the periodic structure of the original sampling data, and carrying out the segmentation processing of the original sampling data based on the periodic structure; for the sampling data in each segment, extracting differential information between adjacent sampling points to form a differential data sequence; dynamically determining a corresponding quantization parameter based on the statistical distribution characteristic of the differential data sequence, and performing adaptive quantization processing on the differential data sequence to generate quantized differential data; and entropy coding processing is carried out on the quantized differential data to generate compressed data used for representing the original sampling data, and characteristic parameters used for reconstructing the original sampling data are stored in the compressed data in an associated mode. Unification of high compression ratio, low power consumption and strict lossless reconstruction is realized, and the application requirements of long-term monitoring and remote transmission of the edge side can be met.

The invention provides a lossless compression and decompression method and system based on a probability model, and belongs to the technical field of data compression. The method comprises the processes of segmentation of input data, probability model training, probabilistic reasoning, entropy coding compression / decompression of an asymmetric digital system and bisection reconstruction input. The system comprises a software module and a hardware module, the software module comprises probabilistic model training, probabilistic reasoning and asymmetric digital system entropy coding compression / decompression, and the hardware module comprises a probabilistic model operation module and a compression / decompression module. According to the method, lossless, high-speed and high-compression-ratio lossless compression can be carried out, and input sequences of different sizes and different types can be compressed through segmentation and parameterization on the input sequences, such as pictures, audios and videos; lossless compression with a high compression ratio is realized through a joint function of probability model reasoning and asymmetric digital system entropy coding; through independent selection or design of hardware modules, high-speed lossless compression is realized.

The invention belongs to the technical field of image acquisition, and discloses a field image acquisition method and system. The method comprises the following steps: acquiring real-time field images and videos by using an edge computing gateway, and extracting a plurality of real-time key frame images; image segmentation is carried out, and the real-time attention of the real-time image area is obtained; lossless compression and dynamic encryption are carried out; collecting real-time network quality data of the transmission network, and generating a real-time data transmission strategy; uploading the plurality of encrypted real-time image compression packets to a cloud data center according to a real-time data transmission strategy; and restoring the plurality of encrypted real-time image compression packets of each real-time foreground image by using the cloud data center to obtain a restored real-time field image video. According to the invention, the problems of low data transmission efficiency, insufficient security, centralized processing bottleneck, high energy consumption and poor transmission reliability in the prior art are solved.

The invention provides a large model KV precision lossless compression method and system, and relates to the field of data processing, and the method comprises the steps: obtaining key value cache data generated in a large language model reasoning process; performing high-dimensional feature extraction and similarity analysis on the key value cache data to generate a corresponding hash signature; based on the Hash signature, performing physical reordering on the key value cache data, so that the key value cache data with similar characteristics are adjacently arranged in a physical storage space; transmitting the reordered key value cache data to an intelligent network card through a high-speed interconnection interface; and calling an integrated hardware lossless compression module through the intelligent network card, and performing streaming compression on the reordered key value cache data. According to the technical scheme, the hardware unloading and assembly line technology is utilized, transmission bandwidth occupation and system overall delay are remarkably reduced, and key technical support is provided for efficient reasoning of a large model in a resource-limited edge environment.

The invention discloses a lossless compression storage method and system based on heterogeneous data in a credential environment, and the method comprises the steps: firstly, extracting a mapping relation between a file type and a structural feature through scanning a target file, carrying out the preliminary lossless compression after classification and grouping, and further employing a sliding window matching and sequence matching algorithm to optimize compressed data for a redundancy mode, and then generating a metadata structure containing a check code and a compression parameter, packaging the metadata structure into a data packet supporting cross-platform transmission, finally transmitting the data packet to a target storage device through a secure write-in protocol, and verifying data consistency and recovering path feasibility by using the check code. According to the method, the data compression efficiency is remarkably improved, the compatibility of cross-platform transmission and the integrity of stored data are ensured, and efficient and reliable technical support is provided for complex electronic file processing.

The invention provides an electroencephalogram data transmission method combining lossless and lossy compression, belongs to the field of electroencephalogram data transmission and compression processing, and is used for solving the problems of low transmission efficiency of pure lossless compression and easy loss of key diagnosis information of pure lossy compression in related technologies. According to the method, multi-dimensional features of electroencephalogram data are extracted, a key area and a non-key area are divided, lossless compression and lossy compression processing are adopted respectively, the area division proportion is dynamically adjusted in combination with transmission bandwidth, signal features and application scenes, and corresponding data are transmitted through dual protocols. The transmission efficiency can be improved while the clinical diagnosis marker and the core characteristics are reserved, and the scene adaptability and the transmission reliability are achieved at the same time.

Raytracing and pathtracing systems use vertex compression including per-component shifts, unsigned arithmetic deltas, and base vertex shortening. Topology encodings include enhanced explicit vertex indexing and implicit triangle strip based indexing including left, right, start tokens, turn back, and degenerate tokens and including rotation. Substantial lossless compression ratio increases are realized, especially for highly quantized vertex data.

A computer-implemented method, according to one approach, includes: receiving one or more streams of system log information from one or more microservices, and comparing the received system log information to compressed templates. One or more blocks of the system log information that correspond to one or more of the compressed templates are identified. All static text is removed from the identified blocks. Moreover, runtime variables from the identified blocks are inserted into respective ones of the compressed templates. The streams of system log information are further updated by replacing the identified blocks with the compressed templates having the runtime variables therein.

The invention discloses a compression and decompression method for DNA sequencing data, and belongs to the technical field of biological information. The technical problems that in the prior art, when third-generation sequencing data are compressed, flexibility is poor, and the compression rate is low are solved. According to the compression method for the DNA sequencing data, global compression or block compression can be dynamically selected before compression, different compression methods are adopted for different types of data, and the flexibility of the compression method is improved; when the base sequence is compressed, the adopted self-indexing structure is a lossless compression method, the initial positions of all the sequences can be compressed and restored by using smaller data volume, and the compression rate is improved. Due to the fact that a special design is adopted in a run length coding structure, a character part is removed, an obtained run length segment is shorter, lossless compression of the position and the sequence is guaranteed, and meanwhile a better compression effect is provided. The method is mainly used for compression and decompression of DNA sequencing data.

The invention relates to the technical field of audio processing, in particular to an audio signal data management method and system of an audio chip. The method comprises the following steps: identifying a real-time multi-channel audio signal and a real-time network monitoring parameter of an audio chip; performing adaptive filtering noise reduction and detail enhancement processing on the real-time multi-channel audio signal to obtain a detail enhanced real-time audio signal; performing band-by-band analysis window definition on the detail enhanced real-time audio signal, performing dynamic spectrum feature mining, and generating a dynamic audio signal feature of each band; performing dynamic compression coding and lossless compression audio coding on the detail enhanced real-time audio signal based on the dynamic audio signal feature of each frequency band, and constructing a distributed stream storage framework; and performing audio channel identification on the real-time multi-channel audio signal, and performing phase alignment processing to obtain a channel phase alignment audio signal. According to the invention, high-efficiency and high-stability audio signal management is realized.

The present application provides a rapid evaluation method and system for generator vibration data of a wind turbine generator set. The method comprises: step 1, acquiring current-day operation data of a wind turbine generator set under test, wherein the current-day operation data comprises a time label, an operation power and a generator vibration signal; step 2, constructing a two-dimensional plane scatter plot on the basis of the current-day operation data; and step 3, combining the obtained two-dimensional plane scatter plot with a preset regression function, and evaluating the vibration state of said wind turbine generator set. The present application utilizes a lossless compression method to process data, so that the current-day vibration state of a generator can be accurately evaluated.

The invention relates to a domain name system (DNS) traffic log lossless compression method and device based on field sorting, and the method comprises the steps: responding to a DNS traffic log compression request, reading data in a segmented manner from a DNS traffic log according to a preset line number, and extracting log information line by line; the codes are replaced by searching the numbers of the domain names and the DNS addresses, and the processed log lines are stored in a memory; sequencing each log segment based on a plurality of fields according to a predetermined rule, converting the sequenced log line number into a 36-system and grouping and writing, converting the time and date into a timestamp or a difference value, and adding the timestamp or the difference value to an index to form a processed DNS traffic log; and performing redundancy elimination class compression on the processed DNS traffic log to generate a compressed file. According to the method, data redundancy is effectively reduced through segmented reading, field coding and multi-field sorting, and meanwhile, the compression ratio is further improved by combining domain name reversal sorting and gzip compression.

The invention discloses a smart factory large-scale data storage and analysis method, and relates to the technical field of industrial Internet of Things, and the method comprises the steps: collecting three-dimensional physical field data of equipment, and carrying out the timestamp alignment and numerical range filtering to generate a standardized data stream; dynamically adjusting a compression level based on the structured data stream, calculating vibration waveform data lossy compression and heat flow field feature vector lossless compression through physical constraint residual errors, and generating a compressed data block; according to the address mapping table, constructing a high-dimensional topological space, extracting bar code life cycle and hole forming position features, and generating a topological feature vector; and performing deviation degree comparison on the topological feature vector and a persistent graph in a historical working condition library, and triggering equipment parameter adjustment according to a preset deviation degree threshold value. Differential compression of vibration waveform data and heat flow field characteristics is achieved through physical constraint residual calculation, key physical field information is reserved while the data size is reduced, and the compression efficiency is improved.

The invention relates to a video lossless compression transmission and storage method and device. The method comprises the following steps: analyzing an obtained video frame sequence; executing quality pre-verification, and if yes, determining that the video frame sequence is a first frame sequence; based on the scene detection target, determining a target attention area in the plurality of first sequence frames; selecting a plurality of target feature frames from the plurality of first sequence frames to execute importance evaluation, and determining corresponding feature levels; mapping the plurality of target feature frames into a one-dimensional pixel sequence; dividing the one-dimensional pixel sequence into a plurality of pixel segments, and performing cross-frame aggregation on the plurality of pixel segments according to the feature level to obtain a plurality of feature sequences; and carrying out hierarchical compression coding on the plurality of feature sequences. By adopting the method, the video can be subjected to quality pre-verification, the target concerned area can be adaptively identified, and differential compression and priority transmission are performed, so that the transmission quality and the storage efficiency of the inspection video can be improved in a complex electromagnetic environment and a limited network condition.

The application discloses a lossless compression storage method and system based on heterogeneous data in a Xinhua environment, first extracts the file type and structure feature mapping relationship by scanning the target file, classifies and groups, and then carries out preliminary lossless compression, further adopts a sliding window matching and sequence matching algorithm to optimize the compressed data for the redundant mode, then generates a metadata structure containing a check code and compression parameters, encapsulates into a data packet supporting cross-platform transmission, finally transmits to the target storage device through a secure write protocol, and verifies the data consistency and recovery path feasibility by using the check code. The application significantly improves the data compression efficiency, ensures the compatibility of cross-platform transmission and the integrity of storage data, and provides efficient and reliable technical support for complex electronic file processing.

The invention relates to the technical field of spacecraft satellite-ground communication, and provides a low-earth-orbit satellite communication time delay optimization method and system aiming at the defects that a traditional low-earth-orbit satellite communication satellite system and a satellite communication frame have high time delay and poor real-time communication function. The core of the invention comprises three parts: firstly, an efficient receiving and transmitting end coding and decoding technology: a transmitting end adopts a subblock coding technology of a Turbo code; and a receiving end uses an MAX-Log-MAP algorithm to carry out rapid iterative decoding on the word block code of the Turbo code. And secondly, compressing the user data: carrying out lossless compression on the user data and a satellite communication frame header by using Huffman coding, reducing the transmission data volume, improving the data processing speed, and thus reducing the processing time delay. And finally, simplifying the frame header design: optimizing the frame header of the communication frame so as to reduce the total length of each frame. The method is particularly suitable for communication satellites (the orbit height is less than or equal to 1500km) in a low earth orbit.

The present invention discloses a dynamic compression method for lightning waveform data and a hardware implementation method, which includes performing targeted compression on data using a dynamic adaptive compression algorithm; the targeted compression includes improving the algorithm principle; using a differential module to process the differential term data stream; using a lookup module to perform data lookup; using an encoding module to form a suitable encoding; and using a control module to transmit the compressed data. By using an FPGA as a processor, the waveform data output by the sampling chip undergoes differential calculation and effective bit truncation, and is compressed and encoded by an improved LZSS algorithm, the present invention realizes lossless compression adapted to lightning waveforms, improves the compression efficiency and saves storage space compared with traditional compression algorithms.

Deep learning data compression using multiple hardware accelerator architectures is provided herein. A system includes a computing device and first and second hardware accelerators coupled thereto. The first and second hardware accelerators may be of different types, such as a tensor streaming processor and a field programmable gate array. The first and second hardware accelerators may be directly connected to one another, such as by a chip-to-chip connection. The first and second accelerators may implement different stages of a data pipeline, such as lossless and lossy compression stages of a learned image compression.

This invention provides a lossless compression and decoding method for depth images. Lossless compression includes: dividing sequential data into multiple independent image blocks according to predetermined partitioning rules; selecting a corresponding thread model based on the characteristics of the depth image; calling a thread based on the thread model and using a reversible encoding algorithm to independently encode each image block; wherein, at the start of encoding for each image block, the encoder state is reset so that the encoding of each image block does not depend on the data of the preceding image block; generating metadata containing position index and compressed data length information and assembling it with the encoded image block for output. Lossless decoding includes: reading the metadata and decoding at least one image block based on the metadata. Through the above methods, parallel encoding and decoding are achieved; precise positioning via physical offsets and random access and on-demand region decoding are supported, improving the processing response of high frame rate depth streams under low computing power environments while ensuring lossless restoration.