617 results about "Compression method" patented technology

The invention provides a context compression method based on multi-round dialogue intention graph construction, which comprises the following steps: S1, dialogue data acquisition and preprocessing: carrying out natural language processing on each dialogue unit; s2, intention atlas construction is achieved through node design and edge design, and each node comprises original text content and structured semantic information; s3, carrying out context compression and graph structure cutting, and only retaining sub-graphs forming a core semantic link; s4, dynamic context management and topic jump processing: in a multi-topic dialogue, when a user jumps or switches to a new topic, a system records a sub-graph of a current active topic, and contextual nodes of an inactive topic are frozen; s5, context sequence generation and model input: linearizing node contents in the cut sub-graph according to a dependent link sequence to generate a compressed context sequence, and transmitting the context sequence and current user input to a large language model for reasoning; and S6, continuous updating and feedback optimization are carried out.

The invention relates to the technical field of artificial intelligence and big language model reasoning optimization, and discloses a big language model-oriented adaptive KV cache compression method and system, and the method comprises the steps: constructing a lexical element importance measurement mechanism; analyzing an attention head distribution structure in large language model reasoning, and constructing a plurality of pruning strategies; based on a lexical element importance measurement mechanism and the attention head distribution structure, designing a self-adaptive key value cache compression hybrid strategy set based on a pruning strategy; constructing a static self-adaptive key value cache compression method, and automatically distributing a key value cache compression strategy in a pre-filling stage of large language model reasoning; and in a decoding stage, performing adaptive compression on the key value cache based on the allocated key value cache compression strategy. According to the method, the key value cache can be efficiently compressed on the premise of not depending on explicit attention score calculation, a system-level reasoning optimization framework is compatible, the generation performance is kept, meanwhile, the video memory consumption is remarkably reduced, and the long context reasoning capability is enhanced.

In one implementation, we propose a scalable framework where a base layer uses NN-based methods to compress the content for computer vision machine tasks and enhancement layer(s) use traditional predictive coding for human viewing. Typically, the based layer performs NN-based analysis to generate a latent tensor, which is entropy coded to produce the base layer bitstream. By performing synthesis on the latent tensor, an inter-layer predictor can be obtained for the enhancement layer(s). Since many machine tasks are not required to be performed for each frame, the base layer may skip analysis for some frames. The synthesis may be performed at the base layer or the enhancement layer(s). In one example, the base layer compresses features optimized for a machine task and the enhancement layer(s) rely on predictive coding. In another example, the enhancement layer(s) can use traditional scalable video compression methods.

The invention relates to a simulation result compression method and system based on semantic segmentation, equipment and a medium. The method comprises the following steps: performing feature extraction on a simulation result file to obtain semantic features; generating a segmentation point sequence according to the features by using a semantic segmentation model, and segmenting the file into a plurality of semantic blocks; extracting a compressibility feature vector for each block, and dynamically distributing an optimal compression strategy for the block through a prediction model; compressing each block by using the distributed strategy to generate a compressed data block; and finally, constructing a block sensing index based on the block type and the used strategy, and integrating the index and the data block to generate a compressed file. By adopting the method, the defects of limited compression ratio and incapability of supporting rapid random access caused by lack of semantic understanding of a general compression algorithm can be overcome, and by understanding the internal logic structure of the file and applying differential compression, the rapid retrieval and analysis capability of specific data is kept while the high compression ratio is realized.

The invention discloses a voice data compression and index storage method based on a voiceprint template, and relates to the technical field of voice compression processing. The voice data compression and index storage method based on the voiceprint template comprises the following steps: S1, collecting and preprocessing acoustic state data, coded data and edge synchronization data, and constructing a standardized voice input data set; s2, evaluating the individual sound channel difference degree, and stripping non-universal components highly related to individuals in the voice; s3, analyzing the audio compression content, and constructing a structured total compression amount; s4, performing joint evaluation on real-time transmission regulation and control requirements of the compressed data, and dynamically adjusting an uploading priority and a continuous transmission strategy; and S5, constructing a multi-dimensional reverse index system at the cloud. The problems that for a high-noise far-field pickup scene, an existing compression method does not consider that individual sound channel characteristics are stripped from original signals, so that compression redundancy is high, and retrieval interference is large are solved.

The invention provides a lossless compression method and decompression method for power grid waveform data, equipment, a medium and a program product, and the method comprises the steps: carrying out the periodic feature analysis of the original sampling data of a power grid waveform, so as to determine the periodic structure of the original sampling data, and carrying out the segmentation processing of the original sampling data based on the periodic structure; for the sampling data in each segment, extracting differential information between adjacent sampling points to form a differential data sequence; dynamically determining a corresponding quantization parameter based on the statistical distribution characteristic of the differential data sequence, and performing adaptive quantization processing on the differential data sequence to generate quantized differential data; and entropy coding processing is carried out on the quantized differential data to generate compressed data used for representing the original sampling data, and characteristic parameters used for reconstructing the original sampling data are stored in the compressed data in an associated mode. Unification of high compression ratio, low power consumption and strict lossless reconstruction is realized, and the application requirements of long-term monitoring and remote transmission of the edge side can be met.

The embodiment of the invention provides a model parameter compression method and device for a large language model, equipment and a storage medium. The method comprises the steps of obtaining verification data and inputting the verification data into a large language model to obtain an input activation tensor received by each network layer; obtaining the reasoning confusion degree of the large language model for reasoning the verification data, and taking the minimization of the reasoning confusion degree as an optimization target to carry out iterative cutting decision to obtain a cutting decision vector; performing outlier clipping processing on the input activation tensor of the network layer based on the clipping decision vector to obtain a target activation tensor; for each network layer, calculating an importance score of a model parameter based on the target activation tensor, and pruning the initial model parameter tensor in combination with the importance score to obtain an intermediate model parameter tensor; and performing quantization processing on the intermediate model parameter tensor of the network layer to obtain a target large language model. Therefore, the storage and calculation complexity of the large language model can be reduced while the performance of the large language model is maintained.

The invention discloses a deep learning image compression method and system based on a semantic discriminator, and mainly solves the problem that the visual task performance of a downstream machine is remarkably reduced due to serious semantic information loss under a high compression rate in the conventional image compression method. According to the implementation scheme, a group of images are selected from an existing image data set and are divided into a training set, a verification set and a test set, and the training set, the verification set and the test set are preprocessed respectively; constructing an image compression network comprising an image codec, a semantic extraction network and a semantic guide discriminator under a Pytorch framework; inputting the training set into an image compression network, carrying out two-stage iterative training, and verifying through a verification set; and inputting the test set into the trained image compression network, and only calling the image codec to output the compressed reconstructed image. According to the method, the structural integrity and semantic consistency of the reconstructed image are remarkably improved, higher visual quality and task accuracy can be kept at a low code rate, better image compression performance is embodied, and the method can be used for efficient transmission and storage of image data.

The invention discloses a dynamic texture atlas recombination compression method and system based on airspace sparsity, and belongs to the technical field of data compression, and the method comprises the steps: obtaining a dynamic texture sequence, carrying out the feature analysis of a texture unit in the dynamic texture sequence, and generating a multi-dimensional feature vector; constructing an initial compressibility potential model for each texture unit based on the multi-dimensional feature vectors; taking the initial compressibility potential model as input, executing an iterative collaborative decision-making process, and generating an optimized texture atlas layout and partition compression strategy mapping table; according to a partition compression strategy mapping table, performing differential compression on the texture data arranged according to the optimized texture atlas layout to generate compressed texture data; and integrating and packaging the layout information of the optimized texture atlas layout, the partition compression strategy mapping table and the compressed texture data to generate a compressed data stream. According to the method, a collaborative decision-making mechanism is adopted, airspace sparsity analysis and a differential compression strategy are combined, and efficient compression of the dynamic texture atlas can be achieved.

The invention provides a difference and dictionary mixed compression method and device for time series data of the Internet of Vehicles, and relates to the field of data processing of the Internet of Vehicles. The method comprises the following steps: acquiring original data corresponding to a target vehicle in the Internet of Vehicles; performing first-order differential coding on the original data to obtain a first-order differential sequence; obtaining a corresponding differential disturbance factor in the first-order differential sequence through a preset model, and calculating the redundancy corresponding to the first-order differential sequence based on the differential disturbance factor; when it is confirmed that the redundancy is greater than a preset redundancy threshold value, performing second-order differential coding on the first-order differential sequence based on a differential disturbance factor to obtain a second-order differential sequence; replacing a target character string in the second-order difference sequence by using a dictionary index through a dynamic dictionary; and carrying out compression packaging on the second-order difference sequence and the dictionary index, and uploading the second-order difference sequence and the dictionary index to an Internet of Vehicles cloud. The problem that the subsequent compression efficiency is greatly reduced due to the fact that an existing method cannot adapt to the time sequence structure difference of the Internet of Vehicles data in different abnormal scenes is solved.

The invention discloses a video code rate dynamic allocation compression method based on content complexity prediction, which comprises the following steps of: S1, acquiring a video frame sequence, and preprocessing; s2, inputting the frame-level feature tensor into a gated residual convolutional network, and outputting a complexity prediction sequence; s3, constructing a frame priority queue, and calculating the complexity jump amplitude between adjacent frames; s4, a compression area is divided, and a corresponding code rate resource scale factor is allocated; s5, configuring a reference frame structure, a prediction interval and an initial quantization step size for each compression region, and calculating a region target bit number; s6, distributing regional bits to each frame in a compression coding process, and dynamically adjusting a frame-level quantization parameter and an entropy coding strategy; and S7, after compression is completed, reversely updating convolution prediction network parameters through bit distribution errors. According to the invention, fine code rate dynamic allocation and adaptive compression control based on content complexity are realized, and the video compression quality and bit utilization efficiency are effectively improved.

The invention provides a compression method and device for soft information in a signal demodulation process and electronic equipment. The method comprises the following steps: acquiring linear soft information output by demapping in a signal demodulation process, wherein the linear soft information has a preset dynamic range; the linear soft information is subjected to dynamic range compression processing through a nonlinear quantization algorithm, nonlinear soft information is obtained, and the number of quantization levels allocated to low-amplitude soft information by the nonlinear quantization algorithm is larger than the number of quantization levels allocated to high-amplitude soft information by the nonlinear quantization algorithm; and outputting the non-linear soft information to a decoder, so that the decoder restores an original hard bit stream transmitted by a signal transmitting end based on a confidence coefficient characteristic in the non-linear soft information, and the bit width of the non-linear soft information is adaptive to the bit width constraint of the decoder on the input information. According to the invention, information loss in a signal demodulation process can be reduced.

The invention discloses a multi-modal corpus storage method and device, computer equipment and a medium, and the method comprises the steps: carrying out the compression processing of text modal data and audio and video modal data of a multi-modal corpus, and employing a suitable compression algorithm for the characteristics of different modal data; similarity calculation is carried out on the text compression features, the audio and video compression features and corresponding modal features of the newly-added multi-modal corpus, semantic redundancy of the newly-added multi-modal corpus and historical compression corpus can be recognized, a fusion compression process is triggered to eliminate cross-modal redundancy, and repeated storage of same semantic information is avoided; corpus features which do not trigger the fusion process are independently stored, the corpus features which trigger the fusion process are fused to generate multi-modal joint features, and the stored data volume is further simplified; the corpus features are migrated to the corresponding data storage layer in combination with the access frequency and the response time, so that the storage cost is reduced.

Lossy methods and hardware for compressing data and the corresponding decompression methods and hardware are described. The lossy compression method comprises dividing a block of pixels into a number of sub-blocks and then analysing, for each sub-block, and selecting one of a candidate set of lossy compression modes. The analysis may, for example, be based on the alpha values for the pixels in the sub-block. In various examples, the candidate set of lossy compression modes comprises at least one mode that uses a fixed alpha channel value for all pixels in the sub-block and one or more modes that encode a variable alpha channel value.

The invention relates to the technical field of computer graphics and image compression, in particular to a multi-moment illumination chartlet compression and decompression method based on two-dimensional Gaussian representation and a computer device.The method comprises the steps that S1, illumination chartlets at multiple target moments are obtained and preprocessed; s2, constructing a shared two-dimensional Gaussian basis set based on the low-frequency component; s3, extracting a residual error at a multi-target moment and features of a highlight and high-frequency region; s4, multi-layer perceptron network construction and two-dimensional Gaussian attribute offset modeling are carried out; s5, performing compression and storage; and S6, decompressing and rendering. The compression rate is greatly improved, the decompression speed is extremely high, the real-time rendering requirement is met, the rendering quality is higher than that of a traditional compression method, the highlight and high-frequency detail modeling capacity is high, the structure is simple, and integration is easy.

The invention discloses a high-definition video low-delay compression method based on self-adaptive adjustment, and aims to solve the problems of insufficient self-adaptive adjustment capability, too high encoding and decoding delay and low code rate and network bandwidth matching degree in the existing high-definition video compression technology. The method comprises the following steps: firstly, acquiring a high-definition video stream and a real-time network bandwidth parameter, extracting intra-frame texture complexity, inter-frame motion intensity and scene switching frequency characteristics, and constructing a two-dimensional adaptive compression strategy in combination with a real-time bandwidth; low-delay coding is executed by adopting parallel frame fragmentation and simplified adaptive arithmetic coding, adaptive code rate control is realized by combining code rate overflow early warning and intra-frame key region priority bit allocation, and finally, fast inverse transformation, quantization recovery and motion compensation optimization are completed at a decoding end based on strategy parameters synchronously transmitted at a coding end so as to reconstruct a high-definition video frame. According to the invention, the adaptive adjustment capability of video compression is improved, low time delay of coding and decoding is realized, and the method is suitable for real-time high-definition video transmission scenes such as remote monitoring and video conferences.

The invention relates to a configurable decompression circuit supporting COO and Bi tmap compression algorithms, and belongs to the field of integrated circuits. According to the hybrid compression method suitable for the circuit, after sparseness analysis is carried out on weight matrixes of all layers of a neural network, COO compression based on a coordinate type sparse matrix or Bitmap compression based on bitmap masks is selected according to the sparseness characteristics of different layers, so that the storage space is optimized; comprising a control module, a first selector, a second selector, a bitmap description memory, a coordinate index memory, a numerical memory, a converter, a COO decoder and a decompression data storage module, decoding of two compression formats of COO and Bi tmap can be supported, and the operation efficiency and flexibility of a neural network processor are improved. The method effectively reduces the weight storage demand of the neural network, enhances the adaptability of the compression algorithm, and is suitable for hardware implementation of an efficient neural network model.

The invention discloses a sparse data compression method, and belongs to the field of big data and real-time calculation. The method comprises the following steps of: 1, optimizing the layout of an UnsafeRow data memory; 2, encoding the UnsafeRow data; step 3, carrying out UnsafeRow data decoding, and carrying out UnsafeRow data decoding; step 4, carrying out serialization and deserialization on the compressed UnsafeRow; step 5, ORC write-in optimization is carried out; wherein the ORC is an efficient and high-performance column type storage format; and step 6, ORC reading optimization is carried out. By optimizing the memory layout of UnsafeRow and compressing the memory use of sparse data, the memory use efficiency of the data is improved, the overhead of a network and a disk is reduced, and the data processing efficiency of Spark SQL (Structured Query Language) is improved; meanwhile, the storage space of data storage is reduced through encoding optimization of ORC.

The present application relates to the technical field of data compression, in particular to a data compression method and system, comprising: an LZ module, a text stream module, a sequence stream module, an interval entropy encoding framework, the LZ module is used for screening repeated data of original data, generating a text stream and a sequence stream; the LZ module screens repeated data through a hash verification method, specifically: calculating a hash value for the input data block, determining repeated data based on hash value comparison to avoid direct data content comparison; the text stream module is used for entropy encoding the text stream; the present application innovatively proposes an interval entropy encoding framework, constructs a more optimal probability model through artificial intelligence and mathematical methods, trains a more optimal finite state machine, and uses the optimal state machine in the compression and decompression process, efficiently solving the problem of slow decompression speed of neural network type methods.

The application discloses a cryptographic algorithm debugging data hierarchical grouping compression method for a reconfigurable cryptographic chip. In the debugging work of the reconfigurable cryptographic chip, the debugging data is divided into reconfigurable cryptographic algorithm configuration information and operation intermediate data. The debugging data is processed by hierarchical grouping, the debugging data of each level module is divided into one or more groups, then the debugging data is filled or spliced, and then the reconfigurable debugging data is compressed by using a Huffman coding mode. After the hierarchical grouping compression method of the debugging data is used, the transmission performance of the debugging data can be effectively improved while the storage resource of the debugging data is reduced.

There is provided a compression method implemented by at least one hardware processor, including: acquiring attention data generated in a layer of a deep learning model based on an attention mechanism; converting a data format of the attention data into a converted data format, wherein a bit pattern of the attention data is maintained in converting the data format of the attention data into the converted data format; realigning, based on data characteristics of input data of the deep learning model, the attention data in the converted data format; and generating first compressed data by inputting the realigned attention data to an encoder model based on an artificial neural network.

The present disclosure belongs to the technical field of audio processing, and relates to an audio compression method and device, a computer storage medium and an electronic device. The method comprises: obtaining a game audio event file corresponding to a target game, and analyzing the game audio event file to obtain an audio event name; counting the number of times that the audio event name appears to obtain an event trigger number corresponding to the audio event name; and determining corresponding compression information for an audio resource corresponding to the audio event name based on the event trigger number, so as to compress the audio resource according to the compression information. In the present disclosure, the compression information corresponding to different audio resources is determined according to different event trigger numbers, thereby avoiding the problem that the prior art cannot simultaneously meet the requirements of audio quality and the size of the compressed file, and further improving the playing quality of the audio resource in the target game, avoiding excessive performance loss, and improving the smoothness of the target game.

The invention discloses a log compression and decompression method and device, equipment and a medium in the technical field of computers. According to the method and the device, the log file not only comprises the content of the current log, but also comprises the position information when the content of the current log subsequently and repeatedly appears, and the position information can be used for querying the log original text of the corresponding compressed field in the log file. Namely, the log file itself serves as a compression index table, so that the method does not need to maintain and store the index table, does not need to use a complex coding compression algorithm, is used for real-time log compression, reduces records of repeated log contents, and retains an original log text.

The invention discloses a hierarchical tensor decomposition and compression method for a radial sensor network. The hierarchical tensor decomposition and compression method comprises the following steps: (1) acquiring basic data; (2) carrying out hierarchical tensor modeling; (3) decomposing in branches; (4) fusing the branches; (5) performing adaptive quantization; (6) multi-hop collaborative coding; and (7) outputting compressed data. In order to solve the three core problems of multi-hop error accumulation, insufficient redundancy utilization among branches and poor dynamic adaptability in the radial sensor network, the method provided by the invention reduces multi-hop transmission error accumulation reduction by establishing a hierarchical tensor model, graph convolution fusion among the branches and adaptive multi-hop quantization, and improves the dynamic adaptability of the radial sensor network. And the transmission efficiency and the life cycle of the radial network are obviously improved.

This invention discloses a multi-level context compression method and system, belonging to the field of information processing technology. Before performing multi-level compression, the method first performs semantic pre-screening, using a lightweight embedding model to evaluate the information entropy and semantic importance of messages, achieving context-aware dynamic budget allocation. Then, it performs multi-level compression sequentially: the first level is content budget control, which enforces an upper limit on the aggregated content size of each message before lightweight compression; the second level is fragment compression, selectively removing historical content with feature gating; the third level is lightweight compression, performing lightweight content compression and supporting delayed boundary message processing; the fourth level is context folding, performing fine-grained selective compression based on the dialogue state graph, retaining key decision nodes; and the fifth level is complete summarization, performing a complete content summary when resource usage exceeds a threshold. This invention also introduces reinforcement learning optimization, dynamically adjusting the compression strategy based on task completion rate feedback. This invention achieves a strategy that prioritizes fine-grained compression over coarse-grained summarization, differential compression based on semantic importance, topological compression based on graph structures, and adaptive strategy optimization.

The application discloses a time series data compression method and device, and relates to the technical field of data compression, and the method comprises the following steps: creating a one-dimensional dictionary according to time series data; sorting the data in the one-dimensional dictionary to obtain a first sorting result, and creating a one-dimensional dictionary index according to the first sorting result; performing traversal XOR on the data in the one-dimensional dictionary to obtain a traversal XOR result, compressing the one-dimensional dictionary according to the traversal XOR result, and obtaining a compressed one-dimensional dictionary; determining a one-dimensional dictionary index sequence corresponding to a time sequence according to the one-dimensional dictionary index and the time sequence corresponding to the time series data, compressing the one-dimensional dictionary index sequence corresponding to the time sequence, and obtaining a compressed index sequence; and merging the compressed one-dimensional dictionary and the compressed index sequence to obtain compressed data of the compressed time series data. The application can further improve the compression ratio of data without losing accuracy, save storage space, and realize efficient storage.

This invention discloses a dual-stream point cloud compression method and system based on semantic scene graphs, relating to the field of point cloud compression. It constructs a point cloud compression network comprising an encoder network and a decoder network: In the encoder network, semantic labels are mapped onto the input point cloud, and scene graphs and octree representations are constructed respectively. Semantic modulation parameters are generated based on the scene graph and encoded to obtain a semantic bitstream. The octree representation, under the influence of semantic modulation parameters, undergoes feature enhancement and attention probability prediction to obtain node occupancy probabilities, and a geometric bitstream is generated through arithmetic encoding. These two bitstreams serve as compressed data. In the decoder network, the bitstreams are decoded, and the octree structure and scene graph information are reconstructed. Node feature representations are recovered by combining the semantic modulation parameters. The reconstructed octree structure is mapped back to the point cloud space through spatial querying and indexing to reconstruct the point cloud. This invention solves the problem of local geometric distortion in low-bitrate octree point cloud compression, ensuring reconstruction quality while reducing the transmission bitrate.

A data compression method and a related device, used for reducing I / O overhead. The data compression method comprises: acquiring attribute information of a logic block, wherein the logic block comprises a plurality of data pages, and the attribute information indicates data sizes of data contained in the plurality of data pages across a plurality of time periods; if the attribute information indicates that the total data size of the data contained in the plurality of data pages during a target time period is greater than or equal to a first threshold, compressing the data contained in the logic block, and storing the result obtained by compression, wherein the target time period includes at least time period among the plurality of time periods, and the time interval between the target time period and the current moment is greater than or equal to a time threshold; and if the attribute information indicates that the total data size of the data contained in the plurality of data pages during the target time period is less than the first threshold, determining not to compress the data contained in the logic block.

The application discloses a dynamic texture atlas reorganization compression method and system based on spatial sparsity, and belongs to the technical field of data compression. The method comprises the following steps: acquiring a dynamic texture sequence, performing feature analysis on texture units in the dynamic texture sequence, and generating a multi-dimensional feature vector; based on the multi-dimensional feature vector, constructing an initial compressibility potential model for each texture unit; taking the initial compressibility potential model as input, performing an iterative collaborative decision process, and generating an optimized texture atlas layout and a partition compression strategy mapping table; according to the partition compression strategy mapping table, performing differential compression on texture data arranged according to the optimized texture atlas layout, and generating compressed texture data; and integrating and packaging layout information of the optimized texture atlas layout, the partition compression strategy mapping table and the compressed texture data, and generating a compressed data stream. The application adopts a collaborative decision mechanism, combines spatial sparsity analysis and a differential compression strategy, and can realize efficient compression of a dynamic texture atlas.

The invention discloses a hybrid neural representation video compression method based on spatial-temporal characteristic modulation, and belongs to the technical field of video compression. The method employs an encoder-decoder framework to map video content into compact neural network parameters. The core of the method is that a dual modulation mechanism of time sequence perception affine transformation (STAT) and wavelet space attention (WSA) is introduced into a decoder. The STAT module generates affine parameters through time sequence embedding, channel-level modulation is carried out on features, and accurate time sequence alignment is achieved; the WSA module decomposes the feature map through two-dimensional discrete wavelet transform, and generates a spatial attention map based on the high-frequency component to adaptively enhance spatial high-frequency details. The two modules work cooperatively, the problem of spectrum deviation in neural representation is effectively solved, and the time sequence coherence and the space detail quality of a reconstructed video are remarkably improved while the advantage of rapid decoding is kept.