Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

523 results about "Compress Algorithm" patented technology

A UNIX compression program based on the LZC compression method.

Application-driven three-dimensional spatial data transmission method and system

The present invention relates to the technical field of data transmission. Disclosed is an application-driven three-dimensional spatial data transmission method. The method comprises: determining system key performance indicators (KPIs) by means of qualitative and quantitative analysis; constructing an AI-driven adaptive three-dimensional data transmission mechanism, and dynamically adjusting a transmission strategy on the basis of a real-time network state, a device capability, an application scenario and the KPIs; developing an adaptive compression algorithm set oriented to three-dimensional data, so as to meet differentiated compression requirements of different application scenarios; performing loop execution of a test, and adjusting and optimizing the data transmission mechanism and the compression algorithm set on the basis of a test feedback result and the real-time network state; and deploying an optimized transmission method to a production environment, and collecting field data to optimize the system performance and verify the achievement of the KPIs. By constructing a qualitative and quantitative analysis framework based on machine learning, the present invention quantifies differentiated transmission requirements of different application scenarios and formulates transmission strategies meeting the scenario requirements.
Owner:GUIZHOU POWER GRID CO LTD

Complex equipment data monitoring method and system based on knowledge graph and edge calculation

The invention relates to the technical field of data monitoring, in particular to a complex equipment data monitoring method and system based on a knowledge graph and edge calculation. The method comprises the steps of multi-modal data acquisition and edge intelligent preprocessing; performing feature extraction on the preprocessed data; fusing multi-source heterogeneous data and constructing a dynamic knowledge graph; performing deep knowledge reasoning and layered fault early warning decision making; and performing self-optimization and knowledge closed-loop management based on early warning decision. According to the differential coding and self-adaptive compression algorithm of the edge layer, the data transmission quantity is reduced by 75%, the network bandwidth requirement is greatly reduced, and the system operation efficiency is improved.
Owner:YANTAI UNIV

Kafka storage and calculation separated data processing method and device

The embodiment of the invention provides a kafka storage and calculation separated data processing method and device, and the method comprises the steps: constructing a multi-layer data processing architecture which comprises four layers: network communication, task distribution, processing scheduling and storage. Cold and hot data separation is realized through hierarchical design of a log pre-writing area and a main storage area, and intelligent scheduling of system resources is realized in combination with a copy manager, a coordinator and a load predictor. An adaptive compression algorithm and a verification mechanism are adopted to ensure the data transmission efficiency and integrity, a message cache region is created to store hotspot data, and flexible access of cold and hot data is realized based on message sites. According to the method, the defects of the traditional technology in storage and calculation coupling, storage efficiency, load balancing and the like are effectively overcome, and the performance and expandability of a message queue system are remarkably improved.
Owner:富盛科技股份有限公司

Cloud data anomaly detection and safety response system based on artificial intelligence

The invention discloses a cloud data anomaly detection and safety response system based on artificial intelligence, relates to the technical field of data processing, and solves the problems that firstly, an incremental compression algorithm is difficult to store and preprocess multi-source heterogeneous data; secondly, it is difficult to fuse statistical analysis and a deep learning model, locate outliers and analyze abnormal semantics in unstructured data, and then it is difficult to effectively predict a potential attack path; then, on the premise that data security and traceability are guaranteed, correlation analysis of cross-node anomalies is difficult to achieve so as to identify distributed attacks; and finally, an attack and defense confrontation model is difficult to construct for safety response, and the safety response effect is difficult to evaluate. According to the method, cloud environment data are processed through an adaptive probe cluster and the like, multiple models are fused to generate anomaly detection features and predict attack paths, and response and evaluation are carried out through reinforcement learning and digital twinning by means of cooperative detection such as federated learning and the like.
Owner:GUANGZHOU PENGJIE TECH CO LTD

Mass data storage optimization method and device and storage medium

The invention relates to the technical field of big data processing, and provides a mass data storage optimization method which comprises the following steps: analyzing target data according to access frequency and / or access time of the data, and dividing the target data into hot data, warm data and cold data; aiming at the hot data, the temperature data and the cold data, performing compression processing by adopting corresponding compression algorithms; before the target data is stored, performing duplicate removal on the target data according to the fingerprint of the target data; monitoring the use condition of the memory, the access frequency of the target data and the performance of the compression algorithm in real time, and deploying system resources according to a monitoring result; and setting a multi-level cache, and caching different types of target data to the multi-level cache according to the divided types of the target data. According to the technical scheme, mass data can be efficiently stored and rapidly accessed, and the storage space utilization rate and the overall system performance are remarkably improved.
Owner:深圳市青云端信息科技有限公司

Interactive large-screen visualization system and method based on multidimensional data real-time rendering

The invention discloses an interactive large-screen visualization system and method based on multi-dimensional data real-time rendering, relates to the technical field of image rendering, and aims to solve the bottleneck problems of a traditional system in the aspects of data throughput efficiency, rendering delay and interactive response. The method comprises the following steps: dividing space blocks by adopting a non-uniform octree, dynamically adjusting the size of the blocks according to data density, and combining a ZSTD + Hoffman hybrid compression algorithm; a CPU-GPU three-level asynchronous processing assembly line is constructed, and task priorities are dynamically adjusted through a fixation point weight algorithm; the master node dynamically allocates LOD parameters, and the slave nodes achieve special-shaped large-screen sub-pixel-level splicing based on a spherical coordinate system model; a Q-learning algorithm is adopted to predict a user operation sequence, and dual-channel bandwidth adaptive allocation is realized in combination with an ASTP protocol. The system comprises a dynamic partitioning module, a pipeline scheduling module, a multi-node cooperation module, an intelligent preloading module and an exception handling module. Real-time interaction visualization of a large amount of data can be realized, the rendering efficiency is improved, and hardware resource occupation is reduced.
Owner:BEIJING HANGYUN SCI & TECH CO LTD

Distributed data storage management system and method

The invention discloses a distributed data storage management system and method, and relates to the technical field of distributed storage, and the method comprises the steps: obtaining an access log, metadata and content features of each node; data object attributes are judged, cold data are marked, and cold data distribution nodes are positioned; the method comprises the following steps: dividing cold data into permanent, periodic and sudden basic access modes through first-layer classification; generating a composite classification label through second-layer classification; matching a predefined strategy matrix according to a classification result, dynamically selecting a compression algorithm and a storage medium, and allocating bandwidth; operation auditing, compliance verification and exception handling are realized through the smart contract; and collecting strategy execution indexes in real time, and adjusting classification model parameters and resource scheduling weights. Through an intelligent cold data judgment and classification mechanism, cold data is accurately identified and managed, occupation of storage resources is reduced, and the overall utilization rate of the storage resources is improved.
Owner:CHANGCHUN INST OF ELECTRONIC TECH

High-performance AIOT real-time data processing method and device based on Flink, Kafka and Dores

The invention discloses a high-performance AIOT real-time data processing method and device based on Flink, Kafka and Dores, and the method comprises the steps: constructing an end-to-end hierarchical architecture, capturing multi-source database change data in real time through Flink CDC, and analyzing semi-structured data in combination with Apache NiFi, and generating a standardized data stream; the method comprises the following steps: realizing message routing based on KafkaTopic division of a service domain, ensuring the consistency of a data structure by using Schema Registry, and optimizing the transmission efficiency through a dynamic compression algorithm; an Flink real-time computing engine is adopted to implement window aggregation, asynchronous dimension table association and incremental state persistence strategies, and the resource fluctuation and state management problems in streaming computing are solved; time pre-partitioning and intelligent cold and hot layering are achieved through a Dores storage layer, the query performance is improved in combination with materialized views and multi-level indexes, and finally a full-link closed loop of collection-transmission-calculation-storage-management is formed. According to the invention, the problems that the real-time performance and the consistency are difficult to balance and the system complexity is high in the prior art are solved.
Owner:SHANGHAI QUZHI NETWORK TECH CO LTD

Data asset management system based on block chain and big data analysis

The invention is suitable for the technical field of data asset management, and provides a data asset management system based on a block chain and big data analysis, and the system comprises a first terminal which carries out the storage of the ownership information of data assets through a block chain network, and generates a storage data package; generating an evaluation result based on a preset technical index analysis model; writing the hash value of the evaluation result into transaction data of the main chain of the block chain, and generating an evidence storage voucher matched with the block chain transaction; an interaction data sequence is packaged for identity signature, encryption and compression to generate a target code stream, and the target code stream is sent to the second terminal; the second terminal decrypts the encrypted data in the target code stream by using a private key, decompresses the data by using a compression algorithm matched with the first terminal, and restores the data into an interactive data sequence; verifying the access authority and the transaction condition of the evidence storage data packet; checking the anchoring state of the evidence storage voucher on the main chain of the block chain; and the data transaction is completed, the data asset state report is generated, and the data element marketization configuration efficiency is improved.
Owner:GUANGXI ZHUANG AUTONOMOUS REGION COMPUTER CO

Large language model distributed training method and system based on dynamic resource scheduling

The invention provides a large language model distributed training method and system based on dynamic resource scheduling, and the method comprises the steps: collecting the resource state data of each calculation node at a periodic time interval, dividing a training task of a large language model into a plurality of types of subtasks in a current training batch, and carrying out the calculation of the subtasks; on the basis of the resource state data of each computing node and the task description data of each type of subtasks, allocating each type of subtasks to the optimal computing node at the optimal proportion by utilizing a reinforcement learning strategy; in addition, gradient data generated on the computational nodes are compressed by adopting a gradient compression algorithm, and the compression ratio of the gradient data is dynamically adjusted in combination with the current network bandwidth utilization rate of the computational nodes; and the parameter server performs weighted fusion on the gradient data compressed by different computing nodes based on the parameter synchronization interval, updates global model parameters based on a fusion result, and broadcasts the global model parameters to each computing node, so that the training efficiency of the large language model can be remarkably improved.
Owner:BENXI IRON & STEEL (GROUP) INFORMATION AUTOMATION CO LTD

Digital twin drive multi-source heterogeneous data fusion and intelligent analysis method for oil and gas pipeline construction

The invention discloses a digital twin-driven multi-source heterogeneous data fusion and intelligent analysis method for oil and gas pipeline construction. The method comprises the following steps: collecting multi-source heterogeneous data related to oil and gas pipeline construction; fusing data by using a space-time diagram neural network and constructing a dynamic offset compensation model to correct coordinate drift; constructing a data credibility dynamic evaluation chain, performing cross validation on sensor data to generate a confidence coefficient weight, and complementing missing data through a GAN complementing module to form a credibility enhanced data set; designing a layered distillation compression algorithm to compress the PINN model into a TinyLSTM engine, and developing a model increment updating protocol to realize efficient collaboration of a cloud end and an edge end; and training and optimizing a TinyLSTM engine by using the data set with enhanced credibility. According to the method, the defects in the aspects of data dynamic correction, credibility evaluation and complementation, model edge deployment real-time response and the like in the prior art are overcome, and the method has remarkable innovativeness and practicability.
Owner:INNER MONGOLIA WESTERN NATURAL GAS CO LTD +1

Attachment packaging and unpackaging method, system and equipment based on compression algorithm and medium

The invention provides an attachment packing and unpacking method, system and device based on a compression algorithm and a medium, and belongs to the technical field of file compression. The method comprises the steps that all attachment files needing to be packaged are obtained by traversing a specified directory, screening and filtering are conducted according to file attributes, and a renamed file list and a path mapping table are generated; according to the renamed file list, using a Brotli algorithm to compress each attachment file based on a preset compression level, and packaging the compressed data into a packaged file containing metadata, compressed data and verification information; when the packed file needs to be unpacked, the legality of the packed file is confirmed through magic number verification and index analysis, file data are recovered through Brotli streaming decompression, and the original directory structure of the file is restored. According to the method, efficient compression and streaming decompression are achieved through the Brotli algorithm, and the attachment storage efficiency, the transmission speed and the data integrity are remarkably improved in combination with the intelligent screening and directory structure reduction technology.
Owner:INSPUR GENERSOFT CO LTD

Load awareness and data dynamic layout method and system based on deep learning

The invention relates to the technical field of data processing, in particular to a load awareness and data dynamic layout method and system based on deep learning, and the method comprises the steps: collecting hardware load indexes of all nodes in a distributed cluster in real time, and carrying out the preprocessing of the hardware load indexes; based on the preprocessed hardware load index, calculating a comprehensive load score of each node through a weighted fuzzy logic algorithm; dividing load intervals according to the comprehensive load score and matching a predefined strategy combination; and dynamically adjusting a compression algorithm, a compression ratio and a fragment merging parameter based on the current service flow peak-valley state of the target data block and a matched predefined strategy combination, the service flow peak-valley state being determined through a request processing rate and response time. The objective of the invention is to realize dynamic and fine-grained load sensing and data layout.
Owner:CHENGDU NEW HOPE FINANCIAL INFORMATION CO LTD

Multi-source heterogeneous data compression and transmission method and system for intelligent fusion terminal

The invention relates to the technical field of data processing and communication, in particular to a multi-source heterogeneous data compression and transmission method and system for an intelligent fusion terminal, and the method comprises the steps: collecting multi-source heterogeneous data from different data sources through the intelligent fusion terminal, carrying out the data preprocessing of the collected multi-source heterogeneous data, and obtaining a data preprocessing result; comprising format unification and noise filtering; establishing a data feature analysis model, and performing feature extraction and classification on the preprocessed data; according to data feature extraction and classification results, dynamic weights are allocated to different types of data, and the data are divided into data blocks of different sizes according to the allocated dynamic weights and data features; for different types of data blocks, the priority of the data blocks is set, a proper compression algorithm is selected for compression, and the compressed data is transmitted through a network. The multi-source heterogeneous data is subjected to feature analysis and targeted compression processing, so that network bandwidth occupation and equipment energy consumption are reduced, and data transmission efficiency is improved.
Owner:NANJING SIYU ELECTRIC TECH CO LTD

Self-adaptive data compression and transmission method for low-power-consumption wide-area Internet of Things

The invention relates to the technical field of data transmission of the Internet of Things, in particular to a self-adaptive data compression and transmission method of a low-power-consumption wide-area Internet of Things. According to the method, a three-layer collaborative architecture comprising an equipment layer, a fog computing layer and a cloud computing layer is constructed, data preprocessing and feature analysis are performed on the equipment layer, and data types, entropy values and repeatability information are extracted; the fog calculation layer selects an optimal compression algorithm based on a multi-dimensional decision engine, and reduces redundancy through spatial-temporal correlation analysis and data aggregation; the cloud computing layer collects compression performance data, adopts reinforcement learning and federal learning to train a global optimization model, and dynamically issues strategy parameters; and the fog node adaptively adjusts a compression strategy in combination with the system state to realize the optimal balance between the compression ratio and the reconstruction precision. The method is suitable for field deployment environments with limited electric quantity and network, has the advantages of low power consumption, high efficiency, strong adaptability and the like, and can be widely applied to Internet of Things scenes such as remote monitoring, smart energy and the like.
Owner:WUXI PROFESSIONAL COLLEGE OF SCI & TECH

Three-dimensional model compression transmission method and system based on dynamic feature perception

The invention relates to a three-dimensional model compression transmission method and system based on dynamic feature perception, and relates to the technical field of three-dimensional live-action modeling, and the method comprises the steps: obtaining the original three-dimensional data of a three-dimensional model, carrying out the multi-modal feature extraction based on a model scene, dividing a key region, a transition region and a non-key region, and generating a three-dimensional model partition map, compressing each region according to the partition map label and a preset compression algorithm to generate a multi-resolution LOD sequence, determining a transmission data hierarchy in combination with a network state and equipment parameters, and finally rendering the model according to the transmission data hierarchy, user behavior information and an environment state to obtain a target three-dimensional model. The technical effects of effectively compressing, transmitting and rendering the three-dimensional model according to various factors such as different area characteristics, network states, equipment parameters and user behaviors of the model and improving the transmission efficiency and the rendering quality are achieved.
Owner:BEIJING ZHIHUI YUNZHOU TECH CO LTD

Efficient data compression and transmission method based on computing power host management operation platform

The invention discloses an efficient data compression and transmission method based on a computing power host management operation platform, and relates to a large-scale data concurrent management technology. Comprising the steps of 1, integrating a lightweight data feature analyzer at a Kafka producer end, extracting feature indexes of data streams in real time through the data feature analyzer, and generating compression strategy parameters according to the feature indexes, and 2, performing feature recognition on structured data and unstructured data, extracting data features, and generating compression strategy parameters according to the compression strategy parameters. And step 3, based on data characteristics, CPU load of producers / consumers and network bandwidth prediction, constructing a multi-objective optimization model, dynamically selecting a compression algorithm and a compression level, selecting compression strategy parameters according to the compression algorithm, introducing a compression income ratio index, calculating a ratio of compression rate increase to compression time consumption, and performing compression strategy optimization. And preferentially selecting a compression algorithm with a high gain ratio to perform data compression and transmission.
Owner:INSPUR COMM TECH CO LTD

Low-power-consumption hardware encryption system based on geological disaster monitoring scene

The invention relates to the field of hardware encryption, and discloses a geological disaster monitoring scene-based low-power-consumption hardware encryption system, which comprises a layered architecture module for constructing a terminal sensing layer and a cloud platform layer, generating a three-dimensional risk thermodynamic diagram, and carrying out short-term early warning model and emergency deduction; the encryption and decryption module is used for carrying out multi-dimensional identity verification based on protocol layering and national secret algorithm fusion, carrying out dynamic management and eavesdropping monitoring on a secret key, optimizing a data processing flow, and establishing a secure channel to dynamically negotiate whether encryption or decryption is needed; the key management module is used for implementing a triple binding mechanism, carrying out processing and risk assessment on the monitoring data, and carrying out key backup and recovery by adopting a dynamic key period according to a generated risk level; and the safety protection module is used for performing intelligent management on the power consumption of the system based on the active state of the system, selecting a compression strategy according to the data type by adopting a dynamic compression algorithm, performing safety monitoring on hardware and firmware, performing intrusion monitoring on the system and performing automatic repair.
Owner:中国地质环境监测院(自然资源部地质灾害技术指导中心)

Time series data lossless compression method and system based on dynamic context awareness

The invention discloses a time series data lossless compression method and system based on dynamic context awareness, and relates to the field of data lossless compression, and the method comprises the steps: extracting target data from a device port, and carrying out the preprocessing of the target data through a data processing technology according to the business dimension corresponding to the target data, constructing a structured data set based on a preprocessing result; capturing local time sequence features based on the structured data set, distributing attention weights in combination with a hidden state, generating a feature vector used for sensing a context, and outputting a dimension vector by using a three-layer full-connection network; and according to the dimension vector, analyzing the compression performance score value of each compression algorithm in the resource library, sorting the compression performance score values, and selecting the compression algorithm meeting the target requirement to compress the target data. According to the method, the global periodic features and the local mutation features of the time series data are analyzed through the space-time attention mechanism, and real-time driving algorithm switching is achieved.
Owner:GUODIAN NANJING AUTOMATION

Intelligent monitor and method for multi-parameter fusion analysis in natural gas pipeline operation

The invention belongs to the field of natural gas pipeline operation and maintenance, and particularly discloses an intelligent monitor for multi-parameter fusion analysis in natural gas pipeline operation. The sensor module is arranged on a natural gas pipeline and is used for acquiring original data and signal intensity; the preprocessing module preprocesses the original data to obtain preprocessed data, and then adopts a self-adaptive compression algorithm to obtain compressed data; the fusion module adopts a multi-source data fusion algorithm for the compressed data to obtain fusion features; the diagnosis module inputs the fusion features into a fault diagnosis model based on a machine learning algorithm, and when it is diagnosed that a fault exists, the fault type is output; the positioning module obtains the position, geographic information data and historical operation data of the sensor module, and obtains a fault position by adopting a positioning algorithm according to the signal intensity, the position, geographic information data and historical operation data of the sensor module and a fault type; and the early warning module generates a fault report according to the fault position and the fault type, sends out an early warning signal and uploads the fault report.
Owner:PRELITE (TIANJIN) GAS EQUIP CO LTD +1

Nerve radiation field rendering method based on dynamic hash coding

The invention discloses a neural radiation field rendering method based on dynamic hash coding, and the method comprises the steps: employing the feature sequence data as the input, calculating the density value and color value of each sampling point through the forward propagation of a neural network, carrying out the volume rendering integral operation according to the ray tracing principle in the direction of a ray, and obtaining the feature sequence data; judging a final color output result of the current pixel point; according to an error value between the color output result and a real image, updating a network parameter weight through a back propagation algorithm, and if the error value is greater than a convergence threshold, continuing to iterate the training process to adjust a feature coding strategy to obtain an optimized neural radiation field model parameter; and after the rendering performance configuration parameters are obtained, optimizing a storage allocation strategy of feature data through a memory pool management mechanism, and if the current memory occupancy rate exceeds a safety threshold, starting a data compression algorithm to reduce the storage space requirement, and obtaining a real-time rendering output result. According to the invention, high-quality real-time rendering of the dynamic scene is realized.
Owner:ZHEJIANG UNIV OF TECH

Industrial energy consumption data compression and storage method and system based on cooperation of multiple edge nodes

The invention discloses a multi-edge-node collaborative industrial energy consumption data compression and storage method and system, particularly relates to the technical field of data compression and storage, and realizes hierarchical compression processing by layering industrial energy consumption data acquisition equipment into a main node, a core slave node and a terminal slave node. According to the method, various compression algorithms are used for carrying out lossy or lossless compression on different types of data, and cooperative compression and data redundancy elimination between edge nodes are realized under the coordination of a main node by introducing a distance calculation mechanism of'node hierarchy + auxiliary parameters'. The system constructs a node correlation graph through data abstracts, optimizes a data transmission path based on reinforcement learning, and adopts a cold and hot data classification strategy to improve the storage efficiency. The data compression efficiency and reliability are improved, and the method is suitable for real-time processing and long-term storage of large-scale industrial energy consumption data.
Owner:BEIJING JIUJIAN TECH CO LTD

Parameter advertisement method and apparatus, device, and system

This application discloses a parameter advertisement method and apparatus. An unpacking end advertises a packing end of a feature of a data flow that needs to be packed and an identifier of a compression algorithm, so that the feature of the data flow and the identifier of the compression algorithm are automatically transferred from the unpacking end to the packing end. Therefore, the packing end can determine packing for some data flows and a to-be-used compression algorithm based on the received feature of the data flow and the received identifier of the compression algorithm, so that the packing end can pack a specified data flow by using a specified compression algorithm, and a process in which the feature of the data flow and the identifier of the compression algorithm are manually configured at the packing end is avoided. Therefore, configuration complexity at the packing end is reduced.
Owner:HUAWEI TECH CO LTD

ZSTD compression algorithm optimization method based on joint coding and ternary decomposition

PendingCN121749995ACode conversionBit fieldStructural decomposition
The invention relates to the technical field of data compression, in particular to a ZSTD compression algorithm optimization method based on joint coding and ternary decomposition, and the method comprises the following steps: S1, building a joint frequency table based on the joint distribution of the literal length and the matching length of a previous section, setting a frequency deviation criterion, and dynamically adjusting a low-frequency combination proportion; s2, decomposing the offset value structure into a leading segment, a trailing segment and a middle segment, performing bit width-entropy ratio analysis on the middle segment, and selecting a symbol index or bit stream coding; and S3, entropy discriminant coding is performed on the joint symbol and the offset symbol respectively, and a structure position is mapped synchronously to generate a structured compressed output stream. According to the method, collaborative optimization of frequency self-adaption, redundancy minimization and structured mapping is realized through joint symbol dynamic modeling and offset ternary structure entropy discriminant coding, so that the ZSTD compression efficiency and the coding stability are remarkably improved.
Owner:LANZHOU UNIV

Memory management method and related equipment

The invention discloses a memory management method and related equipment, and relates to the technical field of data storage, the method comprises the following steps: based on access information of memory pages of a target operating system, determining a dynamic popularity score of each memory page; on the basis of the current memory pressure level and the dynamic popularity score of the target operating system, executing a corresponding hierarchical data migration strategy so as to migrate different memory pages to the corresponding main Swap storage space or auxiliary Swap storage space; adjusting a compression algorithm applied to the auxiliary Swap storage space based on the current central processing unit utilization rate of the target operating system; and detecting real-time states of the main Swap storage space, the auxiliary Swap storage space and the physical memory, and adjusting a hierarchical data migration strategy and a compression algorithm based on a detection result. Through dynamic popularity perception, hierarchical data migration, adaptive compression regulation and cross-layer collaborative optimization, the memory utilization rate, the exchange efficiency and the overall system stability can be improved under different system loads.
Owner:启朔(深圳)科技有限公司

Transmission data compression method and system based on cloud computing

The invention relates to the technical field of electric digital data processing, and provides a transmission data compression method and system based on cloud computing, and the method comprises the steps: collecting system log data, obtaining a to-be-compressed character string and a to-be-compressed character string set, obtaining a high-frequency code according to the to-be-compressed character string set, and calculating the global redundancy high frequency of the to-be-compressed character string; according to the global redundancy high frequency and the high frequency code of the to-be-compressed character string, the redundancy matching density and the global compression gain degree of the to-be-compressed character string are obtained, then the to-be-compressed character string is subjected to global compression according to the global compression gain degree of the to-be-compressed character string, and then the global compression character string is compressed by using an LZ77 compression algorithm. And then transmission of the system log data is completed. The method aims at solving the problems that an existing LZ77 compression algorithm only considers local similarity, and the compression efficiency of system log data with high global repeatability is not high enough.
Owner:SHENZHEN ENCYCLOPEDIA ZHIYUN TECHNOLOGY CO LTD

Simulation result compression method, system and equipment based on semantic segmentation and medium

The invention relates to a simulation result compression method and system based on semantic segmentation, equipment and a medium. The method comprises the following steps: performing feature extraction on a simulation result file to obtain semantic features; generating a segmentation point sequence according to the features by using a semantic segmentation model, and segmenting the file into a plurality of semantic blocks; extracting a compressibility feature vector for each block, and dynamically distributing an optimal compression strategy for the block through a prediction model; compressing each block by using the distributed strategy to generate a compressed data block; and finally, constructing a block sensing index based on the block type and the used strategy, and integrating the index and the data block to generate a compressed file. By adopting the method, the defects of limited compression ratio and incapability of supporting rapid random access caused by lack of semantic understanding of a general compression algorithm can be overcome, and by understanding the internal logic structure of the file and applying differential compression, the rapid retrieval and analysis capability of specific data is kept while the high compression ratio is realized.
Owner:陈建忠

Multi-modal data enhanced storage system based on Parquet format

The invention provides a multi-modal data enhanced storage system based on a Parquet format. Unified storage and optimization of multi-modal data are achieved through native transformation and expansion of the Parquet format. The system comprises a modal perception format expansion component and a modal optimization coding strategy component, wherein the modal perception format expansion component realizes deep fusion of multi-modal semantics and a storage structure by expanding a Parquet logic type system and constructing a four-layer hierarchical metadata architecture from a file level to a page level; and performing instruction level fusion with a Parquet compression algorithm, executing targeted storage optimization based on semantic descriptors and modal characteristic parameters, and solving the problems of unified storage and efficient management of multi-modal data.
Owner:BEIJING TIANHUA XINGHANG TECH

Natural resource archive retrieval and utilization method combining large model and knowledge graph technology

The invention provides a natural resource archive retrieval and utilization method combining a large model and a knowledge graph technology, and relates to the technical field of data processing, the method comprises the following steps: identifying business entities in the field of natural resources and semantic association thereof, and constructing an RDF triple with attribute constraint; performing geometric feature preserving type coordinate simplification on the plot entity vector boundary coordinate point sequence through a Douglas-Peucker compression algorithm to generate a key coordinate point set after noise reduction; on the basis of the key coordinate point set, vector geometric superposition operation is carried out on polygon boundaries of the two plot entities, the area proportion of overlapped regions is calculated according to the intersection-to-union ratio, and when the intersection-to-union ratio exceeds a preset threshold value, entity space alignment is carried out, so that a triple set of space alignment is obtained; and fusing the triple set with the aligned space, and constructing the natural resource domain knowledge graph. According to the invention, redundant information interference is effectively reduced.
Owner:ZHEJIANG WANWEI SPACE INFORMATION TECH CO LTD

Method and system for identifying anti-counterfeit label

The invention discloses a method and a system for identifying an anti-counterfeit label. The method comprises the following steps: acquiring spectral response characteristics of the anti-counterfeit label to be identified through a multiband optical sensor; extracting a dynamic coding area and a base material characteristic area in the spectral response characteristics, wherein the dynamic coding area comprises topological distribution information of a programmable encryption particle array; processing the topological distribution information of the dynamic coding region through a feature compression algorithm to generate a first verification factor, and extracting a spatial frequency feature of the base material feature region as a second verification factor; and uploading the first verification factor and the second verification factor to a cloud verification platform through a secure communication protocol, wherein the cloud verification platform is internally provided with a dynamic spectral feature library and a physical feature association model. According to the invention, through a double-engine architecture of programmable dynamic target and environment adaptive verification, the problems of static replicability and environment vulnerability of physical anti-counterfeiting are solved, and collaborative rise of anti-counterfeiting strength and use reliability is realized.
Owner:GUANGZHOU ZIBAI XINLIAN INTELLIGENT MANUFACTURING TECHNOLOGY CO LTD