Large data compression method and system based on neural network, and storage medium
A neural network and compression method technology, applied to systems and storage media, in the field of big data compression methods based on neural networks, can solve the problems of long training time, complex self-encoding network, long time, etc., and achieve the effect of reducing the time-consuming compression
Pending Publication Date: 2022-06-17
河南伴龙电子科技有限公司
View PDF0 Cites 4 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, the inventors of the embodiments of the present invention found in the process of using the autoencoder network to compress large data that the required training time is relatively long, and the
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0039] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0040] In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be practiced in other embodim...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a big data compression method and system based on a neural network and a storage medium, and relates to the field of artificial intelligence. The method mainly comprises the following steps of: performing arithmetic coding on to-be-compressed data, and respectively obtaining an initial weight of each data; constructing a self-encoding network, wherein the self-encoding network comprises an input layer, an output layer and at least one hidden layer; taking the coded data as the input and output of the self-coding network at the same time, taking the initial weight of each piece of data as the initial weight of a neuron corresponding to each piece of data, and training the self-coding network; and performing network pruning on the self-encoding network according to the weights from small to large until at least one of the compression rate and the accuracy rate of the self-encoding network is out of a corresponding preset threshold range, and taking data corresponding to a hidden layer in the self-encoding network after pruning as compressed data. According to the embodiment of the invention, the processing efficiency of big data compression can be improved.
Description
technical field [0001] The present application relates to the field of data compression, in particular to a neural network-based big data compression method, system and storage medium. Background technique [0002] Big data (big data), or huge amount of data, refers to the amount of data involved that is too large to be captured, managed, processed, and organized within a reasonable time through current mainstream software tools to help enterprises operate. Information for more positive purposes of decision-making. [0003] Generally speaking, there is a lot of redundant information in the data, which is more obvious in the middle of big data. The purpose of data compression is to reduce data redundancy as much as possible. Currently, self-encoding networks are usually used for data compression. However, in the process of compressing big data by using the self-encoding network in the embodiment of the present invention, the inventor found that the required training time is ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): H03M7/30G06N3/04G06N3/08
CPCH03M7/30G06N3/04G06N3/082
Inventor 周杨凡常小梅
Owner 河南伴龙电子科技有限公司