Binary data compression and storage method and system based on attention mechanism and electronic equipment

Abstract

The invention relates to the technical field of data compression and storage, and discloses a binary data compression and storage method and system based on an attention mechanism and electronic equipment, and the method comprises the following steps: S1, carrying out the type recognition of binary data through employing an attention mechanism deep convolutional neural network formula and a data type recognition algorithm; s2, selecting a compression strategy according to a type identification result; s3, analyzing the data structure, and finding out a special structure according to an analysis result of the data structure and a structure searching algorithm for processing so as to improve a compression ratio; s4, compressing the data; and S5, storing the compressed data. The system and the electronic equipment correspond to the method. By the adoption of the method and device, the advantages of the deep neural network can be fully played, intelligent compression and efficient storage of binary data are achieved, and powerful technical support is provided for development of electronic equipment and electronic systems.

Description

Technical Field

[0001] The present application relates to the technical field of data compression and storage, and specifically to a binary data compression and storage method, system and electronic device based on an attention mechanism. Background Art

[0002] With the rapid development of information technology, the generation and storage demand of data has exploded. In various fields, whether it is scientific research, commercial applications or personal life, we are faced with the challenge of processing and storing massive amounts of data. Especially in electronic devices and systems that have strict requirements on storage space, how to efficiently compress and store data has become a key issue.

[0003] In traditional data compression technology, there are many methods, such as Huffman coding, LZ77 algorithm, etc. These methods can achieve data compression to a certain extent, but they often have some limitations. For example, Huffman coding requires statistical analysis of data and the construction of a specific coding table, which has poor adaptability to dynamically changing data. The LZ77 algorithm does not have an ideal compression ratio when processing certain types of data.

[0004] At the same time, with the continuous miniaturization and enhancement of functions of electronic devices, the demand for storage space is more urgent. The storage space of electronic devices such as smartphones, tablets, and wearable devices is limited, while users' demand for storing multimedia files such as photos, videos, music, and various applications is increasing. In addition, in the Internet of Things system, a large number of sensor devices continuously generate data, which needs to be stored and transmitted efficiently. Cloud computing platforms also face the problem of large-scale data storage and management, which puts higher requirements on data compression technology.

[0005] In terms of data storage, although the capacity of traditional storage technologies such as hard disks and flash memory is constantly increasing, the cost and energy consumption are also increasing accordingly. Moreover, as the amount of data continues to grow, simply increasing storage capacity is not a sustainable solution. Therefore, it is necessary to develop more efficient data compression technology to reduce storage space usage and improve storage efficiency.

[0006] In recent years, deep learning technology has achieved great success in image recognition, speech processing and other fields. Deep neural networks have powerful feature extraction and pattern recognition capabilities, can automatically learn the intrinsic characteristics of data, and provide new ideas for data compression. The introduction of the attention mechanism further enhances the performance of deep neural networks, enabling them to pay more attention to key information in the data and improve processing efficiency.

[0007] However, in the field of binary data compression and storage, the advantages of deep neural networks with attention mechanisms have not yet been fully utilized. Most existing binary data compression methods are based on traditional algorithms and lack the ability to intelligently identify data types and adaptively compress them. When faced with different types of binary data, it is impossible to automatically select the most appropriate compression strategy based on their characteristics, resulting in a low compression ratio and a large waste of storage space.

[0008] In addition, existing data compression technologies are often inefficient when processing large-scale data and are difficult to meet real-time requirements. In electronic devices and systems, especially in some application scenarios with high response time requirements, such as real-time monitoring and online games, fast and efficient data compression and storage methods are needed to ensure system performance and user experience.

[0009] In summary, existing data compression storage technologies have many shortcomings when facing the ever-increasing amount of data and strict requirements on storage space. Summary of the invention

[0010] The purpose of this application is to provide a binary data compression and storage method, system and electronic device based on an attention mechanism, so as to give full play to the advantages of deep neural networks, realize intelligent compression and efficient storage of binary data, and provide strong technical support for the development of electronic devices and electronic systems.

[0011] To achieve the above objectives, this application discloses the following technical solutions:

[0012] In a first aspect, the present application discloses a binary data compression storage method based on an attention mechanism, the method comprising the following steps:

[0013] S1: Use the attention mechanism deep convolutional neural network formula and data type recognition algorithm to identify the type of binary data;

[0014] S2: Select compression strategy based on type recognition result;

[0015] S3: Analyze the data structure, and according to the analysis result of the data structure, find out the special structure according to the structure search algorithm to process it to improve the compression ratio;

[0016] S4: compress the data;

[0017] S5: Store the compressed data.

[0018] Preferably, in S1, the attention mechanism deep convolutional neural network formula is:

[0019]

[0020] Among them, X is the input binary data, f(X) is the global feature extraction function, and h i (X) is the local feature extraction function, α i is the attention weight coefficient, g(X) is the variable adjustment function for identifying content and binary related features, and k j (X) is the auxiliary adjustment function, β j is the adjusted weight coefficient, and Y is the adjusted variable.

[0021] Preferably, in S1, the data type identification algorithm is:

[0022]

[0023] Among them, X is the input binary data, p(X) is the main function for extracting data type features, and q l (X) is the auxiliary feature extraction function, γ l is the weight coefficient.

[0024] Preferably, S2 specifically includes: selecting a suitable strategy according to the data type and specific requirements using a strategy selection formula, wherein the strategy selection formula is:

[0025]

[0026] Among them, X is the input binary data, r(X) is the main function of strategy selection, t a (X) is the strategy evaluation function, σ a is the selection weight coefficient.

[0027] Preferably, in S3, the structure search algorithm is:

[0028]

[0029] Among them, X is the input binary data, u(X) is the main function of structural analysis, s v (X) is the structural feature extraction function, ε v is the analysis weight coefficient.

[0030] Preferably, the S4 specifically includes: compressing the data using a compression algorithm, wherein the compression algorithm is:

[0031]

[0032] Where X is the input binary data, d(X) is the compression main function, z e (X) is the compression parameter function, η e is the compression weight coefficient.

[0033] Preferably, the S5 specifically includes: storing the compressed data using a storage algorithm, wherein the storage algorithm is:

[0034]

[0035] Among them, X is the input binary data, W(X) is the storage main function, G h (X) is the storage parameter function, θ h To store weight coefficients.

[0036] In a second aspect, the present application discloses a binary data compression storage system based on an attention mechanism, which is applicable to the binary data compression storage method based on the attention mechanism as described above, including a data type recognition module, a compression strategy selection module, a structure analysis module, a compression module and a storage module;

[0037] The data type identification module is configured to: use the attention mechanism deep convolutional neural network formula and the data type identification algorithm to perform type identification on binary data;

[0038] The compression strategy selection module is configured to: select a compression strategy according to the type identification result;

[0039] The structure analysis module is configured to: analyze the data structure, and according to the analysis result of the data structure, find out the special structure according to the structure search algorithm to process it so as to improve the compression ratio;

[0040] The compression module is configured to: compress data;

[0041] The storage module is configured to store the compressed data.

[0042] In a third aspect, the present application discloses an electronic device comprising at least one memory and at least one processor, wherein the memory is communicatively connected to the processor, and a computer program executable by the processor is stored on the memory. When the computer program is executed by the processor, the binary data compression and storage method based on the attention mechanism as described above is implemented.

[0043] Compared with existing solutions, the binary data compression storage method, system and electronic device based on the attention mechanism of the present application have the following technical effects:

[0044] 1. Improve the compression ratio

[0045] 1. Intelligent recognition and adaptive compression: The present invention uses a deep neural network with an attention mechanism to accurately identify the type of binary data and automatically select the most suitable compression strategy according to the different types. Compared with the traditional fixed algorithm compression method, this intelligent recognition and adaptive compression capability can better adapt to various types of binary data, thereby significantly improving the compression ratio. For example, for repeated patterns and texture features in image data, the system can adopt specific compression strategies to minimize data redundancy.

[0046] 2. Special processing of structural analysis: Through structural analysis, the data structure is deeply analyzed to find special structures for special processing. In this way, the inherent structural characteristics of the data can be better utilized to further improve the compression ratio. For example, for certain data structures with specific rules, such as continuous repeated byte sequences or specific encoding patterns, special optimization processing can be performed through structural analysis to achieve a higher degree of compression.

[0047] 2. Save storage space

[0048] 1. Efficient compression: Since the present invention can achieve a high compression ratio, more data can be stored in the same storage space. For electronic devices and systems, this means that the reliance on external storage devices can be reduced, reducing storage costs. For example, in a smartphone, the compression storage method of the present invention allows users to store more photos, videos, and applications without having to frequently clear storage space or purchase additional memory cards.

[0049] 2. Optimize storage layout: The data storage of the present invention can optimize the storage layout according to the characteristics of the compressed data. By reasonably arranging the storage location and method of the data, the utilization rate of the storage space can be further improved. For example, for frequently accessed data, it can be stored on a faster storage medium to increase the access speed; and for infrequently accessed data, a more compact storage method can be used to save space.

[0050] 3. Improve the efficiency and security of data storage

[0051] 1. Fast compression and decompression: The efficient computing power of the attention mechanism deep neural network enables the present invention to complete the data compression and decompression process in a relatively short time. This is very important for electronic devices and systems that need to process data in real time. For example, in live video applications, fast compression and decompression can ensure smooth transmission of video streams, reduce delays, and improve user experience.

[0052] 2. Data security: Data storage uses a safe and reliable storage method to ensure that the compressed data will not be easily tampered with or lost. Encryption technology can be used to protect data to prevent unauthorized access. At the same time, the system can also perform data backup and recovery to cope with various unexpected situations and ensure data security.

[0053] 4. Broad application prospects

[0054] 1. Applicable to various electronic devices and systems: The method, component device and system of binary data compression and storage based on attention mechanism deep neural network of the present invention are highly versatile and can be applied to various types of electronic devices and systems. Whether it is a mobile terminal such as a smart phone, tablet computer, wearable device, or a large data processing system such as an Internet of Things system and a cloud computing platform, it can benefit from it. For example, in an Internet of Things system, the binary data generated by a large number of sensor devices can be efficiently compressed and stored by the present invention, reducing the cost of data transmission and storage.

[0055] 2. Promote the development of data storage technology: This invention provides new ideas and methods for the development of data storage technology. By introducing the attention mechanism deep neural network, it breaks the limitations of traditional data compression technology and lays the foundation for future data storage technology innovation. At the same time, the successful application of this invention will also encourage more researchers and companies to invest in the research and development of data compression storage technology and promote the development of the entire industry.

[0056] 5. Improve user experience

[0057] 1. Increase device storage capacity: For users, the present invention can significantly increase the storage capacity of electronic devices, allowing users to store more personal data and files. This not only facilitates users' daily use, but also reduces the troubles caused by insufficient storage space. For example, users can store more high-definition photos and videos on their mobile phones without worrying about insufficient storage space.

[0058] 2. Improve device performance: Since the present invention can reduce the occupation of device resources by data storage, the performance of electronic devices can be improved. For example, the storage access time and energy consumption are reduced, making the device run more smoothly and the battery life longer. At the same time, the fast compression and decompression process can also improve the response speed of the device and enhance the user's operating experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 A schematic diagram of a flow chart of a binary data compression and storage method based on an attention mechanism provided in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of a process for identifying data types provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of a flow chart of compression strategy selection provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of a process for searching for a special structure provided in an embodiment of the present application;

[0064] Figure 5 A schematic diagram of the data compression process provided in this embodiment;

[0065] Figure 6 A schematic diagram of the data storage process provided in this embodiment;

[0066] Figure 7 A schematic diagram of the working process of the device corresponding to the binary data compression storage method based on the attention mechanism provided in this embodiment;

[0067] Figure 8 A schematic diagram of the composition of the device corresponding to the binary data compression storage method based on the attention mechanism provided in this embodiment. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0069] In this article, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprising..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0070] In today's digital age, the explosive growth of data has posed a severe challenge to storage technology. In particular, efficient data compression and storage methods are crucial in electronic devices and systems that have strict requirements on storage space.

[0071] To this end, the present embodiment provides a first aspect as follows Figure 1 The binary data compression storage method based on the attention mechanism shown in the figure comprises the following steps:

[0072] S1: Use the attention mechanism deep convolutional neural network formula and data type recognition algorithm to identify the type of binary data;

[0073] S2: Select a compression strategy based on the type recognition result; in this step, according to the binary data type recognized in the previous step, select the compression strategy that best suits the type of data from the preset multiple compression strategies. For example, if the recognized data is image-type binary data, the system may select a compression strategy that targets the characteristics of image data, such as a compression method based on discrete cosine transform (DCT); if it is text-type binary data, a compression strategy based on dictionary encoding may be selected.

[0074] S3: Analyze the data structure. According to the analysis results of the data structure, find the special structure according to the structure search algorithm to improve the compression ratio. In this step, the input binary data is deeply analyzed to find the special structure, such as repeated byte sequences, specific encoding patterns, etc. Once these special structures are found, the system will use special processing methods to optimize and compress them. For example, for continuous and repeated byte sequences, run-length encoding can be used for compression, which greatly reduces the amount of data.

[0075] S4: compress the data; in this step, the actual compression operation is performed on the binary data according to the selected compression strategy and the results of the structural analysis. This process may involve the combined use of multiple compression algorithms to achieve the best compression effect. For example, for image data, discrete cosine transform may be performed first, followed by quantization and entropy coding.

[0076] S5: Storing the compressed data. In this step, the compressed binary data is stored safely and reliably. The storage module can use a variety of storage technologies, such as hard disk storage, flash memory storage, etc., and can perform hierarchical storage according to the importance and access frequency of the data to improve storage efficiency and data availability.

[0077] In this embodiment, the attention mechanism deep convolutional neural network formula is:

[0078]

[0079] Where X is the input binary data, f(X) is the global feature extraction function, and h i (X) is the local feature extraction function, α i is the attention weight coefficient, g(X) is the variable adjustment function for identifying content and binary related features, and k j (X) is the auxiliary adjustment function, β j is the adjusted weight coefficient, and Y is the adjusted variable.

[0080] Through the attention mechanism deep convolutional neural network formula, the variables of recognition content and binary related characteristics are fine-tuned. For example, for image data, color distribution, texture features, etc. may be extracted as recognition features; for text data, features such as character frequency and word frequency may be extracted.

[0081] In this embodiment, correspondingly, in S1, the data type identification algorithm is:

[0082]

[0083] Among them, X is the input binary data, p(X) is the main function for extracting data type features, and q l (X) is the auxiliary feature extraction function, γ l is the weight coefficient.

[0084] Specifically, if Figure 2 As shown, the type of data is determined by extracting and analyzing the input binary data. In this process, the main function p(X) is first used to perform preliminary feature extraction, and then combined with the auxiliary feature extraction function q l (X) Further refine the features. By adjusting the weight coefficient γ l , can optimize the effect of feature extraction and improve the accuracy of data type recognition. For example, for image data, color distribution, texture features, etc. may be extracted as recognition features; for text data, character frequency, word frequency, etc. may be extracted.

[0085] In this embodiment, S2 specifically includes: using a strategy selection formula to select a suitable strategy according to the data type and specific requirements, and the strategy selection formula is:

[0086]

[0087] Among them, X is the input binary data, r(X) is the main function of strategy selection, t a (X) is the strategy evaluation function, σ a is the selection weight coefficient.

[0088] Specifically, Figure 3 As shown in the figure, the strategy selection main function r(X) is responsible for the preliminary screening of possible compression strategies, and then the strategy evaluation function t a (X) Each candidate strategy is evaluated, taking into account factors such as compression ratio, compression time, decompression time, etc. Finally, the selection weight coefficient σ is adjusted a , determine the best compression strategy. For example, if the application scenario has high real-time requirements, you may choose a strategy with fast compression speed but relatively low compression ratio; if the scenario has extremely high storage capacity requirements, you will choose a strategy with high compression ratio but long compression time.

[0089] In this embodiment, in S3, the structure search algorithm is:

[0090]

[0091] Among them, X is the input binary data, u(X) is the main function of structural analysis, s v (X) is the structural feature extraction function, ε v is the analysis weight coefficient.

[0092] Specifically, Figure 4 As shown in Figure 1, the function of the structure search algorithm is to deeply analyze the structure of binary data and find special structures in it. The main function of structure analysis u(X) starts the structure analysis process, and then the structure feature extraction function s v (X) Extract the structural features of the data. Analyze the weight coefficient ε vThe importance of different structural features can be adjusted to better discover special structures. For example, for some data with a specific encoding pattern, the structural analysis module can analyze the encoding rules to find out the parts that can be optimized and compressed. For example, for some data with a specific encoding pattern, the structural analysis module can analyze the encoding rules to find out the parts that can be optimized and compressed. For image data, if there are repeated texture structures or specific color distribution patterns, etc., they can be regarded as special structures for analysis and processing. For text data, if there are repeated character sequences or specific vocabulary patterns, etc., they can also be analyzed as special structures.

[0093] In this embodiment, S4 specifically includes: compressing the data using a compression algorithm, and the compression algorithm is:

[0094]

[0095] Where X is the input binary data, d(X) is the compression main function, z e (X) is the compression parameter function, η e is the compression weight coefficient.

[0096] Specifically, Figure 5 As shown in Figure 1, the compression algorithm is the core part of data compression. It uses an efficient compression algorithm, and the compression main function d(X) controls the entire compression process. The compression parameter function z e (X) can be adjusted according to different data types and compression strategies to achieve the best compression effect. For example, for different types of data, such as image data and text data, the quantization step size, encoding table and other parameters in the compression parameter function can be adjusted. For image data, the quantization step size may need to be adjusted according to factors such as the image resolution and color mode; for text data, the encoding table may need to be adjusted according to factors such as the text character set and language. Compression weight coefficient η e It is used to adjust the importance of different parameters. For example, for different types of data, the quantization step size, encoding table and other parameters in the compression parameter function can be adjusted to achieve a higher compression ratio. For example, if it is found that a certain compression algorithm does not have a good compression effect on a certain type of data, the compression weight coefficient η can be adjusted. e , changing the weights of related parameters to improve the compression effect.

[0097] In this embodiment, S5 specifically includes: storing the compressed data using a storage algorithm, and the storage algorithm is:

[0098]

[0099] Among them, X is the input binary data, W(X) is the storage main function, Gh (X) is the storage parameter function, θ h To store weight coefficients.

[0100] Specifically, Figure 6 As shown in Figure 1, the storage algorithm is responsible for safely and reliably storing the compressed binary data. The storage main function W(X) controls the storage process, and the storage parameter function G h (X) can be adjusted according to the characteristics of the data and storage requirements, such as selecting different storage media, setting storage layout, etc. If the data is important data that is frequently accessed, it may be chosen to be stored in a high-speed storage medium such as a solid-state drive, and a reasonable storage layout is adopted for fast access; if the data is non-important data that is not frequently accessed, it can be chosen to be stored in a large-capacity, low-speed storage medium such as a mechanical hard disk, and a more compact storage method is adopted to save space. Storage weight coefficient θ h Used to adjust the importance of different storage parameters. For example, data can be stored on different storage media according to the importance and access frequency of the data to improve storage efficiency and data availability. In addition, during the storage process, a safe and reliable storage method is used to ensure that the compressed data will not be easily tampered with or lost. Encryption technology can be used to protect data to prevent unauthorized access. At the same time, the system can also perform data backup and recovery to cope with various unexpected situations and ensure data security.

[0101] Accordingly, this embodiment also provides a device using the above method, combined with Figure 7 and 8 As shown, the components of the device include a data type identification component, a compression strategy selection component, a structure analysis component, a compression component and a storage component, and each component works together to realize the compression storage of binary data. Each component has its own specific function, and they work together to realize the compression storage of binary data. For example, the data type identification component is responsible for identifying the type of input data, providing a basis for the subsequent compression strategy selection; the compression strategy selection component selects a suitable compression strategy according to the data type; the structure analysis component analyzes the data structure and finds out the special structure for optimized compression; the compression component performs the actual compression operation; and the storage component stores the compressed data safely.

[0102] In the second aspect, the present application discloses a binary data compression and storage system based on an attention mechanism, which is applicable to the binary data compression and storage method based on the attention mechanism as described above, including a data type identification module, a compression strategy selection module, a structure analysis module, a compression module and a storage module.

[0103] Specifically, the data type identification module is configured to: use the attention mechanism deep convolutional neural network formula and the data type identification algorithm to identify the type of binary data; the compression strategy selection module is configured to: select the compression strategy according to the type identification result; the structure analysis module is configured to: analyze the data structure, and according to the analysis result of the data structure, find the special structure according to the structure search algorithm for processing to improve the compression ratio; the compression module is configured to: compress the data; the storage module is configured to: store the compressed data. When working, first, the data type identification module in each device performs feature extraction and analysis on the input binary data to determine the data type. Then, the compression strategy selection module selects the appropriate compression strategy according to the data type and specific needs. Subsequently, the structure analysis module deeply analyzes the data structure to find the special structure. After that, the compression module compresses the data according to the selected compression strategy and the results of the structure analysis. Finally, the storage module stores the compressed data safely and reliably.

[0104] The system can efficiently compress and store binary data. In particular, the device equipped with the system can be an independent device or a part integrated in other electronic devices. For example, in a smart phone, the system can be integrated to compress and store binary data such as photos and videos in the phone to save storage space. In particular, each component of a device can be equipped with the system to achieve compressed storage of large-scale binary data, and can be applied to various scenarios, such as data centers, cloud computing platforms, etc. For example, in a data center, multiple such systems can be deployed to efficiently compress and store massive amounts of binary data, reduce storage costs, and improve storage efficiency.

[0105] In a third aspect, the present application discloses an electronic device comprising at least one memory and at least one processor, wherein the memory is communicatively connected to the processor, and a computer program executable by the processor is stored on the memory. When the computer program is executed by the processor, the binary data compression and storage method based on the attention mechanism as described above is implemented.

[0106] It should be noted that the devices, systems and electronic devices disclosed in this embodiment all correspond to the aforementioned method. Therefore, for the parts of the devices, systems and electronic devices that are not disclosed in the text, please refer to the relevant descriptions in the aforementioned method, and this text will not elaborate on them.

[0107] Moreover, in the embodiments provided in the present application, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein or their combination. For software implementation, part or all of the flow of the embodiment can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein the communication medium includes any medium that is convenient for transmitting a computer program from one place to another. The storage medium can be any available medium that a computer can access. The computer-readable storage medium can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer.

[0108] The following is a further introduction to the technical solution provided by this embodiment through actual application cases.

[0109] 1. Smartphone applications

[0110] In smart phones, a large number of photos, videos and application data are binary data. Using the method of the present invention, the data is first identified by type, for example, the image features of the photo data can be identified. Then, a suitable compression strategy is selected according to its type, such as a compression algorithm suitable for the image may be used for the photo. Then, the data structure is analyzed, and if there is a special structure, it can be optimized. After that, the compression operation is performed, and finally the compressed data is stored in the mobile phone storage. This allows users to store more photos, videos and applications without frequently clearing storage space or purchasing additional memory cards.

[0111] The detailed process includes: In a smart phone, the photo and video data are binary data. Using the method of the present invention, firstly, through the data type recognition module, for the photo data, the data type feature extraction main function p(X) may extract basic features such as color, and the auxiliary feature extraction function q l (X) Further extract features such as texture by adjusting the weight coefficient γ lAccurately identify it as image type data. Then select a suitable compression strategy based on the image type, such as a compression method based on discrete cosine transform. Then analyze the data structure. If the video data contains continuous and repeated byte sequences (such as inter-frame data in certain video formats), run-length encoding is used for optimized compression. Finally, the compressed data is stored in the mobile phone storage.

[0112] 2. Application of Internet of Things System

[0113] In the Internet of Things system, a large number of sensor devices continuously generate binary data. Using the method of the present invention, these data are type identified, compression strategies are selected, structures are analyzed, compressed and stored. By efficiently compressing and storing these data, data transmission and storage costs can be reduced and the operating efficiency of the Internet of Things system can be improved. For example, environmental data collected by sensors can be more efficiently stored and transmitted after being processed by the method of the present invention for subsequent analysis and processing.

[0114] The detailed process content includes: In the IoT system, the data collected by the sensor is often binary data. For example, the temperature data collected by the temperature sensor can be extracted through the data type recognition module. The data type feature extraction main function p(X) may extract the data value range and other features, and the auxiliary feature extraction function q l (X) Further extract features such as data change frequency by adjusting the weight coefficient γ l Accurately identify it as a specific type of data. Select an appropriate compression strategy based on the data type, such as a compression algorithm for numeric data. Analyze the data structure, and if there is a specific encoding mode (such as a specific format encoding of sensor data), perform special processing, and finally store the compressed data in the storage device of the IoT system.

[0115] 3. Image storage application

[0116] In the image storage of digital cameras or smart phones, the image data is binary data. First, the image features are identified by the data type identification module. Then, according to the storage requirements, if the user wants to store more high-quality images in a limited storage space, the compression strategy selection module may choose a compression strategy based on discrete cosine transform and with more refined quantization parameters. Then, the image data structure is analyzed to find out possible special structures for optimization. After that, the compression module is used to perform compression operations, and finally the compressed data is stored in the storage medium of the device.

[0117] 4. Live video application

[0118] In the live video broadcast scenario, the video stream data is binary data. The data type identification module identifies its image features. Due to the high real-time requirements, the compression strategy selection module will select a strategy with fast compression speed but relatively low compression ratio, such as a method based on a fast encoding algorithm. The video data structure is then analyzed and necessary special processing is performed. The compression module is used to perform compression operations and store the compressed data in the storage device of the live broadcast server to ensure smooth transmission of the video stream, reduce delays, and improve user experience.

[0119] 5. Image data processing

[0120] In image editing software, when processing image data (binary data), the data type is first identified and determined to be an image type. Then a compression strategy is selected based on the purpose and storage requirements of the image. Then a structural analysis is performed. If the image has a repeated texture structure, the structural analysis module will identify this special structure. Then, based on the compression strategy and the structural analysis results, the image is compressed using a compression module, such as using a compression algorithm suitable for the texture structure. Finally, the compressed data is stored in the storage area of ​​the software for subsequent use.

[0121] 6. Text file compression

[0122] When processing text files (binary data), the data type is first identified as text. Then the compression strategy is selected according to the nature of the text and the usage requirements. When performing structural analysis, if there are repeated character sequences in the text, the structural analysis module will identify this special structure. According to the compression strategy and the structural analysis results, the compression module is used to compress the text, for example, using a compression algorithm for repeated character sequences. Finally, the compressed data is stored in the corresponding storage device.

[0123] 7. Enterprise Data Storage

[0124] In an enterprise, there are a large number of documents, images, videos and other data that need to be stored. These data need to be stored in the enterprise's storage system after being compressed. First, the types of various types of data are determined through data type identification. Then, according to the characteristics of different types of data and the storage needs of the enterprise, appropriate compression strategies and structural analysis are selected. For example, for document data, a compression strategy based on dictionary encoding may be selected, and for image data, a compression strategy based on discrete cosine transform may be selected. Then, after the compression operation is completed, the storage module is adjusted according to the importance and access frequency of the data. Important and frequently accessed document data may be stored in a high-speed solid-state hard drive within the enterprise, using a reasonable storage layout; infrequently accessed image data may be stored in a large-capacity mechanical hard drive, using a more compact storage method. Finally, the storage module uses encryption technology to protect the data to prevent unauthorized access, and regularly backs up and restores the data to ensure data security.

[0125] 8. Cloud Storage for Personal Use

[0126] When individual users use cloud storage services to store data such as photos, videos, and documents, they also need to go through these steps. First, identify the data type, such as photos and videos are image types, and documents are text types. Then, select an appropriate compression strategy, such as a compression strategy based on discrete cosine transform for image data and a compression strategy based on dictionary encoding for document data. Next, perform structural analysis and compression operations. After that, the storage module adjusts according to the importance and access frequency of the data. For example, frequently accessed photos may be stored in the high-speed storage area of ​​the cloud storage service provider, and infrequently accessed documents may be stored in a large-capacity area using a more compact storage method. Finally, the storage module uses encryption technology to protect the data to prevent unauthorized access, and provides data backup and recovery functions to ensure data security.

[0127] 9. Data Center Data Storage

[0128] In the data center, there is a huge amount of binary data that needs to be stored. The system is deployed on various devices in the data center, and each device processes the data. For example, for different types of file data, the system can accurately identify its type, select the appropriate strategy, and compress and store it after analyzing the structure, effectively saving storage space and improving storage efficiency.

[0129] 10. Data storage on cloud computing platforms

[0130] The cloud computing platform also faces the problem of processing a large amount of binary data. Each device corresponding to the cloud computing platform is equipped with this system, and each device works together to identify the type, select strategies, analyze the structure, compress and store the data. This helps to reduce storage costs and improve the data processing capabilities and efficiency of the platform.

[0131] Finally, it should be noted that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A binary data compression storage method based on attention mechanism, characterized in that: The method comprises the following steps: S1: Use the attention mechanism deep convolutional neural network formula and data type recognition algorithm to identify the type of binary data; S2: Select compression strategy based on type recognition result; S3: Analyze the data structure, and according to the analysis result of the data structure, find out the special structure according to the structure search algorithm to process it to improve the compression ratio; S4: compress the data; S5: Store the compressed data.

2. The binary data compression storage method based on the attention mechanism according to claim 1 is characterized in that: In S1, the attention mechanism deep convolutional neural network formula is: Where X is the input binary data, f(X) is the global feature extraction function, and h i (X) is the local feature extraction function, α i is the attention weight coefficient, g(X) is the variable adjustment function for identifying content and binary related features, and k j (X) is the auxiliary adjustment function, β j is the adjusted weight coefficient, and Y is the adjusted variable.

3. The binary data compression storage method based on the attention mechanism according to claim 1 is characterized in that: In S1, the data type identification algorithm is: Among them, X is the input binary data, p(X) is the main function for extracting data type features, and q l (X) is the auxiliary feature extraction function, γ l is the weight coefficient.

4. The binary data compression storage method based on the attention mechanism according to claim 1 is characterized in that: S2 specifically includes: using a strategy selection formula to select a suitable strategy according to the data type and specific requirements, and the strategy selection formula is: Among them, X is the input binary data, r(X) is the main function of strategy selection, t a (X) is the strategy evaluation function, σ a is the selection weight coefficient.

5. The binary data compression storage method based on the attention mechanism according to claim 1 is characterized in that: In S3, the structure finding algorithm is: Among them, X is the input binary data, u(X) is the main function of structural analysis, s v (X) is the structural feature extraction function, ε v is the analysis weight coefficient.

6. The binary data compression storage method based on the attention mechanism according to claim 1 is characterized in that: The S4 specifically includes: compressing the data using a compression algorithm, wherein the compression algorithm is: Among them, X is the input binary data, d(X) is the compression main function, z e (X) is the compression parameter function, η e is the compression weight coefficient.

7. The binary data compression storage method based on the attention mechanism according to claim 1 is characterized in that: S5 specifically includes: storing the compressed data using a storage algorithm, wherein the storage algorithm is: Among them, X is the input binary data, W(X) is the storage main function, G h (X) is the storage parameter function, θ h To store weight coefficients.

8. A binary data compression storage system based on an attention mechanism, applicable to the binary data compression storage method based on an attention mechanism as described in any one of claims 1 to 7, characterized in that: It includes a data type identification module, a compression strategy selection module, a structure analysis module, a compression module and a storage module; The data type identification module is configured to: use the attention mechanism deep convolutional neural network formula and the data type identification algorithm to perform type identification on binary data; The compression strategy selection module is configured to: select a compression strategy according to the type identification result; The structure analysis module is configured to: analyze the data structure, and according to the analysis result of the data structure, find out the special structure according to the structure search algorithm to process it so as to improve the compression ratio; The compression module is configured to: compress data; The storage module is configured to store the compressed data.

9. An electronic device, characterized in that: It includes at least one memory and at least one processor, the memory is communicatively connected to the processor, the memory stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, the binary data compression and storage method based on the attention mechanism as described in any one of claims 1 to 7 is implemented.

Images