Data processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a data processing method, device and equipment and a storage medium, which are used for improving the utilization rate of a storage space and the data reading speed. The data processing method comprises the following steps: acquiring environment parameter data, and storing the environment parameter data in a buffer area; judging whether the environmental parameter data volume in the buffer area reaches a preset collection upper limit or a compressed data threshold value or not; if yes, compressing the environment parameter data of the buffer area to obtain target compressed data; and storing the target compressed data into a main storage area of the nonvolatile memory.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, device, equipment and storage medium. Background Art

[0002] In the prior art, the method for processing environmental parameters is generally to compress data at the time of reading through a command control device, but this solution has certain problems. For example, when the communication signal is unstable, it is impossible to achieve fast communication by compressing a large amount of data, especially when different mobile phones communicate with NFC environmental parameter recording devices, it is difficult to achieve good signal matching. In addition, the scheme of compressing data when reading is difficult to determine the length of the compressed data, resulting in the last frame of the original data being compressed and read each time often having only a small amount of data, or it is difficult to fill the effective load of the communication, resulting in many idle small intervals in the communication process. In summary, the prior art will lead to low memory utilization efficiency and long data reading time. Summary of the invention

[0003] The present invention provides a data processing method, device, equipment and storage medium to solve the problems of low memory utilization efficiency and long data reading time caused by compression at the reading time in the prior art.

[0004] A first aspect of the present invention provides a data processing method, which is applied to an environmental parameter recording device, including: acquiring environmental parameter data and storing the environmental parameter data in a buffer; determining whether the amount of environmental parameter data in the buffer reaches a preset collection upper limit or compressed data threshold; if so, compressing the environmental parameter data in the buffer to obtain target compressed data; and storing the target compressed data in a main storage area of ​​a non-volatile memory.

[0005] In a feasible implementation, after storing the target compressed data in the main storage area of ​​the non-volatile memory, it also includes: when a user requests to read data, determining whether there is uncompressed environmental parameter data in the buffer; if so, compressing the uncompressed environmental parameter data in the buffer to obtain temporary compressed data; and writing the temporary compressed data into the temporary storage area in the non-volatile memory.

[0006] In a feasible implementation, after writing the temporary compressed data into the temporary storage area in the non-volatile memory, it also includes: reading the target compressed data from the main storage area; determining whether the target compressed data has been read; if so, reading the temporary compressed data from the temporary storage area.

[0007] In a feasible implementation, compressing the environmental parameter data in the buffer to obtain target compressed data includes: selecting at least one compression algorithm, which is lossless compression or lossy compression; applying the at least one compression algorithm to compress the environmental parameter data in the buffer to obtain target compressed data.

[0008] In a feasible implementation manner, the selecting at least one compression algorithm includes: analyzing data characteristics of the environmental parameter data in the buffer; and selecting at least one compression algorithm from a preset compression algorithm library according to the data characteristics.

[0009] In a feasible implementation, storing the target compressed data in the main storage area of ​​a non-volatile memory includes: determining the available storage space in the main storage area of ​​the non-volatile memory; checking whether the size of the target compressed data is suitable for storage in the main storage area; if suitable, writing the target compressed data into the main storage area; and updating the storage management information in the main storage area to record the location and related information of the target compressed data.

[0010] In a feasible implementation manner, after storing the target compressed data in the main storage area of ​​the non-volatile memory, the method further includes: clearing the original data of the target compressed data that has been compressed and stored in the buffer to release the storage space of the buffer.

[0011] The second aspect of the present invention provides an environmental parameter recording device, comprising: an acquisition module, used to acquire environmental parameter data and store the environmental parameter data in a buffer; a storage module, including a buffer and a non-volatile memory, the buffer being used to receive and store the environmental parameter data sent by the acquisition module, and the non-volatile memory being used to store compressed data; a judgment module, used to judge whether the amount of environmental parameter data in the buffer has reached a preset collection upper limit or compression data threshold; a processing module, when the judgment module determines that the amount of environmental parameter data in the buffer has reached the collection upper limit or compression data threshold, used to compress the environmental parameter data in the buffer to obtain target compressed data; a control module, used to store the target compressed data compressed by the processing module in the main storage area of ​​the non-volatile memory.

[0012] A third aspect of the present invention provides a data processing device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the data processing device executes the above-mentioned data processing method.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned data processing method.

[0014] In the technical solution provided by the present invention, environmental parameter data is obtained and stored in a buffer; it is determined whether the amount of environmental parameter data in the buffer reaches a preset collection upper limit or a compression data threshold; if so, the environmental parameter data in the buffer is compressed to obtain target compressed data; the target compressed data is stored in the main storage area of ​​a non-volatile memory. In the embodiment of the present invention, by pre-storing environmental parameter data and compressing and storing it when specific conditions are met, not only the flexibility of data processing is improved, but also the utilization rate of storage space is significantly optimized, and the idle interval during the communication process is reduced, thereby speeding up the data reading speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of an embodiment of a data processing method in an embodiment of the present invention;

[0016] Figure 2 A schematic diagram of another embodiment of the data processing method in an embodiment of the present invention;

[0017] Figure 3 A schematic diagram of an embodiment of an environmental parameter recording device in an embodiment of the present invention;

[0018] Figure 4 FIG. 1 is a schematic diagram of an embodiment of a data processing device in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The embodiments of the present invention provide a data processing method, apparatus, device and storage medium, which improve the utilization rate of storage space and data reading speed by pre-storing environmental parameter data and compressing and storing it when specific conditions are met.

[0020] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] It is understandable that the execution subject of the present invention may be an environmental parameter recording device or a server, which is not limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0022] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the data processing method in the embodiment of the present invention includes:

[0023] 101. Obtain environmental parameter data, and store the environmental parameter data in a buffer;

[0024] Environmental parameters include but are not limited to temperature, humidity and light intensity. The environmental parameter recording device is equipped with a temperature sensor, a humidity sensor and a light intensity sensor. These sensors form a complete sensor network that can capture key parameter data such as temperature, humidity and light intensity in the surrounding environment in real time and accurately. Whenever the sensor captures new environmental parameter data, the data will be immediately sent to the device's built-in buffer through the internal data transmission channel for temporary storage. The buffer is a temporary data storage area with a storage capacity sufficient to accommodate all environmental parameter data collected by the sensor network within a certain period of time, thereby effectively avoiding data loss or accuracy loss caused by data processing or storage delays.

[0025] 102. Determine whether the amount of environmental parameter data in the buffer reaches a preset collection upper limit or compression data threshold;

[0026] When new environmental parameter data is collected and stored in the buffer, the counter will be incremented accordingly. In this way, the amount of data stored in the buffer can be understood and controlled in real time. When the device is reset or restarted, a buffer scan operation is performed to verify and restore the count value of the counter in the volatile memory to ensure the continuity and accuracy of the data. At the same time, the amount of stored data can be compared with the preset collection upper limit, which is set after considering the physical storage capacity of the buffer, the data processing speed, and the expected data collection frequency. In this way, the amount of data in the buffer can be effectively monitored and managed to ensure the smooth progress of data collection and processing.

[0027] In addition, a series of algorithms are applied to dynamically evaluate the characteristics of the data to determine whether the compression data threshold has been reached. These algorithms may include calculating the redundancy of the data (i.e., the proportion of repeated data), analyzing the growth rate of the data (i.e., the amount of new data added per unit time), detecting outliers or mutation points in the data, etc. For example, if the algorithm detects that the data redundancy in the buffer is continuously higher than 30%, or the data growth rate suddenly surges and exceeds the normal growth rate, then it may be determined that the current data characteristics have met the compression data threshold requirements.

[0028] 103. If yes, compress the environment parameter data in the buffer to obtain target compressed data;

[0029] According to the data characteristics and storage requirements of the currently stored environmental parameter data, the data characteristics include but are not limited to the data type, data volume and data change mode, select a compression algorithm that best suits the current scenario from the built-in multiple lossless compression algorithms, the compression algorithms include but are not limited to the MiniLzo algorithm, LZ77 algorithm, LZW algorithm or Huffman coding, which can effectively compress the data without losing any information. After completing the selection of the compression algorithm, start reading all the environmental parameter data stored in the buffer and organize them according to the time sequence of acquisition or a specific data structure for subsequent processing and analysis.

[0030] The read environmental parameter data is analyzed line by line through the compression algorithm to find and eliminate repeated data sequences and identify and compress predictable numerical change patterns to reduce redundant information in the data. This process can significantly reduce the amount of data that needs to be stored to improve compression efficiency. After completing redundant identification and removal, the remaining data will be converted into a more compact representation form, by using shorter codes to represent symbols with higher frequency of occurrence, or using specific data structures to organize data to reduce the demand for storage space. For example, the MiniLzo algorithm analyzes the frequency of occurrence of each symbol or data block in the environmental parameter data, and dynamically adjusts its encoding length according to the frequency of occurrence of each symbol or data block. Through the encoding and reorganization process of the MiniLzo algorithm, a set of target compressed data that occupies less storage space is generated. The target compressed data not only retains all the information of the original environmental parameter data, but also has a smaller size and is more suitable for long-term storage or remote transmission. In addition, in order to ensure that no errors or data loss are introduced during the compression process, the target compressed data can also be verified and the verified target compressed data can be written into the non-volatile memory of the device.

[0031] 104. Store the target compressed data in a main storage area of ​​the non-volatile memory.

[0032] Checking the remaining space in the main storage area can be achieved by reading the status information of the memory. In order to ensure the reasonable management of data and avoid storage overflow, a clear record upper limit is usually set.

[0033] When the main storage area reaches the upper limit of the record, the non-volatile memory is controlled to stop receiving new data to avoid data loss or overwriting. This method is suitable for application scenarios that have strict requirements on data integrity and can accept the suspension of data recording when the storage space is insufficient. In addition, in other specific application scenarios, especially in environments with a high demand for continuous monitoring, a circular storage mechanism can be used. The circular storage mechanism allows the storage area to automatically overwrite the earliest recorded data after reaching the upper limit of the record, thereby ensuring that the device can continue to work uninterruptedly and continuously capture and store new environmental parameter data.

[0034] In summary, when the environmental parameter recording device reaches the storage upper limit, it can flexibly choose to stop working or start the circular storage mechanism according to actual needs to ensure the stable operation of the device and the effective recording of data.

[0035] If it is confirmed that the main storage area has sufficient storage space, the target compressed data is organized according to a preset file format, which includes but is not limited to binary files, database records or custom data structures to ensure the readability and parsability of the data. At the same time, the target compressed data can also be written to the main storage area according to a preset storage strategy, for example, stored in chronological order so that users can retrieve data by timeline; or stored by data type classification so that users can quickly find data based on data type.

[0036] Furthermore, in the process of writing data, in order to ensure the integrity and consistency of the data, a checksum calculation can be performed before writing each data block, and the checksum can be verified to match after writing to ensure that no errors occur during data transmission and storage. In addition, the storage location and related information of the data can be recorded for subsequent data retrieval, analysis and processing, and the related information includes but is not limited to the starting address, size, data type and timestamp of the data.

[0037] Through this data storage process, it is possible to ensure that the target compressed data is stored in the main storage area of ​​the non-volatile memory in an efficient, secure and orderly manner, which not only optimizes the utilization of storage space but also ensures the integrity, readability and retrievability of the data.

[0038] In an embodiment of the present invention, environmental parameter data is obtained, the environmental parameter data is stored in a buffer, and it is determined whether the amount of environmental parameter data in the buffer reaches a preset collection upper limit or a compressed data threshold. If so, the environmental parameter data in the buffer is compressed to obtain target compressed data, and the target compressed data is stored in a main storage area of ​​a non-volatile memory. By pre-storing the environmental parameter data and compressing and storing it when specific conditions are met, not only the flexibility of data processing is improved, but also the utilization rate of the storage space is significantly optimized, and the idle interval during the communication process is reduced, thereby speeding up the data reading speed.

[0039] See also Figure 2 Another embodiment of the data processing method in the embodiment of the present invention includes:

[0040] 201. Obtain environmental parameter data, and store the environmental parameter data in a buffer;

[0041] 202. Determine whether the amount of environmental parameter data in the buffer reaches a preset collection upper limit or compression data threshold;

[0042] The execution process of steps 201-202 is similar to the above steps 101-102, and will not be repeated here.

[0043] 203. If yes, select at least one compression algorithm, the compression algorithm being lossless compression or lossy compression;

[0044] Analyze the data characteristics of the environmental parameter data in the buffer; and select at least one compression algorithm from a preset compression method library according to the data characteristics.

[0045] The environmental parameter data in the buffer usually include readings of sensors such as temperature, humidity, and air pressure. They are mostly numerical data and may show certain distribution patterns and periodicity. For example, temperature changes during a day may follow a certain pattern. In addition, these environmental parameter data often contain a large number of repeated or similar values, especially when the time interval is short, because environmental changes are usually gradual. After clarifying these data characteristics, a suitable algorithm can be selected from the preset compression algorithm library. For the high accuracy and integrity of the environmental parameter data, lossless compression algorithms are preferred, for example, LZW algorithm or Huffman coding.

[0046] In one embodiment, when selecting a compression algorithm suitable for environmental parameter data, the characteristics of the data need to be considered. Environmental parameter data generally have the following characteristics: they are mostly numerical data, and have high requirements for accuracy, because even a small change in the value may have an important impact on subsequent analysis or decision-making; at the same time, these data often contain a large number of repeated or similar values, especially at continuous time points, because environmental changes are usually gradual rather than sudden. Based on these characteristics, two different types of algorithms can be selected from a preset compression algorithm library at the same time: a lossless compression algorithm and a lossy compression algorithm to meet the needs of different scenarios. For example, the lossless compression algorithm can select the LZW algorithm or Huffman coding, which can identify and compress repeated parts in the data while maintaining the original accuracy and integrity of the data. However, in some cases, completely lossless data may not be required, but a certain degree of error can be tolerated in exchange for a higher compression ratio. Therefore, a lossy compression algorithm can be used for compression. The lossy compression algorithm can be selected from algorithms such as JPEG-LS or PNG. These algorithms introduce tiny errors in exchange for higher compression rates and are suitable for application scenarios with slightly lower accuracy requirements and the pursuit of higher compression ratios. In addition, when selecting a lossy compression algorithm, the impact of errors on data quality and analysis results can be evaluated in advance to ensure that the introduced errors are within an acceptable range.

[0047] By selecting a lossless compression algorithm based on the characteristics of environmental parameter data to maintain data accuracy and integrity, and selecting a lossy compression algorithm in exchange for a higher compression ratio, this strategy not only ensures data quality but also improves storage and transmission efficiency, providing a flexible and efficient data compression solution for different application scenarios.

[0048] 204. Apply at least one compression algorithm to compress the environmental parameter data in the buffer to obtain target compressed data;

[0049] The LZW algorithm achieves efficient compression by building a dictionary to store character sequences and their corresponding unique codes. Specifically, by initializing an empty dictionary and an output sequence, the environment parameter data in the buffer is regarded as a character sequence input, and then the characters are read one by one. If the character is not in the dictionary, it is added to the dictionary and attached to the output sequence, and a new unique code is generated at the same time; if the character already exists in the dictionary, its corresponding code is found and attached to the output sequence; for each character in the character sequence, this process is repeated until all characters are processed; finally, the output sequence is the target data compressed by the LZW algorithm, and the data can completely restore the original environment parameter information when decompressed, ensuring the losslessness of the data.

[0050] In addition, Huffman coding assigns codes of different lengths according to the frequency of occurrence of characters. Characters with high frequency use shorter codes, and characters with low frequency use longer codes. This method can also effectively reduce the data size.

[0051] When a strategy combining lossless and lossy compression algorithms is adopted, the entire data set is initially compressed using a lossless compression algorithm. This step aims to preserve the original accuracy and integrity of the data and ensure that the original information can be fully restored after decompression. For those data with slightly lower accuracy requirements and higher compression rates, a lossy compression algorithm is further applied to obtain a higher data compression ratio at the cost of introducing acceptable small errors. In this way, the lossless preservation of key data is guaranteed, and the size of the overall data is effectively reduced, thereby improving the utilization of storage space and the efficiency of data transmission.

[0052] 205. Store the target compressed data in the main storage area of ​​the non-volatile memory;

[0053] Determine the available storage space of the main storage area of ​​the non-volatile memory; check whether the size of the target compressed data is suitable for storage in the main storage area; if suitable, write the target compressed data into the main storage area; update the storage management information in the main storage area to record the location and related information of the target compressed data.

[0054] A storage management system for accessing a non-volatile memory reads the current free space size of the main storage area from the storage management system to obtain the available storage space, compares the size of the target compressed data with the size of the available storage space in the main storage area to ensure that it does not exceed the available space of the main storage area to avoid data overflow or storage failure; if the size of the target compressed data is larger than the size of the available storage space in the main storage area, it is determined that the target compressed data is not suitable for storage in the current main storage area, and it may be considered to clean up unnecessary data in the main storage area, use a larger storage device, or take other measures to ensure that the storage requirements of the data are met; if the size of the target compressed data is not larger than the size of the available storage space in the main storage area, it is determined that the target compressed data is not suitable for storage in the current main storage area. The data is suitable for storage in the available storage space in the main storage area; if it is determined that the target compressed data is suitable for storage in the available storage space in the main storage area, one or more continuous free address blocks are allocated to the main storage area, and the total size of these address blocks should at least match the size of the target compressed data. These address blocks are marked as "in use" to prevent other data from being written to these areas at the same time, and the target compressed data is written to these allocated address blocks according to a specific data format and encoding method; write verification is performed on the target compressed data written to the main storage area to ensure that the data is stored correctly in the specified location. Furthermore, the storage management information of the main storage area can also be updated to record the location and related information where the target compressed data is written.

[0055] After the target compressed data is stored in the main storage area of ​​the non-volatile memory, the original data of the target compressed data that has been compressed and stored in the buffer is cleared to release the storage space of the buffer.

[0056] The compressed and stored original data can be safely removed from the buffer by deletion or overwriting, ensuring the effective recovery and reuse of the buffer space, avoiding redundant data occupying resources, and ensuring that the system can continue to efficiently provide sufficient temporary storage space for subsequent data processing tasks.

[0057] 206. When the user requests to read data, if there is uncompressed environmental parameter data in the buffer, compress it to obtain temporary compressed data, and write it into a temporary storage area in the non-volatile memory;

[0058] When the user requests to read data, it is determined whether there is uncompressed environmental parameter data in the buffer; if so, the uncompressed environmental parameter data in the buffer is compressed to obtain temporary compressed data; and the temporary compressed data is written into a temporary storage area in the non-volatile memory.

[0059] When a user requests to read data, the storage area of ​​the buffer is accessed, and all data items stored in the buffer are checked one by one or the data structure is used to quickly query, for example, a hash table or index tree is used to query; during the checking process, the metadata or status identifier of each data item is identified, and these metadata or identifiers usually contain key information such as data type information and whether it has been compressed; these metadata or identifiers are compared with the characteristics of the preset uncompressed environmental parameter data to identify whether there is uncompressed environmental parameter data in the buffer; if there is uncompressed environmental parameter data, the preset compression algorithm is called, and the uncompressed data in the buffer is processed block by block or item by item, and the data is compressed into a smaller volume by removing redundant information in the data, using the statistical characteristics of the data or the encoding method for efficient encoding conversion, etc.; finally, the data processed by the compression algorithm will be encapsulated by the system into a temporary compressed data packet, which contains the compressed data and necessary metadata, including but not limited to the data name, compression rate and timestamp; the temporary compressed data is written to the temporary storage area in the non-volatile memory.

[0060] 207. Read the target compressed data and the temporary compressed data.

[0061] Read the target compressed data from the main storage area; determine whether the target compressed data has been read; if so, read the temporary compressed data from the temporary storage area.

[0062] A read request is sent to the main storage area, the target compressed data returned from the main storage area is received and processed, and the integrity of the received target compressed data is checked. If the target compressed data is complete, the reading is confirmed to be complete. If the target compressed data is incomplete, the missing part is continued to be read from the main storage area until the reading is confirmed to be complete, wherein the integrity of the received target compressed data can be checked by verifying the checksum or hash value of the target compressed data; if it is confirmed that the reading of the target compressed data is complete, a read request is sent to the temporary storage area, the temporary compressed data returned from the temporary storage area is received and processed, and after reading the temporary compressed data, the temporary compressed data and the target compressed data are merged to form a complete data set.

[0063] In the embodiment of the present invention, the data storage efficiency is effectively improved through the collection, compression and storage process of environmental parameter data, ensuring that when the data volume reaches a preset upper limit or needs to be compressed, a lossless or lossy compression algorithm can be flexibly selected for processing, and the data is preferentially stored in the main storage area of ​​the non-volatile memory. At the same time, when the user requests to read the data, the uncompressed data is instantly compressed and stored in a temporary area, thereby realizing fast data reading and efficient management, saving storage space and ensuring data integrity and reading speed.

[0064] The above describes the data processing method in the embodiment of the present invention. The following describes the environmental parameter recording device in the embodiment of the present invention. Figure 3 , an embodiment of the environmental parameter recording device in the embodiment of the present invention includes:

[0065] The acquisition module 301 is used to acquire environmental parameter data and store the environmental parameter data in a buffer;

[0066] The storage module 302 includes a buffer and a non-volatile memory, wherein the buffer is used to receive and store the environmental parameter data sent by the acquisition module, and the non-volatile memory is used to store the compressed data;

[0067] The judgment module 303 is used to judge whether the amount of environmental parameter data in the buffer reaches a preset collection upper limit or compression data threshold;

[0068] The processing module 304 is used to compress the environmental parameter data in the buffer to obtain target compressed data when the judgment module determines that the amount of environmental parameter data in the buffer reaches the collection upper limit or the compression data threshold;

[0069] The control module 305 is used to store the target compressed data compressed by the processing module into the main storage area of ​​the non-volatile memory.

[0070] The acquisition module is responsible for capturing various environmental parameter data directly from the environment. These environmental parameter data include but are not limited to temperature, humidity, light intensity, air pressure and wind speed, depending on the design and application requirements of the device. The acquisition module is usually equipped with corresponding sensors that can sense environmental changes in real time and convert the sensed information into electrical signals or other recognizable data formats; after acquiring the environmental parameter data, the acquisition module sends these environmental parameter data to the buffer of the storage module. The buffer is a temporary storage area that receives data from the acquisition module and temporarily saves the data while waiting for further processing. The design of the buffer usually takes into account the real-time and integrity of the data to ensure that it is not lost or damaged before the data is compressed and stored; the judgment module regularly or continuously monitors the amount of data in the buffer to ensure that the storage of data does not exceed the preset collection upper limit or compression data threshold. The judgment module usually judges the data based on the preset The rules or algorithms are used to determine whether the amount of data in the buffer reaches the collection upper limit or the compression data threshold; when the judgment module detects that the amount of data in the buffer reaches the collection upper limit or the compression data threshold, the next compression processing flow will be triggered, and the processing module reads the environmental parameter data in the buffer and applies one or more compression algorithms to compress these data. The selection of compression algorithms depends on many factors, including the type and importance of the data, the limitation of storage space, and the needs of subsequent applications. The lossless compression algorithm can retain all the information of the data, but may not achieve the highest compression ratio; while the lossy compression algorithm may achieve a higher compression ratio by discarding some unimportant information, but may sacrifice a certain amount of data accuracy.

[0071] The processing module compresses the environmental parameter data according to the selected compression algorithm and generates target compressed data. The storage space occupied by these compressed data is greatly reduced, thereby improving storage efficiency. At the same time, the processing module can also encrypt or verify the target compressed data to ensure the security and integrity of the data; the control module is responsible for storing the target compressed data generated by the processing module in the main storage area of ​​the non-volatile memory. The non-volatile memory has the characteristic that the data will not be lost after power failure, so it is very suitable for long-term storage of environmental parameter data. When storing data, the control module is also responsible for managing and maintaining the usage status of the memory. It can ensure the correct storage and efficient reading of data by utilizing some intelligent algorithms or strategies. For example, the control module can optimize the storage layout according to factors such as the access frequency, importance or timestamp of the data to improve the speed and efficiency of data reading.

[0072] In the embodiment of the present invention, the environmental parameter recording device realizes the automatic collection, intelligent compression and efficient storage of environmental parameter data by integrating multiple modules such as acquisition, storage, judgment, processing and control, which significantly improves the efficiency and reliability of data storage, while reducing the storage space occupied and improving the efficiency of data reading.

[0073] above Figure 3 The environmental parameter recording device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the data processing device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0074] See also Figure 4 As shown, the data processing device includes a processor 400 and a memory 401 . The memory 401 stores machine executable instructions that can be executed by the processor 400 . The processor 400 executes the machine executable instructions to implement the above data processing method.

[0075] Further, Figure 4 The data processing device shown further includes a bus 402 and a communication interface 403 , and the processor 400 , the communication interface 403 and the memory 401 are connected via the bus 402 .

[0076] The memory 401 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), for example, at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 403 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 402 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0077] The processor 400 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 400. The above processor 400 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 401 , and the processor 400 reads the information in the memory 401 and completes the method steps of the above-mentioned embodiment in combination with its hardware.

[0078] The present invention also provides a data processing device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the data processing method in the above embodiments.

[0079] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computer, the computer executes the steps of the data processing method.

[0080] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0081] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0082] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method, applied to an environmental parameter recording device, characterized in that: The data processing method comprises: Acquire environmental parameter data, and store the environmental parameter data in a buffer; Determining whether the amount of environmental parameter data in the buffer reaches a preset collection upper limit or compression data threshold; If yes, compress the environmental parameter data in the buffer to obtain target compressed data; The target compressed data is stored in a main storage area of ​​a non-volatile memory.

2. The data processing method according to claim 1, characterized in that: After storing the target compressed data in the main storage area of ​​the non-volatile memory, the method further includes: When a user requests to read data, determining whether there is uncompressed environmental parameter data in the buffer; If it exists, compress the uncompressed environmental parameter data in the buffer to obtain temporary compressed data; The temporary compressed data is written to a temporary storage area in the non-volatile memory.

3. The data processing method according to claim 2, characterized in that: After writing the temporary compressed data into the temporary storage area in the non-volatile memory, the method further includes: Reading the target compressed data from the main storage area; Determining whether the target compressed data has been read; If so, the temporary compressed data is read from the temporary storage area.

4. The data processing method according to claim 1, characterized in that: The step of compressing the environmental parameter data in the buffer to obtain target compressed data includes: Selecting at least one compression algorithm, wherein the compression algorithm is lossless compression or lossy compression; The at least one compression algorithm is applied to compress the environmental parameter data in the buffer to obtain target compressed data.

5. The data processing method according to claim 4, characterized in that: The selecting at least one compression algorithm comprises: Analyzing data characteristics of environmental parameter data in the buffer; At least one compression algorithm is selected from a preset compression algorithm library according to the data characteristics.

6. The data processing method according to claim 1, characterized in that: The storing the target compressed data into a main storage area of ​​a non-volatile memory comprises: determining available storage space in a main storage area of ​​a non-volatile memory; Checking whether the size of the target compressed data is suitable for being stored in the primary storage area; If appropriate, writing the target compressed data into the primary storage area; The storage management information in the main storage area is updated to record the location and related information of the target compressed data.

7. The data processing method according to claim 1, characterized in that: After storing the target compressed data in the main storage area of ​​the non-volatile memory, the method further includes: The original data of the target compressed data that has been compressed and stored in the buffer is cleared to release the storage space of the buffer.

8. An environmental parameter recording device, characterized in that: The environmental parameter recording device comprises: An acquisition module, used for acquiring environmental parameter data and storing the environmental parameter data in a buffer; A storage module, comprising a buffer and a non-volatile memory, wherein the buffer is used to receive and store the environmental parameter data sent by the acquisition module, and the non-volatile memory is used to store compressed data; A judgment module, used to judge whether the amount of environmental parameter data in the buffer reaches a preset collection upper limit or compression data threshold; A processing module, when the judgment module determines that the amount of environmental parameter data in the buffer reaches the collection upper limit or the compression data threshold, is used to compress the environmental parameter data in the buffer to obtain target compressed data; The control module is used to store the target compressed data compressed by the processing module into the main storage area of ​​the non-volatile memory.

9. A data processing device, characterized in that: The data processing device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the data processing device to execute the data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the data processing method according to any one of claims 1 to 7 is implemented.

Images