Data storage and uploading method and device for data acquisition terminal

Abstract

The invention belongs to the technical field of data storage and communication, and particularly relates to a data storage and uploading method and device of a data acquisition terminal. The method comprises the following steps: firstly, after data acquisition is completed by a data acquisition terminal, the acquired data is subjected to MCU master control and then uploaded to a cloud platform through a wireless communication module; secondly, checking whether the data received by the platform is consistent with the sent data or not by adopting an MD5 checking algorithm, and if not, determining that the data sending fails; then writing the data which fails to be sent into an external Flash storage chip for storage; finally, the networking state is detected regularly, when the networking state is good and stored data exists, the data stored in the Flash storage chip are sent to the platform again, and the storage area is reset. The problem of data loss caused by network problems is solved, a lightweight data storage scheme is provided for outdoor small embedded equipment, the cost is reduced, and the service life of a Flash storage chip is prolonged.

Description

Technical Field

[0001] The present invention belongs to the technical field of data storage and communication, and more specifically, relates to a data storage and uploading method and device of a data acquisition terminal. Background Art

[0002] With the development of network technology, modern society has become more and more information-based. The proposal of smart water conservancy, smart industry, smart agriculture, etc. has promoted the development of information technology. All walks of life are undergoing digital transformation and upgrading. The current main data collection method is to collect data through sensors, and then upload the data to the platform network through data collection terminals, or sensors directly collect data and upload it to the platform network; the latter has the disadvantages of high cost and few types of data collection, so data collection terminals are often required for data transfer.

[0003] Chinese patent document CN114257978A discloses a method for transmitting sensor data of outdoor power transmission equipment, the method comprising: when the sensor sends the sensor data to the transfer node, acquiring the sensor data; the sensor data is the sensor data collected by the sensor during the monitoring process of the outdoor power transmission equipment, and the sensor data represents the equipment status of the outdoor power transmission equipment; judging whether the transfer node has received the sensor data; if it is determined that the transfer node has not received the sensor data, sending the sensor data to the intelligent gateway through the candidate node, so that when increasing the transmission distance of each hop, retransmission can be avoided when the transmission distance is too far and causes transmission failure, thereby effectively improving the transmission efficiency of the sensor data. However, this method directly uses sensors to collect data for uploading, which is very costly.

[0004] In urban areas with good network conditions, data acquisition terminals usually do not fail to transmit data due to network problems. However, in actual applications, data acquisition terminals are often installed in vast fields or mountains, and network signals are difficult to guarantee. Therefore, data loss due to poor network conditions often occurs. Currently, the most commonly used method is to store data and then upload it. In embedded systems, due to the limitation of storage resources, external memory is often used for capacity expansion. In small embedded devices, EEPROM or Flash is usually used as external memory. EEPROM has the advantages of long service life and can be erased in bytes, but it is expensive and has a relatively small memory capacity. Flash is relatively cheap and has a large memory capacity, but it needs to be erased in sectors, so its service life is short. The existing data storage methods often have disadvantages such as high cost and complex system design. It is crucial to design a more portable and low-cost data storage and upload method.

[0005] In view of this, the present invention designs a data storage and uploading method of a data acquisition terminal. Summary of the invention

[0006] The present invention aims to overcome the deficiencies of the above-mentioned prior art and provide a data storage and uploading method for a data acquisition terminal, which effectively improves the data stability of a simple embedded acquisition terminal, improves efficiency, and at the same time increases the service life of a Flash memory and reduces terminal costs.

[0007] The invention also discloses a device loaded with a data acquisition terminal and a data storage and uploading method.

[0008] The detailed technical scheme of the present invention is as follows:

[0009] A data storage and uploading method for a data acquisition terminal, the method comprising the following steps:

[0010] S1. After the data acquisition terminal completes data acquisition, the collected data is first controlled by the MCU and then uploaded to the cloud platform through the wireless communication module;

[0011] S2. Use the MD5 verification algorithm to verify whether the data received by the cloud platform is consistent with the data sent. If they are inconsistent, the data transmission fails;

[0012] S3, writing the data that failed to be sent into an external Flash storage chip for storage;

[0013] The Flash storage chip determines memory allocation in the following manner: a sector is divided into N parts, where N represents the number of data stored in the sector; M sectors are selected to store data; if the Flash sector size is X, the storage interval size is X / N; since Flash can only be erased in sectors, a sector is used as a transition area, and M sectors are used to store N*(M-1) pieces of data;

[0014] S4. A heartbeat mechanism is used to periodically detect the networking status. When the networking status is connected, the signal is good, and there is stored data, the data stored in the Flash storage chip is resent to the cloud platform and the storage area is cleared.

[0015] Preferably, according to the present invention, step S2 specifically means that after the data is collected, the MCU main control performs MD5 verification, and the verification code and plaintext data are transmitted. The platform side determines whether the data is complete by performing MD5 calculation again and comparing the verification code. If the data is complete, the cloud platform returns a response code. After the data collection terminal receives the response code, it means that the data transmission is complete; if the data collection terminal does not receive the response code, or the MD5 verification code is inconsistent, the data transmission fails.

[0016] Preferably, according to the present invention, in step S3, writing the data that failed to be sent into the external Flash storage chip comprises the following steps:

[0017] S31, determine whether the number of currently stored data is greater than or equal to the set maximum value, that is, N*(M-1): when the number of stored data is less than the maximum value, it proves that the number of stored data has not reached the maximum value at this time, and there is still storage space left, then the variable of the number of stored data is incremented by one, and the judgment of the storage head address is entered; when the number of stored data is greater than or equal to the maximum value, it means that the data has been stored to the preset maximum number, and the judgment of the storage head address is directly entered; since there is a transition area in the M sectors, when the stored data reaches the preset maximum value, the storage head address and the storage tail address will automatically increase or decrease, and the two will not simultaneously satisfy the storage head address of the N-1th data that is greater than or equal to the Mth sector;

[0018] S32, judging the storage head address and the storage tail address: if both are smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage tail address and the storage head address are both incremented by the storage interval size, and then the write address judgment is entered; if the storage head address is not smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage head address is changed to the head address, the storage tail address is incremented by the storage interval size, and then the write address judgment is entered; if the storage tail address is not smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage tail address is changed to the head address, the storage head address is incremented by the storage interval size, and then the write address judgment is entered;

[0019] S33. Since the transition area is used to receive one more sector of data to reduce the number of erasures and writes of the Flash to extend its service life, it is necessary to determine the sector where the write address is located. When the number of stored data reaches the preset maximum value, if the write address is at the first address of a sector, the sector is a transition area and can be directly erased and then stored with data.

[0020] Determine whether the value of the write address modulo the sector capacity is 0: if it is 0, it indicates that the sector is a transition area and can be erased, and then the data to be written is written, and the number of stored data, storage head address and storage tail address are written at the same time; if it is not 0, it will not be erased, and the data to be written is written directly, and the number of stored data, storage head address and storage tail address are written at the same time.

[0021] Preferably, according to the present invention, in step S4, the specific steps of resending the data stored in the Flash storage chip to the cloud platform are as follows:

[0022] S41, reading the number of stored data, the storage head address and the storage tail address into the system memory;

[0023] S42, read the data according to the number of stored data, and send the read data: if the read data fails to be sent, determine whether the read data is not the first time to send failure and has been sent successfully before, if not, enter the judgment area of ​​whether all data have been read and sent once; if so, write the current data to the space where the storage tail address is located, the storage tail address is incremented by the storage interval size, and the storage tail address is written to the external storage, and then enter the judgment area of ​​whether all data have been read and sent once;

[0024] If the read data is sent successfully, it is determined whether it is the first time to send successfully. If so, the variable of the number of stored data is reduced by one, the storage tail address is changed to the current read address, and the storage tail address and the number of stored data are written to the external storage, and the current read address is judged; if it is not the first time to send successfully, the variable of the number of stored data is reduced by one, the number of stored data is written to the external storage, and the current read address is judged;

[0025] S43. Since the data is stored in a circular manner in the Flash, it is necessary to judge the current read address: if it is less than the storage first address of the N-1th data of the Mth sector, the current read address is incremented by the storage interval size, and then the judgment area of ​​whether all data have been read and sent once is entered; if it is not less than the storage first address of the N-1th data of the Mth sector, the current read address becomes the first address, and the judgment area of ​​whether all data have been read and sent once is entered;

[0026] S44. Because there may be multiple stored data, the stored data needs to be read out and sent one by one: if not all of them have been read out and sent once, continue reading and sending; if all of them have been read and sent once, determine whether all of the stored data has been sent. If all of them have been sent, write the stored data empty flag to the external storage. If not all of them have been sent, then this round of stored data reading and sending ends, and the data that has not been sent successfully will continue and be sent first next time.

[0027] In another aspect of the present invention, a data acquisition device for implementing a data storage and uploading method of a data acquisition terminal is provided, the device comprising:

[0028] External Flash storage chip, used to store data that has not been sent successfully;

[0029] MCU master control is responsible for the overall control and management of the entire system;

[0030] Wireless communication module, used to upload data to the cloud platform.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention aims at the problems of data packet loss, data transmission failure when network conditions are bad, short life of external Flash storage, etc. of data acquisition terminals, and proposes a data storage and upload method for data acquisition terminals. The method adopts MD5 verification algorithm to verify whether each data transmission is complete, thereby ensuring effective data transmission; adopts Flash external storage and sets a transition storage area, thereby ensuring effective storage of failed transmission data and increasing the life of Flash; and regularly detects network conditions, thereby ensuring that the stored data can be sent to the platform in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a working flow diagram of the data acquisition terminal of the present invention;

[0034] Figure 2 It is a structural division diagram of the Flash storage chip of the present invention;

[0035] Figure 3 is a data storage flow chart of the present invention;

[0036] Figure 4 It is a flow chart of memory data reading and sending of the present invention;

[0037] Figure 5 It is a diagram showing the implementation effect of the data acquisition terminal in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0039] Embodiment 1,

[0040] Ginseng Figure 1 , a data storage and uploading method of a data acquisition terminal, the method comprising the following steps:

[0041] S1. After the data acquisition terminal completes data acquisition, the collected data is first controlled by the MCU and then uploaded to the cloud platform through the wireless communication module;

[0042] S2. Use the MD5 verification algorithm to check whether the data received by the cloud platform is consistent with the data sent. If they are inconsistent, the data transmission fails;

[0043] S3, writing the data that failed to be sent into an external Flash storage chip for storage;

[0044] S4. A heartbeat mechanism is used to periodically detect the networking status. When the networking status is connected, the signal is good, and there is stored data, the data stored in the Flash storage chip is resent to the cloud platform and the storage area is cleared.

[0045] Step S2 specifically refers to using the MD5 check algorithm to perform data integrity detection. After the data is collected, the MCU performs an MD5 check, and the check code and plain text data are transmitted. The platform side determines whether the data is complete by performing MD5 calculation again to compare the check code. If the data is complete, the platform returns a response code. After the data acquisition terminal receives the response code, it means that the data has been completely transmitted. If the data acquisition terminal does not receive the response code, or the MD5 check code is inconsistent, it is considered that the data transmission has failed.

[0046] like Figure 2 As shown in the figure, the Flash storage chip determines the memory allocation in the following way: divide a sector into N parts, where N represents the number of data stored in the sector; select M sectors to save data. If the Flash sector size is X, the storage interval size is X / N. Since Flash can only be erased in sectors, a sector is used as a transition area, and M sectors are used to store N*(M-1) pieces of data.

[0047] like Figure 3 As shown, in step S3, writing the data that failed to be sent into the external Flash storage chip includes the following steps:

[0048] S31, first determine whether the number of currently stored data is greater than or equal to the set maximum value, that is, N*(M-1). When the number of stored data is less than N*(M-1), it proves that the number of stored data has not reached the maximum value and there is still storage space. Then, the variable of the number of stored data is incremented by one, and the judgment of the storage head address is entered; when the number of stored data satisfies the condition that it is greater than or equal to the maximum value, it means that the data has been stored to the preset maximum number, and the judgment of the storage head address is directly entered; since there is a transition area in the M sectors, when the stored data reaches the preset maximum value, the storage head address and the storage tail address will automatically increase or decrease, and the two will not simultaneously satisfy the storage first address of the N-1th data greater than or equal to the Mth sector;

[0049] S32, judging the storage head address and the storage tail address, if both are smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage tail address and the storage head address are both incremented by the storage interval size, and then the write address judgment is entered; if the storage head address is not smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage head address is changed to the head address, the storage tail address is incremented by the storage interval size, and then the write address judgment is entered; if the storage tail address is not smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage tail address is changed to the head address, the storage head address is incremented by the storage interval size, and then the write address judgment is entered;

[0050] S33. Since this solution uses the transition area to receive one more sector of data to reduce the number of times the Flash is erased and written to extend its service life, it is necessary to judge the sector where the write address is located. When the number of stored data reaches the preset maximum value, if the write address is at the first address of a sector, then the sector is a transition area and can be directly erased and then stored with data.

[0051] Therefore, it is necessary to determine whether the value of the write address modulo the sector capacity is 0. If so, it indicates that the sector is a transition area and can be erased, and then the data to be written is written, and the number of stored data, storage head address, and storage tail address are written at the same time; if not, it will not be erased, and the data to be written will be written directly, and the number of stored data, storage head address, and storage tail address will be written.

[0052] Figure 3 The number of stored data in indicates the number of data currently stored in the Flash;

[0053] The maximum value indicates the number of data that can be stored in the Flash, which is N*(M-1) here.

[0054] The present invention adopts a circular storage method, and the storage head address represents the first address of the storage space where the earliest data among all the currently stored data is located;

[0055] The storage tail address indicates the first address of the storage space where the last piece of data among all currently stored data is located;

[0056] The last data address indicates the first address of the storage space where the last data of the unsent data is stored, which is allocated in the Flash. Here, it is the first address of the storage of the N-1th data of the Mth sector.

[0057] The write address indicates the first address of the storage space where the data is to be written;

[0058] The first address indicates the first address of all spaces allocated in the Flash for storing the failed-to-send data;

[0059] Sector capacity refers to the number of bytes that a Flash sector can store, which is X here.

[0060] The storage interval size indicates the space occupied by a piece of data in the Flash, which is X / N here.

[0061] like Figure 4 As shown, in step S4, resending the data stored in the Flash storage chip to the platform requires reading and sending the stored data, and the specific steps are as follows:

[0062] S41, read the number of stored data, storage head address, and storage tail address into the system memory;

[0063] S42, reading data according to the number of stored data pieces, and sending the read data;

[0064] S43, if the data fails to be sent, it is determined whether it is not the first time that the data fails to be sent and it has been sent successfully before, if not, it enters the judgment area of ​​whether all data have been read and sent once; if so, the current data is written into the space where the storage tail address is located, the storage tail address is incremented by the storage interval size, and the storage tail address is written into the external storage, and then it enters the judgment area of ​​whether all data have been read and sent once;

[0065] S44, if the read data is sent successfully, determine whether it is the first time to send successfully, if so, reduce the variable of the number of stored data by one, change the storage tail address to the current read address, write the storage tail address and the number of stored data to the external storage, and enter the current read address judgment; if it is not the first time to send successfully, reduce the variable of the number of stored data by one, write the number of stored data to the external storage, and enter the current read address judgment;

[0066] S45. Since the data is stored in a circular manner in the Flash, it is necessary to judge the current read address. If it is less than the storage first address of the N-1th data of the Mth sector, the current read address is incremented by the storage interval size, and then the judgment area of ​​whether all data have been read and sent once is entered; if it is not less than the storage first address of the N-1th data of the Mth sector, the current read address becomes the first address, and the judgment area of ​​whether all data have been read and sent once is entered;

[0067] S46. Since there may be multiple stored data, the stored data needs to be read out and sent one by one. If not all of them have been read out and sent once, continue to read and send. If all of them have been read and sent once, determine whether all of the stored data has been sent. If all of them have been sent, write a stored data empty flag to the external storage. If not all of them have been sent, this round of stored data reading and sending ends. The data that has not been sent successfully will continue to be sent next time and will be given priority.

[0068] Embodiment 2,

[0069] This embodiment provides a data acquisition device that implements a data storage and upload method of a data acquisition terminal, the device comprising:

[0070] External Flash storage chip, used to store data that has not been sent successfully;

[0071] MCU master control is responsible for the overall control and management of the entire system;

[0072] Wireless communication module, used to upload data to the cloud platform.

[0073] The data acquisition terminal is used for data acquisition from agricultural sensors in mountainous areas. Agricultural sensors generally support RS485 protocol transmission, so this data acquisition terminal uses RS485 input, which is packaged after being processed by the MCU main control and uploaded to the monitoring cloud platform through the 4G module. However, in the mountains, the signal is often weak, especially in rainy weather, and the data from agricultural sensors often fails to be sent. Therefore, the data acquisition terminal proposed by the present invention can be used to send data in time when the signal is good, and when the signal is not good, the data from the agricultural sensor can be saved in time. When the signal is restored, the data within that time period can be uploaded, thereby preventing the agricultural sensor data from being discontinuous in time. The implementation effect is as follows: Figure 5 As shown:

[0074] Among them: 1: 4G antenna, 2: pole, 3: waterproof box, 4: data acquisition terminal, 5: power cord, 6: agricultural sensor, 7: RS485 data cable.

Claims

1. A data storage and uploading method for a data acquisition terminal, characterized in that: The method comprises the following steps: S1. After the data acquisition terminal completes data acquisition, the collected data is first controlled by the MCU and then uploaded to the cloud platform through the wireless communication module; S2. Use the MD5 verification algorithm to verify whether the data received by the cloud platform is consistent with the data sent. If they are inconsistent, the data transmission fails; S3, writing the data that failed to be sent into an external Flash storage chip for storage; The Flash storage chip determines memory allocation in the following manner: a sector is divided into N parts, where N represents the number of data stored in the sector; M sectors are selected to store data; if the Flash sector size is X, the storage interval size is X / N; a sector is used as a transition area, and M sectors are used to store N*(M-1) pieces of data; S4. A heartbeat mechanism is used to periodically detect the networking status. When the networking status is connected, the signal is good, and there is stored data, the data stored in the Flash storage chip is resent to the cloud platform and the storage area is cleared.

2. The data storage and uploading method of a data acquisition terminal according to claim 1, characterized in that: Specifically, step S2 means that after the data is collected, the MCU main control performs MD5 verification, and transmits the verification code and plain text data. The platform side determines whether the data is complete by performing MD5 calculation again and comparing the verification code. If the data is complete, the cloud platform returns a response code. After the data collection terminal receives the response code, it means that the data transmission is complete. If the data collection terminal does not receive the response code, or the MD5 verification code is inconsistent, the data transmission fails.

3. The data storage and uploading method of a data acquisition terminal according to claim 2 is characterized in that: In step S3, writing the data that failed to be sent into the external Flash storage chip includes the following steps: S31, determine whether the number of currently stored data is greater than or equal to the set maximum value, that is, N*(M-1): when the number of stored data is less than the maximum value, the variable of the number of stored data is incremented by one, and the storage head address is determined; when the number of stored data is greater than or equal to the maximum value, the storage head address is determined; when the stored data reaches the preset maximum value, the storage head address and the storage tail address are automatically increased or decreased, and the two will not simultaneously satisfy the storage head address of the N-1th data greater than or equal to the Mth sector; S32, judging the storage head address and the storage tail address: if both are smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage tail address and the storage head address are both incremented by the storage interval size, and then the write address judgment is entered; if the storage head address is not smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage head address is changed to the head address, the storage tail address is incremented by the storage interval size, and then the write address judgment is entered; if the storage tail address is not smaller than the storage head address of the N-1th data of the Mth sector, the write address is changed to the storage tail address, the storage tail address is changed to the head address, the storage head address is incremented by the storage interval size, and then the write address judgment is entered; S33. Determine whether the value obtained by taking the remainder of the write address and the sector capacity is 0: if it is 0, it indicates that the sector is a transition area and can be erased, and then the data to be written is written, and the number of stored data, the storage head address and the storage tail address are written at the same time; if it is not 0, it is not erased, and the data to be written is directly written, and the number of stored data, the storage head address and the storage tail address are written at the same time.

4. The data storage and uploading method of a data acquisition terminal according to claim 3 is characterized in that: In step S4, the specific steps of resending the data stored in the Flash storage chip to the cloud platform are as follows: S41, reading the number of stored data, the storage head address and the storage tail address into the system memory; S42, read the data according to the number of stored data, and send the read data: if the read data fails to be sent, determine whether the read data is not the first time to send failure and has been sent successfully before, if not, enter the judgment area of ​​whether all data have been read and sent once; if so, write the current data to the space where the storage tail address is located, the storage tail address is incremented by the storage interval size, and the storage tail address is written to the external storage, and then enter the judgment area of ​​whether all data have been read and sent once; If the read data is sent successfully, it is determined whether it is the first successful transmission. If so, the variable of the number of stored data is reduced by one, the storage tail address is changed to the current read address, and the storage tail address and the number of stored data are written to the external storage, and the current read address is judged; If it is not the first time to send successfully, the variable of the number of stored data items is reduced by one, the number of stored data items is written to the external storage, and the judgment of the current read address is entered; S43, judging the current read address: if it is smaller than the storage first address of the N-1th data of the Mth sector, the current read address is incremented by the storage interval size, and then the judgment area of ​​whether all data has been read and sent is entered; If it is not less than the storage first address of the N-1th data of the Mth sector, the current read address becomes the first address, and the judgment area of ​​whether all data has been read and sent once is entered; S44, read out the stored data one by one and send them: if not all of them have been read out and sent once, continue reading and sending; if all of them have been read and sent once, determine whether all of the stored data have been sent. If all of them have been sent, write the stored data empty flag to the external storage. If not all of them have been sent, then this round of stored data reading and sending ends, and the data that has not been sent successfully will continue and be sent first next time.

5. A data acquisition device for realizing a data storage and uploading method of a data acquisition terminal, characterized in that: The device comprises: External Flash storage chip, used to store data that has not been sent successfully; MCU master control is responsible for the overall control and management of the entire system; Wireless communication module, used to upload data to the cloud platform.

Images