Method, apparatus for data saving and reading

Abstract

Embodiments of the present application provide a method and device for saving and reading data, comprising obtaining correspondence information between a value and a key, the value representing a name of a first data block storing variable data, and the key representing a name of a variable; writing the correspondence information between the value and the key into a target file; storing data of the variable into a second data block of a data cache area; when detecting that a data size of a target variable in the data cache area satisfies a condition, writing the data of the variable into the target file, wherein the condition comprises a ratio between a size of the data cache area and a number of variables in the correspondence information between the key and the value; based on a name of a third data block and correspondence information between a name of a variable stored in the third data block, adding the correspondence information between the value and the key written into the target file; wherein the third data block is a data block other than the first data block in the second data block. According to the embodiments of the present application, the efficiency of saving and reading data is improved by data block storage.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of hardware-in-the-loop simulation, and particularly relates to a data saving and reading method and device. BACKGROUND

[0002] At present, real-time data of a model in hardware-in-the-loop simulation is usually saved as a MAT format type file, and quick reading of a MAT format file containing a large amount of data is supported.

[0003] In the prior art, a plug-in for saving a MAT format file is usually used to process data, and then the data is saved as a MAT format type file. However, the size of each MAT file is preset when saving, and data cannot be appended to the MAT file. Therefore, when saving a large amount of data, the number of generated MAT files is too large, resulting in low efficiency of saving and reading data. SUMMARY

[0004] The embodiments of the present application provide a data saving and reading method and device, which can solve the problem of low efficiency of saving and reading data caused by too many generated MAT files when saving a large amount of data in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a data saving method, which comprises:

[0006] Obtaining corresponding relationship information of values and keys, wherein the value represents the name of a first data block storing variable data, and the key represents the name of a variable;

[0007] Writing the corresponding relationship information of the values and the keys into a target file;

[0008] Storing data of the variable into a second data block of a data cache area, wherein the second data block comprises the first data block;

[0009] When it is detected that the data size of a target variable in the variable of the data cache area meets a condition, writing the data of the variable into the target file, wherein the condition comprises the ratio of the size of the data cache area to the number of variables in the corresponding relationship information of the keys and the values;

[0010] Based on the name of a third data block and the corresponding relationship information of the names of the variables stored in the third data block, adding the corresponding relationship information of the values and the keys written into the target file, wherein the third data block is a data block other than the first data block in the second data block.

[0011] In some embodiments, obtaining the corresponding relationship information of the values and the keys comprises:

[0012] Obtaining a first dictionary, wherein the first dictionary comprises the corresponding relationship information of the values and the keys.

[0013] In some embodiments, the method for saving data further comprises:

[0014] writing, into the first dictionary, the name of the third data block and the corresponding relationship information of the name of the variable storing the data in the third data block, the corresponding value and the corresponding key.

[0015] In some embodiments, based on the name of the third data block and the corresponding relationship information of the name of the variable storing the data in the third data block, the corresponding relationship information of the value and the key written into the target file is added, comprising:

[0016] obtaining the starting position of the first data block in the binary stream;

[0017] based on the starting position of the first data block in the binary stream, the name of the third data block in the first dictionary and the corresponding relationship information of the name of the variable storing the data in the third data block, the corresponding relationship information of the value and the key written into the target file is added.

[0018] In some embodiments, the method for saving data further comprises:

[0019] recording, in the second dictionary, the starting position of the second data block in the binary stream;

[0020] obtaining the starting position of the second data block in the binary stream;

[0021] writing, in the target file, a position matrix, the position matrix comprising the starting position of the second data block in the binary stream.

[0022] In some embodiments, after the corresponding relationship information of the value and the key written into the target file is added based on the name of the third data block and the corresponding relationship information of the name of the variable storing the data in the third data block, the method further comprises:

[0023] emptying the data in the data buffer.

[0024] In a second aspect, an embodiment of the present application provides a data reading method, the method comprising:

[0025] reading, from the target file, the corresponding relationship information of the value and the key, the value representing the name of the second data block storing the variable data and the key representing the name of the variable;

[0026] obtaining, from the position matrix in the target file, the starting position of each second data block in the binary stream according to the name of the second data block;

[0027] based on the starting position of the second data block in the binary stream, reading the data of the variable stored in the second data block.

[0028] In some embodiments, the starting position of each second data block in the binary stream is obtained from the position matrix in the target file according to the name of the second data block, comprising:

[0029] A third dictionary is constructed, and the third dictionary includes the corresponding relationship information of the values and the keys;

[0030] The starting position of each second data block in the binary stream is obtained from the position matrix in the target file according to the name of the second data block in the third dictionary.

[0031] In a third aspect, an embodiment of the present application provides a data saving device, which comprises:

[0032] The obtaining module is configured to obtain the corresponding relationship information of the values and the keys, wherein the values represent the names of the first data blocks storing variable data, and the keys represent the names of the variables.

[0033] The writing module is configured to write the corresponding relationship information of the values and the keys into the target file.

[0034] The storage module is configured to store the data of the variables into the second data blocks of the data cache area, and the second data blocks include the first data blocks.

[0035] The writing module is further configured to write the data of the variables into the target file when it is detected that the data size of the target variable in the data cache area meets a condition, wherein the condition includes the ratio of the size of the data cache area to the number of the variables in the corresponding relationship information of the keys and the values.

[0036] The writing module is further configured to add the corresponding relationship information of the values and the keys written into the target file based on the name of the third data block and the corresponding relationship information of the names of the variables stored in the third data block, wherein the third data block is a data block other than the first data block in the second data blocks.

[0037] In some embodiments, the obtaining module is further configured to obtain a first dictionary, and the first dictionary includes the corresponding relationship information of the values and the keys.

[0038] In some embodiments, the writing module is further configured to write the corresponding relationship information of the values and the keys corresponding to the name of the third data block and the name of the variable storing the data in the third data block into the first dictionary.

[0039] In some embodiments, the obtaining module is further configured to obtain the starting position of the first data block in the binary stream.

[0040] The writing module is further configured to add the value-key corresponding relationship information written into the target file based on the starting position of the first data block in the binary stream, the name of the third data block in the first dictionary, and the name of the variable, the corresponding value and the key corresponding relationship information of the data stored in the third data block.

[0041] In some embodiments, the data saving apparatus further comprises a recording module;

[0042] The recording module is configured to record the starting position of the second data block in the binary stream in the second dictionary.

[0043] The obtaining module is further configured to obtain the starting position of the second data block in the binary stream.

[0044] The writing module is further configured to write the position matrix in the target file, the position matrix comprising the starting position of the second data block in the binary stream.

[0045] In some embodiments, the data saving apparatus further comprises an emptying module;

[0046] The emptying module is configured to empty the data in the data buffer after adding the value-key corresponding relationship information written into the target file based on the name of the third data block and the name of the variable corresponding relationship information stored in the third data block.

[0047] In a fourth aspect, an embodiment of the present application provides a data reading apparatus, which comprises:

[0048] The reading module is configured to read the value-key corresponding relationship information from the target file, the value representing the name of the second data block storing variable data, and the key representing the name of the variable.

[0049] The obtaining module is configured to obtain the starting position of each second data block in the binary stream from the position matrix in the target file according to the name of the second data block.

[0050] The reading module is further configured to read the data of the variable stored in the second data block based on the starting position of the second data block in the binary stream.

[0051] In some embodiments, the data reading apparatus further comprises a constructing module;

[0052] The constructing module is configured to construct a third dictionary, the third dictionary comprising the value-key corresponding relationship information.

[0053] The obtaining module is further configured to obtain the starting position of each second data block in the binary stream from the position matrix in the target file according to the name of the second data block in the third dictionary.

[0054] In a fifth aspect, an electronic device is provided, and the electronic device includes a processor and a memory storing computer program instructions.

[0055] The processor, when executing the computer program instructions, implements the data saving method according to any one of the embodiments of the first aspect, and the data reading method according to any one of the embodiments of the second aspect.

[0056] In a sixth aspect, a computer storage medium is provided, and the computer storage medium stores computer program instructions, and the computer program instructions, when executed by a processor, implement the data saving method according to any one of the embodiments of the first aspect, and the data reading method according to any one of the embodiments of the second aspect.

[0057] In a seventh aspect, a computer program product is provided, and instructions in the computer program product, when executed by a processor of an electronic device, cause the electronic device to perform the data saving method according to any one of the embodiments of the first aspect, and the data reading method according to any one of the embodiments of the second aspect.

[0058] The data saving and reading method and device provided in the embodiments of the present application can obtain the correspondence between the value and the key, that is, the correspondence between the name of the first data block in which the variable data is stored and the variable name, write the correspondence between the value and the key in the target file, and thus can preset the data block corresponding to each variable, which plays a placeholder role and facilitates subsequent writing of complete variable data to the target file. Then, the data of the variable is stored in the second data block of the data cache area. When it is detected that the data size of the target variable in the variable of the data cache area meets a condition, the data of the variable is written to the target file, where the condition includes the ratio of the size of the data cache area to the number of variables in the correspondence between the key and the value. In this way, the data can be stored in blocks. Based on the name of the third data block other than the first data block in the second data block and the correspondence between the name of the variable stored in the third data block, the correspondence between the value and the key written to the target file is added. In this way, the target file can be saved by appending in the data block storage manner, and the efficiency of saving and reading data is improved. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required in the embodiments of the present application will be briefly introduced. Those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0060] Figure 1 is a flowchart of the data saving method provided by an embodiment of the present application.

[0061] Figure 2 is a structural diagram of a data buffer provided by an embodiment of the present application;

[0062] Figure 3 is a flow diagram of a data reading method provided by an embodiment of the present application;

[0063] Figure 4 is a structural diagram of a data saving device provided by an embodiment of the present application;

[0064] Figure 5 is a structural diagram of a data reading device provided by an embodiment of the present application;

[0065] Figure 6 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0066] The features and exemplary embodiments of various aspects of the present application will be described below in detail, in order to make the purposes, technical solutions and advantages of the present application more clear and apparent, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, but not to limit the present application. The present application can be implemented without some of these specific details by those skilled in the art. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0067] It should be noted that in this document, relational terms such as first and second and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprises... " does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0068] As described in the background, the prior art software saves data as a file of MAT format type, mainly by using a plug-in developed by itself to save the MAT format file to save the data after processing, and the size of each MAT file is customized by the developer. Because the software needs to support the saving and reading of large data, but the MAT file generated under this method cannot be appended, so it will cause the number of MAT files generated when saving large data to be too large, and it will be very troublesome to read data and process data again. If the size of the custom MAT file is too large, it will cause the problem of computer memory overflow during saving.

[0069] To solve the above problems, the embodiments of the present application provide a data saving and reading method and device. The data saving method can obtain the correspondence information of values and keys, i.e. the correspondence between the name of the first data block storing variable data and the variable name, write the correspondence information of values and keys in the target file, so that the data block corresponding to each variable can be preset to play a placeholder role, facilitating subsequent writing of complete variable data to the target file. Then, the data of the variable is stored in the second data block of the data cache area. When the data size of the target variable in the data cache area meets the condition, the data of the variable is written to the target file, wherein the condition includes the ratio of the size of the data cache area to the number of variables in the correspondence information of keys and values. In this way, the data can be stored in blocks. Based on the name of the third data block other than the first data block in the second data block and the correspondence information of the variable names stored in the third data block, the correspondence information of values and keys written to the target file is added. In this way, the target file can be saved and appended by means of data block storage, improving the efficiency of saving and reading data. The data saving method provided by the embodiments of the present application will be introduced first.

[0070] Figure 1 A flowchart of the data saving method provided by an embodiment of the present application is shown.

[0071] As shown in Figure 1 , the data saving method can specifically include the following steps:

[0072] S110, obtaining the correspondence information of values and keys, the value representing the name of the first data block storing variable data, and the key representing the name of the variable.

[0073] The correspondence information of values and keys here can be the correspondence between the name of the first data block storing variable data and the variable name, i.e. the name of the first data block storing each variable, wherein the first data block can be a data block storing variable data.

[0074] Exemplarily, the correspondence information of the value and the key can be stored in a digital dictionary for storing variable names and variable information, a key Key of the digital dictionary being a complete name of a variable, and a value Value of the digital dictionary being a set of names of the first data block for storing variable data. For example, when data of multiple variables is stored, a data block of a first variable is named as V_1_1, a data block of a second variable is named as V_2_1, and so on, a first value in the name of the data block indicating that the data block belongs to a variable, a second value indicating that the data block belongs to which variable, and a third value indicating that the data block belongs to which data block of the variable.

[0075] S120, writing the correspondence information of the value and the key into the target file.

[0076] The target file can be a file capable of storing data, for example, can be a file of a MAT format type. The correspondence between the name of the first data block for storing variable data and the variable name is written into the target file.

[0077] As an example, when real-time data in hardware-in-the-loop (HIL) simulation is stored, after the correspondence between the data block for storing variable data and the variable is obtained, a matrix named variable information VARIABLE_INFO is written into a MAT file, the matrix including a name of each data block storing different variables, that is, the correspondence between the data block and the variable, the written VARIABLE_INFO matrix actually playing a placeholder role, since data has not yet been stored in the MAT file, and the data stored in the matrix is incomplete.

[0078] In some embodiments, before the correspondence information of the value and the key is written into the target file, a header file of the target file is written. For example, before data is stored, a 128-byte header file is written into a MAT file, and initialization of the file is completed.

[0079] S130, storing data of the variable into a second data block of the data cache area, the second data block can include the first data block.

[0080] The data cache area can be used for temporarily storing data of the variable, and the data cache area can be, for example, a data buffer Buffer object. The second data block can be a data block of the data cache area for storing data of the variable, and the second data block can include the first data block.

[0081] As an example, when variable data is stored in blocks, the obtained data of the variable is filled into the data buffer Buffer, including a time (Time) of storing the variable and a name (Value) of a data block for storing the data of the variable.

[0082] In some embodiments, the storing of the data of the variables to the second data block of the data buffer can be simultaneous storing of the data of each variable or storing of the data of a single variable at a time.

[0083] S140, when detecting that the data size of the target variable in the variables of the data buffer meets a condition, writing the data of the variables to a target file, wherein the condition can include a ratio of the size of the data buffer to the number of variables in the key-value corresponding information.

[0084] The target variable can be any variable stored in the variables of the data buffer. When storing the data of the variables to the second data block of the data buffer, the data size of each variable stored in the data buffer is detected, and if the data size of any variable (target variable) reaches the ratio of the size of the data buffer to the number of variables, the data of the variables in the data buffer is written to the target file, wherein the ratio of the size of the data buffer to the number of variables represents the space size of each variable occupying the data buffer.

[0085] As an example, when saving multiple variables as a MAT file, since the data size of each variable in the multiple variable storage cannot be predicted, the space of the binary stream file cannot be accurately allocated, and the data of each variable cannot be continuous, so the data needs to be stored in blocks. When storing the data in blocks, the size of the data buffer Buffer is M, and the number of variables stored in the MAT file is C by prewriting the VARIABLE_INFO matrix in the MAT file. When filling the data buffer Buffer, the data block of each variable is pre-allocated to occupy M / C space in Buffer, and after storing each variable, it is detected whether the data filled in Buffer by the variable reaches the size of M / C space, if not, the next variable is saved. When the data of a variable reaches the size of M / C, the variable data in Buffer is written to the MAT file once.

[0086] As another example, as Figure 2The structure of the data buffer area is shown, the number of variables is C, the first data buffer (Buffer) stores data of data blocks V_1_1 to V_C_1 (V_1_1-V_C_1), the second Buffer stores data of V_1_2-V_C_2, and so on, and the Nth Buffer stores data of V_1_N-V_C_N, wherein the first value in the name of the data block indicates that the data block belongs to a variable, the second value indicates which variable the data block belongs to, and the third value indicates which data block of the variable the data block belongs to. For example, the data block V_1_1 indicates that the data block belongs to the first data block of variable 1, that is, the data block stores the first data block of variable 1, so that the MAT file can be written in an additional manner in the form of data block.

[0087] In S150, based on the name of the third data block and the corresponding relationship information of the name of the variable stored in the third data block, the corresponding relationship information of the value and the key written to the target file is added; wherein the third data block is a data block other than the first data block in the second data block.

[0088] After the data of the variable is written to the target file in S140, the name of the second data block in which each variable is specifically stored can be obtained, and the third data block other than the first data block in the second data block is a newly added data block. The corresponding relationship information of the name of the third data block and the name of the variable stored in the third data block can represent a set of the name of the variable stored in each data block in the third data block, and based on the corresponding relationship information, the corresponding relationship information of the value and the key written to the target file is added.

[0089] As an example, after the variable data in the data buffer Buffer is written into the MAT file, the name of the data block A in which each variable data is stored can be obtained, the name of the data block B corresponding to each variable in the VARIABLE_INFO matrix written in advance in the MAT file is obtained, and the data block name of the data block B in the data block A is written into the MAT file. In addition, the name of the last data block in the data block A list can also be written into the MAT file.

[0090] In addition, the corresponding relationship information of the value and the key written to the target file can be added after all the variable data in the data buffer area is written into the target file, or after the variable data in the data buffer area is written into the target file once.

[0091] In the embodiments of the present application, the correspondence between the value and the key is obtained, that is, the correspondence between the name of the first data block in which the variable data is stored and the variable name, and the correspondence between the value and the key is written in the target file. In this way, the data block corresponding to each variable can be preset to play a placeholder role, facilitating subsequent writing of complete variable data into the target file. Then, the data of the variable is stored in the second data block of the data cache area. When it is detected that the data size of the target variable in the data cache area meets the condition, the data of the variable is written into the target file, wherein the condition includes the ratio of the size of the data cache area to the number of variables in the correspondence between the key and the value. In this way, the data can be stored in blocks. Based on the name of the third data block in the second data block except the first data block and the correspondence between the name of the variable stored in the third data block and the value and the key, the correspondence between the value and the key written into the target file is added. In this way, the target file can be saved by appending in the data block storage mode, and the efficiency of saving and reading data is improved.

[0092] In some embodiments, S110 can specifically include:

[0093] The first dictionary can include the correspondence between the value and the key.

[0094] The first dictionary can be a digital dictionary pre-established for storing variable names and variable information, and the first dictionary can include the correspondence between the value and the key, that is, the correspondence between the name of the data block in which the variable data is stored and the variable name.

[0095] As an example, to complete the block storage of data in HIL simulation, a data dictionary D1 for storing variable names and variable information is established, the key Key of the dictionary is the complete name of the variable, and the value Value of the dictionary is a set of variable data block names. Then, the correspondence between the value and the key in the data dictionary D1 is obtained for subsequent writing into the MAT file.

[0096] In the embodiments of the present application, the first dictionary is obtained, the first dictionary can include the correspondence between the value and the key, and the correspondence between the value and the key can be defined in advance before saving the data, facilitating subsequent block storage of the data.

[0097] In some embodiments, the data saving method can further include:

[0098] The name of the third data block and the name of the variable stored in the third data block are written into the first dictionary, and the corresponding value and key correspondence information is written into the first dictionary.

[0099] As an example, after the data dictionary D1 for storing variable names and variable information is established and the variable data in the data buffer Buffer is written into the MAT file, the name of the data block A storing each variable data can be obtained. The name of the data block B corresponding to each variable in the VARIABLE_INFO matrix written in advance in the MAT file is obtained, and the data block name of the data block B is additionally written into the data dictionary D1. In addition, the name of the last data block in the data block list A can also be additionally written into the data dictionary D1.

[0100] In the embodiments of the present application, by writing the corresponding relationship information of the name of the third data block and the name of the variable of the data stored in the third data block, the corresponding value and key into the first dictionary, the corresponding relationship information can be additionally written into the first dictionary first, so as to improve the efficiency of data writing when a large amount of data is written into the target file.

[0101] In some embodiments, based on the corresponding relationship information of the name of the third data block and the name of the variable of the data stored in the third data block, the corresponding relationship information of the value and the key additionally written into the target file can specifically include:

[0102] The starting position of the first data block in the binary stream is obtained;

[0103] Based on the starting position of the first data block in the binary stream, the name of the third data block in the first dictionary, and the name of the variable of the data stored in the third data block, the corresponding relationship information of the value and the key additionally written into the target file is written.

[0104] The starting position of the first data block in the binary stream can be the file pointer position of the first data block in the current file stream. After the corresponding relationship of the value and the key in the first dictionary is established, the starting position of the first data block in the binary stream in the corresponding relationship is recorded, and then the name of the third data block and the name of the variable of the data stored in the third data block are additionally written into the corresponding relationship information of the value and the key in the target file according to the file stream.

[0105] As an example, after the initialization of the MAT file is completed, that is, after the placeholder matrix named variable information VARIABLE_INFO is written into the MAT file using the number dictionary D1, the starting position (Position) P1 of the matrix of VARIABLE_INFO in the binary stream is recorded. The file stream is positioned to the starting position P1, and the complete VARIABLE_INFO matrix is written into the MAT file through the number dictionary D1.

[0106] In the embodiments of the present application, the starting position of the first data block in the binary stream is obtained, and based on the starting position of the first data block in the binary stream, the name of the third data block in the first dictionary, and the corresponding relationship information of the value and the key of the variable of the data stored in the third data block, the corresponding relationship information of the value and the key added and written into the target file. In this way, the starting position of the first data block in the binary stream of the target file can be accurately located, and the third data block can be added and written into the target file according to the file stream.

[0107] In some embodiments, the method of saving data can further include:

[0108] Recording the starting position of the second data block in the binary stream in the second dictionary;

[0109] Obtaining the starting position of the second data block in the binary stream;

[0110] Writing the position matrix in the target file, the position matrix can include the starting position of the second data block in the binary stream.

[0111] The second dictionary can be a data dictionary including the starting position Position of the second data block in the binary stream, the key Key of the second dictionary can be the name of the second data block, and the value Value can be the starting position Position of the second data block in the binary file stream. The starting position of the second data block in the binary stream is obtained from the second dictionary, and the position matrix including the starting position of the second data block in the binary stream is written into the target file.

[0112] As an example, when writing variable data into a MAT file, each data block in the data buffer Buffer is written into the MAT file as a matrix, and the starting position Position of the matrix in the binary stream is recorded, and the starting position Position is saved to the digital dictionary D2. Further, the starting position Position of each data block in the binary stream is obtained from the digital dictionary D2, and a matrix of position information POSITION_INFO is written into the MAT file.

[0113] In addition, after each data block in the data buffer Buffer is written into the MAT file as a matrix, the following three parameters can be saved into the global dictionary, which are the numerical dictionary D3 (the dictionary Key is the name of the data block, and the Value is the minimum value of the Time in the data block) and the numerical D4 (the dictionary Key is the name of the data block, and the Value is the maximum value of the Time in the data block) of the minimum value and the maximum value of the Time in the matrix, and the numerical dictionary D5 (the dictionary Key is the name of the data block, and the Value is the number of values stored in the data block) of the number of HIL variable points contained in the matrix. When the MAT file saving is completed, the Buffer from the last saved data to the present is obtained, and the obtained numerical dictionaries D3, D4 and D5 are written into the MAT file as a cell array Cell in the Buffer matrix.

[0114] In the embodiment of the application, the starting position of the second data block in the binary stream is recorded in the second dictionary, and then the starting position of the second data block in the binary stream is obtained, and the position matrix is written in the target file. The position matrix can include the starting position of the second data block in the binary stream. In this way, the efficiency of reading each data block during data reading can be improved.

[0115] In some embodiments, after the corresponding relationship information of the value and the key written into the target file is added based on the name of the third data block and the corresponding relationship information of the name of the variable stored in the third data block in S150, the data saving method can further include:

[0116] The data in the data buffer area is emptied.

[0117] As an example, when it is detected that the data filled by a variable in the Buffer reaches the space size of M / C during the storage of the data blocks, the variable data in the Buffer is written into the MAT file once, and the Buffer is emptied for the next writing of the variable data.

[0118] In the embodiment of the application, the data in the data buffer area is emptied, which facilitates the next writing of the variable data.

[0119] Based on this, Figure 3 A flowchart of a data reading method provided by an embodiment of the application is shown.

[0120] As Figure 3 shown, the data reading method can specifically include the following steps:

[0121] S310, read the corresponding relationship information of the value and the key from the target file, the value representing the name of the second data block storing the variable data, and the key representing the name of the variable.

[0122] S320, obtaining, according to the name of each second data block, a starting position of each second data block in the binary stream from a position matrix in the target file.

[0123] S330, reading data of the variable stored in the second data block based on the starting position of the second data block in the binary stream.

[0124] As an example, when reading data in the MAT file, first, a matrix named variable information VARIABLE_INFO is read from the MAT file, which includes the correspondence between the name of the data block storing the variable data and the variable name. Then, the starting position of each data block in the binary stream is obtained from the position information POSITION_INFO matrix in the MAT file through the name of the data block, the file stream is positioned to the corresponding starting position, and the variable data in the data block is quickly read.

[0125] In the embodiments of the present application, the correspondence information between the value and the key is read from the target file, the value represents the name of the second data block storing the variable data, the key represents the name of the variable, and the starting position of each second data block in the binary stream is obtained from the position matrix in the target file according to the name of the second data block. Further, the data of the variable stored in the second data block is read based on the starting position of the second data block in the binary stream. In this way, the data block to be read can be quickly read through the starting position of the second data block in the binary stream, and the data readability and timeliness are improved.

[0126] In some embodiments, S130 can specifically include:

[0127] Constructing a third dictionary, the third dictionary can include the correspondence information between the value and the key;

[0128] According to the name of each second data block in the third dictionary, a starting position of each second data block in the binary stream is obtained from a position matrix in the target file.

[0129] The third dictionary can be a numerical dictionary with the same structure as the first dictionary, which can be constructed by reading the correspondence information between the value and the key from the target file. The third dictionary can include the correspondence information between the value and the key, the value Value represents the name of the second data block storing the variable data, and the key KEY represents the name of the variable.

[0130] As an example, when reading data in the MAT file, first, a matrix named variable information VARIABLE_INFO is read from the MAT file, the matrix includes the corresponding relationship between the names of the data blocks storing variable data and the variable names, and a number dictionary D3 including the corresponding relationship is constructed. Then, the starting position of each data block in the binary stream is obtained from the position information POSITION_INFO matrix in the MAT file through the Value value of the number dictionary D3, that is, the name of the data block, the file stream is positioned to the corresponding starting position, and the variable data in the data block is quickly read.

[0131] In the embodiment of the application, by constructing the third dictionary, the third dictionary can include the corresponding relationship information of the values and the keys, and the starting position of each second data block in the binary stream is obtained from the position matrix in the target file according to the name of the second data block in the third dictionary, so that the corresponding relationship information of the values and the keys needs to be read every time the data is read, and the efficiency of reading a large amount of data is improved.

[0132] Figure 4 FIG. 4 is a structural schematic diagram of a data saving apparatus 400 according to an example embodiment.

[0133] As shown in Figure 4 , the data saving apparatus 400 can include:

[0134] The obtaining module 401 is configured to obtain the corresponding relationship information of the values and the keys, the value representing the name of the first data block storing the variable data, and the key representing the name of the variable.

[0135] The writing module 402 is configured to write the corresponding relationship information of the values and the keys in the target file.

[0136] The storage module 403 is configured to store the data of the variable in the second data block of the data cache area, and the second data block includes the first data block.

[0137] The writing module 402 is further configured to write the data of the variable in the target file when detecting that the data size of the target variable in the variable of the data cache area meets a condition, and the condition includes the ratio of the size of the data cache area to the number of variables in the corresponding relationship information of the keys and the values.

[0138] The writing module 402 is further configured to add the corresponding relationship information of the values and the keys written in the target file based on the name of the third data block and the corresponding relationship information of the names of the variables stored in the third data block, and the third data block is a data block other than the first data block in the second data block.

[0139] In some embodiments, the obtaining module 401 is further configured to obtain a first dictionary, the first dictionary comprising the correspondence information between the values and the keys.

[0140] In some embodiments, the writing module 402 is further configured to write, into the first dictionary, the correspondence information between the name of the third data block and the name of the variable of the data stored in the third data block, and the corresponding values and keys.

[0141] In some embodiments, the obtaining module 401 is further configured to obtain the starting position of the first data block in the binary stream.

[0142] The writing module 402 is further configured to add, based on the starting position of the first data block in the binary stream, the correspondence information between the name of the third data block and the name of the variable of the data stored in the third data block, and the corresponding values and keys in the first dictionary, the correspondence information between the values and the keys into the target file.

[0143] In some embodiments, the data saving apparatus 400 can further comprise a recording module.

[0144] The recording module is configured to record, in a second dictionary, the starting position of the second data block in the binary stream.

[0145] The obtaining module 401 is further configured to obtain the starting position of the second data block in the binary stream.

[0146] The writing module 402 is further configured to write, into the target file, a position matrix comprising the starting position of the second data block in the binary stream.

[0147] In some embodiments, the data saving apparatus 400 can further comprise an emptying module.

[0148] The emptying module is configured to, after adding, based on the correspondence information between the name of the third data block and the name of the variable of the data stored in the third data block, the correspondence information between the values and the keys into the target file, empty the data in the data buffer.

[0149] Thus, by obtaining the correspondence information of the value and the key, i.e., the correspondence between the name of the first data block storing the variable data and the variable name, the correspondence information of the value and the key is written in the target file, so that each variable corresponding to the stored data block can be preset, which plays a placeholder role and facilitates subsequent writing of complete variable data to the target file. Then, the data of the variable is stored in the second data block of the data cache area. When it is detected that the data size of the target variable in the data cache area meets the condition, the data of the variable is written to the target file, wherein the condition includes the ratio of the size of the data cache area to the number of variables in the correspondence information of the key and the value. In this way, the data can be stored in blocks. Thus, based on the name of the third data block in the second data block except the first data block and the correspondence information of the name of the variable stored in the third data block, the correspondence information of the value and the key written to the target file is added. In this way, the target file can be saved by appending in the data block storage mode, and the efficiency of saving and reading data is improved.

[0150] Figure 5 is a structural schematic diagram of a data reading device 500 according to an exemplary embodiment.

[0151] As shown in Figure 5 , the data reading device 500 can include:

[0152] The reading module 501 is configured to read the correspondence information of the value and the key from the target file, wherein the value represents the name of the second data block storing the variable data, and the key represents the name of the variable.

[0153] The obtaining module 502 is configured to obtain the starting position of each second data block in the binary stream from the position matrix in the target file according to the name of the second data block.

[0154] The reading module 501 is further configured to read the data of the variable stored in the second data block based on the starting position of the second data block in the binary stream.

[0155] In some embodiments, the data reading device 500 can further include a constructing module.

[0156] The constructing module is configured to construct a third dictionary, wherein the third dictionary includes the correspondence information of the value and the key.

[0157] The obtaining module 502 is further configured to obtain the starting position of each second data block in the binary stream from the position matrix in the target file according to the name of the second data block in the third dictionary.

[0158] Thus, by reading the correspondence information of the value and the key from the target file, the value represents the name of the second data block storing the variable data, the key represents the name of the variable, and the starting position of each second data block in the binary stream is obtained from the position matrix in the target file according to the name of the second data block. Further, based on the starting position of the second data block in the binary stream, the data of the variable stored in the second data block is read. In this way, the data block to be read can be quickly read through the starting position of the second data block in the binary stream, and the data readability and timeliness are improved.

[0159] Figure 6 A hardware structure schematic diagram of an electronic device provided by an embodiment of the present application is shown.

[0160] The electronic device can include a processor 601 and a memory 602 storing computer program instructions.

[0161] Specifically, the processor 601 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0162] The memory 602 can include a mass storage for data or instructions. By way of example and not limitation, the memory 602 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 602 can include removable or non-removable (or fixed) media. Where appropriate, the memory 602 can be internal or external to the integrated gateway disaster recovery device. In certain embodiments, the memory 602 is non-volatile solid-state memory.

[0163] The memory can include read-only memory (ROM), random-access memory (RAM), magnetic disk storage mediums, optical storage mediums, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software that, when executed (by one or more processors), is operable to perform operations described with reference to the methods according to an aspect of the present disclosure.

[0164] The processor 601 reads and executes the computer program instructions stored in the memory 602 to implement the method of any one of the above embodiments of data saving.

[0165] In one example, the electronic device can further include a communication interface 603 and a bus 610. As shown, the processor 601, the memory 602, and the communication interface 603 are connected through the bus 610 and complete communication with each other. Figure 6

[0166] The communication interface 603 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0167] The bus 610 includes hardware, software or both to couple the components of a device that stores data with one another. Without limitation, for example, the bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or combination of two or more of these. Where appropriate, the bus 610 can include one or more buses. Although the present embodiments describe and show a particular bus, the present application contemplates any suitable bus or interconnect.

[0168] The electronic device can execute the data saving method in the embodiments of the present application based on the correspondence information of the obtained value and key, the value representing the name of the first data block storing the variable data, and the key representing the name of the variable, thereby realizing the data saving method described in combination Figure 1 with the above embodiments.

[0169] In addition, in combination with the data saving method in the above embodiments, the embodiments of the present application can provide a computer storage medium to realize. The computer storage medium has computer program instructions stored thereon; the computer program instructions are executed by the processor to realize any one of the data saving methods in the above embodiments.

[0170] It should be noted that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted herein. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0171] ​The functions noted in the description of the structural block diagrams above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium, or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. A "machine-readable medium" includes any medium that can store or transport information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. The code segments can be downloaded via computer networks such as the Internet, intranets, and the like.

[0172] It is also important to note that the examples described herein can be implemented in a variety of systems, including and / or incorporating software, firmware, hardware, and / or circuitry. Also, the example need not necessarily be implemented as part of any example environment. It is to be understood that any feature described in this specification can be implemented alone or in combination with other features. It is to be understood that the terminology used herein is for the purpose of describing particular examples only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Expressions such as "at least one of," when preceding a list of two or more items, denote that at least one of the items, but possibly more than one of the items, is present. Expressions such as "one or more of," when preceding a list of two or more items, denote that at least one of the items, but possibly more than one of the items, is present.

[0173] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. Alternatively, computer program

[0174] The above merely describes a specific implementation of the present application. Those skilled in the art can clearly understand the specific working processes of the system, modules and units described above for the convenience and brevity of description, and can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein again. It should be understood that the protection scope of the present application is not limited to this, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A method for data storage, characterized in that: The method comprises Obtaining correspondence information between a value and a key, wherein the value represents the name of a first data block storing variable data, and the key represents the name of the variable; Write a variable information matrix into the target file, wherein the variable information matrix includes information on the correspondence between the values ​​and the keys. The variable information matrix serves as a placeholder. If data is not saved in the target file, the data in the variable information matrix is ​​incomplete. The target file is a MAT file. Storing the data of the variable in a second data block in a data buffer area, where the second data block includes the first data block; When it is detected that the data of the target variable in the variable of the data buffer area meets a condition, writing the data of the variable into the target file, wherein the condition includes a ratio of the size of the data buffer area to the number of variables in the key-value correspondence information; Based on the name of the third data block and the correspondence information of the name of the variable stored in the third data block, the correspondence information of the value and key written to the target file is added; wherein, the third data block is a data block in the second data block except the first data block.

2. The method according to claim 1, characterized in that The obtained value and key correspondence information includes: A first dictionary is obtained, where the first dictionary includes correspondence information between the values ​​and the keys.

3. The method according to claim 2, characterized in that The method further comprises: The name of the third data block, and the name of the variable of the data stored in the third data block, the corresponding value and the corresponding relationship information of the key are written into the first dictionary.

4. The method according to claim 3, characterized in that The adding of the correspondence information between the value and the key written to the target file based on the name of the third data block and the correspondence information between the name of the variable stored in the third data block includes: Obtaining a starting position of the first data block in the binary stream; Based on the starting position of the first data block in the binary stream, the name of the third data block in the first dictionary, and the name of the variable of the data stored in the third data block, the corresponding value and key correspondence information is added, and the value and key correspondence information written to the target file is added.

5. The method according to claim 1, wherein The method further comprises: Recording the starting position of the second data block in the binary stream in the second dictionary; Obtaining a starting position of the second data block in the binary stream; A position matrix is ​​written into the target file, where the position matrix includes a starting position of the second data block in the binary stream.

6. The method according to claim 1, characterized in that After adding the correspondence information between the value and the key written to the target file based on the correspondence information between the name of the third data block and the name of the variable stored in the third data block, the method further includes: Clear the data in the data buffer area.

7. A data reading method, characterized in that: The method comprises: Reading correspondence information between a value and a key from a target file, wherein the value represents a name of a second data block storing variable data, and the key represents a name of the variable; According to the name of the second data block, obtaining the starting position of each second data block in the binary stream from the position matrix in the target file; Based on the starting position of the second data block in the binary stream, the data of the variable stored in the second data block is read, and the data of the variable is stored by the data saving method according to any one of claims 1 to 6.

8. The method according to claim 7, characterized in that The step of obtaining, according to the name of the second data block, a starting position of each second data block in the binary stream from a position matrix in the target file includes: Constructing a third dictionary, wherein the third dictionary includes correspondence information between the values ​​and the keys; According to the name of the second data block in the third dictionary, the starting position of each second data block in the binary stream is obtained from the position matrix in the target file.

9. A data storage device, characterized in that: The device comprises: an acquisition module, configured to acquire information on a correspondence between a value and a key, wherein the value represents a name of a first data block storing variable data, and the key represents a name of the variable; A writing module is used to write a variable information matrix into a target file, wherein the variable information matrix includes information on the correspondence between the values ​​and the keys. The variable information matrix serves as a placeholder. If data is not saved in the target file, the data in the variable information matrix is ​​incomplete. The target file is a MAT file. A storage module, configured to store the data of the variable into a second data block in a data buffer area, where the second data block includes the first data block; The writing module is further configured to write the data of the variable into the target file when it is detected that the data size of the target variable in the variable in the data buffer area meets a condition, wherein the condition includes a ratio of the size of the data buffer area to the number of variables in the key-value correspondence information; The writing module is also used to add the correspondence information of the value and key written to the target file based on the name of the third data block and the correspondence information of the name of the variable stored in the third data block; wherein the third data block is a data block in the second data block other than the first data block.

10. A data reading device, characterized in that: The device comprises: A reading module, configured to read information on a correspondence between a value and a key from a target file, wherein the value represents a name of a second data block storing variable data, and the key represents a name of the variable; An acquisition module, configured to acquire, according to the name of the second data block, a starting position of each second data block in the binary stream from a position matrix in the target file; The reading module is further used to read the data of the variable stored in the second data block based on the starting position of the second data block in the binary stream, and the data of the variable is stored through the data preservation method described in any one of claims 1 to 6.

Images